fix(metrics): Prometheus metrics output

[m3diumrare]

After the sei-chain to sei_chain prefix change in #3554, it surfaced that otel labels also had invalid Prometheus label characters ('.')

Tested on local Sei node, with this patch the Prometheus remote write endpoint no longer drops any metrics from the node.

Example before:

curl -si -H "Accept: application/openmetrics-text;version=1.0.0;escaping=allow-utf-8" http://127.0.0.1:26660/metrics | grep 'telemetry.sdk.language'
target_info{"service.name"="unknown_service:seid","telemetry.sdk.language"="go","telemetry.sdk.name"="opentelemetry","telemetry.sdk.version"="1.39.0"} 1

after:

 curl -si -H "Accept: application/openmetrics-text;version=1.0.0;escaping=allow-utf-8" http://127.0.0.1:26660/metrics | grep 'telemetry.sdk.language'
target_info{service_name="unknown_service:seid",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.39.0"} 1

[cursor]

PR Summary

Low Risk
Observability-only exporter configuration; main risk is label/name changes for existing Prometheus queries or alerts.

Overview
Fixes Prometheus remote write dropping node metrics by translating OpenTelemetry metric/label names into Prometheus-safe form (dots and similar characters become underscores).

Both OTel→Prometheus setup paths now use WithTranslationStrategy(otlptranslator.UnderscoreEscapingWithSuffixes): SetupOtelPrometheus in sei-db and SetupOtelMetricsProvider in utils (still with the sei_chain namespace). github.com/prometheus/otlptranslator is added as a direct module dependency for that option.

Exported labels change shape—for example telemetry.sdk.language becomes telemetry_sdk_language—so scrapers and dashboards that assumed dotted OTel-style label names may need to match the new names.

^{Reviewed by Cursor Bugbot for commit 70d09f2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[m3diumrare]

@philipsu522 this would be great to get into 6.6 release if possible

[m3diumrare]

Hi @Kbhat1 - is there a process for becoming a contributor? I'd like to get this reviewed and merged, and I'm concerned because my last PR fell by the wayside. I'm thinking the Contributor tag makes a difference in reviews here? Thanks

Also this is non-breaking for app hash, but I can't add that label.

[codecov]

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 58.20%. Comparing base (89fcffe) to head (70d09f2).

Files with missing lines	Patch %	Lines
sei-db/common/metrics/setup.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3640      +/-   ##
==========================================
- Coverage   59.16%   58.20%   -0.96%     
==========================================
  Files        2263     2179      -84     
  Lines      187159   177514    -9645     
==========================================
- Hits       110724   103330    -7394     
+ Misses      66484    65028    -1456     
+ Partials     9951     9156     -795     

Flag	Coverage Δ
sei-chain-pr	35.84% <80.00%> (?)
sei-db	70.41% <ø> (ø)
sei-db-state-db	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
utils/metrics/metrics_util.go	88.55% <100.00%> (+0.17%)	⬆️
sei-db/common/metrics/setup.go	0.00% <0.00%> (ø)

... and 84 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[philipsu522]

/backport

[seidroid]

Successfully created backport PR for release/v6.6:

• Backport release/v6.6: fix(metrics): Prometheus metrics output #3673