INFOPLAT-1575 Implement Prometheus to OTel metrics Forwarder

[pkcll]

Implement Prometheus to OTel metrics conversion and export.

Forwarder:

• Gets metrics from prometheus.Gatherer and exports them to provided OTLP/GRPC endpoint. Uses piped Receiver and Exporter.

Receiver:

• Wraps prometheusreceiver
• Fetches prometheus metrics data via prometheus.Gatherer (same process memory, no HTTP calls)
• Uses custom implementation of prometheus.scraper (from here INFOPLAT-1575 Implement metric scraper from prometheus.Gatherer  pkcll/prometheus#1) to shortcut HTTP request calls and fetch data from prometheus.Gatherer
• Converts Prometheus metrics into OTel format using prometheusreceiver
• Passes OTel metrics data to downstream OTel otlpexporter

Exporter:

• Wraps otlpexporter
• Receives metric data from the receiver
• Export OTel metrics data to otel collector endpoint via otlpexporter

Upstream PR's

• core/services/promotel: add Forwarder service chainlink#14597
  • INFOPLAT-1575 Integrate promotel Forwarder chainlink#16121

Downstream PR's

• INFOPLAT-1575 Implement Prometheus to OTel metrics conversion pkcll/opentelemetry-collector-contrib#1
  • INFOPLAT-1575 Update scraper, textparse to support promotel pkcll/prometheus#3

Run Example

cd chainlink-common/pkg/promotel

# Start OTel Collector
docker run \
 -p 127.0.0.1:4317:4317 \
 otel/opentelemetry-collector-contrib:0.118.0

# Run example test
go test -timeout 30s -run ^TestExample$ github.com/smartcontractkit/chainlink-common/pkg/promotel/internal

OTel Collector logs with exported test metric TestExample_test_counter_metric

Metric #4
otelcollectoragent  | Descriptor:
otelcollectoragent  |      -> Name: TestExample_test_counter_metric
otelcollectoragent  |      -> Description:
otelcollectoragent  |      -> Unit:
otelcollectoragent  |      -> DataType: Sum
otelcollectoragent  |      -> IsMonotonic: true
otelcollectoragent  |      -> AggregationTemporality: Cumulative
otelcollectoragent  | NumberDataPoints #0
otelcollectoragent  | StartTimestamp: 2025-02-12 15:27:56.202 +0000 UTC
otelcollectoragent  | Timestamp: 2025-02-12 15:27:56.202 +0000 UTC
otelcollectoragent  | Value: 1.000000

[jmank88]

How did we end up more than doubling (+150) our dependencies? Are these all from otel?

[jmank88]

The k8s deps in particular are concerning. We have had enough trouble with those just in our test modules. I don't think we can allow it to infect everything like this.

[github-actions]

This PR is stale because it has been open 30 days with no activity.
Remove the stale label or comment or this will be closed in 7 days.

[github-actions]

This PR has been automatically closed because it has been stale for > 30 days.
If you wish to continue working on this PR, please reopen it and make any necessary changes.