Skip to content

Add per-database throughput attribution metrics to EventProducer#1002

Merged
akshayrai merged 1 commit intolinkedin:masterfrom
akshayrai:akrai/per-datastream-bytes-rate
Mar 17, 2026
Merged

Add per-database throughput attribution metrics to EventProducer#1002
akshayrai merged 1 commit intolinkedin:masterfrom
akshayrai:akrai/per-datastream-bytes-rate

Conversation

@akshayrai
Copy link
Collaborator

Summary

  • Adds bytesProducedRate and eventProduceRate metrics keyed by source database (EventProducer.db.<databaseName>.*) to EventProducer. These metrics are emitted on successful Kafka ack, gated by the enableThroughputAttributionMetrics config flag (default off).
  • Covers CDC connectors automatically via URI parsing. No connector-specific changes needed. BMM pipelines (kafka://) emit no database metric since there is no source database concept.
  • Aggregate and per-connector-type meters are also emitted for rollups.

Testing Done

  • Includes unit tests

@akshayrai akshayrai force-pushed the akrai/per-datastream-bytes-rate branch from 76ade02 to 4307f89 Compare March 17, 2026 06:32
@akshayrai akshayrai marked this pull request as ready for review March 17, 2026 06:59
@akshayrai akshayrai merged commit a33006a into linkedin:master Mar 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants