Skip to content

Conversation

@m-nagarajan
Copy link

@m-nagarajan m-nagarajan commented Sep 2, 2025

Problem:
In AggregationTemporalitySelector#deltaPreferred() : GAUGE seems to fallback to default i,e DELTA, which results in no data getting exported if a GAUGE instrument is not updated in that export interval.

Solution:
Make GAUGE to be CUMULATIVE to consistently export the last set data.

@m-nagarajan m-nagarajan requested a review from a team as a code owner September 2, 2025 22:56
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Sep 2, 2025

CLA Signed

  • ✅login: m-nagarajan / name: Manoj Nagarajan / (3c5a9bb)

The committers listed above are authorized under a signed CLA.

@codecov
Copy link

codecov bot commented Sep 2, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.00%. Comparing base (e59dba5) to head (3c5a9bb).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7634      +/-   ##
============================================
+ Coverage     89.99%   90.00%   +0.01%     
- Complexity     7079     7080       +1     
============================================
  Files           803      803              
  Lines         21412    21412              
  Branches       2086     2086              
============================================
+ Hits          19269    19272       +3     
+ Misses         1479     1477       -2     
+ Partials        664      663       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jkwatson
Copy link
Contributor

jkwatson commented Sep 3, 2025

I believe this behavior is intentional. Do you have evidence to the contrary?

@breedx-splk
Copy link
Contributor

Although it's an unfortunate part of the data model, temporality is effectively irrelevant for gauge metric data points. They're just a measurement. They don't represent a delta over time nor do they represent a cumulative aggregation. Just a value.

The first sentence here about temporality describes it as relevant for "additive quantities", which a gauge isn't.

It also sounds like you might be using a synchronous gauge and expecting it to report/export data even when you haven't performed a measurement? Is that the case?

@breedx-splk breedx-splk added the needs author feedback Waiting for additional feedback from the author label Sep 3, 2025
@m-nagarajan
Copy link
Author

m-nagarajan commented Sep 3, 2025

It also sounds like you might be using a synchronous gauge and expecting it to report/export data even when you haven't performed a measurement? Is that the case?

Yes that is correct. I was expecting the Synchronous and Asynchronous GAUGE will be exported the same way (ie last set data to be exported always) and the only difference would be how the value is set (sync vs async).

The 2 alternatives I considered to get there was

  1. Using Async GAUGE and add a state for it to read from, which works, but introduces a slight level of abstraction between the data path and the metric path, which is fine.
  2. use Sync GAUGE and override the deltaPreferred() like this PR, which worked.

As 2 worked (emits last set data every export interval) and as Synchronous Gauge is added at a later point in time in OpenTelemetry, I wanted to check to see if this is by design or a latent bug.

@github-actions github-actions bot removed the needs author feedback Waiting for additional feedback from the author label Sep 3, 2025
@breedx-splk
Copy link
Contributor

I wanted to check to see if this is by design or a latent bug.

I defer to the maintainers, but I do think this is the intentional design.

I was expecting the Synchronous and Asynchronous GAUGE will be exported the same way (ie last set data to be exported always) and the only difference would be how the value is set (sync vs async).

If you haven't recorded a measurement, I think it would be a mistake for the sdk to continue recording the previous measurement. It sounds like you have your head around the situation, tho. 👍🏻

@m-nagarajan
Copy link
Author

m-nagarajan commented Sep 3, 2025

I believe this behavior is intentional. Do you have evidence to the contrary?

@jkwatson I was expecting the Synchronous and Asynchronous GAUGE will be exported the same way (ie last set data to be exported always) and the only difference would be how the value is set (sync vs async). Testing this PR locally helped me get there ie the last data set() for a synchronous gauge was exported every minute. So, wanted to check if its by design or a latent bug.
image the default vs
image with the change.

@jkwatson
Copy link
Contributor

jkwatson commented Sep 3, 2025

I believe this behavior is intentional. Do you have evidence to the contrary?

@jkwatson I was expecting the Synchronous and Asynchronous GAUGE will be exported the same way (ie last set data to be exported always) and the only difference would be how the value is set (sync vs async). Testing this PR locally helped me get there ie the last data set() for a synchronous gauge was exported every minute. So, wanted to check if its by design or a latent bug.

Yep, I understand, but I was looking for evidence from somewhere (hopefully the specification) that this wasn't the intended behavior (as it is now). I don't want to go changing the behavior of this, when it might be what is intended, and, indeed, expected by the majority of users.

@m-nagarajan
Copy link
Author

Thanks @jkwatson and @breedx-splk.
As I understand that the current behavior is the intended behavior, closing this PR and I will be using a custom deltaPreferred() for my purpose.

@m-nagarajan m-nagarajan closed this Sep 4, 2025
@breedx-splk
Copy link
Contributor

Thanks @m-nagarajan.

For future researchers, we concluded in a slack thread that this section of the spec is what governs this behavior: https://github.com/open-telemetry/opentelemetry-specification/blob/4b630d89d43a07d426878866a22b91f7230c00d9/specification/metrics/sdk.md?plain=1#L1300-L1312

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants