Extended Reporting Service Documentation & Service renaming #146

thomas-nicolai-frequenz · 2025-12-18T16:32:54Z

This PR clarifies resampling and streaming semantics of the Reporting API. The primary focus of this change is documentation correctness and consistency. Some breaking API changes are introduced.

Key improvements:

Resampling is now explicitly defined as time-grid materialization over telemetry data, applied to both historical and real-time streams.
sample_time semantics are clarified:
- origin time when not resampled,
- interval start when resampled.
Resampled streams are time-keyed upsert streams:
- metric values may be unset if no data contributed to an interval,
- intervals may be re-emitted with updated values when late telemetry arrives.
Examples and field documentation were updated to reflect:
- empty values,
- interval-based timestamps,
- non-append-only behavior.

Some schema or wire-format changes were made.

…al to align with MarketMetering API. * Formatting to align with other Frequenz APIs Signed-off-by: tomni <[email protected]>

Signed-off-by: tomni <[email protected]>

* Renamed ReceiveMicrogridElectricalComponentsDataStream RPC * Renamed ReceiveAggregatedMicrogridElectricalComponentsDataStream RPC Signed-off-by: tomni <[email protected]>

* Introduced DownsamplingMethod to ResamplingOptions * Enhanced ResamplingOptions documentation Signed-off-by: tomni <[email protected]>

Signed-off-by: tomni <[email protected]>

…e and ReceiveMicrogridSensorsDataStreamResponse Signed-off-by: tomni <[email protected]>

Signed-off-by: tomni <[email protected]>

stefan-brus-frequenz · 2026-01-06T14:15:27Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+//     algorithm and secret key. All requests to this service must be made
+//     over TLS (HTTPS).
+//
+service ReportingService {


Just FYI, the name change of the service and the RPCs below are considered breaking changes (they don't change the message contents "on the wire" per se, but they will cause older clients to be incompatible). If this is intentional, then it should definitely be mentioned in the release notes.

And also in the PR title. Ideally we could split unproblematic doc updates from breaking changes where possible.

And also in the PR title

Thats done.

then it should definitely be mentioned in the release notes.

OK.

stefan-brus-frequenz · 2026-01-06T14:17:22Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+  // Optional; Time window for matching sample interval start timestamps.
+  //
+  // If omitted, samples from all timestamps are included.
+  frequenz.api.common.v1alpha8.types.Interval interval = 1;


This is also a breaking change, and should be mentioned in the release notes.

stefan-brus-frequenz · 2026-01-06T14:19:28Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+// !!! note "Real-time streams"
+//     For open-ended streams, resampling windows advance according to
+//     wall-clock time. Window emission is time-driven, not event-driven.
+//      As telemetry samples may arrive late (e.g. due to


I would suggest mentioning here that there is no fixed guarantee for when a sample may arrive. This seems like a question that would come up from clients.

I've tried to make that a little more precise.

stefan-brus-frequenz · 2026-01-06T14:20:46Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+// !!! note "Resolution constraints"
+//     The service enforces an upper bound on the resampling resolution.
+//
+//     Resolutions larger than one calendar month are not supported, as


I would reword this, because one calendar month is not something that can be expressed in seconds (a 'calendar month' could be anywhere between 28 and 31 days). Maybe "resolutions larger than 30 days"?

I would even go down to a week, i.e. 7 days. 30 days is not a typical resampling period, but monthly is. However, as Stefan pointed out this would be a new feature and I am not sure if this is in the scope of the resampling of the reporting API. Also, in practice it's just a pandas one-liner on the data.

this would be a new feature and I am not sure if this is in the scope of the resampling of the reporting API. Also, in practice it's just a pandas one-liner on the data.

Right the intention was to allow long term analysis but not needed to receive a lot of data points. But OK, I roll back and limit it.

stefan-brus-frequenz · 2026-01-06T14:23:13Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+//     over TLS (HTTPS).
+//
+service ReportingService {
  // Streams metrics for a list of microgrid components.


Should this also be "electrical components" for consistency?

cwasicki · 2026-01-08T21:18:03Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+//     algorithm and secret key. All requests to this service must be made
+//     over TLS (HTTPS).
+//
+service ReportingService {


And also in the PR title. Ideally we could split unproblematic doc updates from breaking changes where possible.

cwasicki · 2026-01-08T21:21:10Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

-
-  // Optional UTC end time for the query.
-  google.protobuf.Timestamp end_time = 2;
+  // Optional; Time window for matching sample interval start timestamps.


If there is only the interval field left, do we need this message at all?

Well time filter is used in many places and might be extended in the future. If we remove this message we have to duplicate the documentation in many places. You just raised exactly that in the Electricity API.

cwasicki · 2026-01-08T21:27:04Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+// !!! note "Resolution constraints"
+//     The service enforces an upper bound on the resampling resolution.
+//
+//     Resolutions larger than one calendar month are not supported, as


I would even go down to a week, i.e. 7 days. 30 days is not a typical resampling period, but monthly is. However, as Stefan pointed out this would be a new feature and I am not sure if this is in the scope of the resampling of the reporting API. Also, in practice it's just a pandas one-liner on the data.

cwasicki · 2026-01-08T21:42:16Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

+//     For each interval k:
+//         Ik = [T0 + k·ΔT,  T0 + (k+1)·ΔT)
+//
+//     The aggregated value A_k is by default the arithmetic mean of all


That's not true, for some metrics there is another aggregation on resampling, e.g. for cumulative energy metrics it is the first value (i.e. the one closest to the time stamp of the window). For some also min or max make sense, but besides bounds I am not sure if we have such in our data off the top of my head.

cwasicki · 2026-01-08T21:58:06Z

proto/frequenz/api/reporting/v1alpha10/reporting.proto

-  //     This is the original timestamp of the samples that were aggregated.
+  // Semantics depend on whether resampling is enabled:
+  //
+  //   • Without resampling:


I don't think we should support formula aggregation without resampling. If we do, we have to specify clearly by which logic the timestamps from independent data sources are aligned before the formula is applied. If they are aggregated without alignment, in most cases the result will not be useful.

True, but only in case metrics from multiple sources are aggregated with a formula. Not true for single source aggregation but I'll adjust it how you suggested it.

…ting API. Signed-off-by: tomni <[email protected]>

thomas-nicolai-frequenz added 12 commits December 18, 2025 16:42

* Changed TimeFilter to use frequenz.api.common.v1alpha8.types.Interv…

0942da8

…al to align with MarketMetering API. * Formatting to align with other Frequenz APIs Signed-off-by: tomni <[email protected]>

* Extended the header documentation

1ce5186

Signed-off-by: tomni <[email protected]>

* Refined terminology around electrical components

0ca93cd

Signed-off-by: tomni <[email protected]>

* Added at Authentication note to service header.

6e72a5e

Signed-off-by: tomni <[email protected]>

* Renamed ReportingService

8a794fc

* Renamed ReceiveMicrogridElectricalComponentsDataStream RPC * Renamed ReceiveAggregatedMicrogridElectricalComponentsDataStream RPC Signed-off-by: tomni <[email protected]>

* Increase version to v1alpha11

6bcc95f

* Introduced DownsamplingMethod to ResamplingOptions * Enhanced ResamplingOptions documentation Signed-off-by: tomni <[email protected]>

* Refined SimpleAggregatedMetricValue documentation

e00b16a

Signed-off-by: tomni <[email protected]>

* Extended service header around streamed resampling requests

a4bfcd9

Signed-off-by: tomni <[email protected]>

* Updated header documentation.

6f8e6a8

Signed-off-by: tomni <[email protected]>

* Reverted to v1alpha10.

910fe3d

Signed-off-by: tomni <[email protected]>

* Added example to ReceiveAggregatedMicrogridSensorsDataStreamRespons…

3c6f1e5

…e and ReceiveMicrogridSensorsDataStreamResponse Signed-off-by: tomni <[email protected]>

* Added two clarifications to FilterOption and ResamplingOptions

30fa97b

Signed-off-by: tomni <[email protected]>

thomas-nicolai-frequenz requested a review from a team as a code owner December 18, 2025 16:32

thomas-nicolai-frequenz requested a review from eduardiazf December 18, 2025 16:32

github-actions bot added the part:proto Affects the protocol buffer definition files label Dec 18, 2025

docs: add release note for reporting API documentation updates

1087bb0

Signed-off-by: tomni <[email protected]>

github-actions bot added the part:docs Affects the documentation label Dec 18, 2025

thomas-nicolai-frequenz added 3 commits December 18, 2025 17:51

style(proto): sort imports in reporting.proto

2f2611d

Signed-off-by: tomni <[email protected]>

style(proto): sort imports in reporting.proto

6edbc73

Signed-off-by: tomni <[email protected]>

style(proto): sort imports in reporting.proto

166af94

Signed-off-by: tomni <[email protected]>

stefan-brus-frequenz reviewed Jan 6, 2026

View reviewed changes

cwasicki reviewed Jan 8, 2026

View reviewed changes

thomas-nicolai-frequenz changed the title ~~Extended Reporting Service Documentation~~ Extended Reporting Service Documentation & Service renaming Jan 10, 2026

Docs: Clarify resampling semantics and resolution limits in the Repor…

1c83e50

…ting API. Signed-off-by: tomni <[email protected]>

Extended Reporting Service Documentation & Service renaming #146

Are you sure you want to change the base?

Extended Reporting Service Documentation & Service renaming #146

Uh oh!

Conversation

thomas-nicolai-frequenz commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomas-nicolai-frequenz Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomas-nicolai-frequenz Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

thomas-nicolai-frequenz commented Dec 18, 2025 •

edited

Loading

thomas-nicolai-frequenz Jan 10, 2026 •

edited

Loading

thomas-nicolai-frequenz Jan 10, 2026 •

edited

Loading