diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 14cd83b..ecf6a6d 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -3,6 +3,9 @@ ## Summary - The behavior of the TimeFilter has changed, and its documentation has been updated accordingly. +- Clarified resampling and streaming semantics in the Reporting API, including interval-based timestamps, empty values, and update (upsert) behavior for resampled streams. +- Renamed the service to align with other Frequenz services. +- TimeFilter has been changed to make use of frequenz.api.common.v1alpha8.types.Interval ## Upgrading diff --git a/proto/frequenz/api/reporting/v1alpha10/reporting.proto b/proto/frequenz/api/reporting/v1alpha10/reporting.proto index ea498a2..ab017b3 100644 --- a/proto/frequenz/api/reporting/v1alpha10/reporting.proto +++ b/proto/frequenz/api/reporting/v1alpha10/reporting.proto @@ -12,69 +12,205 @@ syntax = "proto3"; package frequenz.api.reporting.v1alpha10; -import "google/protobuf/timestamp.proto"; - import "frequenz/api/common/v1alpha8/metrics/metrics.proto"; import "frequenz/api/common/v1alpha8/microgrid/electrical_components/electrical_components.proto"; import "frequenz/api/common/v1alpha8/microgrid/microgrid.proto"; import "frequenz/api/common/v1alpha8/microgrid/sensors/sensors.proto"; +import "frequenz/api/common/v1alpha8/types/interval.proto"; +import "google/protobuf/timestamp.proto"; -// The Reporting service provides services for real-time and historical metrics monitoring -// of various microgrid components like Batteries, EV Chargers, and Inverters. +// The Reporting service provides access to historical and real-time telemetry +// data from microgrids. +// +// It exposes raw metrics originating from microgrid electrical components +// and sensors. +// +// Electrical components represent assets involved in energy conversion, +// storage, or flow within a microgrid (e.g. batteries, inverters, EV +// chargers, meters). Sensors represent auxiliary measurement devices that +// are not part of the electrical topology (e.g. temperature, irradiance, +// environmental, or external reference sensors). +// +// In addition to raw metrics, clients may optionally request server-side +// aggregation and resampling of metrics, applied deterministically to both +// historical and real-time telemetry data based on explicitly provided +// parameters. +// +// Aggregation is defined using user-provided formulas, while resampling +// allows telemetry samples to be grouped into fixed time windows. +// +// The service is designed as a general-purpose data access layer for telemetry +// analysis, notebooks, forecasting, trading, and other downstream analytics +// and optimization systems. The service is streaming-first and enables the +// replay of historical telemetry data for a requested time range, and +// may continue with real-time telemetry updates if specified. +// +// Aggregation, resampling, and filtering are all optional and performed +// server-side according to the parameters provided in the request. +// +// 1. Metrics and telemetry +// +// In the Reporting API, a metric represents a well-defined physical or +// logical quantity (e.g. voltage, active power, state of charge, temperature) +// measured at a specific point in time. +// +// Metrics originate from telemetry samples produced by microgrid electrical +// components or sensors. Each telemetry sample: +// • has a UTC timestamp, +// • represents exactly one metric value, +// • may be associated with a logical connection or channel, +// • may optionally include bounds or operational state. +// +// Aggregated metrics are derived exclusively from telemetry samples. +// No additional inference, estimation, or semantic transformation is +// performed by the Reporting API. +// +// 2. Aggregation formulas +// +// An aggregation formula defines how multiple telemetry samples for the +// same metric are combined into a single aggregated value. +// +// Aggregation formulas are optional and provided explicitly by the client. +// They are evaluated deterministically by the server and may: +// • reference electrical component identifiers, +// • reference sensor identifiers, +// • use basic mathematical operators, +// • use supported aggregation functions such as sum() or avg(). +// +// Aggregation formulas operate only on telemetry samples and do not introduce +// additional inference, estimation, or business logic. +// +// 3. Resampling +// +// Resampling is an optional operation that groups telemetry samples into +// fixed, contiguous time windows and produces at most one output sample per +// window. +// +// Resampling applies uniformly to historical replay and real-time streaming +// data. +// +// When resampling is enabled: +// • samples are grouped using left-closed, right-open time intervals, +// • all samples within a window are aggregated, +// • the resulting timestamp represents the start of the resampling +// interval [t, t + ΔT) +// • every stream must be treated as an upsert stream keyed by +// (aggregation_config, sample.sample_time): +// - intervals may be emitted with an unset sample value, and +// - intervals may be re-emitted later with updated values due to +// late-arriving telemetry. +// +// If no resampling resolution is specified, data is returned in its original +// resolution without modification. +// +// 4. Time windows and stream lifecycle +// +// All time filtering follows a left-closed, right-open interval convention: +// +// [start_time, end_time) // -// Utilize these APIs for tasks ranging from real-time metric streaming to complex, -// formula-driven historical data aggregation. For detailed information on component -// categories and the metrics they provide, please consult the referenced component categories -// imported above. +// A bounded time window results in a finite stream that closes automatically +// once all matching samples have been emitted. // -service Reporting { - // Streams metrics for a list of microgrid components. +// An unbounded time window (i.e. no end_time) results in a stream that may +// continue emitting real-time samples indefinitely. +// +// 5. Data inclusion (Filtering) +// +// Some data categories (e.g. bounds, operational states) are excluded by +// default. +// +// Clients may explicitly request inclusion of this data in order to: +// • reduce payload size when not needed, +// • minimize server-side computation, +// • keep high-frequency streams efficient and predictable. +// +// Unless explicitly requested, optional data MUST NOT be assumed to be +// present in responses. +// +// !!! important "Authentication" +// All requests to the Reporting service must be signed. The key identifier +// and signature must be included in the request metadata (gRPC metadata +// / HTTP headers). The signature must be computed using the HMAC-SHA256 +// algorithm and secret key. All requests to this service must be made +// over TLS (HTTPS). +// +service ReportingService { + // Streams metrics for a list of microgrid electrical components. // // !!! note - // This RPC endpoint streams metrics data for components of one or more microgrids. - // Clients need to provide at least one microgrid ID and one component ID - // to get a result. - rpc ReceiveMicrogridComponentsDataStream(ReceiveMicrogridComponentsDataStreamRequest) - returns (stream ReceiveMicrogridComponentsDataStreamResponse) {} - - // Streams aggregated metrics based on user-defined formulas for a list of microgrid - // components. + // This RPC endpoint streams metrics data for electrical components of one + // or more microgrids. Clients need to provide at least one microgrid + // ID and one electrical component ID to get a result. + rpc ReceiveMicrogridElectricalComponentsDataStream + (ReceiveMicrogridElectricalComponentsDataStreamRequest) + returns (stream ReceiveMicrogridElectricalComponentsDataStreamResponse) {} + + // Streams aggregated metrics based on user-defined formulas for a list of + // microgrid electrical components. // // !!! note - // This RPC endpoint retrieves aggregated historical metrics data based on user-defined - // formulas for microgrid components. At least one formula for one microgrid must be provided - // for a valid result. - rpc ReceiveAggregatedMicrogridComponentsDataStream(ReceiveAggregatedMicrogridComponentsDataStreamRequest) - returns (stream ReceiveAggregatedMicrogridComponentsDataStreamResponse) {} + // This RPC endpoint retrieves aggregated historical metrics data based + // on user-defined formulas for microgrid electrical components. At least + // one formula for one microgrid must be provided for a valid result. + rpc ReceiveAggregatedMicrogridElectricalComponentsDataStream + (ReceiveAggregatedMicrogridElectricalComponentsDataStreamRequest) + returns (stream ReceiveAggregatedMicrogridElectricalComponentsDataStreamResponse) {} // Streams metrics for a list of microgrid sensors. // // !!! note - // This RPC endpoint streams metrics data for sensors of one or more microgrids. - // Clients need to provide at least one microgrid ID and one sensor ID - // to get a result. - rpc ReceiveMicrogridSensorsDataStream(ReceiveMicrogridSensorsDataStreamRequest) + // This RPC endpoint streams metrics data for sensors of one or more + // microgrids. Clients need to provide at least one microgrid ID and + // one sensor ID to get a result. + rpc ReceiveMicrogridSensorsDataStream + (ReceiveMicrogridSensorsDataStreamRequest) returns (stream ReceiveMicrogridSensorsDataStreamResponse) {} - // Streams aggregated metrics based on user-defined formulas for a list of microgrid - // sensors. + // Streams aggregated metrics based on user-defined formulas for a list of + // microgrid sensors. // // !!! note - // This RPC endpoint retrieves aggregated historical metrics data based on user-defined - // formulas for microgrid sensors. At least one formula for one microgrid must be provided - // for a valid result. - rpc ReceiveAggregatedMicrogridSensorsDataStream(ReceiveAggregatedMicrogridSensorsDataStreamRequest) + // This RPC endpoint retrieves aggregated historical metrics data based + // on user-defined formulas for microgrid sensors. At least one formula + // for one microgrid must be provided for a valid result. + rpc ReceiveAggregatedMicrogridSensorsDataStream + (ReceiveAggregatedMicrogridSensorsDataStreamRequest) returns (stream ReceiveAggregatedMicrogridSensorsDataStreamResponse) {} } -// Defines whether to include results in the response message. +// Defines whether a specific data category should be included in the response. +// +// The FilterOption enum is used in `IncludeOptions` to explicitly control +// whether optional data blocks (e.g. bounds, operational states) are emitted +// in stream responses. +// +// If a FilterOption is omitted, the default behavior is equivalent to +// FILTER_OPTION_EXCLUDE. +// enum FilterOption { + // UNSPECIFIED: Default unspecified value — must not be used. FILTER_OPTION_UNSPECIFIED = 0; + + // EXCLUDED: Explicitly exclude the corresponding data from the response. FILTER_OPTION_EXCLUDE = 1; + + // INCLUDED: Explicitly include the corresponding data in the response. FILTER_OPTION_INCLUDE = 2; } -// Time-filter defining the historical window and real-time behaviour of the stream. +// Defines how multiple native samples are combined when downsampling +// to a coarser resolution. +enum DownsamplingMethod { + // UNSPECIFIED: Defaults to MEAN. + DOWNSAMPLING_METHOD_UNSPECIFIED = 0; + + // MEAN: Arithmetic mean of all samples within the interval. + DOWNSAMPLING_METHOD_MEAN = 1; +} + +// Time-filter defining the historical window and real-time behaviour of the +// stream. // // Semantics: // - If `end_time` is provided, it MUST be ≤ the current server time. The @@ -93,39 +229,126 @@ enum FilterOption { // // Validation rules: // - end_time > now() → INVALID_ARGUMENT +// message TimeFilter { - // Optional UTC start time for the query. - google.protobuf.Timestamp start_time = 1; - - // Optional UTC end time for the query. - google.protobuf.Timestamp end_time = 2; + // Optional; Time window for matching sample interval start timestamps. + // + // If omitted, samples from all timestamps are included. + frequenz.api.common.v1alpha8.types.Interval interval = 1; } -// Resampling options for aggregated microgrid components data. -// If data is resampled, all samples that fall in a left-closed time interval -// determined by the resolution will be aggregated. -// At the moment only mean aggregation is supported. -// The timestamp for each aggregated sample represents the beginning of its corresponding -// time interval, marking the earliest point from which data was aggregated. +// Resampling options for microgrid electrical component or sensor telemetry. +// +// Resampling allows clients to request a regular, uniform time grid onto +// which high-resolution telemetry samples are aggregated. The service +// operates on telemetry data that may be sampled at sub-second resolution +// and whose sampling characteristics can differ between microgrids, +// electrical components, and sensors. Resampling therefore materializes a +// fixed time grid and aggregates all telemetry samples whose timestamps fall into each +// grid interval. +// +// The output time series contains at most one sample per interval. The +// timestamp of each output sample equals the start of the corresponding +// interval. +// +// !!! note "Sparse telemetry and fine-grained resolutions" +// If the requested resampling resolution is finer than the effective +// sampling density of the telemetry, some resampling intervals may +// contain no telemetry samples. +// +// In such cases, the corresponding output sample will have no value +// set. The Reporting service does not interpolate, extrapolate, or carry +// values forward across intervals. +// +// !!! note "Resolution constraints" +// Resampling resolutions are fixed-duration intervals expressed in +// seconds and are not calendar-aware. The service enforces an upper +// bound on the resampling resolution. +// +// Resolutions larger than 7 days are not supported, as they exceed the +// intended scope of fixed-duration telemetry resampling. +// +// Typical and recommended resolutions include 5 minutes, 15 minutes, +// 1 hour, 1 day, and 7 days. Longer-horizon or calendar-based aggregation +// (for example monthly analysis) should be performed client-side on the +// resampled data. +// +// !!! example "Example: Downsampling from 1-second to 15-minute intervals" +// The stored time series has a effective sampling resolution of Δt = 1 +// second. Resampling is requested with a resolution of ΔT = 15 minutes. In +// this case all native samples whose timestamps fall within an interval +// [t, t + Δt) contribute to the aggregated value for that interval. The +// timestamp of the aggregated sample is exactly t, the start of the +// interval. +// +// For each interval k: +// Ik = [T0 + k·ΔT, T0 + (k+1)·ΔT) +// +// The aggregated value A_k is computed using the configured aggregation +// method (see `downsampling_method`) and may additionally follow +// metric-specific semantics. +// +// For example, for instantaneous metrics the arithmetic mean is commonly +// used, while for cumulative (counter-like) metrics averaging is not +// meaningful and a representative value within the interval is used +// instead. +// +// The resampled series consists of points: +// (t = T0 + k·ΔT, value = A_k) +// +// !!! note "Real-time streams" +// For open-ended streams, resampling windows advance according to +// wall-clock time. Window emission is time-driven, not event-driven. +// +// The service does not provide a fixed guarantee for when an interval’s +// sample will be emitted. In particular, delivery latency may vary due to +// buffering, transport delays, upstream batching, scheduling, or load. +// Clients MUST NOT rely on sample arrival time to infer measurement time +// or completeness. +// +// Because telemetry samples may arrive late, an interval MAY be emitted +// before all telemetry for that interval has arrived. In consequence, a +// resampling interval that has already been emitted MAY be re-emitted +// later with an updated value when additional samples for that interval +// arrive. +// +// Clients MUST be able to handle interval updates (upserts) and MUST NOT +// assume that emitted intervals are final. +// message ResamplingOptions { - // Optional resampling resolution for the data, represented in seconds. - // If omitted, data will be returned in its original resolution. + // Optional resampling resolution, expressed in seconds. The resolution + // MUST NOT exceed the maximum supported resampling interval. + // + // If omitted, telemetry is returned in its original, event-driven form + // without resampling. optional uint32 resolution = 1; + + // Optional aggregation method applied when downsampling. + // Defaults to DOWNSAMPLING_METHOD_MEAN when omitted. + DownsamplingMethod downsampling_method = 2; } -// Message defining the aggregation configuration for a custom formula within a specific microgrid. +// Message defining the aggregation configuration for a custom formula within +// a specific microgrid. // -// The AggregationConfig allows clients to specify how metrics should be aggregated across -// microgrid components. It serves as the configuration guideline for any aggregation operation. +// The AggregationConfig allows clients to specify how metrics should be +// aggregated across microgrid electrical components. It serves as the +// configuration guideline for any aggregation operation. +// +// Aggregation formulas operate on exactly one metric per configuration. +// Cross-metric aggregation is not supported. // // !!! example -// To calculate the total voltage across three components in series with IDs 1, 2, and 3, the -// configuration could look like: +// To calculate the total voltage across three electrical components in +// series with IDs 1, 2, and 3, the configuration could look like: +// +// ```json // { // microgrid_id: 42, // metric: DC_VOLTAGE_V, // aggregation_formula: "#1 + #2 + #3" // } +// ``` // message AggregationConfig { // ID of the microgrid for which the formula is being specified. @@ -133,71 +356,103 @@ message AggregationConfig { // The metric that is to be aggregated using the specified formula. // - // !!! caution - // Ensure the chosen metric is supported by all relevant microgrid components. - // Failure to meet these conditions might results in an aggregation error. + // !!! caution "Supported metrics" + // Ensure the chosen metric is supported by all relevant microgrid + // electrical components. Failure to meet these conditions might results + // in an aggregation error. frequenz.api.common.v1alpha8.metrics.Metric metric = 2; // The formula used for aggregating the component metric for this microgrid. // // !!! info // Two types of aggregation formulas are supported: - // 1. Aggregate functions: These are standard aggregation functions like `sum()` and `avg()`. + // 1. Aggregate functions: These are standard aggregation functions + // like `sum()` and `avg()`. // Example: `sum(#1,#2)`, `avg(#3,#4)` // - // 2. Dedicated formulas: These support basic math operators while concatenating microgrid - // component IDs. + // 2. Dedicated formulas: These support basic math operators while + // concatenating microgrid electrical component IDs. // Example: `#1 + #2 - #3`, `(#3 * #2) /# 1` // - // Additionally, metric connection identifiers can be used in the formula, for components - // that e.g. provide the same metric from multiple connections/sensors. - // Example: `sum(#1[connection0], #2, #3)`, `#1[connection0] + #2[connection1]` + // Additionally, metric connection identifiers can be used in the + // formula, for electrical components that e.g. provide the same metric + // from multiple connections/sensors. + // + // Example: + // `sum(#1[connection0], #2, #3)`, `#1[connection0] + #2[connection1]` string aggregation_formula = 3; } -// Defines a single aggregated metric sample, which represents a consolidated metric value -// over a collection of microgrid components within a specific timeframe. +// Defines a single aggregated metric value. +// +// This message represents the aggregated - across multiple electrical +// components or sensors- result for one metric and one aggregation +// configuration. It is particularly useful in scenarios where an overall +// metric needs to be calculated for a set of components. For instance, it +// can be used to represent the average voltage across multiple sources +// within a microgrid or to calculate the average state of charge of several +// batteries. // -// This message serves to encapsulate aggregate metrics derived from multiple component -// measurements. It is particularly useful in scenarios where an overall metric needs to be -// calculated for a set of components. For instance, you could use it to represent the average -// voltage across multiple sources within a microgrid or to calculate the average state of -// charge of several batteries. +// !!! note "Resampling is required" +// Aggregated/formula streams always use resampling to align measurements +// from various sources in time. // message SimpleAggregatedMetricValue { - // The UTC sample timestamp for the aggregated metrics. + // The UTC timestamp identifying the resampling interval. // - // !!! info - // This is the original timestamp of the samples that were aggregated. + // This timestamp is the start of the resampling interval [t, t + ΔT) to + // which contributing telemetry measurements are assigned based on their + // origin timestamps. + // + // `sample_time` refers to measurement time (interval start), not processing + // or emission time. google.protobuf.Timestamp sample_time = 1; - // The aggregated value of the metric. - // float value = 2; + // The aggregated metric value. + // + // !!! note "Unset values (resampling)" + // When resampling is enabled, the sample value MAY be unset if no + // telemetry measurements were observed within the corresponding + // resampling interval. + // + // !!! important "Late measurements and updates" + // Telemetry measurements are assigned based on their origin timestamps, + // not their arrival time. As a result, when resampling is enabled, + // previously emitted intervals MAY be re-emitted later with updated + // values if additional measurements arrive for the same interval. + // + // Clients MUST be able to handle such updates and MUST NOT assume that + // emitted values are final. frequenz.api.common.v1alpha8.metrics.SimpleMetricValue sample = 2; } -// Message defining a metric to receive data for, optionally filtering by connections. +// Message defining a metric to receive data for, optionally filtering by +// connections. // -// This message allows clients to request data for a specific metric, and optionally -// filter the data by connection identifiers. This is particularly useful when a metric can -// originate from multiple connections within a component, such as different DC strings for -// batteries or PV arrays connected to a hybrid inverter. +// This message allows clients to request data for a specific metric, and +// optionally filter the data by connection identifiers. This is particularly +// useful when a metric can originate from multiple connections within a +// component, such as different DC strings for batteries or PV arrays +// connected to a hybrid inverter. // // !!! note -// The `MetricConnections` message enables fine-grained control over which are included in the data stream. -// If multiple `MetricConnections` messages are specified for different metrics, the filters apply -// separately to each metric. -// The `connections` specified here correspond to the `connections` field in the `MetricSample` message -// defined in the common API. By filtering on `connections`, you effectively select which `MetricSample` -// messages to receive based on their `connections` value. +// The `MetricConnections` message enables fine-grained control over +// which are included in the data stream. If multiple `MetricConnections` +// messages are specified for different metrics, the filters apply +// separately to each metric. The `connections` specified here correspond +// to the `connections` field in the `MetricSample` message defined in +// the common API. By filtering on `connections`, you effectively select +// which `MetricSample` messages to receive based on their `connections` +// value. // // !!! example -// Suppose you have a hybrid inverter that provides the `VOLTAGE` metric from both its battery and -// PV array. If you're only interested in the voltage measurements from the battery, you can use -// `Metrics` message to specify the `VOLTAGE` metric and set `connections` to `["dc_battery_1"]` -// for the DC string connected to a battery. +// Suppose you have a hybrid inverter that provides the `VOLTAGE` metric +// from both its battery and PV array. If you're only interested in the +// voltage measurements from the battery, you can use `Metrics` message +// to specify the `VOLTAGE` metric and set `connections` to +// `["dc_battery_1"]` for the DC string connected to a battery. // -// ``` +// ```json // { // "metric": "VOLTAGE", // "connections": ["dc_battery_1"] @@ -207,29 +462,35 @@ message SimpleAggregatedMetricValue { message MetricConnections { // The metric for which data is requested. // - // If `connections` are specified, only data for this metric from those connections will be returned. - // If no `connections` are specified, data from all connections for this metric will be returned. + // If `connections` are specified, only data for this metric from those + // connections will be returned. If no `connections` are specified, data + // from all connections for this metric will be returned. frequenz.api.common.v1alpha8.metrics.Metric metric = 1; - // An optional list of connections to filter the data for the specified metric. + // An optional list of connections to filter the data for the specified + // metric. // - // Each connection corresponds to the `connections` field in `MetricSample` messages - // (see the common API definition). - // If this list is empty or not specified, data from all sources for this metric will be returned. + // Each connection corresponds to the `connections` field in `MetricSample` + // messages (see the common API definition). If this list is empty or not + // specified, data from all sources for this metric will be returned. // // !!! important - // The connections identifiers must match exactly those used in the `MetricSample` messages. - // Ensure that the connections specified here are valid for the given metric and component. + // The connections identifiers must match exactly those used in the + // `MetricSample` messages. Ensure that the connections specified here + // are valid for the given metric and component. // // !!! important - // There is an important distinction between these connection strings, and the `ComponentConnection` - // message in the common API. A connection string here may, unlike the `ComponentConnection` message, - // refer to a connection that is not present in the microgrid's component graph. + // There is an important distinction between these connection strings, + // and the `ComponentConnection` message in the common API. A + // connection string here may, unlike the `ComponentConnection` message, + // refer to a connection that is not present in the microgrid's + // component graph. // // !!! example - // For a metric that can come from multiple DC strings in a PV array, you might specify: + // For a metric that can come from multiple DC strings in a PV array, + // you might specify: // - // ``` + // ```json // { // "connections": ["dc_string_1", "dc_string_2"] // } @@ -239,63 +500,76 @@ message MetricConnections { } // Request message for receiving stream of metrics, such as electrical -// measurements, and other information for individual microgrid components. +// measurements, and other information for individual microgrid electrical +// components. // // !!! note -// In addition to the raw metrics, the API can also return additional information -// like errors or operational states of the components during the specified time period. +// In addition to the raw metrics, the API can also return additional +// information like errors or operational states of the +// electrical components during the specified time period. // -message ReceiveMicrogridComponentsDataStreamRequest { - // Include options for filtering microgrid components data. +message ReceiveMicrogridElectricalComponentsDataStreamRequest { + // Include options for filtering microgrid electrical components data. // // !!! note // Specifies which additional fields should be included in the response. // message IncludeOptions { - // Optional bound inclusion. By default, bounds are not included in the response. + // Optional bound inclusion. By default, bounds are not included in the + // response. optional FilterOption bounds = 1; - // Optional operational state inclusion. By default, states are not included in the response. + // Optional operational state inclusion. By default, states are not + // included in the response. optional FilterOption states = 2; } - // General filter criteria for querying microgrid components data. + // General filter criteria for querying microgrid electrical components data. // // !!! note // The filter criteria defined here are applied universally across all - // specified microgrids and their respective components. + // specified microgrids and their respective electrical components. message StreamFilter { - // Optional resampling options like resolution for the data, represented in seconds. + // Optional resampling options like resolution for the data, represented + // in seconds. + // // If omitted, data will be returned in its original representation. ResamplingOptions resampling_options = 1; - // Include options specifying additional fields to be included in the response. + // Include options specifying additional fields to be included in the + // response. IncludeOptions include_options = 2; // Optional time-based filter criteria. - // If omitted, data will start streaming from the timestamp that the request was received. + // If omitted, data will start streaming from the timestamp that the + // request was received. TimeFilter time_filter = 3; } - // Encapsulates the microgrid ID and the component IDs within that microgrid for which - // the historical data should be retrieved. + // Encapsulates the microgrid ID and the component IDs within that + // microgrid for which the historical data should be retrieved. // // !!! note - // Each entry in this repeated field associates a microgrid ID with its respective - // component IDs. At least one such association must be provided for a valid request. - repeated frequenz.api.common.v1alpha8.microgrid.MicrogridElectricalComponentIDs microgrid_components = 1; + // Each entry in this repeated field associates a microgrid ID with its + // respective component IDs. At least one such association must be + // provided for a valid request. + repeated frequenz.api.common.v1alpha8.microgrid. + MicrogridElectricalComponentIDs microgrid_components = 1; // List of metrics to receive data for, each possibly with connection filters. // // !!! note - // At least one metric must be specified. Each metric can have an optional list of connections. - // If connections are provided for a metric, only data from those connections will be returned. - // If no connections are provided for a metric, data from all connections will be returned for that metric. + // At least one metric must be specified. Each metric can have an + // optional list of connections. If connections are provided for a + // metric, only data from those connections will be returned. + // If no connections are provided for a metric, data from all + // connections will be returned for that metric. // // !!! example - // To receive `VOLTAGE` data only from the battery connection and `CURRENT` data from all connections: + // To receive `VOLTAGE` data only from the battery connection and + // `CURRENT` data from all connections: // - // ``` + // ```json // { // "metrics": [ // { @@ -304,94 +578,113 @@ message ReceiveMicrogridComponentsDataStreamRequest { // }, // { // "metric": "CURRENT" - // // No connections specified; data from all connections will be returned for CURRENT + // // No connections specified; data from all connections will be + // // returned for CURRENT // } // ] // } // ``` repeated MetricConnections metrics = 2; - // General filter that applies to the data retrieval for all specified microgrids and components. + // General filter that applies to the data retrieval for all specified + // microgrids and electrical components. // // !!! note - // The filter can specify a list of metrics to be return but also specify bounds, operational - // state, or errors to be returned. + // The filter can specify a list of metrics to be return but also + // specify bounds, operational state, or errors to be returned. StreamFilter filter = 3; } -// Response containing a single data sample for one microgrid's components. +// Response containing telemetry data for one microgrid's electrical components. // -// !!! note -// The microgrid's components are provided as single data samples that -// encapsulate metrics, bounds, errors, and operational states along with their -// associated timestamps. Each response message covers a single microgrid. -// If multiple microgrids are provided in the request, expect sequential messages -// in the stream. +// The microgrid's electrical components are provided as telemetry samples that +// encapsulate metrics, bounds, and operational states along with their +// associated measurement timestamps. // -// !!! example -// Example output structure: +// Each response message covers a single microgrid. If multiple microgrids are +// provided in the request, expect sequential messages in the stream. +// +// !!! example "Example output structure (resampling enabled)" +// This example shows telemetry for a single electrical component when +// resampling is enabled. All metric samples share the same `sample_time`, +// which represents the start of the resampling interval [t, t + ΔT). +// Metric values may be present if telemetry measurements contributed to +// the interval, or unset if no measurements were observed at the time of +// emission. State data is not resampled and always uses the origin time +// at which the state was observed. // ``` // { // microgrid_id: 1, // components: [ // { -// component_id: 13, -// metric_samples: [ -// /* all metrics belong to the same timestamp */ -// /* sample is type MetricSampleVariant */ -// { sample_time: "2023-10-01T00:00:00Z", metric: "DC_VOLTAGE_V", sample: {...}, bounds: {...} }, -// { sample_time: "2023-10-01T00:00:00Z", metric: "DC_CURRENT_A", sample: {...}, bounds: {...} } -// ], -// state: { -// sample_time: "2023-10-01T00:00:13.12Z", -// states: ["COMPONENT_STATE_CHARGING", "CABLE_CONNECTED"], -// warnings: [] -// errors: [], -// ] -// }, -// { -// component_id: 59, +// electrical_component_id: 13, // metric_samples: [ -// /* all metrics belong to the same timestamp */ -// /* sample is type MetricSampleVariant */ -// { sample_time: "2023-10-01T00:00:00Z", metric: "DC_VOLTAGE_V", sample: {...}, bounds: {...} }, -// { sample_time: "2023-10-01T00:00:00Z", metric: "DC_CURRENT_A", sample: {...}, bounds: {...} } +// { +// // Interval start time (resampling enabled) +// sample_time: "2023-10-01T00:00:00Z", +// metric: "METRIC_DC_VOLTAGE", +// value: { +// // value present because telemetry contributed +// simple_metric: { value: 720.5 } +// }, +// bounds: { ... } +// }, +// { +// // Same interval, but no telemetry contributed for this metric +// sample_time: "2023-10-01T00:00:00Z", +// metric: "METRIC_DC_CURRENT", +// // value intentionally unset +// bounds: { ... } +// } // ], -// state: { -// sample_time: "2023-10-01T00:00:13.12Z", -// states: ["COMPONENT_STATE_ERROR"], -// warnings: [] -// errors: ["COMPONENT_ERROR_CODE_BATTERY_RELAY_ERROR"], +// state_snapshots: [ +// { +// // State snapshots are not resampled and use origin time +// origin_time: "2023-10-01T00:00:13.12Z", +// states: ["ELECTRICAL_COMPONENT_STATE_CODE_CHARGING"], +// warnings: [], +// errors: [] +// } // ] // } // ] // } // ``` // -message ReceiveMicrogridComponentsDataStreamResponse { - // Microgrid ID for which the components and samples are reported. +message ReceiveMicrogridElectricalComponentsDataStreamResponse { + // Microgrid ID for which the electrical components and samples are reported. uint64 microgrid_id = 1; - // List of components within this microgrid, each with its associated data samples. - repeated frequenz.api.common.v1alpha8.microgrid.electrical_components.ElectricalComponentTelemetry components = 2; + // List of electrical components within this microgrid, each with its + // associated data samples. + // + // !!! important "Empty values and updates" + // When resampling is enabled, metric sample values MAY be unset for a + // given resampling interval and MAY be updated later if additional + // telemetry measurements arrive. Clients MUST be able to handle empty + // values and updates. + repeated frequenz.api.common.v1alpha8.microgrid.electrical_components. + ElectricalComponentTelemetry components = 2; } -// Message defining the request format for streaming aggregated historical metrics. -// This request allows to specify custom aggregation formulas, along with general -// filtering. +// Message defining the request format for streaming aggregated historical +// metrics. // -// At least one aggregation formula config must be provided. The aggregation -// follows the passive sign convention. +// This request allows to specify custom aggregation formulas, along with +// general filtering. At least one aggregation formula config must be +// provided. The aggregation follows the passive sign convention. // -message ReceiveAggregatedMicrogridComponentsDataStreamRequest { - // General filter criteria for querying microgrid components data. +message ReceiveAggregatedMicrogridElectricalComponentsDataStreamRequest { + // General filter criteria for querying microgrid electrical components data. message AggregationStreamFilter { - // Optional resampling options like resolution for the data, represented in seconds. + // Optional resampling options like resolution for the data, represented + // in seconds. // If omitted, data will be returned in its original representation. ResamplingOptions resampling_options = 1; // Optional time-based filter criteria. - // If omitted, data will start streaming from the timestamp that the request was received. + // If omitted, data will start streaming from the timestamp that the + // request was received. TimeFilter time_filter = 2; } @@ -402,14 +695,15 @@ message ReceiveAggregatedMicrogridComponentsDataStreamRequest { AggregationStreamFilter filter = 2; } -// Message defining the response format for a stream that fetches aggregated real-time metrics -// for the provided custom aggregation formulas. +// Message defining the response format for a stream that fetches aggregated +// real-time metrics for the provided custom aggregation formulas. // // !!! note -// The formula and metric must have been specified in the corresponding request. -// A single aggregated sample for the metric is returned in the sample field. Each message -// covers a single formula. For multiple formulars provided in the request, expect sequential -// messages in the stream. +// The formula and metric must have been specified in the corresponding +// request. A single aggregated sample for the metric is returned in the +// sample field. Each message covers a single formula. For multiple +// formulars provided in the request, expect sequential messages in the +// stream. // // !!! example // Given a stream output, a single sample might be: @@ -427,11 +721,18 @@ message ReceiveAggregatedMicrogridComponentsDataStreamRequest { // } // ``` // -message ReceiveAggregatedMicrogridComponentsDataStreamResponse { +message ReceiveAggregatedMicrogridElectricalComponentsDataStreamResponse { // The metric and formula that has been used to aggregate the sample. AggregationConfig aggregation_config = 1; - // Aggregated sample value and corresponding UTC timestamp when it was sampled. + // Aggregated sample value and corresponding UTC timestamp when it was + // sampled. + // + // !!! important "Empty values and updates" + // When resampling is enabled, metric sample values MAY be unset for a + // given resampling interval and MAY be updated later if additional + // telemetry measurements arrive. Clients MUST be able to handle empty + // values and updates. SimpleAggregatedMetricValue sample = 2; } @@ -439,8 +740,9 @@ message ReceiveAggregatedMicrogridComponentsDataStreamResponse { // measurements, and other information for individual microgrid sensors. // // !!! note -// In addition to the raw metrics, the API can also return additional information -// like errors or operational states of the sensors during the specified time period. +// In addition to the raw metrics, the API can also return additional +// information like errors or operational states of the sensors during +// the specified time period. // message ReceiveMicrogridSensorsDataStreamRequest { // Include options for filtering microgrid sensors data. @@ -449,7 +751,8 @@ message ReceiveMicrogridSensorsDataStreamRequest { // Specifies which additional fields should be included in the response. // message IncludeOptions { - // Optional operational state inclusion. By default, states are not included in the response. + // Optional operational state inclusion. By default, states are not + // included in the response. optional FilterOption states = 1; } @@ -459,74 +762,134 @@ message ReceiveMicrogridSensorsDataStreamRequest { // The filter criteria defined here are applied universally across all // specified microgrids and their respective sensors. message StreamFilter { - // Optional resampling options like resolution for the data, represented in seconds. + // Optional resampling options like resolution for the data, represented + // in seconds. // If omitted, data will be returned in its original representation. ResamplingOptions resampling_options = 1; - // Include options specifying additional fields to be included in the response. + // Include options specifying additional fields to be included in the + // response. IncludeOptions include_options = 2; // Optional time-based filter criteria. - // If omitted, data will start streaming from the timestamp that the request was received. + // If omitted, data will start streaming from the timestamp that the + // request was received. TimeFilter time_filter = 3; } - // Encapsulates the microgrid ID and the sensor IDs within that microgrid for which - // the historical data should be retrieved. + + // Encapsulates the microgrid ID and the sensor IDs within that microgrid + // for which the historical data should be retrieved. // // !!! note - // Each entry in this repeated field associates a microgrid ID with its respective - // sensor IDs. At least one such association must be provided for a valid request. - repeated frequenz.api.common.v1alpha8.microgrid.MicrogridSensorIDs microgrid_sensors = 1; + // Each entry in this repeated field associates a microgrid ID with its + // respective sensor IDs. At least one such association must be + // provided for a valid request. + repeated frequenz.api.common.v1alpha8.microgrid.MicrogridSensorIDs + microgrid_sensors = 1; // List of metrics to receive data for, each possibly with connection filters. // // !!! note - // At least one metric must be specified. Each metric can have an optional list of connections. - // If connections are provided for a metric, only data from those connections will be returned. - // If no connections are provided for a metric, data from all connections will be returned for that metric. + // At least one metric must be specified. Each metric can have an + // optional list of connections. If connections are provided for a + // metric, only data from those connections will be returned. If no + // connections are provided for a metric, data from all connections + // will be returned for that metric. repeated MetricConnections metrics = 2; - // General filter that applies to the data retrieval for all specified microgrids and sensors. + // General filter that applies to the data retrieval for all specified + // microgrids and sensors. // // !!! note - // The filter can specify a list of metrics to be return but also specify bounds, operational - // state, or errors to be returned. + // The filter can specify a list of metrics to be return but also + // specify bounds, operational state, or errors to be returned. StreamFilter filter = 3; } // Response containing a single data sample for one microgrid's sensors. // -// !!! note -// The microgrid's sensors are provided as single data samples that -// encapsulate metrics, bounds, errors, and operational states along with their -// associated timestamps. Each response message covers a single microgrid. -// If multiple microgrids are provided in the request, expect sequential messages -// in the stream. +// The microgrid's sensors are provided as telemetry samples that encapsulate +// metric measurements and sensor state snapshots along with their associated +// timestamps. +// +// Each response message covers a single microgrid. If multiple microgrids are +// provided in the request, expect sequential messages in the stream. +// +// !!! example "Example output structure (resampling enabled)" +// This example shows telemetry for a single sensor when resampling is +// enabled. Both metric samples share the same `sample_time`, which +// represents the start of the resampling interval. One metric has a value +// because telemetry contributed to the interval, while the other metric +// has no value set because no measurements were observed for that metric +// during the interval. The sensor state snapshot uses its own origin time. +// +// ``` +// { +// "microgrid_id": 1, +// "sensors": [ +// { +// "sensor_id": 7, +// "metric_samples": [ +// { +// "sample_time": "2023-10-01T00:00:00Z", +// "metric": "METRIC_TEMPERATURE_C", +// "value": { +// "simple_metric": { "value": 21.4 } +// } +// }, +// { +// "sample_time": "2023-10-01T00:00:00Z", +// "metric": "METRIC_HUMIDITY_PERCENT" +// // value intentionally unset: no telemetry in this interval +// } +// ], +// "state_snapshots": [ +// { +// "origin_time": "2023-10-01T00:00:12.45Z", +// "states": ["SENSOR_STATE_OK"], +// "warnings": [], +// "errors": [] +// } +// ] +// } +// ] +// } +// ``` // message ReceiveMicrogridSensorsDataStreamResponse { // Microgrid ID for which the sensors and samples are reported. uint64 microgrid_id = 1; - // List of sensors within this microgrid, each with its associated data samples. - repeated frequenz.api.common.v1alpha8.microgrid.sensors.SensorTelemetry sensors = 2; + // List of sensors within this microgrid, each with its associated data + // samples. + // + // !!! important "Empty values and updates" + // When resampling is enabled, metric sample values MAY be unset for a + // given resampling interval and MAY be updated later if additional + // telemetry measurements arrive. Clients MUST be able to handle empty + // values and updates. + repeated frequenz.api.common.v1alpha8.microgrid.sensors.SensorTelemetry + sensors = 2; } -// Message defining the request format for streaming aggregated historical metrics. -// This request allows to specify custom aggregation formulas, along with general -// filtering. +// Message defining the request format for streaming aggregated historical +// metrics. // -// At least one aggregation formula config must be provided. The aggregation -// follows the passive sign convention. +// This request allows to specify custom aggregation formulas, along with +// general filtering. At least one aggregation formula config must be +// provided. The aggregation follows the passive sign convention. // message ReceiveAggregatedMicrogridSensorsDataStreamRequest { // General filter criteria for querying microgrid sensors data. message AggregationStreamFilter { - // Optional resampling options like resolution for the data, represented in seconds. + // Optional resampling options like resolution for the data, represented + // in seconds. // If omitted, data will be returned in its original representation. ResamplingOptions resampling_options = 1; // Optional time-based filter criteria. - // If omitted, data will start streaming from the timestamp that the request was received. + // If omitted, data will start streaming from the timestamp that the + // request was received. TimeFilter time_filter = 2; } @@ -537,19 +900,55 @@ message ReceiveAggregatedMicrogridSensorsDataStreamRequest { AggregationStreamFilter filter = 2; } -// Message defining the response format for a stream that fetches aggregated real-time metrics -// for the provided custom aggregation formulas. +// Message defining the response format for a stream that fetches aggregated +// real-time metrics for the provided custom aggregation formulas. // // !!! note -// The formula and metric must have been specified in the corresponding request. -// A single aggregated sample for the metric is returned in the sample field. Each message -// covers a single formula. For multiple formulars provided in the request, expect sequential +// The formula and metric must have been specified in the corresponding +// request. A single aggregated sample for the metric is returned in the +// sample field. Each message covers a single formula. For multiple +// formulars provided in the request, expect sequential // messages in the stream. // +// !!! example +// Example output structure (resampling enabled): +// +// This example shows an aggregated sensor metric for a single resampling +// interval. The `sample_time` represents the start of the resampling +// interval [t, t + ΔT). The aggregated value is present because telemetry +// measurements from one or more sensors contributed to the interval. +// +// If no telemetry measurements had contributed, the `sample` value would +// be unset. The same interval may be re-emitted later with an updated +// value if additional telemetry measurements arrive. +// +// ``` +// { +// "aggregation_config": { +// "microgrid_id": 1, +// "metric": "METRIC_TEMPERATURE_C", +// "aggregation_formula": "avg(#7,#9,#12)" +// }, +// "sample": { +// "sample_time": "2023-10-01T00:00:00Z", +// "sample": { +// "simple_metric": { "value": 21.8 } +// } +// } +// } +// ``` +// message ReceiveAggregatedMicrogridSensorsDataStreamResponse { // The metric and formula that has been used to aggregate the sample. AggregationConfig aggregation_config = 1; - // Aggregated sample value and corresponding UTC timestamp when it was sampled. + // Aggregated sample value and corresponding UTC timestamp when it was + // sampled. + // + // !!! important "Empty values and updates" + // When resampling is enabled, metric sample values MAY be unset for a + // given resampling interval and MAY be updated later if additional + // telemetry measurements arrive. Clients MUST be able to handle empty + // values and updates. SimpleAggregatedMetricValue sample = 2; }