Skip to content

Commit 79726f9

Browse files
cwasickicwasicki
authored
Introduce simple reporting client with support for single metric historical data (#16)
This initial version of the reporting API client offers focused functionality to retrieve historical data for a single metric from a single component. Features of this version: * Pagination Handling: Seamlessly request time series data of a single metric from a single component, with integrated pagination management. * Data Transformation: Utilize a wrapper data class that retains the raw protobuf response while offering transformation capabilities. A generator function is provided for iterating over individual page values. * Structured Data: Streamline data representation through named tuples for timestamp and value pairs, eliminating ambiguity for single component and metric scenarios. * Usage Examples: Code examples demonstrate usage, along with experimental code for upcoming features (considered for removal). * Unit Testing: Basic unit tests are included. Limitations of this version: * Single Metric Focus: Initially supporting queries for individual metrics to support most common use cases, with an extensible design for multiple metrics/components in future. * States and bounds: In line with focus on widely used functionalities, the approach to integrating states and bounds within the data structures is still under exploration. * Metric Sample Variants: Currently supports `SimpleMetricSample` exclusively, with decision on how to integrate `AggregatedMetricSample` pending. * Resampling: not yet exposed (service-side). * Streaming: Functions not yet available (service-side). Current generator implementations on pages and entries can be aligned with streaming output in future. * Aggregation: Support for formulas still missing (service-side)
2 parents 5a0acd1 + 1190207 commit 79726f9

File tree

6 files changed

+487
-27
lines changed

6 files changed

+487
-27
lines changed

‎RELEASE_NOTES.md‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,27 @@
22

33
## Summary
44

5-
<!-- Here goes a general summary of what this release is about -->
5+
This release introduces the initial version of the Reporting API client with support for
6+
retrieving single metric historical data for a single component.
67

78
## Upgrading
89

910
<!-- Here goes notes on how to upgrade from previous versions, including deprecations and what they should be replaced with -->
1011

1112
## New Features
1213

13-
<!-- Here goes the main new features and examples or instructions on how to use them -->
14+
* Introducing the initial version of the Reporting API client, streamlined for
15+
retrieving single metric historical data for a single component. It incorporates
16+
pagination handling and utilizes a wrapper data class that retains the raw
17+
protobuf response while offering transformation capabilities limited here
18+
to generators of structured data representation via named tuples.
19+
20+
* Current limitations include a single metric focus with plans for extensibility,
21+
ongoing development for states and bounds integration, as well as support for
22+
service-side features like resampling, streaming, and formula aggregations.
23+
24+
* Code examples are provided to guide users through the basic usage of the client.
25+
1426

1527
## Bug Fixes
1628

‎examples/client.py‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# License: MIT
2+
# Copyright © 2024 Frequenz Energy-as-a-Service GmbH
3+
4+
"""Examples usage of reporting API."""
5+
6+
import argparse
7+
import asyncio
8+
from datetime import datetime
9+
from pprint import pprint
10+
from typing import AsyncGenerator
11+
12+
import pandas as pd
13+
from frequenz.client.common.metric import Metric
14+
15+
from frequenz.client.reporting import ReportingClient
16+
17+
# Experimental import
18+
from frequenz.client.reporting._client import MetricSample
19+
20+
21+
# pylint: disable=too-many-locals
22+
async def main(microgrid_id: int, component_id: int) -> None:
23+
"""Test the ReportingClient.
24+
25+
Args:
26+
microgrid_id: int
27+
component_id: int
28+
"""
29+
service_address = "localhost:50051"
30+
client = ReportingClient(service_address)
31+
32+
microgrid_components = [(microgrid_id, [component_id])]
33+
metrics = [
34+
Metric.DC_POWER,
35+
Metric.DC_CURRENT,
36+
]
37+
38+
start_dt = datetime.fromisoformat("2023-11-21T12:00:00.00+00:00")
39+
end_dt = datetime.fromisoformat("2023-11-21T12:01:00.00+00:00")
40+
41+
page_size = 10
42+
43+
print("########################################################")
44+
print("Iterate over single metric generator")
45+
46+
async for sample in client.iterate_single_metric(
47+
microgrid_id=microgrid_id,
48+
component_id=component_id,
49+
metric=metrics[0],
50+
start_dt=start_dt,
51+
end_dt=end_dt,
52+
page_size=page_size,
53+
):
54+
print("Received:", sample)
55+
56+
###########################################################################
57+
#
58+
# The following code is experimental and demonstrates potential future
59+
# usage of the ReportingClient.
60+
#
61+
###########################################################################
62+
63+
async def components_data_iter() -> AsyncGenerator[MetricSample, None]:
64+
"""Iterate over components data.
65+
66+
Yields:
67+
Single metric sample
68+
"""
69+
# pylint: disable=protected-access
70+
async for page in client._iterate_components_data_pages(
71+
microgrid_components=microgrid_components,
72+
metrics=metrics,
73+
start_dt=start_dt,
74+
end_dt=end_dt,
75+
page_size=page_size,
76+
):
77+
for entry in page.iterate_metric_samples():
78+
yield entry
79+
80+
async def components_data_dict(
81+
components_data_iter: AsyncGenerator[MetricSample, None]
82+
) -> dict[int, dict[int, dict[datetime, dict[Metric, float]]]]:
83+
"""Convert components data iterator into a single dict.
84+
85+
The nesting structure is:
86+
{
87+
microgrid_id: {
88+
component_id: {
89+
timestamp: {
90+
metric: value
91+
}
92+
}
93+
}
94+
}
95+
96+
Args:
97+
components_data_iter: async generator
98+
99+
Returns:
100+
Single dict with with all components data
101+
"""
102+
ret: dict[int, dict[int, dict[datetime, dict[Metric, float]]]] = {}
103+
104+
async for ts, mid, cid, met, value in components_data_iter:
105+
if mid not in ret:
106+
ret[mid] = {}
107+
if cid not in ret[mid]:
108+
ret[mid][cid] = {}
109+
if ts not in ret[mid][cid]:
110+
ret[mid][cid][ts] = {}
111+
112+
ret[mid][cid][ts][met] = value
113+
114+
return ret
115+
116+
print("########################################################")
117+
print("Iterate over generator")
118+
async for msample in components_data_iter():
119+
print("Received:", msample)
120+
121+
print("########################################################")
122+
print("Dumping all data as a single dict")
123+
dct = await components_data_dict(components_data_iter())
124+
pprint(dct)
125+
126+
print("########################################################")
127+
print("Turn data into a pandas DataFrame")
128+
data = [cd async for cd in components_data_iter()]
129+
df = pd.DataFrame(data).set_index("timestamp")
130+
pprint(df)
131+
132+
133+
if __name__ == "__main__":
134+
parser = argparse.ArgumentParser()
135+
parser.add_argument("microgrid_id", type=int, help="Microgrid ID")
136+
parser.add_argument("component_id", type=int, help="Component ID")
137+
138+
args = parser.parse_args()
139+
asyncio.run(main(args.microgrid_id, args.component_id))

‎pyproject.toml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ requires-python = ">= 3.11, < 4"
2929
# TODO(cookiecutter): Remove and add more dependencies if appropriate
3030
dependencies = [
3131
"typing-extensions >= 4.5.0, < 5",
32+
"frequenz-api-reporting >= 0.1.1, < 1",
33+
"frequenz-client-common @ git+https://github.com/frequenz-floss/[email protected]",
3234
]
3335
dynamic = ["version"]
3436

@@ -62,6 +64,7 @@ dev-mypy = [
6264
"types-Markdown == 3.4.2.10",
6365
# For checking the noxfile, docs/ script, and tests
6466
"frequenz-client-reporting[dev-mkdocs,dev-noxfile,dev-pytest]",
67+
"pandas-stubs >= 2, < 3", # Only required for example
6568
]
6669
dev-noxfile = [
6770
"nox == 2023.4.22",
@@ -71,6 +74,7 @@ dev-pylint = [
7174
"pylint == 3.0.2",
7275
# For checking the noxfile, docs/ script, and tests
7376
"frequenz-client-reporting[dev-mkdocs,dev-noxfile,dev-pytest]",
77+
"pandas >= 2, < 3", # Only required for example
7478
]
7579
dev-pytest = [
7680
"pytest == 8.0.0",
@@ -82,6 +86,10 @@ dev-pytest = [
8286
dev = [
8387
"frequenz-client-reporting[dev-mkdocs,dev-flake8,dev-formatting,dev-mkdocs,dev-mypy,dev-noxfile,dev-pylint,dev-pytest]",
8488
]
89+
examples = [
90+
"grpcio >= 1.51.1, < 2",
91+
"pandas >= 2, < 3",
92+
]
8593

8694
[project.urls]
8795
Documentation = "https://frequenz-floss.github.io/frequenz-client-reporting-python/"

‎src/frequenz/client/reporting/__init__.py‎

Lines changed: 4 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,12 @@
11
# License: MIT
22
# Copyright © 2024 Frequenz Energy-as-a-Service GmbH
33

4-
"""Reporting API client for Python.
4+
"""Client to connect to the Reporting API.
55
6-
TODO(cookiecutter): Add a more descriptive module description.
6+
This package provides a low-level interface for interacting with the reporting API.
77
"""
88

99

10-
# TODO(cookiecutter): Remove this function
11-
def delete_me(*, blow_up: bool = False) -> bool:
12-
"""Do stuff for demonstration purposes.
10+
from ._client import ReportingClient
1311

14-
Args:
15-
blow_up: If True, raise an exception.
16-
17-
Returns:
18-
True if no exception was raised.
19-
20-
Raises:
21-
RuntimeError: if blow_up is True.
22-
"""
23-
if blow_up:
24-
raise RuntimeError("This function should be removed!")
25-
return True
12+
__all__ = ["ReportingClient"]

0 commit comments

Comments
 (0)