fix:use sources param in 5th tutorial due to PV schedules in 3rd tutorial (#1846)

* fix:use sources param in 5th tutorial due to PV schedules in 3rd tutorial

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* remove whitespace

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* add changelog entry

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* no spaces before colons, fix wording

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

---------

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Author: nhoening

documentation/changelog.rst

@@ -33,7 +33,7 @@ Infrastructure / Support
 ----------------------
 * Updated dependencies [see `PR #1752 <https://www.github.com/FlexMeasures/flexmeasures/pull/1752>`_, `PR #1820 <https://www.github.com/FlexMeasures/flexmeasures/pull/1820>`_ and `PR #1842 <https://www.github.com/FlexMeasures/flexmeasures/pull/1842>`_]
 * Speed up data searches by refactoring our filter for selecting data from the latest version of a data generating model [see `PR #1698 <https://github.com/FlexMeasures/flexmeasures/pull/1698>`_]
-* Add new tutorial for PV curtailment and multi-asset scheduling [see `PR #1804 <https://github.com/FlexMeasures/flexmeasures/pull/1804>`_]
+* Add new tutorial for PV curtailment and multi-asset scheduling [see `PR #1804 <https://github.com/FlexMeasures/flexmeasures/pull/1804>`_ and `PR #1846 <https://github.com/FlexMeasures/flexmeasures/pull/1846>`_]
 * Drop need for .env file in favour of more adaptation of flexmeasures.cfg config file [see `PR #1759 <https://github.com/FlexMeasures/flexmeasures/pull/1759>`_]
 * Support saving beliefs with a ``belief_horizon`` in the ``AggregatorReporter`` [see `PR #1735 <https://www.github.com/FlexMeasures/flexmeasures/pull/1735>`_]
 * Support skipping a ``PandasReporter`` transformation if any of its (keyword) arguments has no data [see `PR #1669 <https://www.github.com/FlexMeasures/flexmeasures/pull/1669>`_]

documentation/tut/scripts/run-tutorial5-in-docker.sh

@@ -13,27 +13,27 @@ docker exec -it flexmeasures-server-1 flexmeasures show beliefs --sensor 7 --sta
 
 docker exec -it flexmeasures-server-1 flexmeasures show data-sources --show-attributes --id 6
 
-echo "[TUTORIAL-RUNNER] Configure reporter ..."
+echo "[TUTORIAL-RUNNER] Configure headroom reporter ..."
 
 echo "
 {
-   'weights' : {
-       'grid connection capacity' : 1.0,
-       'PV' : -1.0,
+   'weights': {
+       'grid connection capacity': 1.0,
+       'PV': -1.0,
    }
 }" > headroom-config.json
 docker cp headroom-config.json flexmeasures-server-1:/app
 
 echo "
 {
-    'input' : [{'name' : 'grid connection capacity','sensor' : 7},
-               {'name' : 'PV', 'sensor' : 3}],
-    'output' : [{'sensor' : 8}]
+    'input': [{'name': 'grid connection capacity', 'sensor': 7},
+               {'name': 'PV', 'sensor': 3, 'sources': [4]}],
+    'output': [{'sensor': 8}]
 }" > headroom-parameters.json
 docker cp headroom-parameters.json flexmeasures-server-1:/app
 
 
-echo "[TUTORIAL-RUNNER] add report ..."
+echo "[TUTORIAL-RUNNER] add headroom report ..."
 
 docker exec -it flexmeasures-server-1 flexmeasures add report --reporter AggregatorReporter \
    --parameters headroom-parameters.json --config headroom-config.json \
@@ -49,8 +49,8 @@ echo "[TUTORIAL-RUNNER] now the inflexible process ..."
 
 echo "
 {
-    'input' : [{'sensor' : 4}],
-    'output' : [{'sensor' : 9}]
+    'input': [{'sensor': 4}],
+    'output': [{'sensor': 9}]
 }" > inflexible-parameters.json
 
 docker cp inflexible-parameters.json flexmeasures-server-1:/app
@@ -67,8 +67,8 @@ echo "[TUTORIAL-RUNNER] now the breakable process ..."
 
 echo "
 {
-    'input' : [{'sensor' : 5}],
-    'output' : [{'sensor' : 10}]
+    'input': [{'sensor': 5}],
+    'output': [{'sensor': 10}]
 }" > breakable-parameters.json
 
 docker cp breakable-parameters.json flexmeasures-server-1:/app
@@ -86,8 +86,8 @@ echo "[TUTORIAL-RUNNER] now the breakable process ..."
 
 echo "
 {
-    'input' : [{'sensor' : 6}],
-    'output' : [{'sensor' : 11}]
+    'input' : [{'sensor': 6}],
+    'output' : [{'sensor': 11}]
 }" > shiftable-parameters.json
 
 docker cp shiftable-parameters.json flexmeasures-server-1:/app

documentation/tut/toy-example-reporter.rst

@@ -120,12 +120,14 @@ In practice, we need to create the `config` and `parameters`:
     $ echo "
     $ {
     $     'input': [{'name': 'grid connection capacity', 'sensor': 7},
-    $                {'name': 'PV', 'sensor': 3}],
+    $                {'name': 'PV', 'sensor': 3, 'sources': [4]}],
     $     'output': [{'sensor': 8}]
     $ }" > headroom-parameters.json
 
 The output sensor (ID: 8) is actually the one created just to store that information - the headroom our battery has when considering solar production.
 
+We limit the PV input to data with source 4, which is the one created when we added the toy account and is associated with our solar forecasts. This way, we ensure that only the forecasted values are used in the report, and not (also) schedules from the 3rd tutorial.
+
 Finally, we can create the report with the following command:
 
 .. code-block:: bash

flexmeasures/data/models/reporting/aggregator.py

@@ -83,6 +83,7 @@ def _compute_report(
                 )
 
             # Check for multiple sources within the entire frame (excluding different versions of the same source)
+            # Raise error if that is the case and no source filter was applied - user should be explicit here
             unique_sources = df.lineage.sources
             properties = [
                 "name",
@@ -99,9 +100,11 @@ def _compute_report(
                 and (source is None or len(source) == 0)
             ):
                 raise ValueError(
-                    "Missing attribute source or sources. The fields `source` or `sources` is required when having multiple sources within the time window."
+                    f"Missing attribute 'sources' for input sensor {sensor.id}: {sensor.name} (to identify one specific source). The field  `sources` is required when having data with multiple sources within the time window, to ensure only required data is used in the reporter. "
+                    f"We found data from the following sources: {[source.id for source in unique_sources]}."
                 )
 
+            # drop all indexes but event_start
             df = df.droplevel([1, 2, 3])
 
             # apply weight