Skip to content

Commit 017128d

Browse files
authored
Merge pull request #618 from BDonnot/issue_616
Issue #616
2 parents 36f4586 + 27a266b commit 017128d

File tree

13 files changed

+348
-43
lines changed

13 files changed

+348
-43
lines changed

CHANGELOG.rst

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ Work kind of in progress
4040

4141
Next release
4242
---------------------------------
43+
- TODO bug on maintenance starting at midnight (they are not correctly handled in the observation)
44+
=> cf script test_issue_616
4345
- TODO Notebook for tf_agents
4446
- TODO Notebook for acme
4547
- TODO Notebook using "keras rl" (see https://keras.io/examples/rl/ppo_cartpole/)
@@ -48,7 +50,8 @@ Next release
4850
- TODO jax everything that can be: create a simple env based on jax for topology manipulation, without
4951
redispatching or rules
5052
- TODO backend in jax, maybe ?
51-
53+
- TODO done and truncated properly handled in gym_compat module (when game over
54+
before the end it's probably truncated and not done)
5255

5356
[1.10.3] - 2024-xx-yy
5457
-------------------------
@@ -65,10 +68,23 @@ Next release
6568
(because it should always have been like this)
6669
- [FIXED] a bug in the `MultiFolder` and `MultifolderWithCache` leading to the wrong
6770
computation of `max_iter` on some corner cases
71+
- [FIXED] issue on `seed` and `MultifolderWithCache` which caused
72+
https://github.com/rte-france/Grid2Op/issues/616
73+
- [FIXED] another issue with the seeding of `MultifolderWithCache`: the seed was not used
74+
correctly on the cache data when calling `chronics_handler.reset` multiple times without
75+
any changes
6876
- [ADDED] possibility to skip some step when calling `env.reset(..., options={"init ts": ...})`
6977
- [ADDED] possibility to limit the duration of an episode with `env.reset(..., options={"max step": ...})`
7078
- [ADDED] possibility to specify the "reset_options" used in `env.reset` when
7179
using the runner with `runner.run(..., reset_options=xxx)`
80+
- [ADDED] the time series now are able to regenerate their "random" part
81+
even when "cached" thanks to the addition of the `regenerate_with_new_seed` of the
82+
`GridValue` class (in public API)
83+
- [ADDED] `MultifolderWithCache` now supports `FromHandlers` time series generator
84+
- [IMPROVED] the documentation on the `time series` folder.
85+
- [IMPROVED] now the "maintenance from json" (*eg* the `JSONMaintenanceHandler` or the
86+
`GridStateFromFileWithForecastsWithMaintenance`) can be customized with the day
87+
of the week where the maintenance happens (key `maintenance_day_of_week`)
7288

7389
[1.10.2] - 2024-05-27
7490
-------------------------

docs/chronics.rst

Lines changed: 142 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -54,26 +54,158 @@ come from the :class:`grid2op.GridValue` and are detailed in the
5454
:func:`GridValue.forecasts` method.
5555

5656

57-
More control on the chronics
57+
More control on the time series
5858
-------------------------------
5959
We explained, in the description of the :class:`grid2op.Environment` in sections
6060
:ref:`environment-module-chronics-info` and following how to have more control on which chronics is used,
6161
with steps are used within a chronics etc. We will not detailed here again, please refer to this page
6262
for more information.
6363

64-
However, know that you can have a very detailed control on which chronics are used:
64+
However, know that you can have a very detailed control on which time series using the `options`
65+
kwargs of a call to `env.reset()` (or the `reset_otions` kwargs when calling the
66+
`runner.run()`) :
6567

66-
- use `env.set_id(THE_CHRONIC_ID)` (see :func:`grid2op.Environment.Environment.set_id`) to set the id of the
67-
chronics you want to use
68-
- use `env.chronics_handler.set_filter(a_function)` (see :func:`grid2op.Chronics.GridValue.set_filter`)
68+
69+
Use a specific time serie for an episode
70+
*******************************************
71+
72+
To use a specific time series for a given episode, you can use
73+
`env.reset(options={"time serie id": THE_ID_YOU_WANT)`.
74+
75+
For example:
76+
77+
.. code-block:: python
78+
79+
import grid2op
80+
env_name = "l2rpn_case14_sandbox"
81+
env = grid2op.make(env_name)
82+
83+
# you can use an int:
84+
obs = env.reset(options={"time serie id": 0})
85+
86+
# or the name of the folder (for most grid2op environment)
87+
obs = env.reset(options={"time serie id": "0000"}) # for l2rpn_case14_sandbox
88+
89+
# for say l2rpn_neurips_2020_track1
90+
# obs = env.reset(options={"time serie id": "Scenario_august_008"})
91+
92+
# for say l2rpn_idf_2023
93+
# obs = env.reset(options={"time serie id": "2035-04-23_7"})
94+
95+
96+
.. note::
97+
For oldest grid2op versions (please upgrade if that's the case) you needed to use:
98+
`env.set_id(THE_CHRONIC_ID)` (see :func:`grid2op.Environment.Environment.set_id`) to set the id of the
99+
chronics you want to use.
100+
101+
102+
Skipping the initial few steps
103+
*******************************
104+
105+
Often the time series provided for an environment always start at the same date and time on
106+
the same hour of the day and day of the week. It might not be ideal to learn controler
107+
with such data or might "burn up" computation time during evaluation.
108+
109+
To do that, you can use the `"init ts"` reset options, for example with:
110+
111+
.. code-block:: python
112+
113+
import grid2op
114+
env_name = "l2rpn_case14_sandbox"
115+
env = grid2op.make(env_name)
116+
117+
# you can use an int:
118+
obs = env.reset(options={"init ts": 12})
119+
120+
# obs will skip the first hour of the time series
121+
# 12 steps is equivalent to 1h (5 mins per step in general)
122+
123+
124+
.. note::
125+
126+
For oldest grid2op versions (please upgrade if that's the case) you needed to use:
127+
`env.fast_forward_chronics(nb_time_steps)`
128+
(see :func:`grid2op.Environment.BaseEnv.fast_forward_chronics`) to skip initial
129+
few steps
130+
of a given chronics.
131+
132+
Please be aware that this "legacy" behaviour has some issues and is "less clear"
133+
than the "init ts" above and it can have some weird combination with
134+
`set_max_iter` for example.
135+
136+
137+
Limit the maximum length of the current episode
138+
*************************************************
139+
140+
For most enviroment, the maximum duration of an episode is the equivalent of a week
141+
(~2020 steps) or a month (~8100 steps) which might be too long for some usecase.
142+
143+
Anyway, if you want to reduce it, you can now do it with the `"max step"` reset
144+
option like this:
145+
146+
.. code-block:: python
147+
148+
import grid2op
149+
env_name = "l2rpn_case14_sandbox"
150+
env = grid2op.make(env_name)
151+
152+
# you can use an int:
153+
obs = env.reset(options={"max step": 2*288})
154+
155+
# the maximum duration of the episode is now 2*288 steps
156+
# the equivalent of two days
157+
158+
.. note::
159+
160+
For oldest grid2op versions (please upgrade if that's the case) you needed to use:
161+
`env.chronics_handler.set_max_iter(nb_max_iter)`
162+
(see :func:`grid2op.Chronics.ChronicsHandler.set_max_iter`) to limit the number
163+
of steps within an episode.
164+
165+
Please be aware that this "legacy" behaviour has some issues and is "less clear"
166+
than the "init ts" above and it can have some weird combination with
167+
`fast_forward_chronics` for example.
168+
169+
Discard some time series from the existing folder
170+
**************************************************
171+
172+
The folder containing the time series for a given grid2op environment often contains
173+
dozens (thousands sometimes) different time series.
174+
175+
You might want to use only part of them at some point (whether it's some for training and some
176+
for validation and test, or some for training an agent on a process and some to train the
177+
same agent on another process etc.)
178+
179+
Anyway, if you want to do this (on the majority of released environments) you can do it
180+
thanks to the `env.chronics_handler.set_filter(a_function)`.
181+
182+
For example:
183+
184+
.. code-block:: python
185+
186+
import re
187+
import grid2op
188+
env_name = "l2rpn_case14_sandbox"
189+
env = grid2op.make(env_name)
190+
191+
def keep_only_some_ep(chron_name):
192+
return re.match(r".*00.*", chron_name) is not None
193+
194+
env.chronics_handler.set_filter(keep_only_some_ep)
195+
li_episode_kept = env.chronics_handler.reset()
196+
197+
198+
.. note::
199+
For oldest grid2op versions (please upgrade if that's the case) you needed to use:
200+
use `env.chronics_handler.set_filter(a_function)` (see :func:`grid2op.Chronics.GridValue.set_filter`)
69201
to only use certain chronics
202+
203+
70204
- use `env.chronics_handler.sample_next_chronics(probas)`
71205
(see :func:`grid2op.Chronics.GridValue.sample_next_chronics`) to draw at random some chronics
72-
- use `env.fast_forward_chronics(nb_time_steps)`
73-
(see :func:`grid2op.Environment.BaseEnv.fast_forward_chronics`) to skip initial number of steps
74-
of a given chronics
75-
- use `env.chronics_handler.set_max_iter(nb_max_iter)`
76-
(see :func:`grid2op.Chronics.ChronicsHandler.set_max_iter`) to limit the number of steps within an episode
206+
207+
Performance gain (throughput)
208+
********************************
77209

78210
Chosing the right chronics can also lead to some large advantage in terms of computation time. This is
79211
particularly true if you want to benefit the most from HPC for example. More detailed is given in the

grid2op/Chronics/GSFFWFWM.py

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,14 @@ def initialize(
108108
self.max_daily_number_per_month_maintenance = dict_[
109109
"max_daily_number_per_month_maintenance"
110110
]
111+
112+
if "maintenance_day_of_week" in dict_:
113+
self.maintenance_day_of_week = [int(el) for el in dict_[
114+
"maintenance_day_of_week"
115+
]]
116+
else:
117+
self.maintenance_day_of_week = np.arange(5)
118+
111119
super().initialize(
112120
order_backend_loads,
113121
order_backend_prods,
@@ -133,7 +141,6 @@ def _sample_maintenance(self):
133141
########
134142
# new method to introduce generated maintenance
135143
self.maintenance = self._generate_maintenance() #
136-
137144
##########
138145
# same as before in GridStateFromFileWithForecasts
139146
GridStateFromFileWithForecastsWithMaintenance._fix_maintenance_format(self)
@@ -171,7 +178,12 @@ def _generate_matenance_static(name_line,
171178
daily_proba_per_month_maintenance,
172179
max_daily_number_per_month_maintenance,
173180
space_prng,
181+
maintenance_day_of_week=None
174182
):
183+
if maintenance_day_of_week is None:
184+
# new in grid2op 1.10.3
185+
maintenance_day_of_week = np.arange(5)
186+
175187
# define maintenance dataframe with size (nbtimesteps,nlines)
176188
columnsNames = name_line
177189
nbTimesteps = n_
@@ -203,8 +215,6 @@ def _generate_matenance_static(name_line,
203215
datelist = datelist[:-1]
204216

205217
n_lines_maintenance = len(line_to_maintenance)
206-
207-
_24_h = timedelta(seconds=86400)
208218
nb_rows = int(86400 / time_interval.total_seconds())
209219
selected_rows_beg = int(
210220
maintenance_starting_hour * 3600 / time_interval.total_seconds()
@@ -220,7 +230,7 @@ def _generate_matenance_static(name_line,
220230
maxDailyMaintenance = -1
221231
for nb_day_since_beg, this_day in enumerate(datelist):
222232
dayOfWeek = this_day.weekday()
223-
if dayOfWeek < 5: # only maintenance starting on working days
233+
if dayOfWeek in maintenance_day_of_week:
224234
month = this_day.month
225235

226236
maintenance_me = np.zeros((nb_rows, nb_line_maint))
@@ -279,5 +289,9 @@ def _generate_maintenance(self):
279289
self.maintenance_ending_hour,
280290
self.daily_proba_per_month_maintenance,
281291
self.max_daily_number_per_month_maintenance,
282-
self.space_prng
292+
self.space_prng,
293+
self.maintenance_day_of_week
283294
)
295+
296+
def regenerate_with_new_seed(self):
297+
self._sample_maintenance()

grid2op/Chronics/fromChronix2grid.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,4 +309,12 @@ def next_chronics(self):
309309
GridStateFromFileWithForecastsWithMaintenance._fix_maintenance_format(self)
310310

311311
self.check_validity(backend=None)
312+
313+
def regenerate_with_new_seed(self):
314+
raise ChronicsError("You should not 'cache' the data coming from the "
315+
"`FromChronix2grid`, which is probably why you ended "
316+
"up calling this function. If you want to generate data "
317+
"'on the fly' please do not use the `MultiFolder` or "
318+
"`MultiFolderWithCache` `chronics_class` when making your "
319+
"environment.")
312320

grid2op/Chronics/gridValue.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -856,3 +856,21 @@ def cleanup_action_space(self):
856856
"""
857857
self.__action_space = None
858858
# NB the action space is not closed as it is NOT own by this class
859+
860+
def regenerate_with_new_seed(self):
861+
"""
862+
INTERNAL this function is called by some classes (*eg* :class:`MultifolderWithCache`)
863+
when a new seed has been set.
864+
865+
For example, if you use some 'chronics' that generate part of them randomly (*eg*
866+
:class:`GridStateFromFileWithForecastsWithMaintenance`) they need to be aware of this
867+
so that a reset actually update the seeds.
868+
869+
This is closely related to issue https://github.com/rte-france/Grid2Op/issues/616
870+
871+
.. danger::
872+
This function should be called only once (not 0, not twice) after a "seed" function has been set.
873+
Otherwise results might not be fully reproducible.
874+
875+
"""
876+
pass

grid2op/Chronics/handlers/baseHandler.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -494,3 +494,14 @@ def get_init_dict_action(self) -> Union[dict, None]:
494494
action space.
495495
"""
496496
raise NotImplementedError()
497+
498+
def regenerate_with_new_seed(self):
499+
"""This function is called in case of data being "cached" (for example using the
500+
:class:`grid2op.Chronics.MultifolderWithCache`)
501+
502+
In this case, the data in cache needs to be updated if the seed has changed since
503+
the time they have been added to it.
504+
505+
If your handler has some random part, we recommend you to implement this function.
506+
Otherwise feel free to ignore it"""
507+
pass

grid2op/Chronics/handlers/jsonMaintenanceHandler.py

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,8 @@ def __init__(self,
6363
self.n_line = None # used in one of the GridStateFromFileWithForecastsWithMaintenance functions
6464
self._duration_episode_default = _duration_episode_default
6565
self.current_index = 0
66-
66+
self._order_backend_arrays = None
67+
6768
def get_maintenance_time_1d(self, maintenance):
6869
return GridValue.get_maintenance_time_1d(maintenance)
6970

@@ -82,7 +83,8 @@ def _create_maintenance_arrays(self, current_datetime):
8283
self.dict_meta_data["maintenance_ending_hour"],
8384
self.dict_meta_data["daily_proba_per_month_maintenance"],
8485
self.dict_meta_data["max_daily_number_per_month_maintenance"],
85-
self.space_prng
86+
self.space_prng,
87+
self.dict_meta_data["maintenance_day_of_week"] if "maintenance_day_of_week" in self.dict_meta_data else None
8688
)
8789
GridStateFromFileWithForecastsWithMaintenance._fix_maintenance_format(self)
8890

@@ -128,4 +130,8 @@ def _clear(self):
128130

129131
def done(self):
130132
# maintenance can be generated on the fly so they are never "done"
131-
return False
133+
return False
134+
135+
def regenerate_with_new_seed(self):
136+
if self.dict_meta_data is not None:
137+
self._create_maintenance_arrays(self.init_datetime)

grid2op/Chronics/handlers/noisyForecastHandler.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -212,3 +212,7 @@ def forecast(self,
212212
res *= self._env_loss_ratio(inj_dict_env)
213213
# TODO ramps, pmin, pmax !
214214
return res.astype(dt_float) if res is not None else None
215+
216+
def regenerate_with_new_seed(self):
217+
# there is nothing to do for this handler as things are generated "on the fly"
218+
pass

0 commit comments

Comments
 (0)