Merge pull request #615 from BDonnot/master

Upgrade notebooks on RL

Author: BDonnot

.gitignore

@@ -412,7 +412,7 @@ getting_started/env_py38_grid2op110_ray110.ipynb
 getting_started/env_py38_grid2op110_ray210.ipynb
 grid2op/tests/req_chronix2grid
 grid2op/tests/venv_test_chronix2grid/
-
+getting_started/venv_310_ray/
 
 # profiling files
 **.prof

CHANGELOG.rst

@@ -31,15 +31,28 @@ Change Log
 - [???] "asynch" multienv
 - [???] properly model interconnecting powerlines
 
-
+Work kind of in progress
+----------------------------------
 - TODO A number of max buses per sub
 - TODO in the runner, save multiple times the same sceanrio
+- TODO in the gym env, make the action_space and observation_space attribute
+  filled automatically (see ray integration, it's boring to have to copy paste...)
+
+Next release
+---------------------------------
+- TODO Notebook for tf_agents
+- TODO Notebook for acme
+- TODO Notebook using "keras rl" (see https://keras.io/examples/rl/ppo_cartpole/)
+- TODO put the Grid2opEnvWrapper directly in grid2op as GymEnv
+- TODO example for MCTS https://github.com/bwfbowen/muax et https://github.com/google-deepmind/mctx
+- TODO jax everything that can be: create a simple env based on jax for topology manipulation, without
+  redispatching or rules
+- TODO backend in jax, maybe ?
 
 
 [1.10.3] - 2024-xx-yy
 -------------------------
 - TODO Automatic "experimental_read_from_local_dir"
-- TODO Notebook for stable baselines
 
 - [BREAKING] `env.chronics_hander.set_max_iter(xxx)` is now a private function. Use 
   `env.set_max_iter(xxx)` or even better `env.reset(options={"max step": xxx})`. 
@@ -59,7 +72,7 @@ Change Log
 
 [1.10.2] - 2024-05-27
 -------------------------
-- [BREAKING] the `runner.run_one_episode` now returns an extra first argument: 
+- [BREAKING] the `runner.run_one_episode` now returns an extra argument (first position): 
   `chron_id, chron_name, cum_reward, timestep, max_ts = runner.run_one_episode()` which 
   is consistant with `runner.run(...)` (previously it returned only 
   `chron_name, cum_reward, timestep, max_ts = runner.run_one_episode()`)