Add scaling section to A2C documentation (#1250)

* add scaling section to A2C documentation

* add cross-reference to vectorized envs article

* turn it as note

* update changelog

* add Bonifatius94 to the list of contributors

* fix issue number

---------

Co-authored-by: Quentin Gallouédec <[email protected]>
Co-authored-by: Quentin GALLOUÉDEC <[email protected]>
Co-authored-by: Antonin RAFFIN <[email protected]>

Author: 3 people

docs/misc/changelog.rst

@@ -39,6 +39,7 @@ Others:
 Documentation:
 ^^^^^^^^^^^^^^
 - Renamed ``load_parameters`` to ``set_parameters`` (@DavyMorgan)
+- Clarified documentation about subproc multiprocessing for A2C (@Bonifatius94)
 - Fixed typo in ``A2C`` docstring (@AlexPasqua)
 
 
@@ -1222,4 +1223,4 @@ And all the contributors:
 @Gregwar @ycheng517 @quantitative-technologies @bcollazo @git-thor @TibiGG @cool-RR @MWeltevrede
 @Melanol @qgallouedec @francescoluciano @jlp-ue @burakdmb @timothe-chaumont @honglu2875 @yuanmingqi
 @anand-bala @hughperkins @sidney-tio @AlexPasqua @dominicgkerr @Akhilez @Rocamonde @tobirohrer @ZikangXiong
-@DavyMorgan @luizapozzobon
+@DavyMorgan @luizapozzobon @Bonifatius94

docs/modules/a2c.rst

@@ -76,6 +76,24 @@ Train a A2C agent on ``CartPole-v1`` using 4 environments.
       env.render()
 
 
+.. note::
+
+  A2C is meant to be run primarily on the CPU, especially when you are not using a CNN. To improve CPU utilization, try turning off the GPU and using ``SubprocVecEnv`` instead of the default ``DummyVecEnv``:
+
+  .. code-block::
+
+    from stable_baselines3 import A2C
+    from stable_baselines3.common.env_util import make_vec_env
+    from stable_baselines3.common.vec_env import SubprocVecEnv
+
+    if __name__=="__main__":
+        env = make_vec_env("CartPole-v1", n_envs=8, vec_env_cls=SubprocVecEnv)
+        model = A2C("MlpPolicy", env, device="cpu")
+        model.learn(total_timesteps=25_000)
+  
+  For more information, see :ref:`Vectorized Environments <vec_env>`, `Issue #1245 <https://github.com/DLR-RM/stable-baselines3/issues/1245>`_ or the `Multiprocessing notebook <https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/multiprocessing_rl.ipynb>`_.
+
+
 Results
 -------