fix(async_rl): add docs and fix typos for embodiment async rl (RLinf#790)

* docs: add docs for async ppo

* feat: support multi rollout epoch for async ppo

* chore: fix typos in async ppo's yamls

* fix: let async env worker also support multi rollout epoch

* fix: fix correct critic_warmup_steps

* feat: add resume training for async ppo

* fix: convert version to tensor in case perf regression

* chore: move version update logic to set global step

* fix: fix condition when sac does not set global step

---------

Signed-off-by: Bo Dai <daibo@infini-ai.com>

Author: anHappyDog

docs/source-en/rst_source/tutorials/rlalg/async_ppo.rst

@@ -5,7 +5,7 @@ In this section, we provide an overview of each algorithm, including their core
 
 Each algorithm is implemented with flexibility in mind, allowing researchers and practitioners to apply them to a variety of reinforcement learning tasks. Whether you're exploring standard benchmarks or designing custom environments, RLinf offers streamlined interfaces for training and evaluation.
 
-As of now, RLinf supports seven widely-used reinforcement learning algorithms:
+As of now, RLinf supports eight widely-used reinforcement learning algorithms:
 
 - :doc:`Proximal Policy Optimization (PPO) <ppo>`
 - :doc:`Group Relative Policy Optimization (GRPO) <grpo>`
@@ -14,6 +14,7 @@ As of now, RLinf supports seven widely-used reinforcement learning algorithms:
 - :doc:`Soft Actor-Critic (SAC) <sac>`
 - :doc:`Cross-Q <crossq>`
 - :doc:`RLPD <rlpd>`
+- :doc:`Async Proximal Policy Optimization (Async PPO) <async_ppo>`
 
 We are continuously working to expand the selection of supported algorithms in future releases. Stay tuned for upcoming additions!
 
@@ -28,3 +29,4 @@ We are continuously working to expand the selection of supported algorithms in f
    sac
    crossq
    rlpd
+   async_ppo