shared_train_eval_env #1732
shared_train_eval_env #1732
Conversation
| seed=seed) | ||
| if config.shared_train_eval_env: | ||
| self._env = alf.get_env() | ||
| self._env.reset() |
There was a problem hiding this comment.
assert async_eval = False?
There was a problem hiding this comment.
Good point. Added assertion.
There was a problem hiding this comment.
Commit not pushed to the right remote?
There was a problem hiding this comment.
ah, yes, that was what happened ... now pushed
There was a problem hiding this comment.
Happened to me also. It's hard to remember, especially now that we don't change alf that often. We can probably remove the other remote.
| if config.shared_train_eval_env: | ||
| assert not self._async, "should not use async_eval in shared_train_eval_env mode" | ||
| self._env = alf.get_env() | ||
| self._env.reset() |
There was a problem hiding this comment.
Curious, why do we call env.reset() here but not in the other branch. Maybe add a comment in the code?
| for_evaluation=True, | ||
| num_parallel_environments=num_envs, | ||
| seed=seed) | ||
| if config.shared_train_eval_env: |
There was a problem hiding this comment.
Need to set the step_type in the replay buffer just before evaluation started to StepType.LAST.
There was a problem hiding this comment.
Also need to set the next step type for training to FIRST
Provides an option to train and eval on the same shared env, mimicking the case such as training in real with only one physical env.