SAC: ValueError: setting an array element with a sequence (stable-baselines 2.10)

[marianophielipp]

Code:
[Custom made environment]
import gym
import numpy as np
from stable_baselines.sac.policies import MlpPolicy
from stable_baselines import SAC

model = SAC(MlpPolicy, env, verbose=1)

if train:
model.learn(total_timesteps=total_timesteps, log_interval=10)

Output:

| current_lr | 0.0003 |
| episodes | 10 |
| fps | 0 |
| mean 100 episode reward | -4 |
| n_updates | 0 |
| time_elapsed | 151 |
| total timesteps | 92 |

TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "train_real_arm_perception_1.py", line 43, in
model.learn(total_timesteps=total_timesteps, log_interval=10)
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/stable_baselines/sac/sac.py", line 464, in learn
mb_infos_vals.append(self._train_step(step, writer, current_lr))
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/stable_baselines/sac/sac.py", line 343, in _train_step
out = self.sess.run(self.step_ops, feed_dict)
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1142, in _run
np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/numpy/core/_asarray.py", line 85, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

[Miffyli]

Use env checker to see if your environment works correctly.

[araffin]

PS: Please use and read the issue template (the env checker is mentioned there too)

[araffin]

I think we need to output a better error message in SB3 (see #707 and #712)
Currently, we cannot do that properly because of the Unvecwrapper...

EDIT: the mentioned issue is not the same but it is related in term of unclear message

[marianophielipp]

check_env(env)
<class 'numpy.ndarray'>
Traceback (most recent call last):
File "", line 1, in
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/stable_baselines/common/env_checker.py", line 214, in check_env
_check_returned_values(env, observation_space, action_space)
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/stable_baselines/common/env_checker.py", line 99, in _check_returned_values
_check_obs(obs, observation_space, 'reset')
File "/home/ipc/open_baselines/open_base/lib/python3.6/site-packages/stable_baselines/common/env_checker.py", line 89, in _check_obs
"method does not match the given observation space".format(method_name))
AssertionError: The observation returned by the reset() method does not match the given observation space

a=env.observation_space.sample()
<class 'numpy.ndarray'>
b=env.reset()
<class 'numpy.ndarray'>
a.shape
(6,)
b.shape
(6,)
a
array([0.30428773, 0.42360216, 0.8966984 , 0.4622259 , 0.6768906 ,
0.5416117 ], dtype=float32)
b
array([ 0.74211503, 0.34441176, 0.33516484, 0.2 , -0.25 ,
0.1 ], dtype=float32)

[Miffyli]

As the error message says, observation from reset() differs from the one set by self.observation_space.

This is not a place for technical support, though. Please close this issue if there are no further enhancements/issues related to stable-baselines.

[marianophielipp]

ok. but seems to be the same. I debug everything. even compare with CartPole.

[Miffyli]

That is because SAC does not support discrete actions, only continuous ones (see docs).

[marianophielipp]

I'm using continuous actions

[Miffyli]

That is because SAC does not support discrete actions, only continuous ones (see docs). Indeed this error should be clarified in future updates.

Edit: Github derped my messages.