Using Multiple environment with unity ML agents

[871234342]

Describe the bug
Hi, I am trying to use unity as environment with SB3 algorithm. I have no problem when running with unity editor or without VecEnv. However, I ran into error when trying to use unity environment with SubporcVecEnv. I got the following error.

Traceback (most recent call last):
  File "/media/caig/新增磁碟區/RL-script/main.py", line 76, in <module>
    env = create_envs(2)
  File "/media/caig/新增磁碟區/RL-script/main.py", line 48, in create_envs
    return SubprocVecEnv([create_env(i) for i in range(num_env)])
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 114, in __init__
    process.start()
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/multiprocessing/context.py", line 300, in _Popen
    return Popen(process_obj)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/multiprocessing/popen_forkserver.py", line 47, in _launch
    reduction.dump(process_obj, buf)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/site-packages/stable_baselines3/common/vec_env/base_vec_env.py", line 479, in __getstate__
    return cloudpickle.dumps(self.var)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 1479, in dumps
    cp.dump(obj)
  File "/home/caig/anaconda3/envs/mlagents/lib/python3.10/site-packages/cloudpickle/cloudpickle.py", line 1245, in dump
    return super().dump(obj)
TypeError: cannot pickle '_thread.lock' object

The code I'm trying to run:

def create_envs(num_env: int):
    def create_env(worker_id: int) -> Callable[[], Env]:
        def _init() -> Env:
            unity_env = UnityEnvironment(unity_file, worker_id=worker_id)
            unity_gym_env = UnityToGymWrapper(unity_env, uint8_visual=True,
                                                    allow_multiple_obs=True)
            return unity_gym_env

        return _init()
    
    return SubprocVecEnv([create_env(i) for i in range(num_env)])

env = create_envs(2)

System Info

• Describe how the library was installed pip
• GPU rtx 3090
• Python version 3.10.12
• Pytorch version 2.1.2
• OS Ubuntu 20.04