Simulation consistency between single (num_envs=1) and parallel (num_envs=K) environments with set_state_dict

[luokuang2001]

In my experiments, I have observed behavioral inconsistencies between the CPU and GPU backends.
I would like to verify if PhysX CPU and GPU solvers might produce slightly different contact dynamics, especially for friction-heavy tasks like PushT-v1

If such a discrepancy exists, does this imply that policies (or World Models) trained on data collected from the GPU backend must be evaluated strictly on the GPU backend to ensure valid results?

[luokuang2001]

I am implementing a Ground Truth (GT) dynamics model for an MPPI planner using ManiSkill. My architecture consists of two environments:

• env_real (num_envs=1): Represents the "real world" or interaction environment.
• env_sim (num_envs=K, e.g., 2048): Used as the internal world model for MPPI to perform massive parallel rollouts.

My goal is to copy the exact state from env_real to all 2048 environments in env_sim at every planning step to ensure the rollouts start from the true current state.

Here is my current implementation

def dynamics_gt(state, action):
    """
    Ground Truth dynamics based on state synchronization.
    
    Args:
        state: (K, 1, nx) - Current state.
        action: (K, H, nu) - Action sequences generated by MPPI Planner.
    """
    # ------------------------------------------------------------------
    # Step 1: State Synchronization (Sync Real -> Sim)
    # ------------------------------------------------------------------
    # Get the authoritative physical state from env_real
    real_state_dict = env_real.base_env.get_state_dict()

    # Apply this state to the parallel environment
    env_sim.base_env.set_state_dict(real_state_dict)

    # ------------------------------------------------------------------
    # Step 2: Parallel Rollout
    # ------------------------------------------------------------------
    K, H, nu = action.shape
    predicted_states = []
    
    # Execute H steps in parallel on GPU
    for t in range(H):
        action_t = action[:, t, :] # (K, nu)
        
        # Step simulation
        obs, _, _, _, _ = env_sim.step(action_t)
        
        # Process observation
        predicted_states.append(obs_processed.unsqueeze(1))
    
    # Reset to clear buffers (if necessary)
    env_sim.reset(seed=0)

    # Return trajectory: (K, H, nx)
    return torch.cat(predicted_states, dim=1)

# Initialize the large environment (Sim / World Model)
env_sim = gym.make(
    "PushT-v1",
    obs_mode="state",
    control_mode="pd_ee_delta_pos",
    num_envs=NUM_SAMPLES  # e.g., 2048
)
env_sim = ManiSkillVectorEnv(env_sim, ignore_terminations=True, auto_reset=False)
env_sim.reset(seed=0) 

# Initialize the single environment (Real / Interaction)
print("Initializing env_real (1)...")
env_real = gym.make(
    "PushT-v1",
    obs_mode="state",
    control_mode="pd_ee_delta_pos",
    render_mode="rgb_array", 
    num_envs=1,
    sim_backend="physx_cuda" # Ensure backend consistency
)
env_real = ManiSkillVectorEnv(env_real, ignore_terminations=True, auto_reset=False)

# Wrapper for saving videos
env_real = RecordEpisode(
    env_real,
    output_dir="./plan_videos_gt",
    save_trajectory=False,
    save_video=True,
    video_fps=10,
    info_on_video=True
)

env_real.reset(seed=0)

I have observed some potential subtle inconsistencies between the env_real behavior and the env_sim rollouts. I would like to clarify the following:

• Should I expect the simulation result of env_real (step 0 to T) to be bit-wise identical to env_sim (step 0 to T) if they start from the same state? Are there known issues where parallelization introduces slight numerical divergence compared to a single-environment instance?

• Is there a better or more standard way to clone the state of a single environment into a vectorized environment for planning purposes?

Any guidance or insights on this would be greatly appreciated. Thank you!

[luokuang2001]

I am working with the PushT-v1 environment and encountering state synchronization and determinism issues when transferring the state from a single environment (num_envs=1) to a vectorized environment (num_envs=N, e.g., 2) using get_state_dict() and set_state_dict().

The Workflow:

• Initialize env_A (num_envs=1) and env_B (num_envs=2). Both use physx_cuda.
• Step env_A for 5 steps (warm-up) to reach a non-initial state.
• Sync state: env_B.set_state_dict(env_A.get_state_dict()).
• Run both environments for 30 steps using identical actions (broadcasting the action from A to B).

The Problem:
I observe significant divergence in the object (T-block) pose:

• A vs B: The trajectory in env_B diverges from env_A.
• Internal Divergence in B: Surprisingly, the two parallel environments within env_B (index 0 and index 1) also diverge from each other, despite being initialized with the same state and receiving the exact same actions.

real.mp4

sim.mp4

[StoneT2000]

GPU simulation should not be assumed to be deterministic unless

• The number of parallel environments is the same
• The spacing of these environments are the same
• The state encompassing all of the parallel environments (not just per env) is the same

This can be due to a number of factors including floating point errors. Each parallel env is actually simulated in one giant scene, and is just placed at different locations, so small floating point differences can accumulate and cause a difference.

Moreover, if you use a single environment you default to the physx_cpu backend, which runs different a physics engine and has no guarantee of being the same.

Some environments like PickCube don't have this problem as much (when running actions from a converged pick cube policy) since the robot grasp can be off by a little and the cube will still be grasped, and the rest of the episode the cube's pose is determined by the robot's joint angles. Robot joint angles generally do not drift / have as much error (assuming not many contacts or using an absolute controller) since at each timestep they are driven to some given joint target.