Goal visualization affects physics

[s-bl]

Hey,

I noticed that adding the goal_marker to the scene has an affect on the physics.
If I replay an action sequence with and without the goal_marker (in both cases rendering is enabled), I end up in significantly different environment states.

Best,
Sebastian

[maximilian-maurer]

Could you expand on this? Am also participating in the competition and we trained for the entire duration of the competition with the goal markers turned on, believing it to be a purely visual feature with no effect on the physics, so would be very curious as to what the differences in behavior are (and to what extent it is different). Is this local to the replay procedure (e.g. there is a mismatch) or is there a larger gap between how the physics behave between the two versions that would be generally noticeable?

[luator]

@s-bl Just to be sure, do you restart everything between the two replays or reset and use the same simulator instance? It would be great if you could provide a minimal reproducible example.

[s-bl]

I noticed because when I produce an action sequence with visualization enabled, the replay_action_log script would fail since there is a significant difference between final states.
I will produce a minimal example in a moment.

[s-bl]

Here is a test script:

import json

from rrc_simulation.gym_wrapper.envs import cube_env
from rrc_simulation.tasks import move_cube

action_log_json = 'action_log.json'

initial_pose_json = "{\"position\": [-0.01186604155103307, -0.052602388371748576, 0.0325], \"orientation\": [0.0, 0.0, 0.9235094793209279, 0.38357560089816534]}"
goal_pose_json = "{\"position\": [-0.0313098394360814, -0.010814080543716283, 0.1], \"orientation\": [0, 0, 0, 1]}"
difficulty = 3

initial_pose = move_cube.Pose.from_json(initial_pose_json)
goal_pose = move_cube.Pose.from_json(goal_pose_json)

initializer = cube_env.FixedInitializer(
    difficulty, initial_pose, goal_pose
)

env = cube_env.CubeEnv(
    initializer=initializer,
    action_type=cube_env.ActionType.TORQUE,
    frameskip=1,
    visualization=True
)

with open(action_log_json, 'r') as f:
    action_log = json.load(f)

env.reset()

t = 0
for action in action_log['actions']:
    env.step(action['torque'])
    t +=1

final_state_w_viz = env.platform.get_object_pose(t).position

env = cube_env.CubeEnv(
    initializer=initializer,
    action_type=cube_env.ActionType.TORQUE,
    frameskip=1,
    visualization=False
)

env.reset()

t = 0
for action in action_log['actions']:
    env.step(action['torque'])
    t +=1

final_state_wo_viz = env.platform.get_object_pose(t).position

print(f'Originally reached state: {action_log["final_object_pose"]["position"]}')
print(f'Replayed reached state with visualization on: {final_state_w_viz}')
print(f'Replayed reached state with visualization off: {final_state_wo_viz}')

Please rename action_log.txt to action_log.json.

My result:

Originally reached state: [-0.03479924244723463, -0.011842202911899849, 0.056331027913188496]
Replayed reached state with visualization on: [-0.03458475 -0.011834    0.05632453]
Replayed reached state with visualization off: [-0.12601226 -0.09786351  0.03348581]

If I remove the visual_box from the code, I achieve the same results.

Best,
Sebastian

action_log.txt

[luator]

Thanks, I can reproduce your results. This is very weird... We will investigate this.