Geom orientation sometimes parsed or compiled wrongly with MjSpec

[jbruedigam-bdai]

Intro

Hi!

I am a researcher at RAI, I use MuJoCo for my research on robot manipulation.

My setup

MuJoCo 3.3.3 (switched from 3.2.6), Python, Ubuntu 22.04

My question

We recently switched from Mujoco version 3.2.6 to 3.3.3 and now we are seeing the following issue:

We have one xml for a robot and different xmls for different objects. We dynamically compose an MjSpec to obtain one model with the robot and a specific object.
Sometimes (i.e., not deterministically), the orientation of the collision geoms is wrong after calling spec.compile(). Position and other things are fine. If the error occurs, the orientation is almost at identity, up to some numerical deviation.
I don't have a minimal example at this point because the code for generating the composed scene is a bit involved. But here's some things that may be relevant:

• we are using include statements in the xml files. If I directly add components it appears to not have this issue, but could also just be random
• if spec.compile() gives the wrong results, calling it again also gives the wrong results. So that part is deterministic. But calling the full compose code is not. However, we are not changing any values in the compose code and it should be deterministic as well.
• when the issue occurs and I print the compiled xml model to string, it also has the wrong orientation values.
• If the issue occurs, only the orientations of the primitive collisions shapes are wrong, not the meshes. Maybe mesh orientations are parsed correctly, but primitive shapes not?

Something I'm curious about that might point to the issue is that when loading a spec from file (and not compiling yet), the position of the geoms is set as specified in the xml, but the quat is just set to identity (see code example below). It appears to be set during compile.

• Is that expected?
• Why is the quat not also parsed directly?
• Could you point me to where the quat is parsed in the C code?

Minimal model and/or code that explain my question

import mujoco

xml = """
    <mujoco>
    <worldbody>
        <geom name="my_geom" type="box" size="1 1 1" pos="1 2 3" euler="45 0 0" mass="1"/>
    </worldbody>
    </mujoco>
"""

spec = mujoco.MjSpec().from_string(xml)
print(spec.geoms[0].pos)  # prints "1 2 3" as expected
print(spec.geoms[0].quat)  # prints "1 0 0 0", not what I'd naively expect
model = spec.compile()
print(model.geom(0).pos)  # prints "1 2 3" as expected
print(model.geom(0).quat)  # prints "0.92 0.38 0 0" as expected

Confirmations

• I searched the latest documentation thoroughly before posting.
• I searched previous Issues and Discussions, I am certain this has not been raised before.

[quagla]

Wow really hard to debug :)

Next time it fails, can you check if this quat is correct?

 quat = mujoco.MjSpec.resolve_orientation(
        degree=spec.compiler.degree,
        sequence=spec.compiler.eulerseq,
        orientation=geom.alt,
    )

Or in general, if geom.alt contains the correct values. At least, we would know if it was parsed correctly and nothing else wrote into it.

[jbruedigam-bdai]

Some results:

• In the minimal example I provided, the quat from your code is correct. geom.alt.euler is also correct.
• In our model composition code, this quat value is always wrong, and sometimes, the compiled value also takes on this wrong value.
• In our model composition code, the geom.alt.euler value is correct, however.

For the following code inside our code

quat = mujoco.MjSpec.resolve_orientation(
    degree=self.spec.compiler.degree,
    sequence=self.spec.compiler.eulerseq,
    orientation=self.spec.geoms[47].alt,
)
print("printing values")
print(f"degree: {self.spec.compiler.degree}")
print(f"eulerseq: {self.spec.compiler.eulerseq}")
print(f"alt.euler: {self.spec.geoms[47].alt.euler}")
print(f"quat: {quat}")
model = self.spec.compile()
print(f"compiled quat: {model.geom(47).quat}")

I sometimes get

printing values
degree: 1
eulerseq: <mujoco._specs.MjCharVec object at 0x75f7582475b0>
alt.euler: [0.         1.57079633 0.        ]
quat: [0.9999060498015511, 0.0, 0.013707354604664747, 0.0]
compiled quat: [0.99990605 0.         0.01370735 0.        ]

and sometimes

printing values
degree: 1
eulerseq: <mujoco._specs.MjCharVec object at 0x75f7582475b0>
alt.euler: [0.         1.57079633 0.        ]
quat: [0.9999060498015511, 0.0, 0.013707354604664747, 0.0]
compiled quat: [0.70710678 0.         0.70710678 0.        ]

Not sure, but I think degree should be 0 since we are using radians in our code. But it's always 1.
Looking at the values again, it seems like the "wrong" value would be correct if the Euler angles where specified in degrees, not radians.
Not sure why it works sometimes

[jbruedigam-bdai]

If I set self.spec.compiler.degree = 0, the resolve_orientation quat is correct, but I still get wrong values for the compiled quat from time to time, i.e., for this code:

self.spec.compiler.degree = 0
quat = mujoco.MjSpec.resolve_orientation(
    degree=self.spec.compiler.degree,
    sequence=self.spec.compiler.eulerseq,
    orientation=self.spec.geoms[47].alt,
)
print("printing values")
print(f"degree: {self.spec.compiler.degree}")
print(f"eulerseq: {self.spec.compiler.eulerseq}")
print(f"alt.euler: {self.spec.geoms[47].alt.euler}")
print(f"quat: {quat}")
model = self.spec.compile()
print(f"compiled quat: {model.geom(47).quat}")

I sometimes get

printing values
degree: 0
eulerseq: <mujoco._specs.MjCharVec object at 0x77f7763472b0>
alt.euler: [0.         1.57079633 0.        ]
quat: [0.7071067811882787, 0.0, 0.7071067811848163, 0.0]
compiled quat: [0.70710678 0.         0.70710678 0.        ]

and sometimes

printing values
degree: 0
eulerseq: <mujoco._specs.MjCharVec object at 0x7a372da79a30>
alt.euler: [0.         1.57079633 0.        ]
quat: [0.7071067811882787, 0.0, 0.7071067811848163, 0.0]
compiled quat: [0.99990605 0.         0.01370735 0.        ]

[tzhao-bdai]

Hi @quagla! Thank you for taking a look at this. I have a minimal example of what @jbruedigam-bdai was talking about before, and have filed a bug report #2834 . Would appreciate any ideas you have about what might be causing the issue!