SEGFAULT in libmujoco.so.3.1.3 while using TACTO together with MuJoCo

[Pia314]

Hi,

I'm a masters student and I'm trying to use MuJoCo for simulating tactile sensors mounted on a robot arm together with TACTO for my thesis.
As a setup I have the Sawyer robot Arm, robotiq 2f-85 gripper and Digit sensors. (see attached meshes.zip). I modified the models from mujoco_menagerie here .

For the TACTO extension I use the bridge from TACTO-MuJoCo here
(Attention to reproduce my setup you have to change the coordinate order in tacto/sensor_mujoco.py in line 80 to ordering="YXZ".

My demo_mujoco_digit.py file looks like this:

import logging
import threading
from threading import Thread
import cv2
from pathlib import Path
import hydra
import tacto.sensor_mujoco as sensor  # Import Tacto library correctly
from omegaconf import DictConfig
import mujoco.viewer
from mujoco import MjModel, MjData
from utils import write_f_strings_to_file
import os
from datetime import datetime
import dm_control.mujoco

# Setup logging
logging.basicConfig(level=logging.INFO)
log = logging.getLogger(__name__)

class SimulationRunner(Thread):
    def __init__(self, mj_model: MjModel, mj_data: MjData):
        super().__init__()
        self.mj_model = mj_model
        self.mj_data = mj_data

    def run(self) -> None:
        mujoco.viewer.launch(self.mj_model, self.mj_data)


@hydra.main(config_path="conf", config_name="digit")
def main(cfg):
    log.info("Initializing simulation environment")

    bg = cv2.imread("conf/bg_digit_240_320.jpg")

    physics = dm_control.mujoco.Physics.from_xml_path(cfg.experiment.urdf_path)

    digit_body = physics.model
    digit_body_data = physics.data
    digits = sensor.Sensor(digit_body, digit_body_data, **cfg.tacto, background=bg)

    digits.add_camera(24, [23])
    digits.add_camera(17, [16])

    digits.add_body(cfg.object.urdf_path, digit_body_data.body("base_link").id, 0)
    
    # run p.stepSimulation in another thread
    t = SimulationRunner(mj_model=digit_body._model, mj_data=digit_body_data._data)
    t.start()

    while t.is_alive():
        color, depth = digits.render(visualize_digit=False)
        digits.updateGUI(color, depth)
    t.join()

if __name__ == "__main__":
    main()

Now it happens that from time to time I have a SEGFAULT in the simulation (I have the impression that it occurs more often when the gripper scrapes on the ground while the sensors are in contact with the object)
To me it looks like there is a bug on the Mujoco side. There is a bug with libmujoco.so.3.1.3. There is also a coredump here (using python 3.8 like in the setup fro TACTO-mujoco):
See coredump_3.8.txt

I have now tried to update to python 3.11 to see if it no longer occurs in new software.
But even here the error still occurs, but the coredump is different:
See coredump_3.11.txt

Can you fix this in Mujoco or help me to get rid of the segfault. Thank you.
Pia