Grasping cloth object using Franka gripper #2015

sachinkundu · 2024-03-06T11:39:51Z

sachinkundu
Mar 6, 2024

Hello,

I have flex object with a definition like

Details

<?xml version="1.0" encoding="utf-8"?>
<mujoco model="cloth">
	<compiler angle="radian"></compiler>
	<option timestep="0.01" tolerance="1e-06" viscosity="0.3" integrator="implicitfast" solver="CG"></option>
	<size nkey="10"></size>
	<statistic meansize="0.0703541" extent="2" center="0 0 0"></statistic>
	<extension>
		<plugin plugin="mujoco.elasticity.shell">
			<instance name="cloth_flexcomp_cloth">
				<config key="face" value="0 5 6 0 6 1 1 6 7 1 7 2 2 7 8 2 8 3 3 8 9 3 9 4 5 10 11 5 11 6 6 11 12 6 12 7 7 12 13 7 13 8 8 13 14 8 14 9 10 15 16 10 16 11 11 16 17 11 17 12 12 17 18 12 18 13 13 18 19 13 19 14 15 20 21 15 21 16 16 21 22 16 22 17 17 22 23 17 23 18 18 23 24 18 24 19"></config>
				<config key="edge" value="0 1 2 3 4 1 5 6 3 7 8 6 9 10 7 11 12 10 13 14 11 15 16 14 17 18 19 20 0 18 21 22 20 23 5 22 24 25 23 26 9 25 27 28 26 29 13 28 30 31 32 33 17 31 34 35 33 36 21 35 37 38 36 39 24 38 40 41 39 42 27 41 43 44 45 46 30 44 47 48 46 49 34 48 50 51 49 52 37 51 53 54 52 55 40 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"></config>
				<config key="young" value="85242.02369946481"></config>
				<config key="poisson" value="0.4032740512363587"></config>
				<config key="thickness" value="0.0005"></config>
			</instance>
		</plugin>
	</extension>
	<worldbody>
		<geom name="floor" size="0 0 0.1" type="plane"></geom>
		<light pos="0 0 4" dir="0 0 -1" diffuse="0.6 0.6 0.6" specular="0.2 0.2 0.2"></light>
		<body>
			<body name="object" pos="0 0 0.83">
				<site name="bottom_site" pos="0 0 -0.045" rgba="0 0 0 0"></site>
				<site name="top_site" pos="0 0 0.03" rgba="0 0 0 0"></site>
				<site name="horizontal_radius_site" pos="0.03 0.03 0" rgba="0 0 0 0"></site>
				<body name="cloth_0" pos="0.17 -0.038 0.00076">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_1" pos="0.13 -0.057 0.00067">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_2" pos="0.078 -0.059 0.00076">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_3" pos="0.072 -0.0088 0.00066">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_4" pos="0.022 -0.017 0.00065">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_5" pos="0.13 -0.06 0.0044">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_6" pos="0.11 -0.02 0.03">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_7" pos="0.074 -0.01 0.00085">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_8" pos="0.023 -0.016 0.0026">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_9" pos="0.013 0.032 0.00075">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_10" pos="0.11 -0.085 0.046">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_11" pos="0.077 -0.05 0.05">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_12" pos="0.043 -0.012 0.04">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_13" pos="0.019 0.029 0.023">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_14" pos="0.061 0.045 0.00067">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_15" pos="0.086 -0.098 0.005">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_16" pos="0.076 -0.049 0.00047">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_17" pos="0.07 -0.00053 0.00052">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_18" pos="0.061 0.046 0.0027">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_19" pos="0.045 0.093 0.00091">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_20" pos="0.14 -0.089 0.00076">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_21" pos="0.13 -0.04 0.00075">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_22" pos="0.12 0.0095 0.00076">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_23" pos="0.11 0.059 0.00076">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
				<body name="cloth_24" pos="0.094 0.11 0.00075">
					<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
					<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
					<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
					<plugin instance="cloth_flexcomp_cloth"></plugin>
				</body>
			</body>
		</body>
	</worldbody>
	<deformable>
		<flex name="cloth" radius="0.001" body="cloth_0 cloth_1 cloth_2 cloth_3 cloth_4 cloth_5 cloth_6 cloth_7 cloth_8 cloth_9 cloth_10 cloth_11 cloth_12 cloth_13 cloth_14 cloth_15 cloth_16 cloth_17 cloth_18 cloth_19 cloth_20 cloth_21 cloth_22 cloth_23 cloth_24" element="0 5 6 0 6 1 1 6 7 1 7 2 2 7 8 2 8 3 3 8 9 3 9 4 5 10 11 5 11 6 6 11 12 6 12 7 7 12 13 7 13 8 8 13 14 8 14 9 10 15 16 10 16 11 11 16 17 11 17 12 12 17 18 12 18 13 13 18 19 13 19 14 15 20 21 15 21 16 16 21 22 16 22 17 17 22 23 17 23 18 18 23 24 18 24 19">
			<contact selfcollide="auto" condim="4" friction = "4 0.1 0.001" solimp="0.001"></contact>
<!--			<edge damping="1"></edge>-->
		</flex>
	</deformable>
	<equality>
<!--		<flex flex="cloth"></flex>-->
		<flex flex="cloth_cloth"></flex>
	</equality>
</mujoco>

I am using robosuite which has a model for Franka Gripper with the definition like

Details

            <body name="leftfinger" pos="0 0 0.0524" quat="0.707107 0 0 0.707107">
                <inertial pos="0 0 0.05" mass="0.1" diaginertia="0.01 0.01 0.005" />
                <joint name="finger_joint1" pos="0 0 0" axis="0 1 0" type="slide" limited="true" range="0.0 0.04" damping="100" armature="1.0" frictionloss="1.0"/>
                <geom type="mesh" contype="0" conaffinity="0" group="1" mesh="finger_vis" name="finger1_visual" rgba="0.499 0.499 0.499 1" />
                <geom type="mesh" group="0" conaffinity="1" contype="0" solref="0.02 1" friction="1 0.005 0.0001" condim="4" mesh="finger" name="finger1_collision"/>
                <body name="finger_joint1_tip" pos="0 0.0085 0.056">
                    <inertial pos="0 0 0" quat="0 0 0 1" mass="0.01" diaginertia="0.01 0.01 0.01" />
                    <geom size="0.008 0.004 0.008" pos="0 -0.005 -0.015" quat="0 0 0 1" type="box" group="0" solref="0.01 0.5" friction = "2 0.05 0.0001" conaffinity="1" contype="1" name="finger1_pad_collision"/>
                </body>
            </body>
            <body name="rightfinger" pos="0 0 0.0524" quat="0.707107 0 0 0.707107">
                <inertial pos="0 0 0.05" mass="0.1" diaginertia="0.01 0.01 0.005" />
                <joint name="finger_joint2" pos="0 0 0" axis="0 1 0" type="slide" limited="true" range="-0.04 0.0" damping="100" armature="1.0" frictionloss="1.0"/>
                <geom quat="0 0 0 1" type="mesh" contype="0" conaffinity="0" group="1" mesh="finger_vis" name="finger2_visual" rgba="0.499 0.499 0.499 1" />
                <geom quat="0 0 0 1" type="mesh" group="0" conaffinity="1" contype="0" solref="0.02 1" friction="1 0.005 0.0001" condim="4" mesh="finger" name="finger2_collision"/>
                <body name="finger_joint2_tip" pos="0 -0.0085 0.056">
                    <inertial pos="0 0 0" quat="0 0 0 1" mass="0.01" diaginertia="0.01 0.01 0.01" />
                    <geom size="0.008 0.004 0.008" pos="0 0.005 -0.015" quat="0 0 0 1" type="box" group="0" solref="0.01 0.5" friction = "2 0.05 0.0001" conaffinity="1" contype="1" name="finger2_pad_collision"/>
                </body>
            </body>

I have tried loads of contact parameters but I cannot get the cloth to stick to the gripper when grasping. You can see the failed video here

Does anyone know what parameters could work? These fingers can reliably grasp a cube whose solimp=[0.99, 0.99, 0.01], and solref=[0.01, 1]. I am using a very simple cloth object with only 25 bodies(it was automatically generated by flexcomp macro) because my laptop is too slow to simulate more detailed cloth.

dblanm · 2024-03-20T15:42:07Z

dblanm
Mar 20, 2024

I have experienced the same when trying to grasp cloth-like objects with a robotic gripper.
Rather than this being related to the cloth parameters I wonder if it has something to do with the model itself.

I guess @quagla can point to what can this issue relate to and whether this is something the team is working on addressing.

0 replies

quagla · 2024-03-21T16:57:48Z

quagla
Mar 21, 2024
Collaborator

Hi, I cannot tell from the video what exactly the issue is. Do you still have the same problem if you make the cloth thicker by increasing radius in the flex parameters?

0 replies

dblanm · 2024-05-15T08:20:05Z

dblanm
May 15, 2024

I tried by increasing the radius + different contact parameters and the issue remains:

The franka applies a constant force trying to close the grippers for grasping the object, and when moving the franka to a new position with the cloth "grasped" the cloth slips from the fingers of the robot.
In the same environment grasping works well with a rigid object (box).

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Grasping cloth object using Franka gripper #2015

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Grasping cloth object using Franka gripper #2015

Uh oh!

sachinkundu Mar 6, 2024

Replies: 3 comments

Uh oh!

dblanm Mar 20, 2024

Uh oh!

quagla Mar 21, 2024 Collaborator

Uh oh!

dblanm May 15, 2024

sachinkundu
Mar 6, 2024

dblanm
Mar 20, 2024

quagla
Mar 21, 2024
Collaborator

dblanm
May 15, 2024