UR5e moves to (0,0,0,0,0,0) during normal execution

[iydv]

Affected ROS2 Driver version(s)

latest humble commit 1828fe0

Used ROS distribution.

Humble

Which combination of platform is the ROS driver running on.

Ubuntu Linux with standard kernel, Ubuntu Linux with realtime patch

How is the UR ROS2 Driver installed.

Build both the ROS driver and UR Client Library from source

Which robot platform is the driver connected to.

UR E-series robot, Real robot

Robot SW / URSim version(s)

5.13

How is the ROS driver used.

Through the robot teach pendant using External Control URCap

Issue details

Summary

We are using UR5e for a small picking demonstration. We are sending basic trajectories to move the robot inside a box, pick part, move outside and place in another box. Trajectories are tested and verified on ros2 control simulator.

Issue details

Once a new trajectory goal is published by our node to the driver, instead of moving along desired trajectory, robot suddenly moves to initial 0,0,0,0,0,0 position with very high speed. This behavior occurs randomly and is completely unpredictable. Here is a detailed description of our setup:

1. We use Ubuntu 22.04 with PREEMPT_RT kernel 6.1.46. ROS2 humble is obtained from latest available binary packages. The limits in /etc/security/limits.conf also correctly configured. We have also tested on standard 6.2 kernel without realtime and got same result.
2. We have built both ROS2 driver and Client Library from source. We used latest commit in humble branch for the driver and latest commit for Client Library.
3. We are using scaled_joint_trajectory_controller . We are publishing simple trajectories using node with rclcpp_action client. Each trajectory is either 2 points (start joint state, goal joint state and duration) or 6 points with spline time parameterization.
4. We have installed latest update SW 5.13 on the PolyScope.
5. Robot is running in remote mode. We have installed External Control URcap v1.0.5 on the PolyScope.
6. We start robot (power, break release etc) using dashboard ROS2 services. We load and play the URCap program also using dashboard ROS2 services.
7. Robot controller is connected to the PC using single dedicated network cable.

The error occured in following cases:

1. Normal operation. We are sending a trajectory of 6 points and instead of executing it, the robot shows error attached in the log and moves to 0,0,0,0,0,0 with high speed.
2. We stopped the robot using emergency button on teach panel. Then we resume the robot power. Stop URCap program and play it again. When the new trajectory is published the robot moves to to 0,0,0,0,0,0 with high speed. This occurs randomly and typically the robot continues operate normally after stop.
3. We stop and start the driver multiple times during the day. If the driver did not started correctly (no indication in the log) same error occurs.
4. We have switched robot to the local control, moved robot using teach panel, switched back to remote mode, started URCap program using dashboard play service.

Steps to Reproduce

The error occurs randomly, you can try to replicate cases above and send single trajectory of >6 points.

Expected Behavior

According to the error log the driver lost connection or not getting correct feedback from the robot. If the connection to the robot is lost/interrupted the driver should cancel motion and stop. The robot should never randomly move to 0,0,0,0,0,0 position in any circumstance.

Actual Behavior

The robot starts moving to 0,0,0,0,0,0 position at very high speed.

Workaround Suggestion

We have not tested to run in headless mode. But our current configuration is not safe to use.

Relevant log output

Sep 12 13:21:23  [ur_ros2_control_node-1] [INFO] [1694517683.420553366] [scaled_joint_trajectory_controller]: Received new action goal
Sep 12 13:21:23  [ur_ros2_control_node-1] [INFO] [1694517683.420580925] [scaled_joint_trajectory_controller]: Accepted new action goal
Sep 12 13:21:27  [ur_ros2_control_node-1] [INFO] [1694517687.421991020] [scaled_joint_trajectory_controller]: Goal reached, success!
Sep 12 13:21:27  [ur_ros2_control_node-1] [INFO] [1694517687.615490226] [io_and_status_controller]: Setting digital output '4' to state: '1'.
Sep 12 13:21:27  [ur_ros2_control_node-1] [INFO] [1694517687.715651625] [io_and_status_controller]: Setting digital output '5' to state: '1'.
Sep 12 13:21:27  [ur_ros2_control_node-1] [INFO] [1694517687.826102946] [scaled_joint_trajectory_controller]: Received new action goal
Sep 12 13:21:27  [ur_ros2_control_node-1] [INFO] [1694517687.826135399] [scaled_joint_trajectory_controller]: Accepted new action goal
Sep 12 13:21:29  [ur_ros2_control_node-1] [INFO] [1694517689.827904775] [scaled_joint_trajectory_controller]: Goal reached, success!
Sep 12 13:21:30  [ur_ros2_control_node-1] [INFO] [1694517690.078498089] [scaled_joint_trajectory_controller]: Received new action goal
Sep 12 13:21:30  [ur_ros2_control_node-1] [INFO] [1694517690.078525009] [scaled_joint_trajectory_controller]: Accepted new action goal
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079830850] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079843569] [tolerances]: Position Error: -1.376591, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079851533] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079856422] [tolerances]: Position Error: -3.068104, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079862062] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079866394] [tolerances]: Position Error: 2.691050, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079872186] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079876501] [tolerances]: Position Error: 0.216523, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079882046] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079886363] [tolerances]: Position Error: 2.558916, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079891795] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.079896119] [tolerances]: Position Error: -1.021753, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [WARN] [1694517690.079904305] [scaled_joint_trajectory_controller]: Aborted due to state tolerance violation
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082024796] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082036927] [tolerances]: Position Error: -1.376542, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082044798] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082049471] [tolerances]: Position Error: -3.068166, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082055378] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082059531] [tolerances]: Position Error: 2.691178, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.082065143] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.470052114] [tolerances]: Position Error: -0.569223, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.470057709] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.470061775] [tolerances]: Position Error: 0.228544, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471914121] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471926233] [tolerances]: Position Error: 0.297476, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471934133] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471938513] [tolerances]: Position Error: -0.667458, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471944444] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471948457] [tolerances]: Position Error: 0.762986, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471954221] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471958270] [tolerances]: Position Error: -0.412087, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471963702] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471985670] [tolerances]: Position Error: -0.568199, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471993699] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.471998195] [tolerances]: Position Error: 0.228196, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473942959] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473955017] [tolerances]: Position Error: 0.297177, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473962961] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473967194] [tolerances]: Position Error: -0.666427, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473973301] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473977343] [tolerances]: Position Error: 0.761690, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473983126] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473987215] [tolerances]: Position Error: -0.410350, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473992665] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.473996750] [tolerances]: Position Error: -0.566963, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.474002126] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.474006196] [tolerances]: Position Error: 0.227767, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475890952] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475903010] [tolerances]: Position Error: 0.296810, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475910881] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475915041] [tolerances]: Position Error: -0.665174, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475921132] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475925199] [tolerances]: Position Error: 0.760203, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475930979] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475935165] [tolerances]: Position Error: -0.408503, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475940801] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475945148] [tolerances]: Position Error: -0.565656, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475950617] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.475954688] [tolerances]: Position Error: 0.227153, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478053292] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478064858] [tolerances]: Position Error: 0.296360, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478072562] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478077198] [tolerances]: Position Error: -0.663898, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478083348] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478087609] [tolerances]: Position Error: 0.758444, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478093414] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478097472] [tolerances]: Position Error: -0.406415, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478102888] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478106997] [tolerances]: Position Error: -0.564262, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478112308] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.478116428] [tolerances]: Position Error: 0.226592, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479941096] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479952377] [tolerances]: Position Error: 0.295812, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479959667] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479964588] [tolerances]: Position Error: -0.662379, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479970440] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479974623] [tolerances]: Position Error: 0.756439, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479980490] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.479984663] [tolerances]: Position Error: -0.404391, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.480004967] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.480010836] [tolerances]: Position Error: -0.562843, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.480016861] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.480021407] [tolerances]: Position Error: 0.226027, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.481974296] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.481985726] [tolerances]: Position Error: 0.295222, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.481993216] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.481997834] [tolerances]: Position Error: -0.660668, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482003955] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482008175] [tolerances]: Position Error: 0.754250, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482013919] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482017980] [tolerances]: Position Error: -0.402048, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482023416] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482027563] [tolerances]: Position Error: -0.561459, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482032795] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.482036905] [tolerances]: Position Error: 0.225460, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483943994] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483955249] [tolerances]: Position Error: 0.294606, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483962649] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483966748] [tolerances]: Position Error: -0.658887, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483973323] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483977317] [tolerances]: Position Error: 0.751917, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483983052] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483987178] [tolerances]: Position Error: -0.399607, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483992886] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.483996853] [tolerances]: Position Error: -0.559984, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.484002297] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.484006230] [tolerances]: Position Error: 0.224704, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486014147] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486026965] [tolerances]: Position Error: 0.293825, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486035152] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486039059] [tolerances]: Position Error: -0.656974, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486045332] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486049425] [tolerances]: Position Error: 0.749406, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486054707] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486058832] [tolerances]: Position Error: -0.397084, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486064205] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486068291] [tolerances]: Position Error: -0.558572, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486073667] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.486077723] [tolerances]: Position Error: 0.223852, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.487960349] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.487971731] [tolerances]: Position Error: 0.293112, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.487979077] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.487983827] [tolerances]: Position Error: -0.654920, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.487989857] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.487994014] [tolerances]: Position Error: 0.746772, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.488018148] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.488023601] [tolerances]: Position Error: -0.394531, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.488030000] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.488034317] [tolerances]: Position Error: -0.557090, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.488040180] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.488044192] [tolerances]: Position Error: 0.223154, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.489966426] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.489978176] [tolerances]: Position Error: 0.292391, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.489985907] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.489990038] [tolerances]: Position Error: -0.652722, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.489996229] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490000422] [tolerances]: Position Error: 0.744054, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490006058] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490010180] [tolerances]: Position Error: -0.391610, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490015700] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490019740] [tolerances]: Position Error: -0.555535, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490025121] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.490029164] [tolerances]: Position Error: 0.222396, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.491960983] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.491972349] [tolerances]: Position Error: 0.291492, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.491979868] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.491984433] [tolerances]: Position Error: -0.650344, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.491990462] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.491994657] [tolerances]: Position Error: 0.741191, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.492000540] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.492004503] [tolerances]: Position Error: -0.388656, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.492009897] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.492013911] [tolerances]: Position Error: -0.553941, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.492019400] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.492023459] [tolerances]: Position Error: 0.221532, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.493971754] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.493983060] [tolerances]: Position Error: 0.290687, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.493990633] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.493994683] [tolerances]: Position Error: -0.647899, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494000861] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494004946] [tolerances]: Position Error: 0.738089, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494010564] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494014615] [tolerances]: Position Error: -0.385509, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494019988] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494024039] [tolerances]: Position Error: -0.552174, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494029404] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.494033448] [tolerances]: Position Error: 0.220722, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594155308] [URPositionHardwareInterface]: Unable to read from hardware...
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594214084] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594221503] [tolerances]: Position Error: 0.290632, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594229349] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594233718] [tolerances]: Position Error: -0.657372, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594265938] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594271534] [tolerances]: Position Error: 0.703701, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594276994] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594280921] [tolerances]: Position Error: -0.341642, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594286820] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594290854] [tolerances]: Position Error: -0.552165, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594296265] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.594300252] [tolerances]: Position Error: 0.220924, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694472576] [URPositionHardwareInterface]: Unable to read from hardware...
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694535869] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694543408] [tolerances]: Position Error: 0.290590, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694552535] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694556079] [tolerances]: Position Error: -0.664292, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694561565] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694565478] [tolerances]: Position Error: 0.671923, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694570432] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694574122] [tolerances]: Position Error: -0.302942, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694578767] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694582569] [tolerances]: Position Error: -0.552157, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694587429] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.694591374] [tolerances]: Position Error: 0.221108, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794752414] [URPositionHardwareInterface]: Unable to read from hardware...
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794811853] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794819742] [tolerances]: Position Error: 0.290553, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794828760] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794832846] [tolerances]: Position Error: -0.669839, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794837754] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794842130] [tolerances]: Position Error: 0.638816, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794847141] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794850834] [tolerances]: Position Error: -0.264292, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794855555] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794859165] [tolerances]: Position Error: -0.552149, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794864190] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794868747] [tolerances]: Position Error: 0.221296, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.794993505] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795001294] [tolerances]: Position Error: 0.290559, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795007870] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795012011] [tolerances]: Position Error: -0.668989, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795016946] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795020739] [tolerances]: Position Error: 0.644577, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795025718] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795029523] [tolerances]: Position Error: -0.270903, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795034354] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795038551] [tolerances]: Position Error: -0.552150, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795043788] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.795047510] [tolerances]: Position Error: 0.221264, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797819067] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797831748] [tolerances]: Position Error: 0.290558, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797839442] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797843652] [tolerances]: Position Error: -0.669147, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797850022] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797854025] [tolerances]: Position Error: 0.643533, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797859243] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797863600] [tolerances]: Position Error: -0.269701, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797868842] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797876574] [tolerances]: Position Error: -0.552150, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797882123] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.797886036] [tolerances]: Position Error: 0.221269, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799811757] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799823795] [tolerances]: Position Error: 0.290558, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799831796] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799836191] [tolerances]: Position Error: -0.669096, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799842433] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799846696] [tolerances]: Position Error: 0.643868, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799852499] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799856769] [tolerances]: Position Error: -0.270087, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799862254] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799866526] [tolerances]: Position Error: -0.552150, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799871920] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.799875971] [tolerances]: Position Error: 0.221268, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801853235] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801866216] [tolerances]: Position Error: 0.290558, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801873959] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801878192] [tolerances]: Position Error: -0.669099, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801884396] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801888439] [tolerances]: Position Error: 0.643850, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801893595] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801897972] [tolerances]: Position Error: -0.270065, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801903316] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801907374] [tolerances]: Position Error: -0.552150, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801912574] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.801916585] [tolerances]: Position Error: 0.221268, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [INFO] [1694517690.802097511] [UR_Client_Library:]: Connection to reverse interface dropped.
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803894951] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803906300] [tolerances]: Position Error: 0.290558, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803914064] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803918382] [tolerances]: Position Error: -0.669098, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803924594] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803928714] [tolerances]: Position Error: 0.643854, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803934554] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803938771] [tolerances]: Position Error: -0.270071, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803944621] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803948859] [tolerances]: Position Error: -0.552150, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803975183] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.803981340] [tolerances]: Position Error: 0.221268, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [INFO] [1694517690.804131093] [io_and_status_controller]: Configure UR gpio controller with tf_prefix:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805794439] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805806854] [tolerances]: Position Error: 0.290558, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805814448] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805818513] [tolerances]: Position Error: -0.669092, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805824779] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805828827] [tolerances]: Position Error: 0.643898, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805834427] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805838407] [tolerances]: Position Error: -0.270121, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805843697] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805847569] [tolerances]: Position Error: -0.552150, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805852823] [tolerances]: Path state tolerances failed:
Sep 12 13:21:30  [ur_ros2_control_node-1] [ERROR] [1694517690.805856786] [tolerances]: Position Error: 0.221267, Position Tolerance: 0.200000
Sep 12 13:21:30  [ur_ros2_control_node-1] [INFO] [1694517690.805965141] [io_and_status_controller]: Configure UR gpio controller with tf_prefix:
Sep 12 13:21:35  [ur_ros2_control_node-1] [INFO] [1694517695.439057342] [UR_Client_Library:]: Robot requested program
Sep 12 13:21:35  [ur_ros2_control_node-1] [INFO] [1694517695.439086989] [UR_Client_Library:]: Sent program to robot

Accept Public visibility

• I agree to make this context public

[fmauch]

Hi there, thanks for reporting this. The output regarding failed tolerances indicates that there is velocity scaling active. Could you provide a complete sample trajectory with which this is reproducible?

Since you said, you confirmed this is working with the simulator: this does not happen when using ursim?

[urmahp]

Hi, thanks for reporting this. For know I haven't been able to reproduce this issue with a ursim. As @fmauch suggested is it possible that you could provide a complete sample trajectory where this issue occurs? Or even better can you provide the code you use to generate and publish the trajectory?

[iydv]

this does not happen when using ursim?

I have tested with ursim today extensively and the problem did not happen. I even simulated various network failures via tc-netem tool and everything behaved as expected (robot stopped when the connection to the interface was lost)
But the behavior of the simulator is ideal. With real hardware the behavior was different.

Here is an example of short trajectory cycle we are sending to the robot.
The problem occurred when we sent MOVE_LINEAR trajectory.

Unfortunately, I cannot provide an example which can be reproduced on ursim, since the issue happens only on real robot hardware.

[test_node-1] [INFO] [1694786693.356491537] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: 0.043128, -1.28824, 1.37179, -1.82208, -1.63632, -2.13071, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 4
[test_node-1] joint positions: -0.195016, -1.70093, 0.902027, -0.944217, -1.52982, -0.195171, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786697.370322140] [test_node]: Trajectory was executed in 4.00366 seconds.

[test_node-1] [INFO] [1694786699.063698768] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: -0.195016, -1.70094, 0.902027, -0.944217, -1.52982, -0.195171, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 8
[test_node-1] joint positions: 0.30493, -0.982258, 0.955637, -1.48215, -1.72737, 0.204445, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786707.180767196] [test_node]: Trajectory was executed in 8.10695 seconds.

[test_node-1] [INFO] [1694786707.273103585] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: 0.30493, -0.982258, 0.955637, -1.48215, -1.72737, 0.204445, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 4
[test_node-1] joint positions: 0.298479, -0.988688, 1.04161, -1.56271, -1.72777, 0.197927, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786711.286847325] [test_node]: Trajectory was executed in 4.00365 seconds.

[test_node-1] [INFO] [1694786711.576626438] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: 0.298479, -0.988688, 1.04161, -1.56271, -1.72777, 0.197927, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 2
[test_node-1] joint positions: 0.30493, -0.982258, 0.955637, -1.48215, -1.72737, 0.204445, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786713.688662269] [test_node]: Trajectory was executed in 2.10191 seconds.

[test_node-1] [INFO] [1694786713.778641016] [test_node]: received 'MOVE_LINEAR' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: 0.30493, -0.982258, 0.955637, -1.48215, -1.72737, 0.204445, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 0.257005
[test_node-1] joint positions: 0.304943, -0.978438, 0.891114, -1.42144, -1.72737, 0.204458, 
[test_node-1] joint velocities: -5.57244e-05, 0.0461017, -0.498813, 0.452714, 1.13123e-05, -5.70261e-05, 
[test_node-1] Trajectory point N 2
[test_node-1] time from start: 0.382078
[test_node-1] joint positions: 0.30493, -0.969934, 0.818666, -1.3575, -1.72737, 0.204445, 
[test_node-1] joint velocities: -6.99323e-05, 0.0885967, -0.621143, 0.53255, 1.41965e-05, -7.15658e-05, 
[test_node-1] Trajectory point N 3
[test_node-1] time from start: 0.514344
[test_node-1] joint positions: 0.30493, -0.955802, 0.736308, -1.28927, -1.72737, 0.204445, 
[test_node-1] joint velocities: 2.02079e-05, 0.122921, -0.622149, 0.499228, -4.10228e-06, 2.06799e-05, 
[test_node-1] Trajectory point N 4
[test_node-1] time from start: 0.668323
[test_node-1] joint positions: 0.30493, -0.934311, 0.640431, -1.21489, -1.72737, 0.204445, 
[test_node-1] joint velocities: -6.05417e-06, 0.156698, -0.626669, 0.469971, 1.22901e-06, -6.19559e-06, 
[test_node-1] Trajectory point N 5
[test_node-1] time from start: 0.85796
[test_node-1] joint positions: 0.30493, -0.902023, 0.523852, -1.1306, -1.72737, 0.204445, 
[test_node-1] joint velocities: 2.13286e-06, 0.175704, -0.581675, 0.405972, -4.32984e-07, 2.18268e-06, 
[test_node-1] Trajectory point N 6
[test_node-1] time from start: 1.3103
[test_node-1] joint positions: 0.30493, -0.850002, 0.367943, -1.02671, -1.72737, 0.204445, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786715.240074438] [test_node]: Trajectory was executed in 1.40097 seconds.

[test_node-1] [INFO] [1694786715.279868288] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: 0.30493, -0.850006, 0.367955, -1.02671, -1.72737, 0.204445, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 4
[test_node-1] joint positions: -1.73736, -2.26675, 1.99658, -1.296, -1.57509, -2.32499, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786719.393518545] [test_node]: Trajectory was executed in 4.10352 seconds.

[test_node-1] [INFO] [1694786719.483745222] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: -1.73736, -2.26675, 1.99658, -1.296, -1.57509, -2.32499, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 3
[test_node-1] joint positions: -1.66795, -1.87629, 1.8111, -1.5007, -1.57476, -2.25559, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786722.496195495] [test_node]: Trajectory was executed in 3.00232 seconds.

[test_node-1] [INFO] [1694786722.586470823] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: -1.66795, -1.87629, 1.8111, -1.5007, -1.57476, -2.25559, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 4
[test_node-1] joint positions: -1.73736, -2.26675, 1.99658, -1.296, -1.57509, -2.32499, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786726.700448778] [test_node]: Trajectory was executed in 4.10383 seconds.

[test_node-1] [INFO] [1694786726.790036854] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: -1.73736, -2.26675, 1.99658, -1.296, -1.57509, -2.32499, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 6
[test_node-1] joint positions: 1.16829, -1.86254, 1.60861, -1.31265, -1.57557, -0.440643, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786732.905519651] [test_node]: Trajectory was executed in 6.10539 seconds.

[test_node-1] [INFO] [1694786733.195629718] [test_node]: received 'MOVE' request
[test_node-1] Trajectory point N 0
[test_node-1] time from start: 0
[test_node-1] joint positions: 1.16829, -1.86254, 1.60861, -1.31265, -1.57557, -0.440643, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] Trajectory point N 1
[test_node-1] time from start: 4
[test_node-1] joint positions: -0.195016, -1.70093, 0.902023, -0.944215, -1.52982, -0.19517, 
[test_node-1] joint velocities: 0, 0, 0, 0, 0, 0, 
[test_node-1] [INFO] [1694786737.309067384] [test_node]: Trajectory was executed in 4.10333 seconds.

[fmauch]

I'll try to put together a script testing this on one of our robots.

Here is an example of short trajectory cycle we are sending to the robot.
The problem occurred when we sent MOVE_LINEAR trajectory.

What's the difference between your MOVE and MOVE_LINEAR? Are both simply goals sent to the scaled JTC?

[iydv]

What's the difference between your MOVE and MOVE_LINEAR? Are both simply goals sent to the scaled JTC?

Yes, there is no difference. Both are joint trajectories sent to the JTC. The naming is only for internal usage

[urmahp]

I have investigated the issue further and managed to reproduce an error similar to what you are experiencing. Is the first point in each trajectory defined with a time of zero? If yes then this could be causing your issue.

If the first point is defined with a time of zero and differs from the current robot position, it would result in the robot moving very quickly towards the target. This is because the commands sent to the robot won't be achievable within a single control cycle, which is expected. I can see that the first point in each trajectory is equal to the last point in the previous trajectory, ensuring that the robot should always start in the correct position. This could explain why you are only experiencing it intermittently, as the robot might have been moved to a different position than expected before you started the driver, or for some reason the target from the previous trajectory hasn't been reached.

This however does not answer why the robot is moving towards joint positions [0,0,0,0,0,0]. Where do you see that the robot is moving towards joint position [0,0,0,0,0,0]? I haven't been able to find it in the logs you have provided.

When you have a chance to test it again you can either try to remove the first point with time zero or set the time to be larger than zero to give the robot time to reach the point, to see if that fixes the issue.

When the current robot position deviates from the path and prints the error you see in the log the robot should definitely stop moving, which is currently not the case, so I will have a look at implementing a fix for that.

[fmauch]

@urmahp note that the upstream JTC recently got updates regarding trajectory_start time. They are also planning on integrating the scaling functionality upstream soon.

[iydv]

Is the first point in each trajectory defined with a time of zero?

Yes, The first point in each trajectory is current joint states, from which robot should start the motion. And time from the start is also zero according to this comment in joint trajectory controller. Unfortunately, I do not know if scaled joint trajectory controller behaves same way. Is there any example trajectory for scaled JTC ?

Where do you see that the robot is moving towards joint position [0,0,0,0,0,0]? I haven't been able to find it in the logs you have provided.

We saw it visually, when the real robot moved, this is not indicated in the driver logs and I cannot repeat the experiment since I don't have access to the robot anymore. We will get another UR10 in October and I will check if the issue still persists.

[urrsk]

@iydv we have not succeed to reproduce and understand where the [0,0,0,0,0,0] are coming from as @fmauch and @urmahp mentioned.
But we have identified that scale JTC does not cancel the motion when the tolerance is too high, which we will fix asap.
In addition, we are also planning to put some checks in the Client library to avid this in the future.

[fmauch]

I've created a possible fix in #810. This is targeting Rolling, but I'll prepare something similar for Humble, as well.

Edit: Humble PR: #811

[fmauch]

I'll go ahead and close this, since I think this should be fixed by the changes in #810 / #811. @iydv feel free to reopen if you still experience the problem, though.