Skip to content

[BugFix] Fix shared mem updater with many policies#3442

Merged
vmoens merged 2 commits intomainfrom
fix-collector-factory
Feb 4, 2026
Merged

[BugFix] Fix shared mem updater with many policies#3442
vmoens merged 2 commits intomainfrom
fix-collector-factory

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Feb 4, 2026

No description provided.

@pytorch-bot
Copy link

pytorch-bot bot commented Feb 4, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3442

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 3 Pending, 1 Unrelated Failure

As of commit 8a06ff9 with merge base 7f24887 (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens

This comment was marked as resolved.

Sign in to view

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 173. Improved: $\large\color{#35bf28}18$. Worsened: $\large\color{#d91a1a}17$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 79.2957μs 77.6580μs 12.8770 KOps/s 12.7780 KOps/s $\color{#35bf28}+0.77\%$
test_tensor_to_bytestream_speed[torch.save] 0.1420ms 0.1412ms 7.0809 KOps/s 7.2410 KOps/s $\color{#d91a1a}-2.21\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1085s 0.1081s 9.2535 Ops/s 9.0902 Ops/s $\color{#35bf28}+1.80\%$
test_tensor_to_bytestream_speed[numpy] 2.7654μs 2.7545μs 363.0452 KOps/s 370.4243 KOps/s $\color{#d91a1a}-1.99\%$
test_tensor_to_bytestream_speed[safetensors] 38.4305μs 38.3045μs 26.1066 KOps/s 26.9073 KOps/s $\color{#d91a1a}-2.98\%$
test_simple 0.5426s 0.5384s 1.8574 Ops/s 1.7663 Ops/s $\textbf{\color{#35bf28}+5.16\%}$
test_transformed 1.1167s 1.1122s 0.8991 Ops/s 0.8770 Ops/s $\color{#35bf28}+2.52\%$
test_serial 1.6656s 1.6492s 0.6064 Ops/s 0.6006 Ops/s $\color{#35bf28}+0.96\%$
test_parallel 1.1879s 1.1142s 0.8975 Ops/s 0.8667 Ops/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[True-True-True-True-True] 0.1540ms 42.7063μs 23.4157 KOps/s 21.7411 KOps/s $\textbf{\color{#35bf28}+7.70\%}$
test_step_mdp_speed[True-True-True-True-False] 51.1910μs 24.1431μs 41.4197 KOps/s 40.9618 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-True-True-False-True] 54.3510μs 23.8234μs 41.9756 KOps/s 41.4128 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-True-True-False-False] 39.2110μs 13.0565μs 76.5904 KOps/s 74.1425 KOps/s $\color{#35bf28}+3.30\%$
test_step_mdp_speed[True-True-False-True-True] 79.5810μs 45.6002μs 21.9297 KOps/s 21.4418 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-False-True-False] 68.1810μs 26.3847μs 37.9008 KOps/s 36.8198 KOps/s $\color{#35bf28}+2.94\%$
test_step_mdp_speed[True-True-False-False-True] 54.3210μs 26.7255μs 37.4175 KOps/s 36.7913 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-True-False-False-False] 41.8210μs 15.8229μs 63.1996 KOps/s 60.7989 KOps/s $\color{#35bf28}+3.95\%$
test_step_mdp_speed[True-False-True-True-True] 94.1720μs 48.6144μs 20.5700 KOps/s 20.4488 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-True-True-False] 60.6910μs 28.9503μs 34.5419 KOps/s 33.2991 KOps/s $\color{#35bf28}+3.73\%$
test_step_mdp_speed[True-False-True-False-True] 57.8810μs 26.2456μs 38.1016 KOps/s 37.1292 KOps/s $\color{#35bf28}+2.62\%$
test_step_mdp_speed[True-False-True-False-False] 52.0220μs 15.7254μs 63.5915 KOps/s 61.3700 KOps/s $\color{#35bf28}+3.62\%$
test_step_mdp_speed[True-False-False-True-True] 86.6020μs 50.2284μs 19.9090 KOps/s 19.2373 KOps/s $\color{#35bf28}+3.49\%$
test_step_mdp_speed[True-False-False-True-False] 63.3020μs 30.9647μs 32.2949 KOps/s 30.6574 KOps/s $\textbf{\color{#35bf28}+5.34\%}$
test_step_mdp_speed[True-False-False-False-True] 66.4020μs 28.7645μs 34.7650 KOps/s 33.8367 KOps/s $\color{#35bf28}+2.74\%$
test_step_mdp_speed[True-False-False-False-False] 72.5710μs 17.7847μs 56.2280 KOps/s 53.7731 KOps/s $\color{#35bf28}+4.57\%$
test_step_mdp_speed[False-True-True-True-True] 86.5220μs 46.9284μs 21.3090 KOps/s 20.2035 KOps/s $\textbf{\color{#35bf28}+5.47\%}$
test_step_mdp_speed[False-True-True-True-False] 59.1310μs 28.6023μs 34.9623 KOps/s 33.2290 KOps/s $\textbf{\color{#35bf28}+5.22\%}$
test_step_mdp_speed[False-True-True-False-True] 2.3647ms 30.2633μs 33.0433 KOps/s 32.0006 KOps/s $\color{#35bf28}+3.26\%$
test_step_mdp_speed[False-True-True-False-False] 43.5700μs 17.2815μs 57.8654 KOps/s 54.9523 KOps/s $\textbf{\color{#35bf28}+5.30\%}$
test_step_mdp_speed[False-True-False-True-True] 96.5520μs 49.4692μs 20.2146 KOps/s 19.4480 KOps/s $\color{#35bf28}+3.94\%$
test_step_mdp_speed[False-True-False-True-False] 58.2710μs 31.7527μs 31.4934 KOps/s 30.8177 KOps/s $\color{#35bf28}+2.19\%$
test_step_mdp_speed[False-True-False-False-True] 69.2310μs 31.7593μs 31.4868 KOps/s 30.3914 KOps/s $\color{#35bf28}+3.60\%$
test_step_mdp_speed[False-True-False-False-False] 49.7710μs 20.1179μs 49.7070 KOps/s 48.6834 KOps/s $\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-False-True-True-True] 92.4010μs 52.2154μs 19.1514 KOps/s 18.2870 KOps/s $\color{#35bf28}+4.73\%$
test_step_mdp_speed[False-False-True-True-False] 87.0820μs 34.4102μs 29.0612 KOps/s 28.3448 KOps/s $\color{#35bf28}+2.53\%$
test_step_mdp_speed[False-False-True-False-True] 64.8610μs 32.2331μs 31.0240 KOps/s 30.6340 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-False-True-False-False] 45.4210μs 19.7960μs 50.5153 KOps/s 48.7064 KOps/s $\color{#35bf28}+3.71\%$
test_step_mdp_speed[False-False-False-True-True] 0.1069ms 54.6088μs 18.3121 KOps/s 17.6944 KOps/s $\color{#35bf28}+3.49\%$
test_step_mdp_speed[False-False-False-True-False] 68.0810μs 36.3245μs 27.5296 KOps/s 26.3606 KOps/s $\color{#35bf28}+4.43\%$
test_step_mdp_speed[False-False-False-False-True] 74.9020μs 34.5951μs 28.9059 KOps/s 28.0587 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[False-False-False-False-False] 49.2110μs 22.2862μs 44.8708 KOps/s 43.7840 KOps/s $\color{#35bf28}+2.48\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8470s 0.7445s 1.3431 Ops/s 1.3296 Ops/s $\color{#35bf28}+1.01\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7112s 0.6205s 1.6117 Ops/s 1.6067 Ops/s $\color{#35bf28}+0.31\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7099s 1.6297s 0.6136 Ops/s 0.6131 Ops/s $\color{#35bf28}+0.08\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5086s 1.4196s 0.7044 Ops/s 0.7013 Ops/s $\color{#35bf28}+0.45\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9611s 1.8767s 0.5329 Ops/s 0.5319 Ops/s $\color{#35bf28}+0.19\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7569s 1.6701s 0.5988 Ops/s 0.5994 Ops/s $\color{#d91a1a}-0.10\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6945s 4.5622s 0.2192 Ops/s 0.2172 Ops/s $\color{#35bf28}+0.92\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.4646s 4.3794s 0.2283 Ops/s 0.2271 Ops/s $\color{#35bf28}+0.55\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 1.9653s 1.8964s 0.5273 Ops/s 0.5280 Ops/s $\color{#d91a1a}-0.13\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7019s 1.6192s 0.6176 Ops/s 0.6117 Ops/s $\color{#35bf28}+0.97\%$
test_values[generalized_advantage_estimate-True-True] 11.0785ms 10.9210ms 91.5669 Ops/s 91.9822 Ops/s $\color{#d91a1a}-0.45\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.5596ms 17.6475ms 56.6653 Ops/s 90.7065 Ops/s $\textbf{\color{#d91a1a}-37.53\%}$
test_values[td0_return_estimate-False-False] 0.2243ms 0.1294ms 7.7252 KOps/s 7.6083 KOps/s $\color{#35bf28}+1.54\%$
test_values[td1_return_estimate-False-False] 30.0454ms 29.4435ms 33.9634 Ops/s 33.6013 Ops/s $\color{#35bf28}+1.08\%$
test_values[vec_td1_return_estimate-False-False] 17.8884ms 17.6523ms 56.6497 Ops/s 89.3854 Ops/s $\textbf{\color{#d91a1a}-36.62\%}$
test_values[td_lambda_return_estimate-True-False] 44.6929ms 43.5596ms 22.9570 Ops/s 22.6531 Ops/s $\color{#35bf28}+1.34\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.9332ms 17.7690ms 56.2778 Ops/s 90.7632 Ops/s $\textbf{\color{#d91a1a}-37.99\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.7964ms 9.6099ms 104.0598 Ops/s 103.2535 Ops/s $\color{#35bf28}+0.78\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7305ms 1.4676ms 681.4068 Ops/s 653.9363 Ops/s $\color{#35bf28}+4.20\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5810ms 0.4374ms 2.2862 KOps/s 2.3271 KOps/s $\color{#d91a1a}-1.76\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.4064ms 34.9316ms 28.6274 Ops/s 37.2548 Ops/s $\textbf{\color{#d91a1a}-23.16\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8960ms 1.7271ms 578.9957 Ops/s 580.0903 Ops/s $\color{#d91a1a}-0.19\%$
test_dqn_speed[False-None] 1.7273ms 1.4033ms 712.6191 Ops/s 721.4322 Ops/s $\color{#d91a1a}-1.22\%$
test_dqn_speed[False-backward] 2.0770ms 1.9464ms 513.7784 Ops/s 529.6461 Ops/s $\color{#d91a1a}-3.00\%$
test_dqn_speed[True-None] 0.9608ms 0.5480ms 1.8249 KOps/s 1.7842 KOps/s $\color{#35bf28}+2.28\%$
test_dqn_speed[True-backward] 1.0502ms 1.0031ms 996.8970 Ops/s 902.5499 Ops/s $\textbf{\color{#35bf28}+10.45\%}$
test_dqn_speed[reduce-overhead-None] 0.9444ms 0.5390ms 1.8552 KOps/s 1.8546 KOps/s $\color{#35bf28}+0.04\%$
test_ddpg_speed[False-None] 3.2088ms 2.8055ms 356.4474 Ops/s 352.7679 Ops/s $\color{#35bf28}+1.04\%$
test_ddpg_speed[False-backward] 4.2323ms 4.0355ms 247.7996 Ops/s 247.4999 Ops/s $\color{#35bf28}+0.12\%$
test_ddpg_speed[True-None] 1.8086ms 1.3985ms 715.0533 Ops/s 688.4946 Ops/s $\color{#35bf28}+3.86\%$
test_ddpg_speed[True-backward] 2.4781ms 2.3818ms 419.8534 Ops/s 388.8162 Ops/s $\textbf{\color{#35bf28}+7.98\%}$
test_ddpg_speed[reduce-overhead-None] 1.7691ms 1.3879ms 720.5025 Ops/s 703.2405 Ops/s $\color{#35bf28}+2.45\%$
test_sac_speed[False-None] 8.5626ms 8.0123ms 124.8089 Ops/s 125.1362 Ops/s $\color{#d91a1a}-0.26\%$
test_sac_speed[False-backward] 11.7042ms 11.2371ms 88.9909 Ops/s 88.7208 Ops/s $\color{#35bf28}+0.30\%$
test_sac_speed[True-None] 2.5724ms 2.1346ms 468.4736 Ops/s 471.3548 Ops/s $\color{#d91a1a}-0.61\%$
test_sac_speed[True-backward] 4.1236ms 3.9991ms 250.0587 Ops/s 248.0883 Ops/s $\color{#35bf28}+0.79\%$
test_sac_speed[reduce-overhead-None] 2.5356ms 2.1030ms 475.5153 Ops/s 460.8615 Ops/s $\color{#35bf28}+3.18\%$
test_redq_speed[False-None] 15.7715ms 11.0942ms 90.1376 Ops/s 97.0584 Ops/s $\textbf{\color{#d91a1a}-7.13\%}$
test_redq_speed[False-backward] 18.3485ms 17.6569ms 56.6351 Ops/s 57.8880 Ops/s $\color{#d91a1a}-2.16\%$
test_redq_speed[True-None] 4.7923ms 4.3755ms 228.5461 Ops/s 229.8710 Ops/s $\color{#d91a1a}-0.58\%$
test_redq_speed[True-backward] 9.8109ms 9.5206ms 105.0351 Ops/s 95.3856 Ops/s $\textbf{\color{#35bf28}+10.12\%}$
test_redq_speed[reduce-overhead-None] 4.7719ms 4.3202ms 231.4718 Ops/s 228.4134 Ops/s $\color{#35bf28}+1.34\%$
test_redq_deprec_speed[False-None] 11.2736ms 10.8998ms 91.7446 Ops/s 92.6735 Ops/s $\color{#d91a1a}-1.00\%$
test_redq_deprec_speed[False-backward] 16.3782ms 15.8024ms 63.2817 Ops/s 64.5900 Ops/s $\color{#d91a1a}-2.03\%$
test_redq_deprec_speed[True-None] 4.0188ms 3.6206ms 276.2005 Ops/s 281.0455 Ops/s $\color{#d91a1a}-1.72\%$
test_redq_deprec_speed[True-backward] 7.8750ms 7.5290ms 132.8192 Ops/s 135.7598 Ops/s $\color{#d91a1a}-2.17\%$
test_redq_deprec_speed[reduce-overhead-None] 3.9748ms 3.5572ms 281.1203 Ops/s 258.2053 Ops/s $\textbf{\color{#35bf28}+8.87\%}$
test_td3_speed[False-None] 8.1433ms 7.9868ms 125.2067 Ops/s 124.8148 Ops/s $\color{#35bf28}+0.31\%$
test_td3_speed[False-backward] 11.4201ms 10.9117ms 91.6444 Ops/s 91.3034 Ops/s $\color{#35bf28}+0.37\%$
test_td3_speed[True-None] 1.8753ms 1.8254ms 547.8106 Ops/s 537.1469 Ops/s $\color{#35bf28}+1.99\%$
test_td3_speed[True-backward] 3.8050ms 3.6321ms 275.3259 Ops/s 272.8825 Ops/s $\color{#35bf28}+0.90\%$
test_td3_speed[reduce-overhead-None] 1.8385ms 1.7981ms 556.1510 Ops/s 550.3286 Ops/s $\color{#35bf28}+1.06\%$
test_cql_speed[False-None] 29.6901ms 26.9240ms 37.1416 Ops/s 39.1991 Ops/s $\textbf{\color{#d91a1a}-5.25\%}$
test_cql_speed[False-backward] 37.6928ms 35.0905ms 28.4978 Ops/s 29.0003 Ops/s $\color{#d91a1a}-1.73\%$
test_cql_speed[True-None] 13.1779ms 12.3785ms 80.7853 Ops/s 81.8799 Ops/s $\color{#d91a1a}-1.34\%$
test_cql_speed[True-backward] 18.7898ms 18.1544ms 55.0830 Ops/s 55.7077 Ops/s $\color{#d91a1a}-1.12\%$
test_cql_speed[reduce-overhead-None] 13.1562ms 12.3378ms 81.0520 Ops/s 81.3082 Ops/s $\color{#d91a1a}-0.32\%$
test_a2c_speed[False-None] 7.6193ms 5.5576ms 179.9341 Ops/s 183.5830 Ops/s $\color{#d91a1a}-1.99\%$
test_a2c_speed[False-backward] 12.1203ms 11.8217ms 84.5904 Ops/s 85.0883 Ops/s $\color{#d91a1a}-0.59\%$
test_a2c_speed[True-None] 3.9596ms 3.7069ms 269.7650 Ops/s 260.7590 Ops/s $\color{#35bf28}+3.45\%$
test_a2c_speed[True-backward] 8.7691ms 8.5378ms 117.1267 Ops/s 115.3329 Ops/s $\color{#35bf28}+1.56\%$
test_a2c_speed[reduce-overhead-None] 4.3145ms 3.7021ms 270.1191 Ops/s 271.4634 Ops/s $\color{#d91a1a}-0.50\%$
test_ppo_speed[False-None] 6.3217ms 5.8813ms 170.0291 Ops/s 168.4246 Ops/s $\color{#35bf28}+0.95\%$
test_ppo_speed[False-backward] 12.6817ms 12.3024ms 81.2851 Ops/s 80.8786 Ops/s $\color{#35bf28}+0.50\%$
test_ppo_speed[True-None] 3.9591ms 3.5853ms 278.9179 Ops/s 264.8768 Ops/s $\textbf{\color{#35bf28}+5.30\%}$
test_ppo_speed[True-backward] 8.5831ms 8.3851ms 119.2585 Ops/s 112.2820 Ops/s $\textbf{\color{#35bf28}+6.21\%}$
test_ppo_speed[reduce-overhead-None] 3.7026ms 3.5495ms 281.7281 Ops/s 271.3421 Ops/s $\color{#35bf28}+3.83\%$
test_reinforce_speed[False-None] 4.8507ms 4.4816ms 223.1342 Ops/s 214.9610 Ops/s $\color{#35bf28}+3.80\%$
test_reinforce_speed[False-backward] 7.4888ms 7.2579ms 137.7816 Ops/s 133.6924 Ops/s $\color{#35bf28}+3.06\%$
test_reinforce_speed[True-None] 3.2623ms 2.8809ms 347.1154 Ops/s 347.9132 Ops/s $\color{#d91a1a}-0.23\%$
test_reinforce_speed[True-backward] 7.8330ms 7.6044ms 131.5029 Ops/s 125.3685 Ops/s $\color{#35bf28}+4.89\%$
test_reinforce_speed[reduce-overhead-None] 3.2208ms 2.8492ms 350.9730 Ops/s 350.4692 Ops/s $\color{#35bf28}+0.14\%$
test_iql_speed[False-None] 26.3618ms 20.7702ms 48.1459 Ops/s 49.9882 Ops/s $\color{#d91a1a}-3.69\%$
test_iql_speed[False-backward] 35.5735ms 30.3319ms 32.9686 Ops/s 33.4071 Ops/s $\color{#d91a1a}-1.31\%$
test_iql_speed[True-None] 8.7297ms 8.4243ms 118.7045 Ops/s 117.6871 Ops/s $\color{#35bf28}+0.86\%$
test_iql_speed[True-backward] 16.7351ms 16.5059ms 60.5843 Ops/s 60.9270 Ops/s $\color{#d91a1a}-0.56\%$
test_iql_speed[reduce-overhead-None] 8.7153ms 8.4511ms 118.3281 Ops/s 118.1278 Ops/s $\color{#35bf28}+0.17\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1004ms 5.8973ms 169.5694 Ops/s 169.2523 Ops/s $\color{#35bf28}+0.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.0426ms 0.3640ms 2.7474 KOps/s 3.2161 KOps/s $\textbf{\color{#d91a1a}-14.57\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5918ms 0.3410ms 2.9322 KOps/s 3.0421 KOps/s $\color{#d91a1a}-3.61\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8483ms 5.6197ms 177.9439 Ops/s 175.7958 Ops/s $\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0842ms 0.3314ms 3.0180 KOps/s 3.2282 KOps/s $\textbf{\color{#d91a1a}-6.51\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5313ms 0.3157ms 3.1678 KOps/s 3.2102 KOps/s $\color{#d91a1a}-1.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5308ms 1.3523ms 739.4925 Ops/s 746.0154 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5131ms 1.2748ms 784.4474 Ops/s 792.0098 Ops/s $\color{#d91a1a}-0.95\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.6749ms 5.8894ms 169.7970 Ops/s 171.9919 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0160ms 0.4708ms 2.1240 KOps/s 2.1016 KOps/s $\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7696ms 0.4800ms 2.0833 KOps/s 2.2857 KOps/s $\textbf{\color{#d91a1a}-8.86\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.6519ms 5.5488ms 180.2200 Ops/s 176.1266 Ops/s $\color{#35bf28}+2.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4874ms 0.3178ms 3.1465 KOps/s 3.5988 KOps/s $\textbf{\color{#d91a1a}-12.57\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4646ms 0.2590ms 3.8607 KOps/s 3.3462 KOps/s $\textbf{\color{#35bf28}+15.37\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.7684ms 5.5066ms 181.5997 Ops/s 177.1269 Ops/s $\color{#35bf28}+2.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8645s 1.8474ms 541.2899 Ops/s 2.8009 KOps/s $\textbf{\color{#d91a1a}-80.67\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5278ms 0.3554ms 2.8136 KOps/s 3.0247 KOps/s $\textbf{\color{#d91a1a}-6.98\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9587ms 5.8081ms 172.1743 Ops/s 172.4824 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0995ms 0.4309ms 2.3206 KOps/s 2.0994 KOps/s $\textbf{\color{#35bf28}+10.53\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6088ms 0.4090ms 2.4452 KOps/s 2.1516 KOps/s $\textbf{\color{#35bf28}+13.64\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.3842ms 4.9434ms 202.2894 Ops/s 57.0496 Ops/s $\textbf{\color{#35bf28}+254.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.4316ms 1.8830ms 531.0571 Ops/s 531.1691 Ops/s $\color{#d91a1a}-0.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.4967ms 0.8951ms 1.1172 KOps/s 787.2381 Ops/s $\textbf{\color{#35bf28}+41.91\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5442s 15.7739ms 63.3959 Ops/s 199.7202 Ops/s $\textbf{\color{#d91a1a}-68.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 11.6795ms 2.0227ms 494.3908 Ops/s 537.1603 Ops/s $\textbf{\color{#d91a1a}-7.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.2855ms 1.2023ms 831.7711 Ops/s 1.1762 KOps/s $\textbf{\color{#d91a1a}-29.28\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.6523ms 5.1714ms 193.3696 Ops/s 192.7075 Ops/s $\color{#35bf28}+0.34\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 13.5927ms 2.0514ms 487.4641 Ops/s 496.1584 Ops/s $\color{#d91a1a}-1.75\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.4379ms 1.0059ms 994.1517 Ops/s 945.4600 Ops/s $\textbf{\color{#35bf28}+5.15\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 38.2511ms 35.9006ms 27.8547 Ops/s 27.7552 Ops/s $\color{#35bf28}+0.36\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.5912ms 17.9963ms 55.5670 Ops/s 56.0312 Ops/s $\color{#d91a1a}-0.83\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 40.6214ms 37.2397ms 26.8531 Ops/s 26.8611 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.4480ms 18.4283ms 54.2644 Ops/s 54.8444 Ops/s $\color{#d91a1a}-1.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.7890ms 38.9516ms 25.6729 Ops/s 25.7454 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.4741ms 19.8435ms 50.3942 Ops/s 50.9109 Ops/s $\color{#d91a1a}-1.01\%$
test_storage_write_lazystack[50-img_shape0-small] 0.8502ms 0.2218ms 4.5086 KOps/s 4.6238 KOps/s $\color{#d91a1a}-2.49\%$
test_storage_write_lazystack[100-img_shape1-atari] 1.6616ms 1.3577ms 736.5665 Ops/s 731.1171 Ops/s $\color{#35bf28}+0.75\%$
test_storage_write_lazystack[100-img_shape2-large_img] 2.6930ms 2.3014ms 434.5180 Ops/s 433.4639 Ops/s $\color{#35bf28}+0.24\%$
test_storage_write_lazystack[200-img_shape3-large_batch] 3.0117ms 2.8514ms 350.7083 Ops/s 348.0964 Ops/s $\color{#35bf28}+0.75\%$
test_storage_write_contiguous[50-img_shape0-small] 0.2212ms 0.1345ms 7.4334 KOps/s 7.6018 KOps/s $\color{#d91a1a}-2.22\%$
test_storage_write_contiguous[100-img_shape1-atari] 0.3629ms 0.2081ms 4.8050 KOps/s 5.3287 KOps/s $\textbf{\color{#d91a1a}-9.83\%}$
test_storage_write_contiguous[100-img_shape2-large_img] 2.0821ms 1.7879ms 559.3207 Ops/s 575.0498 Ops/s $\color{#d91a1a}-2.74\%$
test_storage_write_contiguous[200-img_shape3-large_batch] 1.4480ms 1.3023ms 767.9000 Ops/s 793.1576 Ops/s $\color{#d91a1a}-3.18\%$
test_collector_stack_then_write[50-img_shape0-small] 1.2411ms 1.0836ms 922.8097 Ops/s 912.3243 Ops/s $\color{#35bf28}+1.15\%$
test_collector_stack_then_write[100-img_shape1-atari] 3.8839ms 3.4810ms 287.2777 Ops/s 276.7175 Ops/s $\color{#35bf28}+3.82\%$
test_collector_stack_then_write[100-img_shape2-large_img] 11.0061ms 5.5725ms 179.4535 Ops/s 176.1384 Ops/s $\color{#35bf28}+1.88\%$
test_collector_stack_then_write[200-img_shape3-large_batch] 7.1861ms 6.8202ms 146.6231 Ops/s 146.5651 Ops/s $\color{#35bf28}+0.04\%$
test_collector_lazystack_then_write[50-img_shape0-small] 0.4239ms 0.2743ms 3.6453 KOps/s 3.6467 KOps/s $\color{#d91a1a}-0.04\%$
test_collector_lazystack_then_write[100-img_shape1-atari] 1.7407ms 1.5345ms 651.6597 Ops/s 669.2364 Ops/s $\color{#d91a1a}-2.63\%$
test_collector_lazystack_then_write[100-img_shape2-large_img] 2.8369ms 2.4111ms 414.7445 Ops/s 412.0708 Ops/s $\color{#35bf28}+0.65\%$
test_collector_lazystack_then_write[200-img_shape3-large_batch] 3.3122ms 3.0566ms 327.1568 Ops/s 323.7081 Ops/s $\color{#35bf28}+1.07\%$
test_collector_without_rb[100-img_shape0-atari] 0.4992s 49.8566ms 20.0575 Ops/s 29.4240 Ops/s $\textbf{\color{#d91a1a}-31.83\%}$
test_collector_without_rb[200-img_shape1-large_batch] 68.9823ms 68.2387ms 14.6544 Ops/s 14.8232 Ops/s $\color{#d91a1a}-1.14\%$
test_collector_with_rb[100-img_shape0-atari] 40.0687ms 39.1322ms 25.5544 Ops/s 26.0166 Ops/s $\color{#d91a1a}-1.78\%$
test_collector_with_rb[200-img_shape1-large_batch] 77.1375ms 76.0607ms 13.1474 Ops/s 13.2878 Ops/s $\color{#d91a1a}-1.06\%$

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 172. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 79.3414μs 78.0058μs 12.8196 KOps/s 12.8477 KOps/s $\color{#d91a1a}-0.22\%$
test_tensor_to_bytestream_speed[torch.save] 0.1364ms 0.1355ms 7.3781 KOps/s 7.3867 KOps/s $\color{#d91a1a}-0.12\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1028s 0.1026s 9.7491 Ops/s 9.7000 Ops/s $\color{#35bf28}+0.51\%$
test_tensor_to_bytestream_speed[numpy] 2.4265μs 2.4190μs 413.3882 KOps/s 420.6919 KOps/s $\color{#d91a1a}-1.74\%$
test_tensor_to_bytestream_speed[safetensors] 35.5979μs 35.2840μs 28.3414 KOps/s 28.1729 KOps/s $\color{#35bf28}+0.60\%$
test_simple 0.7708s 0.7696s 1.2993 Ops/s 1.2297 Ops/s $\textbf{\color{#35bf28}+5.66\%}$
test_transformed 1.5014s 1.4074s 0.7106 Ops/s 0.7070 Ops/s $\color{#35bf28}+0.50\%$
test_serial 2.3556s 2.2692s 0.4407 Ops/s 0.4445 Ops/s $\color{#d91a1a}-0.86\%$
test_parallel 2.0189s 1.9492s 0.5130 Ops/s 0.5235 Ops/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-True-True-True] 0.1795ms 42.8343μs 23.3458 KOps/s 23.1524 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-True-True-True-False] 46.3110μs 23.8863μs 41.8651 KOps/s 40.5150 KOps/s $\color{#35bf28}+3.33\%$
test_step_mdp_speed[True-True-True-False-True] 53.4000μs 23.9106μs 41.8225 KOps/s 41.0855 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-True-False-False] 40.3300μs 13.2347μs 75.5590 KOps/s 74.5645 KOps/s $\color{#35bf28}+1.33\%$
test_step_mdp_speed[True-True-False-True-True] 80.5110μs 46.1220μs 21.6816 KOps/s 21.7872 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-True-False-True-False] 56.6610μs 26.3777μs 37.9108 KOps/s 36.5972 KOps/s $\color{#35bf28}+3.59\%$
test_step_mdp_speed[True-True-False-False-True] 51.9900μs 26.6243μs 37.5597 KOps/s 37.6731 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-False-False] 43.5710μs 15.8334μs 63.1578 KOps/s 62.3480 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-False-True-True-True] 81.4310μs 48.0301μs 20.8203 KOps/s 20.7065 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-False-True-True-False] 56.8600μs 28.7982μs 34.7243 KOps/s 33.9775 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[True-False-True-False-True] 53.7400μs 26.5033μs 37.7312 KOps/s 36.6621 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-False-True-False-False] 41.6510μs 16.1463μs 61.9337 KOps/s 62.0018 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-False-False-True-True] 82.7000μs 51.3481μs 19.4749 KOps/s 19.2831 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-False-False-True-False] 60.1500μs 31.7393μs 31.5067 KOps/s 30.4462 KOps/s $\color{#35bf28}+3.48\%$
test_step_mdp_speed[True-False-False-False-True] 58.4400μs 28.4950μs 35.0939 KOps/s 33.8091 KOps/s $\color{#35bf28}+3.80\%$
test_step_mdp_speed[True-False-False-False-False] 46.1600μs 18.5707μs 53.8482 KOps/s 53.0582 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-True-True-True-True] 78.6710μs 48.7898μs 20.4961 KOps/s 20.1091 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[False-True-True-True-False] 62.4100μs 29.2529μs 34.1847 KOps/s 33.4208 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[False-True-True-False-True] 2.4974ms 30.8441μs 32.4212 KOps/s 33.8230 KOps/s $\color{#d91a1a}-4.14\%$
test_step_mdp_speed[False-True-True-False-False] 49.8110μs 17.9919μs 55.5804 KOps/s 57.2834 KOps/s $\color{#d91a1a}-2.97\%$
test_step_mdp_speed[False-True-False-True-True] 86.0910μs 50.8491μs 19.6660 KOps/s 19.8543 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-False-True-False] 58.9510μs 31.9604μs 31.2887 KOps/s 31.0151 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-True-False-False-True] 71.3110μs 32.2787μs 30.9801 KOps/s 30.4628 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-True-False-False-False] 44.4100μs 19.8451μs 50.3903 KOps/s 49.1855 KOps/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[False-False-True-True-True] 80.1910μs 52.4905μs 19.0511 KOps/s 18.3695 KOps/s $\color{#35bf28}+3.71\%$
test_step_mdp_speed[False-False-True-True-False] 65.3910μs 33.9719μs 29.4361 KOps/s 28.2011 KOps/s $\color{#35bf28}+4.38\%$
test_step_mdp_speed[False-False-True-False-True] 59.5800μs 31.8093μs 31.4374 KOps/s 30.5595 KOps/s $\color{#35bf28}+2.87\%$
test_step_mdp_speed[False-False-True-False-False] 40.6710μs 19.8455μs 50.3892 KOps/s 49.9916 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-False-True-True] 91.0210μs 54.0921μs 18.4870 KOps/s 17.7673 KOps/s $\color{#35bf28}+4.05\%$
test_step_mdp_speed[False-False-False-True-False] 62.8610μs 36.1282μs 27.6792 KOps/s 26.4461 KOps/s $\color{#35bf28}+4.66\%$
test_step_mdp_speed[False-False-False-False-True] 77.1100μs 33.8449μs 29.5466 KOps/s 28.5490 KOps/s $\color{#35bf28}+3.49\%$
test_step_mdp_speed[False-False-False-False-False] 83.2100μs 21.6800μs 46.1255 KOps/s 44.5498 KOps/s $\color{#35bf28}+3.54\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8447s 0.7443s 1.3435 Ops/s 1.3345 Ops/s $\color{#35bf28}+0.68\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7051s 0.6126s 1.6325 Ops/s 1.6251 Ops/s $\color{#35bf28}+0.45\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.6952s 1.6163s 0.6187 Ops/s 0.6171 Ops/s $\color{#35bf28}+0.26\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.4828s 1.3993s 0.7146 Ops/s 0.7118 Ops/s $\color{#35bf28}+0.40\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9395s 1.8636s 0.5366 Ops/s 0.5375 Ops/s $\color{#d91a1a}-0.16\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7333s 1.6495s 0.6062 Ops/s 0.6083 Ops/s $\color{#d91a1a}-0.34\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6816s 4.5917s 0.2178 Ops/s 0.2164 Ops/s $\color{#35bf28}+0.63\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5242s 4.4209s 0.2262 Ops/s 0.2264 Ops/s $\color{#d91a1a}-0.08\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 1.9906s 1.8974s 0.5270 Ops/s 0.5267 Ops/s $\color{#35bf28}+0.07\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.6941s 1.6110s 0.6207 Ops/s 0.6009 Ops/s $\color{#35bf28}+3.30\%$
test_values[generalized_advantage_estimate-True-True] 21.5985ms 20.6592ms 48.4047 Ops/s 50.2244 Ops/s $\color{#d91a1a}-3.62\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1361s 3.6338ms 275.1941 Ops/s 264.8675 Ops/s $\color{#35bf28}+3.90\%$
test_values[td0_return_estimate-False-False] 0.1113ms 82.8561μs 12.0691 KOps/s 12.0568 KOps/s $\color{#35bf28}+0.10\%$
test_values[td1_return_estimate-False-False] 51.1164ms 48.8659ms 20.4642 Ops/s 21.1701 Ops/s $\color{#d91a1a}-3.33\%$
test_values[vec_td1_return_estimate-False-False] 1.3488ms 1.0731ms 931.9150 Ops/s 930.2262 Ops/s $\color{#35bf28}+0.18\%$
test_values[td_lambda_return_estimate-True-False] 84.1599ms 80.0343ms 12.4946 Ops/s 12.9174 Ops/s $\color{#d91a1a}-3.27\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2904ms 1.0734ms 931.6486 Ops/s 932.2278 Ops/s $\color{#d91a1a}-0.06\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 21.6262ms 20.8936ms 47.8615 Ops/s 47.0506 Ops/s $\color{#35bf28}+1.72\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0135ms 0.7465ms 1.3396 KOps/s 1.3466 KOps/s $\color{#d91a1a}-0.52\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8400ms 0.6761ms 1.4790 KOps/s 1.5016 KOps/s $\color{#d91a1a}-1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5642ms 1.4830ms 674.2910 Ops/s 675.9603 Ops/s $\color{#d91a1a}-0.25\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8316ms 0.7086ms 1.4111 KOps/s 1.4174 KOps/s $\color{#d91a1a}-0.44\%$
test_dqn_speed[False-None] 1.5900ms 1.4956ms 668.6159 Ops/s 670.2366 Ops/s $\color{#d91a1a}-0.24\%$
test_dqn_speed[False-backward] 2.2362ms 2.1489ms 465.3503 Ops/s 463.6858 Ops/s $\color{#35bf28}+0.36\%$
test_dqn_speed[True-None] 1.1276ms 0.5475ms 1.8266 KOps/s 1.7983 KOps/s $\color{#35bf28}+1.57\%$
test_dqn_speed[True-backward] 1.0978ms 1.0629ms 940.7928 Ops/s 938.3088 Ops/s $\color{#35bf28}+0.26\%$
test_dqn_speed[reduce-overhead-None] 0.6701ms 0.5681ms 1.7601 KOps/s 1.6948 KOps/s $\color{#35bf28}+3.85\%$
test_ddpg_speed[False-None] 3.1807ms 2.8293ms 353.4391 Ops/s 354.9381 Ops/s $\color{#d91a1a}-0.42\%$
test_ddpg_speed[False-backward] 4.5802ms 4.1421ms 241.4213 Ops/s 244.7105 Ops/s $\color{#d91a1a}-1.34\%$
test_ddpg_speed[True-None] 1.3824ms 1.2762ms 783.5644 Ops/s 754.8220 Ops/s $\color{#35bf28}+3.81\%$
test_ddpg_speed[True-backward] 2.3560ms 2.2986ms 435.0404 Ops/s 434.1633 Ops/s $\color{#35bf28}+0.20\%$
test_ddpg_speed[reduce-overhead-None] 1.4085ms 1.3099ms 763.4174 Ops/s 727.6286 Ops/s $\color{#35bf28}+4.92\%$
test_sac_speed[False-None] 8.8654ms 8.2562ms 121.1218 Ops/s 120.0904 Ops/s $\color{#35bf28}+0.86\%$
test_sac_speed[False-backward] 11.7324ms 11.3004ms 88.4922 Ops/s 89.2506 Ops/s $\color{#d91a1a}-0.85\%$
test_sac_speed[True-None] 1.8475ms 1.7618ms 567.6143 Ops/s 566.2120 Ops/s $\color{#35bf28}+0.25\%$
test_sac_speed[True-backward] 3.4945ms 3.3773ms 296.0984 Ops/s 283.1921 Ops/s $\color{#35bf28}+4.56\%$
test_sac_speed[reduce-overhead-None] 19.5039ms 10.9468ms 91.3506 Ops/s 90.7944 Ops/s $\color{#35bf28}+0.61\%$
test_redq_deprec_speed[False-None] 9.9524ms 9.2240ms 108.4132 Ops/s 109.4319 Ops/s $\color{#d91a1a}-0.93\%$
test_redq_deprec_speed[False-backward] 12.7460ms 12.3571ms 80.9249 Ops/s 79.7138 Ops/s $\color{#35bf28}+1.52\%$
test_redq_deprec_speed[True-None] 2.6187ms 2.4677ms 405.2395 Ops/s 407.3865 Ops/s $\color{#d91a1a}-0.53\%$
test_redq_deprec_speed[True-backward] 4.4712ms 4.1297ms 242.1492 Ops/s 245.6807 Ops/s $\color{#d91a1a}-1.44\%$
test_redq_deprec_speed[reduce-overhead-None] 15.9486ms 9.7884ms 102.1615 Ops/s 101.8952 Ops/s $\color{#35bf28}+0.26\%$
test_td3_speed[False-None] 8.3611ms 8.1578ms 122.5817 Ops/s 116.4869 Ops/s $\textbf{\color{#35bf28}+5.23\%}$
test_td3_speed[False-backward] 10.9077ms 10.5282ms 94.9833 Ops/s 94.3174 Ops/s $\color{#35bf28}+0.71\%$
test_td3_speed[True-None] 1.6253ms 1.5905ms 628.7245 Ops/s 629.3728 Ops/s $\color{#d91a1a}-0.10\%$
test_td3_speed[True-backward] 3.1347ms 3.0292ms 330.1173 Ops/s 332.5997 Ops/s $\color{#d91a1a}-0.75\%$
test_td3_speed[reduce-overhead-None] 58.4688ms 24.1989ms 41.3242 Ops/s 41.2730 Ops/s $\color{#35bf28}+0.12\%$
test_cql_speed[False-None] 17.3823ms 17.0553ms 58.6328 Ops/s 58.8371 Ops/s $\color{#d91a1a}-0.35\%$
test_cql_speed[False-backward] 22.9283ms 22.4007ms 44.6415 Ops/s 44.6286 Ops/s $\color{#35bf28}+0.03\%$
test_cql_speed[True-None] 3.3680ms 3.1780ms 314.6586 Ops/s 318.5857 Ops/s $\color{#d91a1a}-1.23\%$
test_cql_speed[True-backward] 5.6706ms 5.2313ms 191.1569 Ops/s 184.1889 Ops/s $\color{#35bf28}+3.78\%$
test_cql_speed[reduce-overhead-None] 19.9649ms 11.9693ms 83.5471 Ops/s 84.6412 Ops/s $\color{#d91a1a}-1.29\%$
test_a2c_speed[False-None] 4.2939ms 3.1946ms 313.0309 Ops/s 316.0097 Ops/s $\color{#d91a1a}-0.94\%$
test_a2c_speed[False-backward] 6.5313ms 6.1115ms 163.6267 Ops/s 157.1269 Ops/s $\color{#35bf28}+4.14\%$
test_a2c_speed[True-None] 1.4060ms 1.3153ms 760.3030 Ops/s 764.4122 Ops/s $\color{#d91a1a}-0.54\%$
test_a2c_speed[True-backward] 3.2140ms 3.0530ms 327.5497 Ops/s 347.0156 Ops/s $\textbf{\color{#d91a1a}-5.61\%}$
test_a2c_speed[reduce-overhead-None] 1.1173ms 0.9643ms 1.0371 KOps/s 1.0358 KOps/s $\color{#35bf28}+0.13\%$
test_ppo_speed[False-None] 3.9417ms 3.7900ms 263.8514 Ops/s 264.3504 Ops/s $\color{#d91a1a}-0.19\%$
test_ppo_speed[False-backward] 7.5524ms 7.1683ms 139.5036 Ops/s 145.7780 Ops/s $\color{#d91a1a}-4.30\%$
test_ppo_speed[True-None] 1.5151ms 1.4038ms 712.3558 Ops/s 718.2619 Ops/s $\color{#d91a1a}-0.82\%$
test_ppo_speed[True-backward] 3.3547ms 3.1907ms 313.4105 Ops/s 314.0502 Ops/s $\color{#d91a1a}-0.20\%$
test_ppo_speed[reduce-overhead-None] 1.1260ms 1.0319ms 969.0751 Ops/s 949.5875 Ops/s $\color{#35bf28}+2.05\%$
test_reinforce_speed[False-None] 2.3914ms 2.2375ms 446.9242 Ops/s 448.9192 Ops/s $\color{#d91a1a}-0.44\%$
test_reinforce_speed[False-backward] 3.7977ms 3.3796ms 295.8907 Ops/s 308.5217 Ops/s $\color{#d91a1a}-4.09\%$
test_reinforce_speed[True-None] 1.4048ms 1.2569ms 795.6102 Ops/s 795.2350 Ops/s $\color{#35bf28}+0.05\%$
test_reinforce_speed[True-backward] 3.3144ms 2.9685ms 336.8691 Ops/s 346.9854 Ops/s $\color{#d91a1a}-2.92\%$
test_reinforce_speed[reduce-overhead-None] 17.0947ms 9.3830ms 106.5757 Ops/s 107.2931 Ops/s $\color{#d91a1a}-0.67\%$
test_iql_speed[False-None] 10.0565ms 9.2966ms 107.5661 Ops/s 107.5558 Ops/s $+0.01\%$
test_iql_speed[False-backward] 13.7418ms 13.2773ms 75.3166 Ops/s 77.2505 Ops/s $\color{#d91a1a}-2.50\%$
test_iql_speed[True-None] 2.3321ms 2.1113ms 473.6405 Ops/s 472.0555 Ops/s $\color{#35bf28}+0.34\%$
test_iql_speed[True-backward] 4.7882ms 4.7140ms 212.1359 Ops/s 213.5321 Ops/s $\color{#d91a1a}-0.65\%$
test_iql_speed[reduce-overhead-None] 18.7993ms 10.4299ms 95.8779 Ops/s 95.3697 Ops/s $\color{#35bf28}+0.53\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1084ms 5.7126ms 175.0510 Ops/s 172.0165 Ops/s $\color{#35bf28}+1.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.3670ms 0.3250ms 3.0769 KOps/s 3.3820 KOps/s $\textbf{\color{#d91a1a}-9.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6743ms 0.3172ms 3.1524 KOps/s 3.3661 KOps/s $\textbf{\color{#d91a1a}-6.35\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.7897ms 5.5323ms 180.7575 Ops/s 178.4208 Ops/s $\color{#35bf28}+1.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6319ms 0.3250ms 3.0768 KOps/s 3.6821 KOps/s $\textbf{\color{#d91a1a}-16.44\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6116ms 0.3304ms 3.0264 KOps/s 3.9678 KOps/s $\textbf{\color{#d91a1a}-23.73\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4522ms 1.2246ms 816.5700 Ops/s 817.7944 Ops/s $\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4581ms 1.1602ms 861.9408 Ops/s 876.3468 Ops/s $\color{#d91a1a}-1.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7848ms 5.6659ms 176.4952 Ops/s 175.5830 Ops/s $\color{#35bf28}+0.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7785ms 0.4747ms 2.1067 KOps/s 1.9623 KOps/s $\textbf{\color{#35bf28}+7.36\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6545ms 0.4493ms 2.2255 KOps/s 2.0864 KOps/s $\textbf{\color{#35bf28}+6.67\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.6892ms 5.5156ms 181.3045 Ops/s 177.7583 Ops/s $\color{#35bf28}+1.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2580ms 0.3100ms 3.2259 KOps/s 2.7712 KOps/s $\textbf{\color{#35bf28}+16.41\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8468s 1.3163ms 759.6882 Ops/s 2.8629 KOps/s $\textbf{\color{#d91a1a}-73.46\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8812ms 5.5985ms 178.6187 Ops/s 180.2668 Ops/s $\color{#d91a1a}-0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9784ms 0.2708ms 3.6924 KOps/s 3.6940 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4720ms 0.2937ms 3.4044 KOps/s 3.9419 KOps/s $\textbf{\color{#d91a1a}-13.64\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.8775ms 5.7903ms 172.7041 Ops/s 173.1568 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.3706ms 0.4790ms 2.0879 KOps/s 588.6566 Ops/s $\textbf{\color{#35bf28}+254.68\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8845ms 0.4696ms 2.1297 KOps/s 2.0636 KOps/s $\color{#35bf28}+3.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.5976ms 4.9925ms 200.2987 Ops/s 196.5425 Ops/s $\color{#35bf28}+1.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 11.4045ms 2.2267ms 449.1015 Ops/s 604.1690 Ops/s $\textbf{\color{#d91a1a}-25.67\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0168ms 0.8841ms 1.1311 KOps/s 894.5234 Ops/s $\textbf{\color{#35bf28}+26.45\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5913s 16.7540ms 59.6872 Ops/s 198.3983 Ops/s $\textbf{\color{#d91a1a}-69.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 3.9607ms 1.8601ms 537.5968 Ops/s 558.7046 Ops/s $\color{#d91a1a}-3.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.0460ms 1.0812ms 924.8695 Ops/s 1.1036 KOps/s $\textbf{\color{#d91a1a}-16.20\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 8.1366ms 5.2437ms 190.7053 Ops/s 50.4896 Ops/s $\textbf{\color{#35bf28}+277.71\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 4.0359ms 1.9430ms 514.6664 Ops/s 517.6713 Ops/s $\color{#d91a1a}-0.58\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.3216ms 1.0318ms 969.1668 Ops/s 956.4980 Ops/s $\color{#35bf28}+1.32\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 38.3021ms 35.5357ms 28.1407 Ops/s 28.1308 Ops/s $\color{#35bf28}+0.04\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.7182ms 17.7900ms 56.2113 Ops/s 56.4387 Ops/s $\color{#d91a1a}-0.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 41.7088ms 37.0149ms 27.0162 Ops/s 27.2790 Ops/s $\color{#d91a1a}-0.96\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.5016ms 18.3360ms 54.5374 Ops/s 55.4274 Ops/s $\color{#d91a1a}-1.61\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.3434ms 38.7075ms 25.8348 Ops/s 25.8126 Ops/s $\color{#35bf28}+0.09\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.4528ms 19.8762ms 50.3115 Ops/s 50.8249 Ops/s $\color{#d91a1a}-1.01\%$
test_storage_write_lazystack[50-img_shape0-small] 0.9849ms 0.2166ms 4.6163 KOps/s 4.6935 KOps/s $\color{#d91a1a}-1.64\%$
test_storage_write_lazystack[100-img_shape1-atari] 1.5589ms 1.3918ms 718.4690 Ops/s 701.2815 Ops/s $\color{#35bf28}+2.45\%$
test_storage_write_lazystack[100-img_shape2-large_img] 2.7471ms 2.2938ms 435.9660 Ops/s 431.1388 Ops/s $\color{#35bf28}+1.12\%$
test_storage_write_lazystack[200-img_shape3-large_batch] 3.0775ms 2.9088ms 343.7808 Ops/s 339.6729 Ops/s $\color{#35bf28}+1.21\%$
test_storage_write_contiguous[50-img_shape0-small] 0.2375ms 0.1477ms 6.7716 KOps/s 6.8700 KOps/s $\color{#d91a1a}-1.43\%$
test_storage_write_contiguous[100-img_shape1-atari] 0.5898ms 0.2054ms 4.8685 KOps/s 4.9385 KOps/s $\color{#d91a1a}-1.42\%$
test_storage_write_contiguous[100-img_shape2-large_img] 1.9160ms 1.8053ms 553.9101 Ops/s 559.1844 Ops/s $\color{#d91a1a}-0.94\%$
test_storage_write_contiguous[200-img_shape3-large_batch] 1.7127ms 1.3448ms 743.6152 Ops/s 738.3668 Ops/s $\color{#35bf28}+0.71\%$
test_collector_stack_then_write[50-img_shape0-small] 1.1735ms 1.1039ms 905.8923 Ops/s 913.5881 Ops/s $\color{#d91a1a}-0.84\%$
test_collector_stack_then_write[100-img_shape1-atari] 7.0787ms 3.6496ms 273.9992 Ops/s 278.8705 Ops/s $\color{#d91a1a}-1.75\%$
test_collector_stack_then_write[100-img_shape2-large_img] 11.0057ms 5.6353ms 177.4528 Ops/s 171.9850 Ops/s $\color{#35bf28}+3.18\%$
test_collector_stack_then_write[200-img_shape3-large_batch] 6.9367ms 6.8516ms 145.9511 Ops/s 145.5017 Ops/s $\color{#35bf28}+0.31\%$
test_collector_lazystack_then_write[50-img_shape0-small] 0.4312ms 0.2722ms 3.6737 KOps/s 3.6986 KOps/s $\color{#d91a1a}-0.67\%$
test_collector_lazystack_then_write[100-img_shape1-atari] 1.6579ms 1.5053ms 664.3391 Ops/s 649.4746 Ops/s $\color{#35bf28}+2.29\%$
test_collector_lazystack_then_write[100-img_shape2-large_img] 2.8726ms 2.4295ms 411.6154 Ops/s 408.5377 Ops/s $\color{#35bf28}+0.75\%$
test_collector_lazystack_then_write[200-img_shape3-large_batch] 3.3992ms 3.0891ms 323.7192 Ops/s 316.8512 Ops/s $\color{#35bf28}+2.17\%$
test_collector_without_rb[100-img_shape0-atari] 34.6410ms 33.9628ms 29.4440 Ops/s 29.8041 Ops/s $\color{#d91a1a}-1.21\%$
test_collector_without_rb[200-img_shape1-large_batch] 66.7927ms 66.3464ms 15.0724 Ops/s 15.1940 Ops/s $\color{#d91a1a}-0.80\%$
test_collector_with_rb[100-img_shape0-atari] 38.5877ms 38.0878ms 26.2551 Ops/s 26.6012 Ops/s $\color{#d91a1a}-1.30\%$
test_collector_with_rb[200-img_shape1-large_batch] 75.4445ms 74.0099ms 13.5117 Ops/s 13.4331 Ops/s $\color{#35bf28}+0.59\%$
test_collector_without_rb_cuda[100-img_shape0-atari] 55.7160ms 55.4961ms 18.0193 Ops/s 18.1288 Ops/s $\color{#d91a1a}-0.60\%$
test_collector_without_rb_cuda[200-img_shape1-large_batch] 0.1112s 0.1108s 9.0250 Ops/s 9.0623 Ops/s $\color{#d91a1a}-0.41\%$
test_collector_with_rb_cuda[100-img_shape0-atari] 58.2356ms 57.7302ms 17.3220 Ops/s 17.4788 Ops/s $\color{#d91a1a}-0.90\%$
test_collector_with_rb_cuda[200-img_shape1-large_batch] 0.1156s 0.1151s 8.6895 Ops/s 8.7584 Ops/s $\color{#d91a1a}-0.79\%$

@vmoens vmoens merged commit 5ef7cc2 into main Feb 4, 2026
117 of 121 checks passed
vmoens added a commit that referenced this pull request Feb 4, 2026
@vmoens
[BugFix] Fix shared mem updater with many policies (#3442)
3669969
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

BugFix Collectors WeightUpdate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens