Skip to content

[Refactor] Move WEIGHT_SYNC_TIMEOUT to collectors._constants #3329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 11 commits into from
Jan 25, 2026
Merged

[Refactor] Move WEIGHT_SYNC_TIMEOUT to collectors._constants #3329

vmoens merged 11 commits into from
Jan 25, 2026

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Jan 12, 2026 •
edited
Loading

Stack from ghstack (oldest at bottom):

Move WEIGHT_SYNC_TIMEOUT from torchrl._utils.py to
torchrl/collectors/_constants.py to match the target branch.

@vmoens
Update
a6808c1 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Jan 12, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3329

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Jan 12, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 12, 2026
vmoens added 7 commits January 12, 2026 17:37
@vmoens
Update
d3ca7c3 
[ghstack-poisoned]
@vmoens
Update
bbf6608 
[ghstack-poisoned]
@vmoens
Update
aab273a 
[ghstack-poisoned]
@vmoens
Update
1e6e2a1 
[ghstack-poisoned]
@vmoens
Update
33cff30 
[ghstack-poisoned]
@vmoens
Update
2c1dbde 
[ghstack-poisoned]
@vmoens
Update
bee136d 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Jan 13, 2026 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 153. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 81.1917μs 80.3097μs 12.4518 KOps/s 12.4486 KOps/s $\color{#35bf28}+0.03\%$
test_tensor_to_bytestream_speed[torch.save] 0.1413ms 0.1410ms 7.0907 KOps/s 7.0455 KOps/s $\color{#35bf28}+0.64\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1305s 0.1300s 7.6919 Ops/s 8.7931 Ops/s $\textbf{\color{#d91a1a}-12.52\%}$
test_tensor_to_bytestream_speed[numpy] 2.7267μs 2.7236μs 367.1637 KOps/s 394.9566 KOps/s $\textbf{\color{#d91a1a}-7.04\%}$
test_tensor_to_bytestream_speed[safetensors] 36.9248μs 36.7306μs 27.2253 KOps/s 25.6146 KOps/s $\textbf{\color{#35bf28}+6.29\%}$
test_simple 0.6704s 0.5786s 1.7283 Ops/s 1.7985 Ops/s $\color{#d91a1a}-3.90\%$
test_transformed 1.2464s 1.1592s 0.8627 Ops/s 0.8683 Ops/s $\color{#d91a1a}-0.65\%$
test_serial 1.7049s 1.7023s 0.5874 Ops/s 0.5900 Ops/s $\color{#d91a1a}-0.44\%$
test_parallel 1.2427s 1.1784s 0.8486 Ops/s 0.8716 Ops/s $\color{#d91a1a}-2.64\%$
test_step_mdp_speed[True-True-True-True-True] 0.3350ms 45.0877μs 22.1790 KOps/s 22.3366 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-True-True-True-False] 50.0110μs 25.4779μs 39.2497 KOps/s 40.6784 KOps/s $\color{#d91a1a}-3.51\%$
test_step_mdp_speed[True-True-True-False-True] 58.1410μs 25.8003μs 38.7592 KOps/s 40.3893 KOps/s $\color{#d91a1a}-4.04\%$
test_step_mdp_speed[True-True-True-False-False] 41.7300μs 14.0541μs 71.1538 KOps/s 73.0126 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[True-True-False-True-True] 0.1247ms 48.1293μs 20.7774 KOps/s 21.0551 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-True-False-True-False] 56.7710μs 28.1619μs 35.5090 KOps/s 35.7542 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-True-False-False-True] 72.1910μs 28.2521μs 35.3957 KOps/s 35.7436 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-True-False-False-False] 45.6310μs 16.9080μs 59.1438 KOps/s 58.9659 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-False-True-True-True] 99.6420μs 51.1140μs 19.5641 KOps/s 19.6041 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-False-True-True-False] 73.9210μs 31.1746μs 32.0774 KOps/s 32.8247 KOps/s $\color{#d91a1a}-2.28\%$
test_step_mdp_speed[True-False-True-False-True] 70.7110μs 28.0415μs 35.6615 KOps/s 35.4210 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-False-True-False-False] 61.9710μs 16.9316μs 59.0611 KOps/s 60.1294 KOps/s $\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-False-False-True-True] 97.1920μs 53.4418μs 18.7119 KOps/s 18.8369 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[True-False-False-True-False] 74.4320μs 33.1734μs 30.1446 KOps/s 30.3895 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[True-False-False-False-True] 98.1420μs 30.5408μs 32.7431 KOps/s 33.5718 KOps/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-False-False-False-False] 63.8410μs 19.3802μs 51.5990 KOps/s 51.6782 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-True-True-True] 97.7220μs 50.0935μs 19.9627 KOps/s 19.7169 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-True-True-False] 77.2510μs 30.6122μs 32.6667 KOps/s 32.5105 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-True-True-False-True] 71.8610μs 31.8352μs 31.4118 KOps/s 31.2444 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-True-True-False-False] 68.2320μs 18.3594μs 54.4681 KOps/s 54.7873 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-True-False-True-True] 2.6609ms 52.9392μs 18.8896 KOps/s 16.7518 KOps/s $\textbf{\color{#35bf28}+12.76\%}$
test_step_mdp_speed[False-True-False-True-False] 71.8620μs 33.3694μs 29.9676 KOps/s 30.2166 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-True-False-False-True] 74.6610μs 33.9062μs 29.4931 KOps/s 29.4558 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-True-False-False-False] 61.2810μs 20.9807μs 47.6628 KOps/s 48.0604 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-False-True-True-True] 0.1090ms 55.9824μs 17.8628 KOps/s 18.3079 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[False-False-True-True-False] 99.2920μs 35.6367μs 28.0610 KOps/s 27.8126 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-False-True-False-True] 0.4387ms 34.3715μs 29.0939 KOps/s 29.1618 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-False-True-False-False] 57.2810μs 20.9901μs 47.6414 KOps/s 48.6855 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-False-False-True-True] 0.4660ms 57.3641μs 17.4325 KOps/s 17.4993 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-False-False-True-False] 0.4500ms 38.3341μs 26.0864 KOps/s 26.3700 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-False-False-False-True] 0.4516ms 36.1897μs 27.6322 KOps/s 28.4553 KOps/s $\color{#d91a1a}-2.89\%$
test_step_mdp_speed[False-False-False-False-False] 51.9910μs 23.5896μs 42.3915 KOps/s 42.3501 KOps/s $\color{#35bf28}+0.10\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8620s 0.7676s 1.3028 Ops/s 1.3084 Ops/s $\color{#d91a1a}-0.43\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7314s 0.6341s 1.5771 Ops/s 1.5854 Ops/s $\color{#d91a1a}-0.52\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7503s 1.6732s 0.5977 Ops/s 0.5946 Ops/s $\color{#35bf28}+0.51\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5201s 1.4457s 0.6917 Ops/s 0.6852 Ops/s $\color{#35bf28}+0.94\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 2.0088s 1.9312s 0.5178 Ops/s 0.5182 Ops/s $\color{#d91a1a}-0.08\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7779s 1.7016s 0.5877 Ops/s 0.5862 Ops/s $\color{#35bf28}+0.26\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6449s 4.6131s 0.2168 Ops/s 0.2160 Ops/s $\color{#35bf28}+0.34\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5372s 4.4448s 0.2250 Ops/s 0.2254 Ops/s $\color{#d91a1a}-0.17\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0384s 1.9623s 0.5096 Ops/s 0.5089 Ops/s $\color{#35bf28}+0.13\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7775s 1.6719s 0.5981 Ops/s 0.5986 Ops/s $\color{#d91a1a}-0.08\%$
test_values[generalized_advantage_estimate-True-True] 11.4735ms 11.3409ms 88.1764 Ops/s 97.8255 Ops/s $\textbf{\color{#d91a1a}-9.86\%}$
test_values[vec_generalized_advantage_estimate-True-True] 14.9569ms 11.1001ms 90.0896 Ops/s 56.7481 Ops/s $\textbf{\color{#35bf28}+58.75\%}$
test_values[td0_return_estimate-False-False] 0.2321ms 0.1327ms 7.5342 KOps/s 7.6929 KOps/s $\color{#d91a1a}-2.06\%$
test_values[td1_return_estimate-False-False] 30.9180ms 30.5753ms 32.7062 Ops/s 35.6397 Ops/s $\textbf{\color{#d91a1a}-8.23\%}$
test_values[vec_td1_return_estimate-False-False] 17.8047ms 11.2674ms 88.7515 Ops/s 56.3690 Ops/s $\textbf{\color{#35bf28}+57.45\%}$
test_values[td_lambda_return_estimate-True-False] 45.9016ms 45.3228ms 22.0639 Ops/s 23.9013 Ops/s $\textbf{\color{#d91a1a}-7.69\%}$
test_values[vec_td_lambda_return_estimate-True-False] 11.2849ms 11.0790ms 90.2606 Ops/s 56.6647 Ops/s $\textbf{\color{#35bf28}+59.29\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.2368ms 10.1354ms 98.6643 Ops/s 110.3057 Ops/s $\textbf{\color{#d91a1a}-10.55\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.8819ms 1.5228ms 656.6807 Ops/s 643.7724 Ops/s $\color{#35bf28}+2.01\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4844ms 0.4369ms 2.2888 KOps/s 2.2866 KOps/s $\color{#35bf28}+0.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 29.9020ms 29.3383ms 34.0851 Ops/s 32.8195 Ops/s $\color{#35bf28}+3.86\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.0868ms 1.7385ms 575.2071 Ops/s 580.7410 Ops/s $\color{#d91a1a}-0.95\%$
test_dqn_speed[False-None] 1.7008ms 1.4192ms 704.6361 Ops/s 706.7618 Ops/s $\color{#d91a1a}-0.30\%$
test_dqn_speed[False-backward] 1.9851ms 1.9407ms 515.2776 Ops/s 515.3114 Ops/s $-0.01\%$
test_dqn_speed[True-None] 0.9478ms 0.5471ms 1.8279 KOps/s 1.8429 KOps/s $\color{#d91a1a}-0.81\%$
test_dqn_speed[True-backward] 1.0644ms 1.0092ms 990.9299 Ops/s 848.1913 Ops/s $\textbf{\color{#35bf28}+16.83\%}$
test_dqn_speed[reduce-overhead-None] 0.9354ms 0.5342ms 1.8720 KOps/s 1.8761 KOps/s $\color{#d91a1a}-0.22\%$
test_ddpg_speed[False-None] 3.1734ms 2.8838ms 346.7660 Ops/s 348.9699 Ops/s $\color{#d91a1a}-0.63\%$
test_ddpg_speed[False-backward] 4.2076ms 4.1439ms 241.3162 Ops/s 243.5702 Ops/s $\color{#d91a1a}-0.93\%$
test_ddpg_speed[True-None] 1.5224ms 1.4168ms 705.8043 Ops/s 685.7649 Ops/s $\color{#35bf28}+2.92\%$
test_ddpg_speed[True-backward] 2.4576ms 2.4196ms 413.2908 Ops/s 411.0935 Ops/s $\color{#35bf28}+0.53\%$
test_ddpg_speed[reduce-overhead-None] 1.8255ms 1.4072ms 710.6539 Ops/s 719.1722 Ops/s $\color{#d91a1a}-1.18\%$
test_sac_speed[False-None] 8.6760ms 8.1495ms 122.7069 Ops/s 123.2184 Ops/s $\color{#d91a1a}-0.42\%$
test_sac_speed[False-backward] 11.8961ms 11.4777ms 87.1252 Ops/s 87.4100 Ops/s $\color{#d91a1a}-0.33\%$
test_sac_speed[True-None] 2.6163ms 2.1629ms 462.3381 Ops/s 460.5948 Ops/s $\color{#35bf28}+0.38\%$
test_sac_speed[True-backward] 4.1293ms 4.0413ms 247.4459 Ops/s 244.0954 Ops/s $\color{#35bf28}+1.37\%$
test_sac_speed[reduce-overhead-None] 2.5175ms 2.1281ms 469.9128 Ops/s 454.4565 Ops/s $\color{#35bf28}+3.40\%$
test_redq_speed[False-None] 10.9215ms 10.4977ms 95.2587 Ops/s 91.0802 Ops/s $\color{#35bf28}+4.59\%$
test_redq_speed[False-backward] 18.5440ms 17.9132ms 55.8249 Ops/s 55.1516 Ops/s $\color{#35bf28}+1.22\%$
test_redq_speed[True-None] 4.9778ms 4.4781ms 223.3090 Ops/s 218.9516 Ops/s $\color{#35bf28}+1.99\%$
test_redq_speed[True-backward] 10.0967ms 9.8408ms 101.6182 Ops/s 100.4772 Ops/s $\color{#35bf28}+1.14\%$
test_redq_speed[reduce-overhead-None] 4.6459ms 4.3498ms 229.8956 Ops/s 231.0573 Ops/s $\color{#d91a1a}-0.50\%$
test_redq_deprec_speed[False-None] 11.8254ms 11.2881ms 88.5888 Ops/s 91.7056 Ops/s $\color{#d91a1a}-3.40\%$
test_redq_deprec_speed[False-backward] 16.6746ms 16.3217ms 61.2683 Ops/s 63.8082 Ops/s $\color{#d91a1a}-3.98\%$
test_redq_deprec_speed[True-None] 4.1823ms 3.6974ms 270.4639 Ops/s 265.4313 Ops/s $\color{#35bf28}+1.90\%$
test_redq_deprec_speed[True-backward] 8.0024ms 7.6926ms 129.9952 Ops/s 130.0078 Ops/s $-0.01\%$
test_redq_deprec_speed[reduce-overhead-None] 4.0716ms 3.6033ms 277.5227 Ops/s 275.6815 Ops/s $\color{#35bf28}+0.67\%$
test_td3_speed[False-None] 8.3502ms 8.1860ms 122.1596 Ops/s 122.1737 Ops/s $\color{#d91a1a}-0.01\%$
test_td3_speed[False-backward] 11.5520ms 11.1236ms 89.8993 Ops/s 90.5753 Ops/s $\color{#d91a1a}-0.75\%$
test_td3_speed[True-None] 1.8914ms 1.8491ms 540.8049 Ops/s 541.2121 Ops/s $\color{#d91a1a}-0.08\%$
test_td3_speed[True-backward] 3.7992ms 3.6651ms 272.8434 Ops/s 239.3837 Ops/s $\textbf{\color{#35bf28}+13.98\%}$
test_td3_speed[reduce-overhead-None] 1.8519ms 1.8102ms 552.4316 Ops/s 549.4579 Ops/s $\color{#35bf28}+0.54\%$
test_cql_speed[False-None] 29.6779ms 26.6281ms 37.5543 Ops/s 38.5421 Ops/s $\color{#d91a1a}-2.56\%$
test_cql_speed[False-backward] 37.1567ms 35.6965ms 28.0139 Ops/s 28.1542 Ops/s $\color{#d91a1a}-0.50\%$
test_cql_speed[True-None] 12.9288ms 12.3963ms 80.6694 Ops/s 82.2598 Ops/s $\color{#d91a1a}-1.93\%$
test_cql_speed[True-backward] 18.7792ms 18.4548ms 54.1864 Ops/s 55.7029 Ops/s $\color{#d91a1a}-2.72\%$
test_cql_speed[reduce-overhead-None] 12.7565ms 12.4586ms 80.2658 Ops/s 80.1970 Ops/s $\color{#35bf28}+0.09\%$
test_a2c_speed[False-None] 5.8085ms 5.5007ms 181.7958 Ops/s 184.4431 Ops/s $\color{#d91a1a}-1.44\%$
test_a2c_speed[False-backward] 12.4286ms 12.0946ms 82.6812 Ops/s 84.2340 Ops/s $\color{#d91a1a}-1.84\%$
test_a2c_speed[True-None] 4.0522ms 3.7408ms 267.3191 Ops/s 270.3017 Ops/s $\color{#d91a1a}-1.10\%$
test_a2c_speed[True-backward] 9.4016ms 8.6850ms 115.1414 Ops/s 112.4217 Ops/s $\color{#35bf28}+2.42\%$
test_a2c_speed[reduce-overhead-None] 4.0497ms 3.6789ms 271.8238 Ops/s 265.6094 Ops/s $\color{#35bf28}+2.34\%$
test_ppo_speed[False-None] 6.4838ms 5.9935ms 166.8473 Ops/s 169.6369 Ops/s $\color{#d91a1a}-1.64\%$
test_ppo_speed[False-backward] 13.0589ms 12.7372ms 78.5099 Ops/s 80.4242 Ops/s $\color{#d91a1a}-2.38\%$
test_ppo_speed[True-None] 4.0533ms 3.6242ms 275.9256 Ops/s 264.4557 Ops/s $\color{#35bf28}+4.34\%$
test_ppo_speed[True-backward] 8.8625ms 8.5535ms 116.9113 Ops/s 116.4849 Ops/s $\color{#35bf28}+0.37\%$
test_ppo_speed[reduce-overhead-None] 4.0346ms 3.6950ms 270.6342 Ops/s 278.5270 Ops/s $\color{#d91a1a}-2.83\%$
test_reinforce_speed[False-None] 5.3064ms 4.7244ms 211.6673 Ops/s 220.8189 Ops/s $\color{#d91a1a}-4.14\%$
test_reinforce_speed[False-backward] 7.9478ms 7.5862ms 131.8177 Ops/s 135.1876 Ops/s $\color{#d91a1a}-2.49\%$
test_reinforce_speed[True-None] 3.2612ms 2.8899ms 346.0342 Ops/s 344.0624 Ops/s $\color{#35bf28}+0.57\%$
test_reinforce_speed[True-backward] 8.0828ms 7.8117ms 128.0126 Ops/s 121.6248 Ops/s $\textbf{\color{#35bf28}+5.25\%}$
test_reinforce_speed[reduce-overhead-None] 3.2758ms 2.8734ms 348.0185 Ops/s 342.5137 Ops/s $\color{#35bf28}+1.61\%$
test_iql_speed[False-None] 20.4495ms 19.6873ms 50.7942 Ops/s 49.3521 Ops/s $\color{#35bf28}+2.92\%$
test_iql_speed[False-backward] 31.4397ms 30.7062ms 32.5667 Ops/s 32.3385 Ops/s $\color{#35bf28}+0.71\%$
test_iql_speed[True-None] 8.9549ms 8.5103ms 117.5047 Ops/s 117.3238 Ops/s $\color{#35bf28}+0.15\%$
test_iql_speed[True-backward] 17.4991ms 16.8497ms 59.3484 Ops/s 58.1637 Ops/s $\color{#35bf28}+2.04\%$
test_iql_speed[reduce-overhead-None] 8.9821ms 8.5979ms 116.3077 Ops/s 116.3001 Ops/s $+0.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1874ms 5.9996ms 166.6775 Ops/s 165.5332 Ops/s $\color{#35bf28}+0.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7443s 0.8387ms 1.1924 KOps/s 2.8879 KOps/s $\textbf{\color{#d91a1a}-58.71\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7154ms 0.2697ms 3.7085 KOps/s 3.1185 KOps/s $\textbf{\color{#35bf28}+18.92\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9454ms 5.7483ms 173.9658 Ops/s 171.6164 Ops/s $\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7310ms 0.2838ms 3.5240 KOps/s 2.9573 KOps/s $\textbf{\color{#35bf28}+19.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7360ms 0.2641ms 3.7860 KOps/s 3.0904 KOps/s $\textbf{\color{#35bf28}+22.51\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5066ms 1.3084ms 764.2715 Ops/s 759.6864 Ops/s $\color{#35bf28}+0.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5309ms 1.2887ms 775.9491 Ops/s 814.1732 Ops/s $\color{#d91a1a}-4.69\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0201ms 5.9267ms 168.7273 Ops/s 166.8401 Ops/s $\color{#35bf28}+1.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8589ms 0.4399ms 2.2735 KOps/s 1.9249 KOps/s $\textbf{\color{#35bf28}+18.11\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6188ms 0.4232ms 2.3632 KOps/s 2.0085 KOps/s $\textbf{\color{#35bf28}+17.66\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9505ms 5.8612ms 170.6139 Ops/s 169.5121 Ops/s $\color{#35bf28}+0.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.1165ms 0.3352ms 2.9836 KOps/s 3.4629 KOps/s $\textbf{\color{#d91a1a}-13.84\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6021ms 0.3314ms 3.0176 KOps/s 3.2316 KOps/s $\textbf{\color{#d91a1a}-6.62\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9465ms 5.7607ms 173.5885 Ops/s 172.9077 Ops/s $\color{#35bf28}+0.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9826ms 0.3369ms 2.9681 KOps/s 3.1882 KOps/s $\textbf{\color{#d91a1a}-6.90\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5084ms 0.2786ms 3.5895 KOps/s 3.4639 KOps/s $\color{#35bf28}+3.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0342ms 5.9455ms 168.1933 Ops/s 166.7552 Ops/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7080ms 0.4665ms 2.1435 KOps/s 2.1754 KOps/s $\color{#d91a1a}-1.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0721ms 0.4859ms 2.0580 KOps/s 2.3615 KOps/s $\textbf{\color{#d91a1a}-12.85\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.5515s 15.9717ms 62.6107 Ops/s 51.5256 Ops/s $\textbf{\color{#35bf28}+21.51\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 3.9537ms 1.7672ms 565.8553 Ops/s 491.4675 Ops/s $\textbf{\color{#35bf28}+15.14\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.1618ms 0.8925ms 1.1204 KOps/s 799.8714 Ops/s $\textbf{\color{#35bf28}+40.08\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.0989ms 5.0521ms 197.9359 Ops/s 197.9048 Ops/s $\color{#35bf28}+0.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 3.9569ms 1.7998ms 555.6139 Ops/s 554.4425 Ops/s $\color{#35bf28}+0.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.8751ms 1.2164ms 822.0999 Ops/s 853.6730 Ops/s $\color{#d91a1a}-3.70\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5415s 16.0096ms 62.4626 Ops/s 190.7066 Ops/s $\textbf{\color{#d91a1a}-67.25\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.8616ms 2.2250ms 449.4448 Ops/s 496.4927 Ops/s $\textbf{\color{#d91a1a}-9.48\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.2070ms 1.2484ms 800.9987 Ops/s 803.4900 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 39.0983ms 36.1965ms 27.6270 Ops/s 27.7498 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 20.2636ms 18.6427ms 53.6404 Ops/s 54.2385 Ops/s $\color{#d91a1a}-1.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 40.7008ms 37.2755ms 26.8273 Ops/s 27.0416 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.5585ms 18.8757ms 52.9782 Ops/s 53.3921 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.4244ms 38.7224ms 25.8248 Ops/s 25.7491 Ops/s $\color{#35bf28}+0.29\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 22.0375ms 20.4636ms 48.8672 Ops/s 49.5942 Ops/s $\color{#d91a1a}-1.47\%$

vmoens added 2 commits January 13, 2026 16:35
@vmoens
Update
cb853a6 
[ghstack-poisoned]
@vmoens
Update
d053934 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Jan 24, 2026 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 148. Improved: $\large\color{#35bf28}19$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 78.9349μs 77.7415μs 12.8631 KOps/s 12.8055 KOps/s $\color{#35bf28}+0.45\%$
test_tensor_to_bytestream_speed[torch.save] 0.1416ms 0.1381ms 7.2423 KOps/s 7.2834 KOps/s $\color{#d91a1a}-0.56\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1123s 0.1119s 8.9350 Ops/s 8.1697 Ops/s $\textbf{\color{#35bf28}+9.37\%}$
test_tensor_to_bytestream_speed[numpy] 2.5241μs 2.5202μs 396.7968 KOps/s 401.4744 KOps/s $\color{#d91a1a}-1.17\%$
test_tensor_to_bytestream_speed[safetensors] 36.0512μs 35.8810μs 27.8699 KOps/s 27.5415 KOps/s $\color{#35bf28}+1.19\%$
test_simple 0.9021s 0.8058s 1.2410 Ops/s 1.2656 Ops/s $\color{#d91a1a}-1.95\%$
test_transformed 1.5254s 1.4163s 0.7061 Ops/s 0.7111 Ops/s $\color{#d91a1a}-0.71\%$
test_serial 2.2460s 2.2412s 0.4462 Ops/s 0.4356 Ops/s $\color{#35bf28}+2.43\%$
test_parallel 1.9963s 1.9263s 0.5191 Ops/s 0.5090 Ops/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-True-True-True-True] 0.1297ms 43.2976μs 23.0960 KOps/s 23.1738 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-True-False] 70.2620μs 24.1646μs 41.3828 KOps/s 40.7381 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-True-True-False-True] 63.4020μs 24.9138μs 40.1385 KOps/s 40.6754 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-True-True-False-False] 41.7910μs 13.3167μs 75.0936 KOps/s 73.9963 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[True-True-False-True-True] 0.1035ms 46.4856μs 21.5120 KOps/s 21.2487 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-False-True-False] 74.5030μs 26.8326μs 37.2681 KOps/s 37.7441 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[True-True-False-False-True] 84.0230μs 26.8671μs 37.2203 KOps/s 36.7930 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-False-False-False] 54.6930μs 16.1996μs 61.7298 KOps/s 62.5764 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[True-False-True-True-True] 88.5540μs 48.9581μs 20.4256 KOps/s 20.4168 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-False-True-True-False] 87.7630μs 29.9631μs 33.3744 KOps/s 33.5015 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-False-True-False-True] 62.3330μs 27.0515μs 36.9665 KOps/s 36.4010 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[True-False-True-False-False] 48.4120μs 15.9777μs 62.5872 KOps/s 62.3618 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-False-True-True] 0.1251ms 51.4538μs 19.4349 KOps/s 19.4695 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-False-True-False] 85.5130μs 32.0090μs 31.2412 KOps/s 31.3902 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[True-False-False-False-True] 69.8730μs 29.4717μs 33.9309 KOps/s 33.8584 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-False-False-False-False] 65.7330μs 18.4280μs 54.2651 KOps/s 53.8907 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-True-True-True-True] 0.1031ms 49.2093μs 20.3213 KOps/s 20.4173 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-True-True-True-False] 83.4130μs 29.1843μs 34.2650 KOps/s 33.9805 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[False-True-True-False-True] 69.4920μs 30.8528μs 32.4120 KOps/s 32.5050 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-True-True-False-False] 52.9730μs 17.8395μs 56.0555 KOps/s 56.3884 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-False-True-True] 3.3266ms 51.6852μs 19.3479 KOps/s 19.0736 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-False-True-False] 67.8720μs 32.6942μs 30.5864 KOps/s 31.1864 KOps/s $\color{#d91a1a}-1.92\%$
test_step_mdp_speed[False-True-False-False-True] 74.1330μs 33.4599μs 29.8865 KOps/s 30.1899 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-True-False-False-False] 55.7020μs 20.3702μs 49.0914 KOps/s 48.5498 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-False-True-True-True] 0.1087ms 54.0177μs 18.5125 KOps/s 18.4650 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-False-True-True-False] 0.1033ms 34.2125μs 29.2291 KOps/s 28.8874 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[False-False-True-False-True] 70.4730μs 32.9430μs 30.3554 KOps/s 30.2173 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-False-True-False-False] 64.2630μs 20.3623μs 49.1104 KOps/s 48.5945 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-False-True-True] 0.1247ms 54.9960μs 18.1831 KOps/s 17.6148 KOps/s $\color{#35bf28}+3.23\%$
test_step_mdp_speed[False-False-False-True-False] 77.9440μs 36.9844μs 27.0385 KOps/s 26.4976 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-False-False-False-True] 0.1006ms 34.7674μs 28.7626 KOps/s 28.2029 KOps/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[False-False-False-False-False] 72.2830μs 22.3123μs 44.8184 KOps/s 43.3570 KOps/s $\color{#35bf28}+3.37\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.7363s 0.7276s 1.3743 Ops/s 1.3228 Ops/s $\color{#35bf28}+3.89\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7128s 0.6216s 1.6088 Ops/s 1.6113 Ops/s $\color{#d91a1a}-0.15\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7097s 1.6292s 0.6138 Ops/s 0.6065 Ops/s $\color{#35bf28}+1.21\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.4939s 1.4189s 0.7048 Ops/s 0.6946 Ops/s $\color{#35bf28}+1.47\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9753s 1.8696s 0.5349 Ops/s 0.5280 Ops/s $\color{#35bf28}+1.29\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7300s 1.6384s 0.6103 Ops/s 0.5954 Ops/s $\color{#35bf28}+2.50\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6276s 4.5140s 0.2215 Ops/s 0.2159 Ops/s $\color{#35bf28}+2.60\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.4713s 4.3117s 0.2319 Ops/s 0.2284 Ops/s $\color{#35bf28}+1.54\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 1.9957s 1.9160s 0.5219 Ops/s 0.5235 Ops/s $\color{#d91a1a}-0.29\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7189s 1.6373s 0.6108 Ops/s 0.6109 Ops/s $\color{#d91a1a}-0.02\%$
test_values[generalized_advantage_estimate-True-True] 22.0286ms 21.1470ms 47.2880 Ops/s 48.3463 Ops/s $\color{#d91a1a}-2.19\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1284s 3.5013ms 285.6047 Ops/s 268.3602 Ops/s $\textbf{\color{#35bf28}+6.43\%}$
test_values[td0_return_estimate-False-False] 0.1075ms 83.6245μs 11.9582 KOps/s 11.9058 KOps/s $\color{#35bf28}+0.44\%$
test_values[td1_return_estimate-False-False] 51.3922ms 49.9965ms 20.0014 Ops/s 20.3333 Ops/s $\color{#d91a1a}-1.63\%$
test_values[vec_td1_return_estimate-False-False] 1.3426ms 1.1146ms 897.2062 Ops/s 907.9702 Ops/s $\color{#d91a1a}-1.19\%$
test_values[td_lambda_return_estimate-True-False] 83.7618ms 81.3595ms 12.2911 Ops/s 12.4103 Ops/s $\color{#d91a1a}-0.96\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2978ms 1.0930ms 914.8920 Ops/s 909.9861 Ops/s $\color{#35bf28}+0.54\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.1663ms 21.2728ms 47.0083 Ops/s 48.0449 Ops/s $\color{#d91a1a}-2.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0589ms 0.7880ms 1.2690 KOps/s 1.2960 KOps/s $\color{#d91a1a}-2.09\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7615ms 0.6815ms 1.4673 KOps/s 1.4470 KOps/s $\color{#35bf28}+1.40\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5714ms 1.4987ms 667.2552 Ops/s 664.6874 Ops/s $\color{#35bf28}+0.39\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7704ms 0.7040ms 1.4204 KOps/s 1.4017 KOps/s $\color{#35bf28}+1.34\%$
test_dqn_speed[False-None] 1.7157ms 1.5339ms 651.9286 Ops/s 651.6662 Ops/s $\color{#35bf28}+0.04\%$
test_dqn_speed[False-backward] 2.3839ms 2.1893ms 456.7757 Ops/s 453.9425 Ops/s $\color{#35bf28}+0.62\%$
test_dqn_speed[True-None] 0.6395ms 0.5560ms 1.7986 KOps/s 1.8204 KOps/s $\color{#d91a1a}-1.20\%$
test_dqn_speed[True-backward] 1.1020ms 1.0700ms 934.6134 Ops/s 906.2634 Ops/s $\color{#35bf28}+3.13\%$
test_dqn_speed[reduce-overhead-None] 0.6978ms 0.5588ms 1.7896 KOps/s 1.6885 KOps/s $\textbf{\color{#35bf28}+5.99\%}$
test_ddpg_speed[False-None] 3.2493ms 2.8955ms 345.3598 Ops/s 339.6421 Ops/s $\color{#35bf28}+1.68\%$
test_ddpg_speed[False-backward] 4.6237ms 4.2509ms 235.2444 Ops/s 231.6444 Ops/s $\color{#35bf28}+1.55\%$
test_ddpg_speed[True-None] 1.3612ms 1.2769ms 783.1380 Ops/s 764.6281 Ops/s $\color{#35bf28}+2.42\%$
test_ddpg_speed[True-backward] 2.5039ms 2.4493ms 408.2834 Ops/s 395.3166 Ops/s $\color{#35bf28}+3.28\%$
test_ddpg_speed[reduce-overhead-None] 1.7783ms 1.3117ms 762.3548 Ops/s 750.8392 Ops/s $\color{#35bf28}+1.53\%$
test_sac_speed[False-None] 9.1485ms 8.3702ms 119.4721 Ops/s 118.9840 Ops/s $\color{#35bf28}+0.41\%$
test_sac_speed[False-backward] 12.3714ms 11.6929ms 85.5221 Ops/s 86.4246 Ops/s $\color{#d91a1a}-1.04\%$
test_sac_speed[True-None] 1.8860ms 1.7618ms 567.6144 Ops/s 557.2680 Ops/s $\color{#35bf28}+1.86\%$
test_sac_speed[True-backward] 3.4216ms 3.3195ms 301.2528 Ops/s 279.2628 Ops/s $\textbf{\color{#35bf28}+7.87\%}$
test_sac_speed[reduce-overhead-None] 0.4146s 11.7278ms 85.2678 Ops/s 94.6931 Ops/s $\textbf{\color{#d91a1a}-9.95\%}$
test_redq_deprec_speed[False-None] 10.1992ms 9.3355ms 107.1182 Ops/s 106.3263 Ops/s $\color{#35bf28}+0.74\%$
test_redq_deprec_speed[False-backward] 12.9668ms 12.5524ms 79.6659 Ops/s 77.2877 Ops/s $\color{#35bf28}+3.08\%$
test_redq_deprec_speed[True-None] 2.5849ms 2.4782ms 403.5254 Ops/s 396.8629 Ops/s $\color{#35bf28}+1.68\%$
test_redq_deprec_speed[True-backward] 4.2001ms 4.0238ms 248.5224 Ops/s 233.0222 Ops/s $\textbf{\color{#35bf28}+6.65\%}$
test_redq_deprec_speed[reduce-overhead-None] 15.5182ms 9.4777ms 105.5106 Ops/s 125.1071 Ops/s $\textbf{\color{#d91a1a}-15.66\%}$
test_td3_speed[False-None] 8.2591ms 8.1966ms 122.0019 Ops/s 120.2049 Ops/s $\color{#35bf28}+1.49\%$
test_td3_speed[False-backward] 11.1525ms 10.6578ms 93.8282 Ops/s 90.7330 Ops/s $\color{#35bf28}+3.41\%$
test_td3_speed[True-None] 1.6538ms 1.5856ms 630.6937 Ops/s 618.5029 Ops/s $\color{#35bf28}+1.97\%$
test_td3_speed[True-backward] 3.0723ms 2.9832ms 335.2072 Ops/s 308.5762 Ops/s $\textbf{\color{#35bf28}+8.63\%}$
test_td3_speed[reduce-overhead-None] 43.2566ms 22.2992ms 44.8447 Ops/s 42.5097 Ops/s $\textbf{\color{#35bf28}+5.49\%}$
test_cql_speed[False-None] 18.1403ms 17.1146ms 58.4295 Ops/s 57.8313 Ops/s $\color{#35bf28}+1.03\%$
test_cql_speed[False-backward] 24.6972ms 22.5324ms 44.3805 Ops/s 43.5173 Ops/s $\color{#35bf28}+1.98\%$
test_cql_speed[True-None] 3.2511ms 3.1387ms 318.6025 Ops/s 306.5573 Ops/s $\color{#35bf28}+3.93\%$
test_cql_speed[True-backward] 5.2714ms 5.1694ms 193.4443 Ops/s 184.2845 Ops/s $\color{#35bf28}+4.97\%$
test_cql_speed[reduce-overhead-None] 18.1270ms 11.3541ms 88.0737 Ops/s 86.5587 Ops/s $\color{#35bf28}+1.75\%$
test_a2c_speed[False-None] 4.1870ms 3.2316ms 309.4478 Ops/s 302.4723 Ops/s $\color{#35bf28}+2.31\%$
test_a2c_speed[False-backward] 6.6043ms 6.1951ms 161.4178 Ops/s 159.3054 Ops/s $\color{#35bf28}+1.33\%$
test_a2c_speed[True-None] 1.4002ms 1.2923ms 773.7929 Ops/s 764.4496 Ops/s $\color{#35bf28}+1.22\%$
test_a2c_speed[True-backward] 2.9292ms 2.8774ms 347.5404 Ops/s 316.3393 Ops/s $\textbf{\color{#35bf28}+9.86\%}$
test_a2c_speed[reduce-overhead-None] 1.0001ms 0.9443ms 1.0590 KOps/s 1.0352 KOps/s $\color{#35bf28}+2.29\%$
test_ppo_speed[False-None] 3.9512ms 3.8390ms 260.4818 Ops/s 257.3579 Ops/s $\color{#35bf28}+1.21\%$
test_ppo_speed[False-backward] 7.6674ms 7.0803ms 141.2379 Ops/s 136.0163 Ops/s $\color{#35bf28}+3.84\%$
test_ppo_speed[True-None] 1.6147ms 1.3556ms 737.6910 Ops/s 710.3940 Ops/s $\color{#35bf28}+3.84\%$
test_ppo_speed[True-backward] 3.1028ms 3.0371ms 329.2577 Ops/s 310.3751 Ops/s $\textbf{\color{#35bf28}+6.08\%}$
test_ppo_speed[reduce-overhead-None] 1.0988ms 1.0065ms 993.5781 Ops/s 958.8627 Ops/s $\color{#35bf28}+3.62\%$
test_reinforce_speed[False-None] 2.4325ms 2.2703ms 440.4722 Ops/s 433.7653 Ops/s $\color{#35bf28}+1.55\%$
test_reinforce_speed[False-backward] 3.3808ms 3.2940ms 303.5817 Ops/s 288.3068 Ops/s $\textbf{\color{#35bf28}+5.30\%}$
test_reinforce_speed[True-None] 1.2981ms 1.2193ms 820.1710 Ops/s 778.8409 Ops/s $\textbf{\color{#35bf28}+5.31\%}$
test_reinforce_speed[True-backward] 2.9234ms 2.8490ms 351.0057 Ops/s 330.4266 Ops/s $\textbf{\color{#35bf28}+6.23\%}$
test_reinforce_speed[reduce-overhead-None] 0.4915s 10.0185ms 99.8149 Ops/s 95.5629 Ops/s $\color{#35bf28}+4.45\%$
test_iql_speed[False-None] 10.3570ms 9.5120ms 105.1300 Ops/s 105.4301 Ops/s $\color{#d91a1a}-0.28\%$
test_iql_speed[False-backward] 13.8179ms 13.3756ms 74.7630 Ops/s 73.3113 Ops/s $\color{#35bf28}+1.98\%$
test_iql_speed[True-None] 2.2386ms 2.1264ms 470.2711 Ops/s 465.4379 Ops/s $\color{#35bf28}+1.04\%$
test_iql_speed[True-backward] 4.7154ms 4.6029ms 217.2555 Ops/s 213.4065 Ops/s $\color{#35bf28}+1.80\%$
test_iql_speed[reduce-overhead-None] 17.0691ms 9.9356ms 100.6487 Ops/s 75.1690 Ops/s $\textbf{\color{#35bf28}+33.90\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2464ms 5.7644ms 173.4797 Ops/s 168.3995 Ops/s $\color{#35bf28}+3.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4734ms 0.3935ms 2.5415 KOps/s 2.8730 KOps/s $\textbf{\color{#d91a1a}-11.54\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7949ms 0.3825ms 2.6146 KOps/s 2.8396 KOps/s $\textbf{\color{#d91a1a}-7.92\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0054ms 5.6943ms 175.6155 Ops/s 172.4006 Ops/s $\color{#35bf28}+1.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9980ms 0.3767ms 2.6544 KOps/s 3.3709 KOps/s $\textbf{\color{#d91a1a}-21.26\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7780ms 0.3572ms 2.7996 KOps/s 3.6618 KOps/s $\textbf{\color{#d91a1a}-23.54\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7084ms 1.4407ms 694.0864 Ops/s 734.3630 Ops/s $\textbf{\color{#d91a1a}-5.48\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4960ms 1.2628ms 791.9073 Ops/s 782.2614 Ops/s $\color{#35bf28}+1.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9952ms 5.8031ms 172.3225 Ops/s 168.5286 Ops/s $\color{#35bf28}+2.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8769ms 0.4933ms 2.0273 KOps/s 1.8718 KOps/s $\textbf{\color{#35bf28}+8.31\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6932ms 0.4408ms 2.2687 KOps/s 1.8863 KOps/s $\textbf{\color{#35bf28}+20.28\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8488ms 5.6486ms 177.0355 Ops/s 170.9385 Ops/s $\color{#35bf28}+3.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0224ms 0.3996ms 2.5024 KOps/s 3.0380 KOps/s $\textbf{\color{#d91a1a}-17.63\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5777ms 0.3511ms 2.8486 KOps/s 3.0080 KOps/s $\textbf{\color{#d91a1a}-5.30\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0552ms 5.6829ms 175.9661 Ops/s 173.1520 Ops/s $\color{#35bf28}+1.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0198ms 0.3643ms 2.7449 KOps/s 3.1253 KOps/s $\textbf{\color{#d91a1a}-12.17\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6211ms 0.3390ms 2.9499 KOps/s 3.1055 KOps/s $\textbf{\color{#d91a1a}-5.01\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.8752ms 5.7740ms 173.1887 Ops/s 168.9905 Ops/s $\color{#35bf28}+2.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9041ms 0.5398ms 1.8526 KOps/s 2.0693 KOps/s $\textbf{\color{#d91a1a}-10.47\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7896ms 0.5248ms 1.9055 KOps/s 2.1467 KOps/s $\textbf{\color{#d91a1a}-11.24\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.4786ms 5.0192ms 199.2352 Ops/s 45.5958 Ops/s $\textbf{\color{#35bf28}+336.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 4.0090ms 1.8136ms 551.4021 Ops/s 496.2379 Ops/s $\textbf{\color{#35bf28}+11.12\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.7760ms 0.9673ms 1.0338 KOps/s 1.0560 KOps/s $\color{#d91a1a}-2.10\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 8.8247ms 5.0037ms 199.8519 Ops/s 196.0295 Ops/s $\color{#35bf28}+1.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.5271ms 1.8990ms 526.6050 Ops/s 504.5114 Ops/s $\color{#35bf28}+4.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.0915ms 0.9583ms 1.0435 KOps/s 754.4027 Ops/s $\textbf{\color{#35bf28}+38.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.6333s 17.8238ms 56.1048 Ops/s 189.0048 Ops/s $\textbf{\color{#d91a1a}-70.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 4.1437ms 1.9932ms 501.6939 Ops/s 57.6174 Ops/s $\textbf{\color{#35bf28}+770.73\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9634ms 1.1445ms 873.7468 Ops/s 838.7676 Ops/s $\color{#35bf28}+4.17\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 39.8137ms 35.4011ms 28.2477 Ops/s 27.8003 Ops/s $\color{#35bf28}+1.61\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 20.6537ms 18.3531ms 54.4866 Ops/s 54.5182 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 40.0819ms 36.5971ms 27.3246 Ops/s 26.8125 Ops/s $\color{#35bf28}+1.91\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.3054ms 18.5267ms 53.9761 Ops/s 52.7802 Ops/s $\color{#35bf28}+2.27\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.5985ms 37.9383ms 26.3586 Ops/s 25.3403 Ops/s $\color{#35bf28}+4.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.9593ms 19.6134ms 50.9855 Ops/s 49.4477 Ops/s $\color{#35bf28}+3.11\%$

@vmoens
Update
7493a91 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jan 25, 2026
@vmoens
[Refactor] Move WEIGHT_SYNC_TIMEOUT to collectors._constants
3ceb6b9 
Move `WEIGHT_SYNC_TIMEOUT` from `torchrl._utils.py` to
`torchrl/collectors/_constants.py` to match the target branch.

ghstack-source-id: 5b47c5c
Pull-Request: #3329
@vmoens vmoens merged commit 7493a91 into gh/vmoens/197/base Jan 25, 2026
41 of 62 checks passed
@vmoens vmoens deleted the gh/vmoens/197/head branch January 25, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens