Skip to content

[Feature] Add RayEvalWorker for async evaluation #3474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 3 commits into from
Feb 10, 2026
Merged

[Feature] Add RayEvalWorker for async evaluation #3474

vmoens merged 3 commits into from
Feb 10, 2026

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Feb 10, 2026 •
edited
Loading

Stack from ghstack (oldest at bottom):

Add a new torchrl.eval module with RayEvalWorker, a Ray-backed async
evaluation helper that runs environment + policy in a separate process.
This is useful when the evaluation environment requires special
process-level initialisation (e.g. Isaac Lab AppLauncher) or when
evaluation should happen concurrently with training on a separate GPU.

API: submit(weights, max_steps) / poll() -> {reward, frames}.
Co-authored-by: Cursor [email protected]

@vmoens
Update
d7fe686 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Feb 10, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3474

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 49 Pending, 2 Unrelated Failures

As of commit aea4611 with merge base 0bc6d20 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions github-actions bot added Feature New feature Environments Adds or modifies an environment wrapper labels Feb 10, 2026
This was referenced Feb 10, 2026
Merged
Merged
Merged
Closed
Closed
Closed
Closed
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 10, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 10, 2026 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 172. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}15$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 81.9769μs 80.8689μs 12.3657 KOps/s 12.0089 KOps/s $\color{#35bf28}+2.97\%$
test_tensor_to_bytestream_speed[torch.save] 0.1423ms 0.1415ms 7.0687 KOps/s 7.2149 KOps/s $\color{#d91a1a}-2.03\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1325s 0.1323s 7.5580 Ops/s 8.7268 Ops/s $\textbf{\color{#d91a1a}-13.39\%}$
test_tensor_to_bytestream_speed[numpy] 2.8757μs 2.8698μs 348.4618 KOps/s 368.7303 KOps/s $\textbf{\color{#d91a1a}-5.50\%}$
test_tensor_to_bytestream_speed[safetensors] 36.8036μs 36.5951μs 27.3260 KOps/s 26.8610 KOps/s $\color{#35bf28}+1.73\%$
test_simple 0.8149s 0.8089s 1.2362 Ops/s 1.2109 Ops/s $\color{#35bf28}+2.09\%$
test_transformed 1.5661s 1.4696s 0.6805 Ops/s 0.6849 Ops/s $\color{#d91a1a}-0.64\%$
test_serial 2.4548s 2.3826s 0.4197 Ops/s 0.4281 Ops/s $\color{#d91a1a}-1.96\%$
test_parallel 1.9503s 1.8458s 0.5418 Ops/s 0.5541 Ops/s $\color{#d91a1a}-2.23\%$
test_step_mdp_speed[True-True-True-True-True] 0.4664ms 43.9521μs 22.7520 KOps/s 22.0977 KOps/s $\color{#35bf28}+2.96\%$
test_step_mdp_speed[True-True-True-True-False] 0.4342ms 24.9986μs 40.0022 KOps/s 39.4157 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-True-True-False-True] 68.7810μs 24.9248μs 40.1207 KOps/s 39.6503 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-True-True-False-False] 0.4432ms 13.8016μs 72.4553 KOps/s 72.6962 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-True-False-True-True] 0.4615ms 47.0182μs 21.2684 KOps/s 21.0863 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-False-True-False] 70.5410μs 27.5134μs 36.3459 KOps/s 36.1217 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-False-False-True] 0.4490ms 27.4421μs 36.4403 KOps/s 35.9320 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-False-False] 0.4342ms 16.7271μs 59.7832 KOps/s 60.0016 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[True-False-True-True-True] 0.4867ms 49.9749μs 20.0101 KOps/s 19.3808 KOps/s $\color{#35bf28}+3.25\%$
test_step_mdp_speed[True-False-True-True-False] 57.7110μs 30.2903μs 33.0139 KOps/s 32.1118 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-False-True-False-True] 0.4436ms 27.6463μs 36.1712 KOps/s 35.6245 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[True-False-True-False-False] 0.4495ms 16.4138μs 60.9245 KOps/s 60.1386 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[True-False-False-True-True] 90.3710μs 52.9057μs 18.9016 KOps/s 18.5422 KOps/s $\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-False-False-True-False] 0.4745ms 33.0839μs 30.2262 KOps/s 30.0007 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-False-True] 0.4470ms 30.0764μs 33.2486 KOps/s 32.6476 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[True-False-False-False-False] 0.4352ms 19.1267μs 52.2828 KOps/s 51.6060 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[False-True-True-True-True] 0.1204ms 50.0299μs 19.9880 KOps/s 19.8007 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-True-True-True-False] 0.4534ms 30.4570μs 32.8332 KOps/s 32.3959 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-True-False-True] 2.3366ms 32.2133μs 31.0431 KOps/s 31.2795 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-True-True-False-False] 0.4339ms 18.2096μs 54.9160 KOps/s 53.5488 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-True-False-True-True] 0.1573ms 52.5375μs 19.0340 KOps/s 18.8819 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-True-False-True-False] 0.4566ms 33.1648μs 30.1524 KOps/s 29.8591 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[False-True-False-False-True] 0.4523ms 33.5150μs 29.8374 KOps/s 28.8798 KOps/s $\color{#35bf28}+3.32\%$
test_step_mdp_speed[False-True-False-False-False] 58.7010μs 20.8204μs 48.0298 KOps/s 47.5502 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[False-False-True-True-True] 0.4806ms 55.2377μs 18.1036 KOps/s 17.8626 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-False-True-True-False] 0.4630ms 36.2054μs 27.6202 KOps/s 27.3969 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-False-True-False-True] 0.4510ms 33.8236μs 29.5651 KOps/s 29.0473 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-False-True-False-False] 53.4910μs 20.6595μs 48.4038 KOps/s 47.3841 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[False-False-False-True-True] 0.4763ms 57.5460μs 17.3774 KOps/s 17.2826 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-False-True-False] 0.4612ms 38.8376μs 25.7483 KOps/s 25.5751 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-False-False-False-True] 0.4592ms 36.5713μs 27.3438 KOps/s 27.0266 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-False-False-False] 0.4331ms 22.9381μs 43.5956 KOps/s 42.6350 KOps/s $\color{#35bf28}+2.25\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8788s 0.7794s 1.2830 Ops/s 1.2918 Ops/s $\color{#d91a1a}-0.68\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7337s 0.6422s 1.5573 Ops/s 1.5644 Ops/s $\color{#d91a1a}-0.46\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7721s 1.6960s 0.5896 Ops/s 0.5933 Ops/s $\color{#d91a1a}-0.62\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5443s 1.4662s 0.6820 Ops/s 0.6817 Ops/s $\color{#35bf28}+0.05\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 2.0238s 1.9486s 0.5132 Ops/s 0.5153 Ops/s $\color{#d91a1a}-0.41\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.8030s 1.7241s 0.5800 Ops/s 0.5831 Ops/s $\color{#d91a1a}-0.53\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.8284s 4.7535s 0.2104 Ops/s 0.2144 Ops/s $\color{#d91a1a}-1.87\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.7006s 4.5351s 0.2205 Ops/s 0.2259 Ops/s $\color{#d91a1a}-2.40\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 1.9711s 1.8870s 0.5299 Ops/s 0.5256 Ops/s $\color{#35bf28}+0.83\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.6933s 1.6102s 0.6211 Ops/s 0.6178 Ops/s $\color{#35bf28}+0.53\%$
test_values[generalized_advantage_estimate-True-True] 22.2308ms 21.5122ms 46.4852 Ops/s 47.4663 Ops/s $\color{#d91a1a}-2.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1429s 3.7950ms 263.5045 Ops/s 273.7908 Ops/s $\color{#d91a1a}-3.76\%$
test_values[td0_return_estimate-False-False] 0.1099ms 85.3325μs 11.7189 KOps/s 11.8046 KOps/s $\color{#d91a1a}-0.73\%$
test_values[td1_return_estimate-False-False] 53.2179ms 51.2938ms 19.4955 Ops/s 19.9653 Ops/s $\color{#d91a1a}-2.35\%$
test_values[vec_td1_return_estimate-False-False] 1.3348ms 1.1138ms 897.8271 Ops/s 898.9621 Ops/s $\color{#d91a1a}-0.13\%$
test_values[td_lambda_return_estimate-True-False] 86.9669ms 84.0206ms 11.9018 Ops/s 11.6367 Ops/s $\color{#35bf28}+2.28\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2561ms 1.1066ms 903.6628 Ops/s 904.4822 Ops/s $\color{#d91a1a}-0.09\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.7164ms 21.8113ms 45.8477 Ops/s 45.3516 Ops/s $\color{#35bf28}+1.09\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0657ms 0.7892ms 1.2670 KOps/s 1.2889 KOps/s $\color{#d91a1a}-1.70\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7454ms 0.6987ms 1.4313 KOps/s 1.4395 KOps/s $\color{#d91a1a}-0.57\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6186ms 1.5156ms 659.8010 Ops/s 664.5600 Ops/s $\color{#d91a1a}-0.72\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9301ms 0.7140ms 1.4005 KOps/s 1.3779 KOps/s $\color{#35bf28}+1.64\%$
test_dqn_speed[False-None] 1.6563ms 1.5696ms 637.1050 Ops/s 629.6883 Ops/s $\color{#35bf28}+1.18\%$
test_dqn_speed[False-backward] 2.4198ms 2.2369ms 447.0559 Ops/s 450.3627 Ops/s $\color{#d91a1a}-0.73\%$
test_dqn_speed[True-None] 1.0891ms 0.5800ms 1.7242 KOps/s 1.6891 KOps/s $\color{#35bf28}+2.08\%$
test_dqn_speed[True-backward] 1.1758ms 1.1206ms 892.4132 Ops/s 890.4646 Ops/s $\color{#35bf28}+0.22\%$
test_dqn_speed[reduce-overhead-None] 1.0296ms 0.5954ms 1.6796 KOps/s 1.6240 KOps/s $\color{#35bf28}+3.42\%$
test_ddpg_speed[False-None] 3.4082ms 2.9669ms 337.0533 Ops/s 335.4425 Ops/s $\color{#35bf28}+0.48\%$
test_ddpg_speed[False-backward] 4.7350ms 4.3057ms 232.2512 Ops/s 234.4162 Ops/s $\color{#d91a1a}-0.92\%$
test_ddpg_speed[True-None] 1.8482ms 1.3515ms 739.9151 Ops/s 750.2410 Ops/s $\color{#d91a1a}-1.38\%$
test_ddpg_speed[True-backward] 2.4933ms 2.4414ms 409.6045 Ops/s 390.0619 Ops/s $\textbf{\color{#35bf28}+5.01\%}$
test_ddpg_speed[reduce-overhead-None] 1.5664ms 1.3754ms 727.0514 Ops/s 731.6619 Ops/s $\color{#d91a1a}-0.63\%$
test_sac_speed[False-None] 9.1814ms 8.5266ms 117.2801 Ops/s 114.2656 Ops/s $\color{#35bf28}+2.64\%$
test_sac_speed[False-backward] 12.2121ms 11.6224ms 86.0410 Ops/s 84.3811 Ops/s $\color{#35bf28}+1.97\%$
test_sac_speed[True-None] 2.3660ms 1.8585ms 538.0641 Ops/s 534.8735 Ops/s $\color{#35bf28}+0.60\%$
test_sac_speed[True-backward] 3.6309ms 3.5539ms 281.3836 Ops/s 276.4775 Ops/s $\color{#35bf28}+1.77\%$
test_sac_speed[reduce-overhead-None] 0.3646s 11.9977ms 83.3496 Ops/s 92.5600 Ops/s $\textbf{\color{#d91a1a}-9.95\%}$
test_redq_deprec_speed[False-None] 10.0759ms 9.5394ms 104.8283 Ops/s 104.2608 Ops/s $\color{#35bf28}+0.54\%$
test_redq_deprec_speed[False-backward] 13.7754ms 13.0036ms 76.9019 Ops/s 77.5633 Ops/s $\color{#d91a1a}-0.85\%$
test_redq_deprec_speed[True-None] 3.0078ms 2.5953ms 385.3180 Ops/s 385.3319 Ops/s $-0.00\%$
test_redq_deprec_speed[True-backward] 4.4871ms 4.2603ms 234.7263 Ops/s 227.2466 Ops/s $\color{#35bf28}+3.29\%$
test_redq_deprec_speed[reduce-overhead-None] 16.1565ms 9.8094ms 101.9434 Ops/s 101.7410 Ops/s $\color{#35bf28}+0.20\%$
test_td3_speed[False-None] 8.5275ms 8.3745ms 119.4104 Ops/s 119.1945 Ops/s $\color{#35bf28}+0.18\%$
test_td3_speed[False-backward] 11.1986ms 10.8224ms 92.4008 Ops/s 89.9718 Ops/s $\color{#35bf28}+2.70\%$
test_td3_speed[True-None] 1.8022ms 1.7690ms 565.2997 Ops/s 576.7795 Ops/s $\color{#d91a1a}-1.99\%$
test_td3_speed[True-backward] 3.3830ms 3.2065ms 311.8627 Ops/s 302.1374 Ops/s $\color{#35bf28}+3.22\%$
test_td3_speed[reduce-overhead-None] 46.2197ms 24.0339ms 41.6079 Ops/s 40.3222 Ops/s $\color{#35bf28}+3.19\%$
test_cql_speed[False-None] 18.5326ms 17.9064ms 55.8461 Ops/s 56.8281 Ops/s $\color{#d91a1a}-1.73\%$
test_cql_speed[False-backward] 23.9691ms 23.1380ms 43.2190 Ops/s 43.0800 Ops/s $\color{#35bf28}+0.32\%$
test_cql_speed[True-None] 3.7129ms 3.4741ms 287.8453 Ops/s 302.4260 Ops/s $\color{#d91a1a}-4.82\%$
test_cql_speed[True-backward] 5.7803ms 5.4998ms 181.8242 Ops/s 182.8536 Ops/s $\color{#d91a1a}-0.56\%$
test_cql_speed[reduce-overhead-None] 19.0647ms 11.9561ms 83.6391 Ops/s 84.3155 Ops/s $\color{#d91a1a}-0.80\%$
test_a2c_speed[False-None] 3.9794ms 3.3424ms 299.1830 Ops/s 296.5302 Ops/s $\color{#35bf28}+0.89\%$
test_a2c_speed[False-backward] 6.4723ms 6.3413ms 157.6960 Ops/s 157.2831 Ops/s $\color{#35bf28}+0.26\%$
test_a2c_speed[True-None] 1.5422ms 1.3657ms 732.1994 Ops/s 732.8924 Ops/s $\color{#d91a1a}-0.09\%$
test_a2c_speed[True-backward] 3.1948ms 3.0773ms 324.9553 Ops/s 331.6855 Ops/s $\color{#d91a1a}-2.03\%$
test_a2c_speed[reduce-overhead-None] 1.4215ms 1.0009ms 999.0610 Ops/s 1.0032 KOps/s $\color{#d91a1a}-0.41\%$
test_ppo_speed[False-None] 4.2596ms 4.0562ms 246.5388 Ops/s 254.5638 Ops/s $\color{#d91a1a}-3.15\%$
test_ppo_speed[False-backward] 7.7005ms 7.2125ms 138.6487 Ops/s 138.2809 Ops/s $\color{#35bf28}+0.27\%$
test_ppo_speed[True-None] 2.0183ms 1.4626ms 683.7015 Ops/s 687.8051 Ops/s $\color{#d91a1a}-0.60\%$
test_ppo_speed[True-backward] 3.3274ms 3.1613ms 316.3275 Ops/s 318.8559 Ops/s $\color{#d91a1a}-0.79\%$
test_ppo_speed[reduce-overhead-None] 1.2272ms 1.0546ms 948.1852 Ops/s 915.4033 Ops/s $\color{#35bf28}+3.58\%$
test_reinforce_speed[False-None] 2.4911ms 2.3444ms 426.5433 Ops/s 429.6443 Ops/s $\color{#d91a1a}-0.72\%$
test_reinforce_speed[False-backward] 3.9665ms 3.5407ms 282.4287 Ops/s 286.9963 Ops/s $\color{#d91a1a}-1.59\%$
test_reinforce_speed[True-None] 1.5050ms 1.3202ms 757.4736 Ops/s 775.8014 Ops/s $\color{#d91a1a}-2.36\%$
test_reinforce_speed[True-backward] 3.1645ms 3.1062ms 321.9369 Ops/s 333.7767 Ops/s $\color{#d91a1a}-3.55\%$
test_reinforce_speed[reduce-overhead-None] 17.5809ms 9.4873ms 105.4040 Ops/s 105.7394 Ops/s $\color{#d91a1a}-0.32\%$
test_iql_speed[False-None] 11.2820ms 9.6434ms 103.6982 Ops/s 104.3008 Ops/s $\color{#d91a1a}-0.58\%$
test_iql_speed[False-backward] 14.1041ms 13.4338ms 74.4389 Ops/s 74.6375 Ops/s $\color{#d91a1a}-0.27\%$
test_iql_speed[True-None] 2.4243ms 2.2331ms 447.8051 Ops/s 450.4622 Ops/s $\color{#d91a1a}-0.59\%$
test_iql_speed[True-backward] 5.5118ms 5.0281ms 198.8816 Ops/s 199.4163 Ops/s $\color{#d91a1a}-0.27\%$
test_iql_speed[reduce-overhead-None] 17.5287ms 10.3599ms 96.5257 Ops/s 74.8906 Ops/s $\textbf{\color{#35bf28}+28.89\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.5375ms 6.1041ms 163.8248 Ops/s 165.1383 Ops/s $\color{#d91a1a}-0.80\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7178ms 0.3375ms 2.9630 KOps/s 3.5090 KOps/s $\textbf{\color{#d91a1a}-15.56\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6009ms 0.3179ms 3.1452 KOps/s 3.3542 KOps/s $\textbf{\color{#d91a1a}-6.23\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1508ms 5.8257ms 171.6522 Ops/s 169.2171 Ops/s $\color{#35bf28}+1.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7907ms 0.3461ms 2.8890 KOps/s 3.1027 KOps/s $\textbf{\color{#d91a1a}-6.89\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6643ms 0.3227ms 3.0986 KOps/s 3.3375 KOps/s $\textbf{\color{#d91a1a}-7.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7123ms 1.4605ms 684.7111 Ops/s 751.7490 Ops/s $\textbf{\color{#d91a1a}-8.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6225ms 1.4052ms 711.6562 Ops/s 812.2612 Ops/s $\textbf{\color{#d91a1a}-12.39\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2985ms 6.0205ms 166.1001 Ops/s 164.2331 Ops/s $\color{#35bf28}+1.14\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0452ms 0.5170ms 1.9344 KOps/s 2.1430 KOps/s $\textbf{\color{#d91a1a}-9.74\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7624ms 0.4968ms 2.0129 KOps/s 2.2559 KOps/s $\textbf{\color{#d91a1a}-10.77\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1033ms 5.9296ms 168.6458 Ops/s 167.2034 Ops/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.9913ms 0.3399ms 2.9421 KOps/s 3.1933 KOps/s $\textbf{\color{#d91a1a}-7.87\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5992ms 0.3045ms 3.2840 KOps/s 3.2547 KOps/s $\color{#35bf28}+0.90\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0652ms 5.8328ms 171.4456 Ops/s 169.8575 Ops/s $\color{#35bf28}+0.93\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5349ms 0.2799ms 3.5733 KOps/s 3.1625 KOps/s $\textbf{\color{#35bf28}+12.99\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4949ms 0.2933ms 3.4089 KOps/s 2.9260 KOps/s $\textbf{\color{#35bf28}+16.51\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.1455ms 6.0182ms 166.1616 Ops/s 163.9066 Ops/s $\color{#35bf28}+1.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7531ms 0.4359ms 2.2940 KOps/s 2.0035 KOps/s $\textbf{\color{#35bf28}+14.50\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6325ms 0.4176ms 2.3949 KOps/s 2.1835 KOps/s $\textbf{\color{#35bf28}+9.68\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6038ms 5.1282ms 195.0018 Ops/s 50.8316 Ops/s $\textbf{\color{#35bf28}+283.62\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 4.0005ms 1.8453ms 541.9284 Ops/s 521.3251 Ops/s $\color{#35bf28}+3.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 3.4696ms 0.9799ms 1.0206 KOps/s 862.7950 Ops/s $\textbf{\color{#35bf28}+18.28\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.8118ms 5.1615ms 193.7409 Ops/s 193.8757 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 4.0500ms 1.8572ms 538.4587 Ops/s 538.4296 Ops/s $+0.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.1354ms 1.2529ms 798.1584 Ops/s 1.0564 KOps/s $\textbf{\color{#d91a1a}-24.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5913s 17.1578ms 58.2827 Ops/s 185.1511 Ops/s $\textbf{\color{#d91a1a}-68.52\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 4.0378ms 2.0612ms 485.1612 Ops/s 467.7025 Ops/s $\color{#35bf28}+3.73\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.4263ms 1.1482ms 870.8968 Ops/s 777.1887 Ops/s $\textbf{\color{#35bf28}+12.06\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 39.3183ms 36.6041ms 27.3193 Ops/s 27.2993 Ops/s $\color{#35bf28}+0.07\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 20.0378ms 18.5187ms 53.9994 Ops/s 53.9267 Ops/s $\color{#35bf28}+0.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 41.1496ms 37.8425ms 26.4253 Ops/s 25.6524 Ops/s $\color{#35bf28}+3.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 21.2860ms 19.0557ms 52.4778 Ops/s 53.3197 Ops/s $\color{#d91a1a}-1.58\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 41.6211ms 39.7161ms 25.1787 Ops/s 25.1546 Ops/s $\color{#35bf28}+0.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.2471ms 20.2772ms 49.3165 Ops/s 48.7544 Ops/s $\color{#35bf28}+1.15\%$
test_storage_write_lazystack[50-img_shape0-small] 0.9090ms 0.2302ms 4.3436 KOps/s 4.3562 KOps/s $\color{#d91a1a}-0.29\%$
test_storage_write_lazystack[100-img_shape1-atari] 1.6633ms 1.3404ms 746.0631 Ops/s 714.3265 Ops/s $\color{#35bf28}+4.44\%$
test_storage_write_lazystack[100-img_shape2-large_img] 2.5802ms 2.2512ms 444.2173 Ops/s 414.6945 Ops/s $\textbf{\color{#35bf28}+7.12\%}$
test_storage_write_lazystack[200-img_shape3-large_batch] 3.0619ms 2.8845ms 346.6787 Ops/s 340.0792 Ops/s $\color{#35bf28}+1.94\%$
test_storage_write_contiguous[50-img_shape0-small] 0.4806ms 0.1648ms 6.0673 KOps/s 6.0989 KOps/s $\color{#d91a1a}-0.52\%$
test_storage_write_contiguous[100-img_shape1-atari] 0.3849ms 0.2371ms 4.2173 KOps/s 3.3284 KOps/s $\textbf{\color{#35bf28}+26.71\%}$
test_storage_write_contiguous[100-img_shape2-large_img] 2.1009ms 1.7992ms 555.7924 Ops/s 561.3725 Ops/s $\color{#d91a1a}-0.99\%$
test_storage_write_contiguous[200-img_shape3-large_batch] 1.6244ms 1.4200ms 704.2449 Ops/s 725.6548 Ops/s $\color{#d91a1a}-2.95\%$
test_collector_stack_then_write[50-img_shape0-small] 1.3483ms 1.1697ms 854.9213 Ops/s 864.5450 Ops/s $\color{#d91a1a}-1.11\%$
test_collector_stack_then_write[100-img_shape1-atari] 3.7119ms 3.5723ms 279.9308 Ops/s 273.2060 Ops/s $\color{#35bf28}+2.46\%$
test_collector_stack_then_write[100-img_shape2-large_img] 6.2018ms 5.7877ms 172.7790 Ops/s 176.0931 Ops/s $\color{#d91a1a}-1.88\%$
test_collector_stack_then_write[200-img_shape3-large_batch] 7.4799ms 7.3072ms 136.8518 Ops/s 141.5441 Ops/s $\color{#d91a1a}-3.32\%$
test_collector_lazystack_then_write[50-img_shape0-small] 0.4188ms 0.2736ms 3.6549 KOps/s 3.5803 KOps/s $\color{#35bf28}+2.08\%$
test_collector_lazystack_then_write[100-img_shape1-atari] 1.7594ms 1.4391ms 694.8806 Ops/s 659.5613 Ops/s $\textbf{\color{#35bf28}+5.35\%}$
test_collector_lazystack_then_write[100-img_shape2-large_img] 2.7802ms 2.3560ms 424.4423 Ops/s 395.7514 Ops/s $\textbf{\color{#35bf28}+7.25\%}$
test_collector_lazystack_then_write[200-img_shape3-large_batch] 3.2526ms 3.0815ms 324.5131 Ops/s 317.0295 Ops/s $\color{#35bf28}+2.36\%$
test_collector_without_rb[100-img_shape0-atari] 35.1059ms 34.6512ms 28.8590 Ops/s 28.8345 Ops/s $\color{#35bf28}+0.08\%$
test_collector_without_rb[200-img_shape1-large_batch] 68.6148ms 68.0988ms 14.6846 Ops/s 14.6146 Ops/s $\color{#35bf28}+0.48\%$
test_collector_with_rb[100-img_shape0-atari] 39.7943ms 39.2632ms 25.4691 Ops/s 25.5007 Ops/s $\color{#d91a1a}-0.12\%$
test_collector_with_rb[200-img_shape1-large_batch] 78.2143ms 77.4644ms 12.9091 Ops/s 12.9841 Ops/s $\color{#d91a1a}-0.58\%$
test_collector_without_rb_cuda[100-img_shape0-atari] 0.7564s 96.2918ms 10.3851 Ops/s 17.2823 Ops/s $\textbf{\color{#d91a1a}-39.91\%}$
test_collector_without_rb_cuda[200-img_shape1-large_batch] 0.1151s 0.1146s 8.7229 Ops/s 8.7649 Ops/s $\color{#d91a1a}-0.48\%$
test_collector_with_rb_cuda[100-img_shape0-atari] 59.9700ms 59.6152ms 16.7742 Ops/s 16.8885 Ops/s $\color{#d91a1a}-0.68\%$
test_collector_with_rb_cuda[200-img_shape1-large_batch] 0.1194s 0.1187s 8.4272 Ops/s 8.3538 Ops/s $\color{#35bf28}+0.88\%$

@github-actions
Copy link
Contributor

github-actions bot commented Feb 10, 2026 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 173. Improved: $\large\color{#35bf28}18$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 81.9989μs 80.2280μs 12.4645 KOps/s 12.5974 KOps/s $\color{#d91a1a}-1.06\%$
test_tensor_to_bytestream_speed[torch.save] 0.1374ms 0.1367ms 7.3174 KOps/s 7.2827 KOps/s $\color{#35bf28}+0.48\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1139s 0.1137s 8.7974 Ops/s 9.7979 Ops/s $\textbf{\color{#d91a1a}-10.21\%}$
test_tensor_to_bytestream_speed[numpy] 2.4255μs 2.4184μs 413.4901 KOps/s 411.1800 KOps/s $\color{#35bf28}+0.56\%$
test_tensor_to_bytestream_speed[safetensors] 36.7826μs 36.5642μs 27.3492 KOps/s 26.3514 KOps/s $\color{#35bf28}+3.79\%$
test_simple 0.5431s 0.5422s 1.8442 Ops/s 1.7329 Ops/s $\textbf{\color{#35bf28}+6.42\%}$
test_transformed 1.1158s 1.1135s 0.8981 Ops/s 0.8633 Ops/s $\color{#35bf28}+4.03\%$
test_serial 1.6422s 1.6375s 0.6107 Ops/s 0.5991 Ops/s $\color{#35bf28}+1.93\%$
test_parallel 1.1211s 1.0287s 0.9721 Ops/s 0.9708 Ops/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-True-True-True-True] 0.1353ms 43.7479μs 22.8582 KOps/s 23.1552 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-True-True-True-False] 56.0230μs 25.1905μs 39.6975 KOps/s 40.7292 KOps/s $\color{#d91a1a}-2.53\%$
test_step_mdp_speed[True-True-True-False-True] 51.1840μs 24.6206μs 40.6164 KOps/s 40.8946 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-True-False-False] 36.7720μs 13.2535μs 75.4519 KOps/s 73.6706 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-True-False-True-True] 75.7050μs 45.5997μs 21.9300 KOps/s 20.9158 KOps/s $\color{#35bf28}+4.85\%$
test_step_mdp_speed[True-True-False-True-False] 79.1260μs 26.5906μs 37.6073 KOps/s 36.5093 KOps/s $\color{#35bf28}+3.01\%$
test_step_mdp_speed[True-True-False-False-True] 52.5040μs 26.5799μs 37.6224 KOps/s 35.9879 KOps/s $\color{#35bf28}+4.54\%$
test_step_mdp_speed[True-True-False-False-False] 47.4730μs 15.9114μs 62.8479 KOps/s 61.5721 KOps/s $\color{#35bf28}+2.07\%$
test_step_mdp_speed[True-False-True-True-True] 81.2260μs 49.7321μs 20.1077 KOps/s 20.0298 KOps/s $\color{#35bf28}+0.39\%$
test_step_mdp_speed[True-False-True-True-False] 59.5240μs 29.2613μs 34.1748 KOps/s 33.8901 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-False-True-False-True] 57.4240μs 26.1403μs 38.2552 KOps/s 36.2986 KOps/s $\textbf{\color{#35bf28}+5.39\%}$
test_step_mdp_speed[True-False-True-False-False] 38.6530μs 16.2071μs 61.7013 KOps/s 61.7095 KOps/s $\color{#d91a1a}-0.01\%$
test_step_mdp_speed[True-False-False-True-True] 79.4460μs 50.6399μs 19.7473 KOps/s 19.0115 KOps/s $\color{#35bf28}+3.87\%$
test_step_mdp_speed[True-False-False-True-False] 55.7830μs 31.5873μs 31.6583 KOps/s 31.1589 KOps/s $\color{#35bf28}+1.60\%$
test_step_mdp_speed[True-False-False-False-True] 58.2830μs 28.9370μs 34.5578 KOps/s 33.4169 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[True-False-False-False-False] 41.1330μs 18.3461μs 54.5076 KOps/s 53.7603 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[False-True-True-True-True] 75.8050μs 48.3351μs 20.6889 KOps/s 20.5911 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-True-True-True-False] 58.7440μs 29.4829μs 33.9179 KOps/s 33.8981 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-True-False-True] 2.4347ms 30.8747μs 32.3890 KOps/s 31.7692 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-True-True-False-False] 45.4730μs 18.0164μs 55.5050 KOps/s 57.1976 KOps/s $\color{#d91a1a}-2.96\%$
test_step_mdp_speed[False-True-False-True-True] 78.5150μs 50.8515μs 19.6651 KOps/s 19.5670 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-True-False-True-False] 58.7240μs 32.8359μs 30.4544 KOps/s 30.8529 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-True-False-False-True] 69.0650μs 33.3851μs 29.9535 KOps/s 30.5354 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-True-False-False-False] 54.5340μs 20.2135μs 49.4719 KOps/s 49.6708 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-False-True-True-True] 84.9750μs 54.4522μs 18.3647 KOps/s 18.4599 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-False-True-True-False] 63.7850μs 35.3415μs 28.2953 KOps/s 28.5511 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-True-False-True] 57.7840μs 33.5917μs 29.7693 KOps/s 30.3069 KOps/s $\color{#d91a1a}-1.77\%$
test_step_mdp_speed[False-False-True-False-False] 49.0830μs 20.0443μs 49.8896 KOps/s 49.3651 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-False-True-True] 91.4360μs 55.6880μs 17.9572 KOps/s 17.8145 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-False-True-False] 0.1047ms 36.9870μs 27.0366 KOps/s 27.1225 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-False-False-False-True] 62.2950μs 35.2932μs 28.3341 KOps/s 29.0208 KOps/s $\color{#d91a1a}-2.37\%$
test_step_mdp_speed[False-False-False-False-False] 53.2530μs 22.5009μs 44.4427 KOps/s 44.2134 KOps/s $\color{#35bf28}+0.52\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.7373s 0.7356s 1.3593 Ops/s 1.3045 Ops/s $\color{#35bf28}+4.20\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7226s 0.6272s 1.5943 Ops/s 1.5882 Ops/s $\color{#35bf28}+0.39\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7298s 1.6478s 0.6069 Ops/s 0.5987 Ops/s $\color{#35bf28}+1.37\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5024s 1.4263s 0.7011 Ops/s 0.6890 Ops/s $\color{#35bf28}+1.75\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9702s 1.8930s 0.5283 Ops/s 0.5177 Ops/s $\color{#35bf28}+2.03\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7483s 1.6675s 0.5997 Ops/s 0.5899 Ops/s $\color{#35bf28}+1.66\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6614s 4.5885s 0.2179 Ops/s 0.2139 Ops/s $\color{#35bf28}+1.89\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5014s 4.4147s 0.2265 Ops/s 0.2221 Ops/s $\color{#35bf28}+2.00\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0043s 1.8860s 0.5302 Ops/s 0.5122 Ops/s $\color{#35bf28}+3.51\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7780s 1.6211s 0.6169 Ops/s 0.6182 Ops/s $\color{#d91a1a}-0.22\%$
test_values[generalized_advantage_estimate-True-True] 10.7896ms 10.3894ms 96.2517 Ops/s 99.4265 Ops/s $\color{#d91a1a}-3.19\%$
test_values[vec_generalized_advantage_estimate-True-True] 18.6701ms 12.4022ms 80.6308 Ops/s 89.6302 Ops/s $\textbf{\color{#d91a1a}-10.04\%}$
test_values[td0_return_estimate-False-False] 0.2222ms 0.1260ms 7.9392 KOps/s 7.8331 KOps/s $\color{#35bf28}+1.35\%$
test_values[td1_return_estimate-False-False] 29.8387ms 28.6601ms 34.8917 Ops/s 36.7770 Ops/s $\textbf{\color{#d91a1a}-5.13\%}$
test_values[vec_td1_return_estimate-False-False] 17.9648ms 12.4276ms 80.4661 Ops/s 88.6835 Ops/s $\textbf{\color{#d91a1a}-9.27\%}$
test_values[td_lambda_return_estimate-True-False] 44.6037ms 42.5025ms 23.5280 Ops/s 25.2944 Ops/s $\textbf{\color{#d91a1a}-6.98\%}$
test_values[vec_td_lambda_return_estimate-True-False] 17.8024ms 12.3903ms 80.7086 Ops/s 88.8958 Ops/s $\textbf{\color{#d91a1a}-9.21\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.5096ms 9.2371ms 108.2587 Ops/s 111.9840 Ops/s $\color{#d91a1a}-3.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9047ms 1.4882ms 671.9468 Ops/s 671.0074 Ops/s $\color{#35bf28}+0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8553ms 0.4219ms 2.3703 KOps/s 2.3827 KOps/s $\color{#d91a1a}-0.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.9798ms 31.8798ms 31.3678 Ops/s 41.1416 Ops/s $\textbf{\color{#d91a1a}-23.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8749ms 1.7282ms 578.6293 Ops/s 576.4437 Ops/s $\color{#35bf28}+0.38\%$
test_dqn_speed[False-None] 1.8446ms 1.4189ms 704.7731 Ops/s 719.7198 Ops/s $\color{#d91a1a}-2.08\%$
test_dqn_speed[False-backward] 2.1137ms 1.9772ms 505.7765 Ops/s 530.3149 Ops/s $\color{#d91a1a}-4.63\%$
test_dqn_speed[True-None] 0.6869ms 0.5648ms 1.7707 KOps/s 1.8075 KOps/s $\color{#d91a1a}-2.04\%$
test_dqn_speed[True-backward] 1.0985ms 1.0364ms 964.8347 Ops/s 898.0378 Ops/s $\textbf{\color{#35bf28}+7.44\%}$
test_dqn_speed[reduce-overhead-None] 0.9463ms 0.5548ms 1.8026 KOps/s 1.7625 KOps/s $\color{#35bf28}+2.27\%$
test_ddpg_speed[False-None] 3.3493ms 2.9253ms 341.8415 Ops/s 353.2847 Ops/s $\color{#d91a1a}-3.24\%$
test_ddpg_speed[False-backward] 4.3693ms 4.1541ms 240.7250 Ops/s 247.5485 Ops/s $\color{#d91a1a}-2.76\%$
test_ddpg_speed[True-None] 1.5635ms 1.4392ms 694.8244 Ops/s 705.8520 Ops/s $\color{#d91a1a}-1.56\%$
test_ddpg_speed[True-backward] 2.5420ms 2.4115ms 414.6877 Ops/s 376.3719 Ops/s $\textbf{\color{#35bf28}+10.18\%}$
test_ddpg_speed[reduce-overhead-None] 1.8378ms 1.4048ms 711.8539 Ops/s 698.7818 Ops/s $\color{#35bf28}+1.87\%$
test_sac_speed[False-None] 8.6149ms 8.0500ms 124.2235 Ops/s 124.2317 Ops/s $-0.01\%$
test_sac_speed[False-backward] 12.0082ms 11.2942ms 88.5411 Ops/s 87.3832 Ops/s $\color{#35bf28}+1.33\%$
test_sac_speed[True-None] 2.3409ms 2.1563ms 463.7622 Ops/s 463.0815 Ops/s $\color{#35bf28}+0.15\%$
test_sac_speed[True-backward] 4.1885ms 4.0514ms 246.8291 Ops/s 243.3735 Ops/s $\color{#35bf28}+1.42\%$
test_sac_speed[reduce-overhead-None] 2.2967ms 2.1416ms 466.9320 Ops/s 463.7961 Ops/s $\color{#35bf28}+0.68\%$
test_redq_speed[False-None] 10.8817ms 10.3849ms 96.2939 Ops/s 95.3526 Ops/s $\color{#35bf28}+0.99\%$
test_redq_speed[False-backward] 18.5702ms 17.8456ms 56.0363 Ops/s 55.3952 Ops/s $\color{#35bf28}+1.16\%$
test_redq_speed[True-None] 4.7579ms 4.5097ms 221.7446 Ops/s 209.2177 Ops/s $\textbf{\color{#35bf28}+5.99\%}$
test_redq_speed[True-backward] 10.1357ms 9.8106ms 101.9304 Ops/s 102.1605 Ops/s $\color{#d91a1a}-0.23\%$
test_redq_speed[reduce-overhead-None] 5.0901ms 4.3920ms 227.6874 Ops/s 226.8432 Ops/s $\color{#35bf28}+0.37\%$
test_redq_deprec_speed[False-None] 11.7068ms 11.1787ms 89.4561 Ops/s 91.3343 Ops/s $\color{#d91a1a}-2.06\%$
test_redq_deprec_speed[False-backward] 0.3917s 23.3734ms 42.7837 Ops/s 63.5840 Ops/s $\textbf{\color{#d91a1a}-32.71\%}$
test_redq_deprec_speed[True-None] 3.9501ms 3.6786ms 271.8426 Ops/s 269.3591 Ops/s $\color{#35bf28}+0.92\%$
test_redq_deprec_speed[True-backward] 8.0407ms 7.6490ms 130.7367 Ops/s 126.0713 Ops/s $\color{#35bf28}+3.70\%$
test_redq_deprec_speed[reduce-overhead-None] 3.7698ms 3.5809ms 279.2620 Ops/s 269.4610 Ops/s $\color{#35bf28}+3.64\%$
test_td3_speed[False-None] 48.6199ms 8.3499ms 119.7615 Ops/s 125.0312 Ops/s $\color{#d91a1a}-4.21\%$
test_td3_speed[False-backward] 11.1782ms 10.9053ms 91.6982 Ops/s 91.9373 Ops/s $\color{#d91a1a}-0.26\%$
test_td3_speed[True-None] 1.9915ms 1.8738ms 533.6628 Ops/s 541.4065 Ops/s $\color{#d91a1a}-1.43\%$
test_td3_speed[True-backward] 3.8675ms 3.6895ms 271.0376 Ops/s 270.4698 Ops/s $\color{#35bf28}+0.21\%$
test_td3_speed[reduce-overhead-None] 1.8866ms 1.8132ms 551.5255 Ops/s 543.7613 Ops/s $\color{#35bf28}+1.43\%$
test_cql_speed[False-None] 29.0910ms 26.0665ms 38.3633 Ops/s 37.6762 Ops/s $\color{#35bf28}+1.82\%$
test_cql_speed[False-backward] 40.7216ms 36.8921ms 27.1061 Ops/s 28.3031 Ops/s $\color{#d91a1a}-4.23\%$
test_cql_speed[True-None] 13.8475ms 13.2181ms 75.6536 Ops/s 79.8294 Ops/s $\textbf{\color{#d91a1a}-5.23\%}$
test_cql_speed[True-backward] 18.9393ms 18.4304ms 54.2581 Ops/s 55.6838 Ops/s $\color{#d91a1a}-2.56\%$
test_cql_speed[reduce-overhead-None] 13.4167ms 12.6612ms 78.9814 Ops/s 80.3100 Ops/s $\color{#d91a1a}-1.65\%$
test_a2c_speed[False-None] 6.0788ms 5.5973ms 178.6565 Ops/s 182.0236 Ops/s $\color{#d91a1a}-1.85\%$
test_a2c_speed[False-backward] 12.7166ms 11.9705ms 83.5387 Ops/s 83.7384 Ops/s $\color{#d91a1a}-0.24\%$
test_a2c_speed[True-None] 4.4324ms 3.8076ms 262.6333 Ops/s 264.1147 Ops/s $\color{#d91a1a}-0.56\%$
test_a2c_speed[True-backward] 8.8577ms 8.6583ms 115.4959 Ops/s 115.0250 Ops/s $\color{#35bf28}+0.41\%$
test_a2c_speed[reduce-overhead-None] 4.0219ms 3.8372ms 260.6056 Ops/s 269.2131 Ops/s $\color{#d91a1a}-3.20\%$
test_ppo_speed[False-None] 6.5079ms 6.1012ms 163.9028 Ops/s 166.5395 Ops/s $\color{#d91a1a}-1.58\%$
test_ppo_speed[False-backward] 13.7947ms 12.8009ms 78.1195 Ops/s 79.4612 Ops/s $\color{#d91a1a}-1.69\%$
test_ppo_speed[True-None] 4.1277ms 3.6626ms 273.0280 Ops/s 268.6036 Ops/s $\color{#35bf28}+1.65\%$
test_ppo_speed[True-backward] 9.2580ms 8.8326ms 113.2169 Ops/s 113.4734 Ops/s $\color{#d91a1a}-0.23\%$
test_ppo_speed[reduce-overhead-None] 4.1750ms 3.7476ms 266.8399 Ops/s 274.3374 Ops/s $\color{#d91a1a}-2.73\%$
test_reinforce_speed[False-None] 5.1617ms 4.6900ms 213.2209 Ops/s 215.0523 Ops/s $\color{#d91a1a}-0.85\%$
test_reinforce_speed[False-backward] 7.8989ms 7.5160ms 133.0486 Ops/s 134.0609 Ops/s $\color{#d91a1a}-0.76\%$
test_reinforce_speed[True-None] 3.4779ms 3.0211ms 331.0066 Ops/s 340.3324 Ops/s $\color{#d91a1a}-2.74\%$
test_reinforce_speed[True-backward] 8.3125ms 7.9075ms 126.4627 Ops/s 126.6588 Ops/s $\color{#d91a1a}-0.15\%$
test_reinforce_speed[reduce-overhead-None] 3.1494ms 2.8976ms 345.1079 Ops/s 346.9521 Ops/s $\color{#d91a1a}-0.53\%$
test_iql_speed[False-None] 21.1491ms 20.1423ms 49.6468 Ops/s 49.4007 Ops/s $\color{#35bf28}+0.50\%$
test_iql_speed[False-backward] 37.2526ms 31.2434ms 32.0068 Ops/s 32.4422 Ops/s $\color{#d91a1a}-1.34\%$
test_iql_speed[True-None] 8.9765ms 8.5437ms 117.0455 Ops/s 112.6247 Ops/s $\color{#35bf28}+3.93\%$
test_iql_speed[True-backward] 18.6164ms 17.0608ms 58.6138 Ops/s 59.3913 Ops/s $\color{#d91a1a}-1.31\%$
test_iql_speed[reduce-overhead-None] 8.8383ms 8.5854ms 116.4767 Ops/s 113.9473 Ops/s $\color{#35bf28}+2.22\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.3226ms 5.9459ms 168.1829 Ops/s 163.9741 Ops/s $\color{#35bf28}+2.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.8790ms 0.2939ms 3.4020 KOps/s 3.4805 KOps/s $\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5002ms 0.2665ms 3.7526 KOps/s 3.7607 KOps/s $\color{#d91a1a}-0.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9655ms 5.6462ms 177.1093 Ops/s 169.5267 Ops/s $\color{#35bf28}+4.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7520ms 0.3166ms 3.1588 KOps/s 2.8131 KOps/s $\textbf{\color{#35bf28}+12.29\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5546ms 0.3250ms 3.0767 KOps/s 2.9288 KOps/s $\textbf{\color{#35bf28}+5.05\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6283ms 1.3870ms 720.9687 Ops/s 687.8204 Ops/s $\color{#35bf28}+4.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.8280ms 1.2977ms 770.5912 Ops/s 737.8195 Ops/s $\color{#35bf28}+4.44\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2275ms 5.8055ms 172.2501 Ops/s 163.6874 Ops/s $\textbf{\color{#35bf28}+5.23\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.3026ms 0.4413ms 2.2662 KOps/s 1.9871 KOps/s $\textbf{\color{#35bf28}+14.05\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8845ms 0.4289ms 2.3318 KOps/s 1.9637 KOps/s $\textbf{\color{#35bf28}+18.75\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0799ms 5.6712ms 176.3283 Ops/s 168.9324 Ops/s $\color{#35bf28}+4.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.8944ms 0.3020ms 3.3109 KOps/s 2.5133 KOps/s $\textbf{\color{#35bf28}+31.74\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7687ms 0.2791ms 3.5829 KOps/s 3.3202 KOps/s $\textbf{\color{#35bf28}+7.91\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8964ms 5.6237ms 177.8189 Ops/s 168.9235 Ops/s $\textbf{\color{#35bf28}+5.27\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7742ms 0.3513ms 2.8464 KOps/s 3.0237 KOps/s $\textbf{\color{#d91a1a}-5.86\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4471ms 0.2589ms 3.8626 KOps/s 2.9205 KOps/s $\textbf{\color{#35bf28}+32.26\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2427ms 5.8299ms 171.5297 Ops/s 163.7970 Ops/s $\color{#35bf28}+4.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3423ms 0.4762ms 2.0998 KOps/s 2.3025 KOps/s $\textbf{\color{#d91a1a}-8.80\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7400ms 0.4721ms 2.1184 KOps/s 2.0340 KOps/s $\color{#35bf28}+4.15\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.5625s 16.2121ms 61.6822 Ops/s 57.9612 Ops/s $\textbf{\color{#35bf28}+6.42\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.9443ms 1.9295ms 518.2792 Ops/s 508.6891 Ops/s $\color{#35bf28}+1.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.5286ms 1.2303ms 812.8087 Ops/s 1.1132 KOps/s $\textbf{\color{#d91a1a}-26.99\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.8237ms 5.0803ms 196.8390 Ops/s 196.3247 Ops/s $\color{#35bf28}+0.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 12.3557ms 1.9273ms 518.8545 Ops/s 530.5112 Ops/s $\color{#d91a1a}-2.20\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1264ms 0.9011ms 1.1098 KOps/s 759.9768 Ops/s $\textbf{\color{#35bf28}+46.03\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.5876ms 5.1791ms 193.0847 Ops/s 59.3972 Ops/s $\textbf{\color{#35bf28}+225.07\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 4.1957ms 1.8321ms 545.8356 Ops/s 526.9804 Ops/s $\color{#35bf28}+3.58\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.6157ms 1.0843ms 922.2307 Ops/s 970.9089 Ops/s $\textbf{\color{#d91a1a}-5.01\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 38.5881ms 35.6159ms 28.0774 Ops/s 28.4021 Ops/s $\color{#d91a1a}-1.14\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.9673ms 18.4588ms 54.1747 Ops/s 55.7873 Ops/s $\color{#d91a1a}-2.89\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 39.6159ms 36.8374ms 27.1463 Ops/s 27.2400 Ops/s $\color{#d91a1a}-0.34\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.4771ms 18.5059ms 54.0368 Ops/s 55.0638 Ops/s $\color{#d91a1a}-1.87\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.4359ms 38.5359ms 25.9498 Ops/s 25.7447 Ops/s $\color{#35bf28}+0.80\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.5467ms 19.9530ms 50.1178 Ops/s 49.2281 Ops/s $\color{#35bf28}+1.81\%$
test_storage_write_lazystack[50-img_shape0-small] 0.8212ms 0.2152ms 4.6467 KOps/s 4.4523 KOps/s $\color{#35bf28}+4.37\%$
test_storage_write_lazystack[100-img_shape1-atari] 1.6996ms 1.4222ms 703.1359 Ops/s 720.9292 Ops/s $\color{#d91a1a}-2.47\%$
test_storage_write_lazystack[100-img_shape2-large_img] 2.8083ms 2.3663ms 422.6065 Ops/s 416.2919 Ops/s $\color{#35bf28}+1.52\%$
test_storage_write_lazystack[200-img_shape3-large_batch] 3.3049ms 2.9381ms 340.3560 Ops/s 343.5534 Ops/s $\color{#d91a1a}-0.93\%$
test_storage_write_contiguous[50-img_shape0-small] 0.2633ms 0.1346ms 7.4270 KOps/s 7.4053 KOps/s $\color{#35bf28}+0.29\%$
test_storage_write_contiguous[100-img_shape1-atari] 0.3446ms 0.2012ms 4.9693 KOps/s 5.3616 KOps/s $\textbf{\color{#d91a1a}-7.32\%}$
test_storage_write_contiguous[100-img_shape2-large_img] 1.8200ms 1.7195ms 581.5805 Ops/s 571.5595 Ops/s $\color{#35bf28}+1.75\%$
test_storage_write_contiguous[200-img_shape3-large_batch] 1.4809ms 1.2712ms 786.6631 Ops/s 720.7539 Ops/s $\textbf{\color{#35bf28}+9.14\%}$
test_collector_stack_then_write[50-img_shape0-small] 1.2108ms 1.0813ms 924.7991 Ops/s 907.9320 Ops/s $\color{#35bf28}+1.86\%$
test_collector_stack_then_write[100-img_shape1-atari] 4.5540ms 3.4894ms 286.5827 Ops/s 275.5987 Ops/s $\color{#35bf28}+3.99\%$
test_collector_stack_then_write[100-img_shape2-large_img] 11.0198ms 5.7304ms 174.5068 Ops/s 182.2380 Ops/s $\color{#d91a1a}-4.24\%$
test_collector_stack_then_write[200-img_shape3-large_batch] 7.2005ms 6.8247ms 146.5262 Ops/s 146.0161 Ops/s $\color{#35bf28}+0.35\%$
test_collector_lazystack_then_write[50-img_shape0-small] 0.6910ms 0.2687ms 3.7223 KOps/s 3.5528 KOps/s $\color{#35bf28}+4.77\%$
test_collector_lazystack_then_write[100-img_shape1-atari] 1.9544ms 1.5154ms 659.8909 Ops/s 660.5710 Ops/s $\color{#d91a1a}-0.10\%$
test_collector_lazystack_then_write[100-img_shape2-large_img] 2.8092ms 2.4718ms 404.5702 Ops/s 397.1399 Ops/s $\color{#35bf28}+1.87\%$
test_collector_lazystack_then_write[200-img_shape3-large_batch] 3.3819ms 3.1038ms 322.1902 Ops/s 321.0719 Ops/s $\color{#35bf28}+0.35\%$
test_collector_without_rb[100-img_shape0-atari] 34.2599ms 33.6006ms 29.7614 Ops/s 29.6600 Ops/s $\color{#35bf28}+0.34\%$
test_collector_without_rb[200-img_shape1-large_batch] 67.0041ms 66.3757ms 15.0657 Ops/s 15.0073 Ops/s $\color{#35bf28}+0.39\%$
test_collector_with_rb[100-img_shape0-atari] 39.1604ms 38.2102ms 26.1710 Ops/s 26.3115 Ops/s $\color{#d91a1a}-0.53\%$
test_collector_with_rb[200-img_shape1-large_batch] 75.1044ms 74.6039ms 13.4041 Ops/s 13.4521 Ops/s $\color{#d91a1a}-0.36\%$

This was referenced Feb 10, 2026
Closed
Closed
Closed
Merged
Merged
Merged
Merged
Merged
Merged
@vmoens
Update
e2f67ca 
[ghstack-poisoned]
@vmoens
Update
aea4611 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Feb 10, 2026
@vmoens @cursoragent
[Feature] Add RayEvalWorker for async evaluation (#3474)
2123ce8 
Add a new torchrl.eval module with RayEvalWorker, a Ray-backed async
evaluation helper that runs environment + policy in a separate process.
This is useful when the evaluation environment requires special
process-level initialisation (e.g. Isaac Lab AppLauncher) or when
evaluation should happen concurrently with training on a separate GPU.

API: submit(weights, max_steps) / poll() -> {reward, frames}.
Co-authored-by: Cursor <[email protected]>
ghstack-source-id: f3574c2
Pull-Request: #3474
Co-authored-by: Cursor <[email protected]>
@vmoens vmoens merged commit aea4611 into gh/vmoens/226/base Feb 10, 2026
98 of 104 checks passed
@vmoens vmoens deleted the gh/vmoens/226/head branch February 10, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Collectors Environments Adds or modifies an environment wrapper Feature New feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens