Skip to content

[BugFix] Fix device initialization in CrossQLoss.maybe_init_target_entropy #3311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 3 commits into from
Jan 8, 2026
Merged

[BugFix] Fix device initialization in CrossQLoss.maybe_init_target_entropy #3311

vmoens merged 3 commits into from
Jan 8, 2026

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Jan 8, 2026 •
edited
Loading

@vmoens
Update
eb41b61 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Jan 8, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3311

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Jan 8, 2026
Merged
Merged
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 8, 2026
@vmoens vmoens linked an issue Jan 8, 2026 that may be closed by this pull request
[BUG] giving target_entroy = N (only "auto" work) with crossQLoss do not work #3309
Closed
3 tasks
@vmoens
Update
8d55bc8 
[ghstack-poisoned]
This was referenced Jan 8, 2026
Merged
Merged
@vmoens
Update
8ae652f 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jan 8, 2026
@vmoens
[BugFix] Fix device initialization in CrossQLoss.maybe_init_target_en…
416e454 
…tropy

Move device retrieval outside the 'if target_entropy == auto' block to ensure
the device is available when registering the target_entropy buffer.


ghstack-source-id: 5d570b9
Pull-Request: #3311
@vmoens vmoens merged commit 8ae652f into gh/vmoens/189/base Jan 8, 2026
54 of 60 checks passed
@vmoens vmoens deleted the gh/vmoens/189/head branch January 8, 2026 11:18
@github-actions
Copy link

github-actions bot commented Jan 8, 2026

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}24$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 80.0534μs 78.7286μs 12.7019 KOps/s 12.3823 KOps/s $\color{#35bf28}+2.58\%$
test_tensor_to_bytestream_speed[torch.save] 0.1385ms 0.1370ms 7.2991 KOps/s 7.1907 KOps/s $\color{#35bf28}+1.51\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1143s 0.1133s 8.8278 Ops/s 7.7429 Ops/s $\textbf{\color{#35bf28}+14.01\%}$
test_tensor_to_bytestream_speed[numpy] 2.6302μs 2.6239μs 381.1078 KOps/s 358.3991 KOps/s $\textbf{\color{#35bf28}+6.34\%}$
test_tensor_to_bytestream_speed[safetensors] 39.6431μs 39.2909μs 25.4512 KOps/s 24.9708 KOps/s $\color{#35bf28}+1.92\%$
test_simple 0.5458s 0.5389s 1.8555 Ops/s 1.7823 Ops/s $\color{#35bf28}+4.11\%$
test_transformed 1.0982s 1.0958s 0.9126 Ops/s 0.8845 Ops/s $\color{#35bf28}+3.17\%$
test_serial 1.6480s 1.6388s 0.6102 Ops/s 0.6022 Ops/s $\color{#35bf28}+1.34\%$
test_parallel 1.2222s 1.1572s 0.8642 Ops/s 0.8529 Ops/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-True-True-True] 0.3143ms 44.0385μs 22.7074 KOps/s 23.0211 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-True-True-True-False] 64.1310μs 24.4329μs 40.9284 KOps/s 40.7596 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-True-True-False-True] 65.9510μs 24.3867μs 41.0059 KOps/s 40.8702 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-True-True-False-False] 47.2510μs 13.4028μs 74.6115 KOps/s 76.1320 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-False-True-True] 91.3610μs 46.1620μs 21.6628 KOps/s 21.6020 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-True-False-True-False] 55.2910μs 27.0271μs 37.0000 KOps/s 36.8519 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-True-False-False-True] 75.7810μs 27.2221μs 36.7349 KOps/s 36.0647 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[True-True-False-False-False] 65.1110μs 16.2778μs 61.4332 KOps/s 60.6216 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-False-True-True-True] 92.9320μs 49.4359μs 20.2282 KOps/s 20.2542 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-False-True-True-False] 65.9110μs 29.9179μs 33.4249 KOps/s 33.6166 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-False-True-False-True] 93.3110μs 26.9733μs 37.0737 KOps/s 36.7378 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-True-False-False] 47.2910μs 16.1037μs 62.0977 KOps/s 61.8683 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-False-False-True-True] 94.6720μs 51.4088μs 19.4519 KOps/s 19.1485 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-False-False-True-False] 62.6510μs 32.0037μs 31.2464 KOps/s 31.1472 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-False-False-False-True] 61.4910μs 29.8710μs 33.4772 KOps/s 33.9371 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-False-False-False] 45.6910μs 18.2534μs 54.7843 KOps/s 52.6365 KOps/s $\color{#35bf28}+4.08\%$
test_step_mdp_speed[False-True-True-True-True] 0.1133ms 48.1468μs 20.7698 KOps/s 19.9621 KOps/s $\color{#35bf28}+4.05\%$
test_step_mdp_speed[False-True-True-True-False] 64.7710μs 29.4914μs 33.9082 KOps/s 33.1441 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-True-True-False-True] 2.3314ms 31.2139μs 32.0371 KOps/s 31.8569 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[False-True-True-False-False] 41.9700μs 18.0014μs 55.5513 KOps/s 55.7355 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-False-True-True] 0.1254ms 52.1001μs 19.1938 KOps/s 19.0754 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-True-False-True-False] 57.3810μs 32.9211μs 30.3757 KOps/s 30.5508 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-False-True] 52.6310μs 33.4014μs 29.9389 KOps/s 29.8905 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[False-True-False-False-False] 57.3210μs 20.6962μs 48.3181 KOps/s 48.6644 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-False-True-True-True] 97.6520μs 54.8845μs 18.2201 KOps/s 18.1986 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[False-False-True-True-False] 65.1610μs 34.9747μs 28.5921 KOps/s 28.6969 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-True-False-True] 69.5020μs 33.7926μs 29.5923 KOps/s 29.8214 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-True-False-False] 52.3110μs 20.6069μs 48.5273 KOps/s 49.1927 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[False-False-False-True-True] 0.1237ms 56.4981μs 17.6997 KOps/s 17.4853 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-False-True-False] 74.6710μs 37.9964μs 26.3183 KOps/s 26.5384 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-False-False-False-True] 65.2310μs 35.9697μs 27.8012 KOps/s 28.2093 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-False-False-False-False] 47.6310μs 23.1923μs 43.1177 KOps/s 43.5124 KOps/s $\color{#d91a1a}-0.91\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8532s 0.7527s 1.3286 Ops/s 1.3205 Ops/s $\color{#35bf28}+0.61\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7171s 0.6193s 1.6148 Ops/s 1.6046 Ops/s $\color{#35bf28}+0.64\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7257s 1.6451s 0.6079 Ops/s 0.6014 Ops/s $\color{#35bf28}+1.08\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5158s 1.4374s 0.6957 Ops/s 0.6947 Ops/s $\color{#35bf28}+0.14\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9711s 1.8905s 0.5290 Ops/s 0.5224 Ops/s $\color{#35bf28}+1.27\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7529s 1.6730s 0.5977 Ops/s 0.5920 Ops/s $\color{#35bf28}+0.97\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.7625s 4.6200s 0.2165 Ops/s 0.2148 Ops/s $\color{#35bf28}+0.75\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.4976s 4.3982s 0.2274 Ops/s 0.2244 Ops/s $\color{#35bf28}+1.32\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0566s 1.9336s 0.5172 Ops/s 0.5150 Ops/s $\color{#35bf28}+0.42\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7366s 1.6473s 0.6071 Ops/s 0.6015 Ops/s $\color{#35bf28}+0.93\%$
test_values[generalized_advantage_estimate-True-True] 9.9351ms 9.7015ms 103.0764 Ops/s 98.3210 Ops/s $\color{#35bf28}+4.84\%$
test_values[vec_generalized_advantage_estimate-True-True] 13.3874ms 11.0510ms 90.4896 Ops/s 89.7772 Ops/s $\color{#35bf28}+0.79\%$
test_values[td0_return_estimate-False-False] 0.2378ms 0.1288ms 7.7664 KOps/s 7.6445 KOps/s $\color{#35bf28}+1.60\%$
test_values[td1_return_estimate-False-False] 27.1014ms 26.0198ms 38.4323 Ops/s 36.4728 Ops/s $\textbf{\color{#35bf28}+5.37\%}$
test_values[vec_td1_return_estimate-False-False] 11.9834ms 11.0744ms 90.2986 Ops/s 90.3897 Ops/s $\color{#d91a1a}-0.10\%$
test_values[td_lambda_return_estimate-True-False] 40.1356ms 38.6425ms 25.8782 Ops/s 24.4020 Ops/s $\textbf{\color{#35bf28}+6.05\%}$
test_values[vec_td_lambda_return_estimate-True-False] 12.1884ms 11.1561ms 89.6374 Ops/s 90.0566 Ops/s $\color{#d91a1a}-0.47\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.1984ms 8.6273ms 115.9109 Ops/s 110.3899 Ops/s $\textbf{\color{#35bf28}+5.00\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.6969ms 1.5119ms 661.4078 Ops/s 676.6936 Ops/s $\color{#d91a1a}-2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5491ms 0.4133ms 2.4193 KOps/s 2.3995 KOps/s $\color{#35bf28}+0.83\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 30.1936ms 24.4531ms 40.8947 Ops/s 33.6769 Ops/s $\textbf{\color{#35bf28}+21.43\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1178ms 1.7467ms 572.5148 Ops/s 566.4177 Ops/s $\color{#35bf28}+1.08\%$
test_dqn_speed[False-None] 1.8892ms 1.3754ms 727.0683 Ops/s 732.7885 Ops/s $\color{#d91a1a}-0.78\%$
test_dqn_speed[False-backward] 2.0329ms 1.8918ms 528.6048 Ops/s 538.9348 Ops/s $\color{#d91a1a}-1.92\%$
test_dqn_speed[True-None] 0.7008ms 0.5255ms 1.9031 KOps/s 1.8289 KOps/s $\color{#35bf28}+4.05\%$
test_dqn_speed[True-backward] 1.0287ms 0.9782ms 1.0223 KOps/s 931.2039 Ops/s $\textbf{\color{#35bf28}+9.78\%}$
test_dqn_speed[reduce-overhead-None] 0.9024ms 0.5214ms 1.9179 KOps/s 1.8485 KOps/s $\color{#35bf28}+3.75\%$
test_dqn_speed[reduce-overhead-backward] 0.9889ms 0.9582ms 1.0436 KOps/s 1.0235 KOps/s $\color{#35bf28}+1.96\%$
test_ddpg_speed[False-None] 3.1398ms 2.8051ms 356.4898 Ops/s 357.2966 Ops/s $\color{#d91a1a}-0.23\%$
test_ddpg_speed[False-backward] 4.0642ms 3.9636ms 252.2953 Ops/s 252.8127 Ops/s $\color{#d91a1a}-0.20\%$
test_ddpg_speed[True-None] 1.7710ms 1.3748ms 727.3795 Ops/s 714.9461 Ops/s $\color{#35bf28}+1.74\%$
test_ddpg_speed[True-backward] 2.4711ms 2.3487ms 425.7728 Ops/s 424.7448 Ops/s $\color{#35bf28}+0.24\%$
test_ddpg_speed[reduce-overhead-None] 1.7731ms 1.4008ms 713.8981 Ops/s 728.6585 Ops/s $\color{#d91a1a}-2.03\%$
test_ddpg_speed[reduce-overhead-backward] 2.4906ms 2.3855ms 419.2059 Ops/s 427.1789 Ops/s $\color{#d91a1a}-1.87\%$
test_sac_speed[False-None] 8.3642ms 7.7638ms 128.8037 Ops/s 129.2740 Ops/s $\color{#d91a1a}-0.36\%$
test_sac_speed[False-backward] 11.3637ms 10.9786ms 91.0859 Ops/s 91.9809 Ops/s $\color{#d91a1a}-0.97\%$
test_sac_speed[True-None] 2.4018ms 2.1669ms 461.4815 Ops/s 471.0438 Ops/s $\color{#d91a1a}-2.03\%$
test_sac_speed[True-backward] 4.1305ms 4.0348ms 247.8418 Ops/s 232.4020 Ops/s $\textbf{\color{#35bf28}+6.64\%}$
test_sac_speed[reduce-overhead-None] 3.7364ms 2.1703ms 460.7680 Ops/s 455.3296 Ops/s $\color{#35bf28}+1.19\%$
test_sac_speed[reduce-overhead-backward] 4.3624ms 4.0334ms 247.9306 Ops/s 240.4867 Ops/s $\color{#35bf28}+3.10\%$
test_redq_speed[False-None] 11.0377ms 10.2551ms 97.5127 Ops/s 97.4880 Ops/s $\color{#35bf28}+0.03\%$
test_redq_speed[False-backward] 21.3563ms 17.8457ms 56.0358 Ops/s 56.4295 Ops/s $\color{#d91a1a}-0.70\%$
test_redq_speed[True-None] 4.7221ms 4.4558ms 224.4255 Ops/s 223.3111 Ops/s $\color{#35bf28}+0.50\%$
test_redq_speed[True-backward] 10.2805ms 9.8943ms 101.0682 Ops/s 105.6920 Ops/s $\color{#d91a1a}-4.37\%$
test_redq_speed[reduce-overhead-None] 4.5632ms 4.3817ms 228.2208 Ops/s 229.5660 Ops/s $\color{#d91a1a}-0.59\%$
test_redq_speed[reduce-overhead-backward] 10.1953ms 9.9401ms 100.6022 Ops/s 103.7909 Ops/s $\color{#d91a1a}-3.07\%$
test_redq_deprec_speed[False-None] 11.3677ms 10.8354ms 92.2902 Ops/s 93.9616 Ops/s $\color{#d91a1a}-1.78\%$
test_redq_deprec_speed[False-backward] 15.9219ms 15.5108ms 64.4713 Ops/s 66.5034 Ops/s $\color{#d91a1a}-3.06\%$
test_redq_deprec_speed[True-None] 4.2227ms 3.7882ms 263.9765 Ops/s 284.7817 Ops/s $\textbf{\color{#d91a1a}-7.31\%}$
test_redq_deprec_speed[True-backward] 9.0777ms 7.8389ms 127.5691 Ops/s 134.8688 Ops/s $\textbf{\color{#d91a1a}-5.41\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.9294ms 3.5508ms 281.6283 Ops/s 287.6273 Ops/s $\color{#d91a1a}-2.09\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.7932ms 7.5613ms 132.2518 Ops/s 122.7587 Ops/s $\textbf{\color{#35bf28}+7.73\%}$
test_td3_speed[False-None] 7.8967ms 7.7846ms 128.4595 Ops/s 117.5187 Ops/s $\textbf{\color{#35bf28}+9.31\%}$
test_td3_speed[False-backward] 11.2376ms 10.6001ms 94.3384 Ops/s 93.2076 Ops/s $\color{#35bf28}+1.21\%$
test_td3_speed[True-None] 1.8367ms 1.7922ms 557.9732 Ops/s 546.9184 Ops/s $\color{#35bf28}+2.02\%$
test_td3_speed[True-backward] 3.7439ms 3.6019ms 277.6337 Ops/s 264.1061 Ops/s $\textbf{\color{#35bf28}+5.12\%}$
test_td3_speed[reduce-overhead-None] 1.8292ms 1.7783ms 562.3422 Ops/s 556.2340 Ops/s $\color{#35bf28}+1.10\%$
test_td3_speed[reduce-overhead-backward] 3.7870ms 3.6657ms 272.8013 Ops/s 235.2276 Ops/s $\textbf{\color{#35bf28}+15.97\%}$
test_cql_speed[False-None] 28.6617ms 25.7417ms 38.8475 Ops/s 38.4541 Ops/s $\color{#35bf28}+1.02\%$
test_cql_speed[False-backward] 38.1572ms 34.7944ms 28.7402 Ops/s 28.4747 Ops/s $\color{#35bf28}+0.93\%$
test_cql_speed[True-None] 12.9099ms 12.3727ms 80.8231 Ops/s 78.7470 Ops/s $\color{#35bf28}+2.64\%$
test_cql_speed[True-backward] 18.8245ms 18.4225ms 54.2816 Ops/s 55.3348 Ops/s $\color{#d91a1a}-1.90\%$
test_cql_speed[reduce-overhead-None] 12.9656ms 12.4580ms 80.2699 Ops/s 79.4592 Ops/s $\color{#35bf28}+1.02\%$
test_cql_speed[reduce-overhead-backward] 18.9158ms 18.3680ms 54.4426 Ops/s 52.2126 Ops/s $\color{#35bf28}+4.27\%$
test_a2c_speed[False-None] 5.8295ms 5.3187ms 188.0143 Ops/s 184.8970 Ops/s $\color{#35bf28}+1.69\%$
test_a2c_speed[False-backward] 12.0299ms 11.6279ms 86.0000 Ops/s 86.0099 Ops/s $\color{#d91a1a}-0.01\%$
test_a2c_speed[True-None] 3.8546ms 3.6825ms 271.5535 Ops/s 263.5683 Ops/s $\color{#35bf28}+3.03\%$
test_a2c_speed[True-backward] 8.7527ms 8.3528ms 119.7204 Ops/s 117.7748 Ops/s $\color{#35bf28}+1.65\%$
test_a2c_speed[reduce-overhead-None] 3.9452ms 3.6877ms 271.1700 Ops/s 271.0619 Ops/s $\color{#35bf28}+0.04\%$
test_a2c_speed[reduce-overhead-backward] 8.8626ms 8.6062ms 116.1954 Ops/s 114.7127 Ops/s $\color{#35bf28}+1.29\%$
test_ppo_speed[False-None] 6.1353ms 5.7851ms 172.8593 Ops/s 169.8096 Ops/s $\color{#35bf28}+1.80\%$
test_ppo_speed[False-backward] 12.6444ms 12.2569ms 81.5866 Ops/s 81.2986 Ops/s $\color{#35bf28}+0.35\%$
test_ppo_speed[True-None] 3.8778ms 3.5796ms 279.3605 Ops/s 277.8613 Ops/s $\color{#35bf28}+0.54\%$
test_ppo_speed[True-backward] 8.5376ms 8.3362ms 119.9588 Ops/s 108.6200 Ops/s $\textbf{\color{#35bf28}+10.44\%}$
test_ppo_speed[reduce-overhead-None] 4.0252ms 3.5803ms 279.3049 Ops/s 279.0419 Ops/s $\color{#35bf28}+0.09\%$
test_ppo_speed[reduce-overhead-backward] 8.6099ms 8.4605ms 118.1968 Ops/s 115.9829 Ops/s $\color{#35bf28}+1.91\%$
test_reinforce_speed[False-None] 4.8912ms 4.5172ms 221.3755 Ops/s 219.6355 Ops/s $\color{#35bf28}+0.79\%$
test_reinforce_speed[False-backward] 7.6669ms 7.2683ms 137.5835 Ops/s 136.4637 Ops/s $\color{#35bf28}+0.82\%$
test_reinforce_speed[True-None] 3.3249ms 2.8660ms 348.9182 Ops/s 340.6802 Ops/s $\color{#35bf28}+2.42\%$
test_reinforce_speed[True-backward] 7.9604ms 7.5718ms 132.0691 Ops/s 119.9997 Ops/s $\textbf{\color{#35bf28}+10.06\%}$
test_reinforce_speed[reduce-overhead-None] 3.1322ms 2.8293ms 353.4504 Ops/s 353.2175 Ops/s $\color{#35bf28}+0.07\%$
test_reinforce_speed[reduce-overhead-backward] 8.1389ms 7.8313ms 127.6931 Ops/s 112.7635 Ops/s $\textbf{\color{#35bf28}+13.24\%}$
test_iql_speed[False-None] 25.9990ms 20.2092ms 49.4823 Ops/s 49.8322 Ops/s $\color{#d91a1a}-0.70\%$
test_iql_speed[False-backward] 32.0290ms 29.9752ms 33.3609 Ops/s 33.4854 Ops/s $\color{#d91a1a}-0.37\%$
test_iql_speed[True-None] 11.2672ms 8.6428ms 115.7030 Ops/s 113.4694 Ops/s $\color{#35bf28}+1.97\%$
test_iql_speed[True-backward] 17.2121ms 16.5857ms 60.2928 Ops/s 59.8468 Ops/s $\color{#35bf28}+0.75\%$
test_iql_speed[reduce-overhead-None] 9.0967ms 8.5071ms 117.5496 Ops/s 117.1196 Ops/s $\color{#35bf28}+0.37\%$
test_iql_speed[reduce-overhead-backward] 17.3784ms 17.1311ms 58.3733 Ops/s 57.5685 Ops/s $\color{#35bf28}+1.40\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.9568ms 5.9196ms 168.9305 Ops/s 170.2842 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5348ms 0.2920ms 3.4241 KOps/s 2.6650 KOps/s $\textbf{\color{#35bf28}+28.49\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6917ms 0.2696ms 3.7092 KOps/s 2.8221 KOps/s $\textbf{\color{#35bf28}+31.43\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9528ms 5.6846ms 175.9137 Ops/s 177.4602 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0681ms 0.3341ms 2.9935 KOps/s 3.6371 KOps/s $\textbf{\color{#d91a1a}-17.70\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6220ms 0.3134ms 3.1912 KOps/s 3.5893 KOps/s $\textbf{\color{#d91a1a}-11.09\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7202ms 1.2821ms 779.9799 Ops/s 794.2312 Ops/s $\color{#d91a1a}-1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5472ms 1.2675ms 788.9580 Ops/s 842.1372 Ops/s $\textbf{\color{#d91a1a}-6.31\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.9531ms 5.9125ms 169.1340 Ops/s 172.9678 Ops/s $\color{#d91a1a}-2.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2197ms 0.4584ms 2.1816 KOps/s 2.2949 KOps/s $\color{#d91a1a}-4.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6936ms 0.4492ms 2.2263 KOps/s 2.3402 KOps/s $\color{#d91a1a}-4.87\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8228ms 5.6384ms 177.3551 Ops/s 177.0329 Ops/s $\color{#35bf28}+0.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0450ms 0.2760ms 3.6235 KOps/s 2.9956 KOps/s $\textbf{\color{#35bf28}+20.96\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6509ms 0.2624ms 3.8106 KOps/s 3.0471 KOps/s $\textbf{\color{#35bf28}+25.06\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8128ms 5.5705ms 179.5169 Ops/s 177.8970 Ops/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8029ms 0.3045ms 3.2846 KOps/s 3.1480 KOps/s $\color{#35bf28}+4.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6374ms 0.3093ms 3.2334 KOps/s 3.5553 KOps/s $\textbf{\color{#d91a1a}-9.05\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9040ms 5.7648ms 173.4658 Ops/s 171.3373 Ops/s $\color{#35bf28}+1.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2826ms 0.5071ms 1.9720 KOps/s 2.1482 KOps/s $\textbf{\color{#d91a1a}-8.20\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6595ms 0.4512ms 2.2162 KOps/s 2.1686 KOps/s $\color{#35bf28}+2.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.3693ms 4.9372ms 202.5459 Ops/s 53.0222 Ops/s $\textbf{\color{#35bf28}+282.00\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.2700ms 2.3271ms 429.7156 Ops/s 481.7649 Ops/s $\textbf{\color{#d91a1a}-10.80\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.3815ms 1.1890ms 841.0316 Ops/s 828.4369 Ops/s $\color{#35bf28}+1.52\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.6259s 17.3394ms 57.6720 Ops/s 198.5863 Ops/s $\textbf{\color{#d91a1a}-70.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 12.3264ms 2.0208ms 494.8625 Ops/s 483.4591 Ops/s $\color{#35bf28}+2.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1405ms 1.0861ms 920.7496 Ops/s 836.5224 Ops/s $\textbf{\color{#35bf28}+10.07\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 8.1410ms 5.1819ms 192.9786 Ops/s 191.8424 Ops/s $\color{#35bf28}+0.59\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.6178ms 2.1734ms 460.1179 Ops/s 465.5146 Ops/s $\color{#d91a1a}-1.16\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.9704ms 1.3915ms 718.6592 Ops/s 806.6341 Ops/s $\textbf{\color{#d91a1a}-10.91\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 37.3593ms 33.6624ms 29.7068 Ops/s 28.8253 Ops/s $\color{#35bf28}+3.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.8652ms 17.6826ms 56.5527 Ops/s 56.4026 Ops/s $\color{#35bf28}+0.27\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.4228ms 34.4405ms 29.0356 Ops/s 27.1101 Ops/s $\textbf{\color{#35bf28}+7.10\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.9692ms 18.0976ms 55.2559 Ops/s 53.9030 Ops/s $\color{#35bf28}+2.51\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 38.3703ms 36.5850ms 27.3336 Ops/s 25.8825 Ops/s $\textbf{\color{#35bf28}+5.61\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.0816ms 18.8978ms 52.9163 Ops/s 49.4018 Ops/s $\textbf{\color{#35bf28}+7.11\%}$

@vmoens vmoens mentioned this pull request Jan 8, 2026
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[BUG] giving target_entroy = N (only "auto" work) with crossQLoss do not work

2 participants

@vmoens