Skip to content

[BugFix] Fix target_entropy computation for composite action specs #3312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 3 commits into from
Jan 8, 2026
Merged

[BugFix] Fix target_entropy computation for composite action specs #3312

vmoens merged 3 commits into from
Jan 8, 2026

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Jan 8, 2026 •
edited
Loading

Stack from ghstack (oldest at bottom):

  • Handle composite action distributions in SACLoss.target_entropy by summing
    numel of all leaf specs
  • Add warning in SafeProbabilisticModule when out_keys don't match spec structure
    (helps catch misconfigured CompositeDistribution setups)
  • Update tests to pass action_spec explicitly for composite action distributions
  • Improve docstrings for target_entropy parameter in SACLoss, REDQLoss

@vmoens
Update
a527251 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Jan 8, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3312

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Jan 8, 2026
@vmoens
[BugFix] Fix target_entropy computation for composite action specs
e4b6c92 
- Handle composite action distributions in SACLoss.target_entropy by summing
  numel of all leaf specs
- Add warning in SafeProbabilisticModule when out_keys don't match spec structure
  (helps catch misconfigured CompositeDistribution setups)
- Update tests to pass action_spec explicitly for composite action distributions
- Improve docstrings for target_entropy parameter in SACLoss, REDQLoss


ghstack-source-id: f5a5fa5
Pull-Request: #3312
This was referenced Jan 8, 2026
Merged
Merged
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 8, 2026
@vmoens
Update
0974bb2 
[ghstack-poisoned]
This was referenced Jan 8, 2026
Merged
Merged
@vmoens
Update
7643ce0 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jan 8, 2026
@vmoens
[BugFix] Fix target_entropy computation for composite action specs
06960c6 
- Handle composite action distributions in SACLoss.target_entropy by summing
  numel of all leaf specs
- Add warning in SafeProbabilisticModule when out_keys don't match spec structure
  (helps catch misconfigured CompositeDistribution setups)
- Update tests to pass action_spec explicitly for composite action distributions
- Improve docstrings for target_entropy parameter in SACLoss, REDQLoss


ghstack-source-id: e0b9ac3
Pull-Request: #3312
@vmoens vmoens merged commit 7643ce0 into gh/vmoens/190/base Jan 8, 2026
53 of 60 checks passed
@vmoens vmoens deleted the gh/vmoens/190/head branch January 8, 2026 11:18
@github-actions
Copy link

github-actions bot commented Jan 8, 2026

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 81.6461μs 80.2211μs 12.4655 KOps/s 12.2122 KOps/s $\color{#35bf28}+2.07\%$
test_tensor_to_bytestream_speed[torch.save] 0.1415ms 0.1412ms 7.0825 KOps/s 7.1138 KOps/s $\color{#d91a1a}-0.44\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1196s 0.1195s 8.3704 Ops/s 8.2845 Ops/s $\color{#35bf28}+1.04\%$
test_tensor_to_bytestream_speed[numpy] 2.8400μs 2.8379μs 352.3792 KOps/s 369.1317 KOps/s $\color{#d91a1a}-4.54\%$
test_tensor_to_bytestream_speed[safetensors] 37.3882μs 37.2169μs 26.8695 KOps/s 26.7616 KOps/s $\color{#35bf28}+0.40\%$
test_simple 0.5512s 0.5500s 1.8182 Ops/s 1.7380 Ops/s $\color{#35bf28}+4.62\%$
test_transformed 1.1244s 1.1228s 0.8906 Ops/s 0.8665 Ops/s $\color{#35bf28}+2.79\%$
test_serial 1.6742s 1.6720s 0.5981 Ops/s 0.5886 Ops/s $\color{#35bf28}+1.62\%$
test_parallel 1.1119s 1.0943s 0.9138 Ops/s 0.8843 Ops/s $\color{#35bf28}+3.34\%$
test_step_mdp_speed[True-True-True-True-True] 0.1321ms 44.3934μs 22.5259 KOps/s 22.3747 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-True-True-True-False] 0.4428ms 25.6260μs 39.0229 KOps/s 39.9957 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-True-True-False-True] 0.4341ms 25.5247μs 39.1778 KOps/s 40.0951 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[True-True-True-False-False] 35.4310μs 13.9255μs 71.8106 KOps/s 72.2606 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-True-False-True-True] 0.4570ms 48.7591μs 20.5090 KOps/s 20.6909 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-True-False-True-False] 0.4388ms 28.0688μs 35.6267 KOps/s 35.7731 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-True-False-False-True] 0.4425ms 28.1123μs 35.5717 KOps/s 35.5222 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-True-False-False-False] 45.1810μs 16.9523μs 58.9892 KOps/s 59.6833 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-False-True-True-True] 0.4699ms 51.0527μs 19.5876 KOps/s 19.4694 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-False-True-True-False] 0.4450ms 31.0231μs 32.2341 KOps/s 32.4346 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[True-False-True-False-True] 0.4476ms 28.6618μs 34.8897 KOps/s 35.8885 KOps/s $\color{#d91a1a}-2.78\%$
test_step_mdp_speed[True-False-True-False-False] 43.3410μs 16.2657μs 61.4790 KOps/s 59.9052 KOps/s $\color{#35bf28}+2.63\%$
test_step_mdp_speed[True-False-False-True-True] 99.5230μs 52.6918μs 18.9783 KOps/s 18.8307 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-False-False-True-False] 68.5220μs 33.6710μs 29.6992 KOps/s 30.2128 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-False-False-False-True] 62.8820μs 31.0041μs 32.2537 KOps/s 33.0296 KOps/s $\color{#d91a1a}-2.35\%$
test_step_mdp_speed[True-False-False-False-False] 55.3210μs 19.3272μs 51.7405 KOps/s 51.6701 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[False-True-True-True-True] 97.6830μs 51.0423μs 19.5916 KOps/s 19.7611 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[False-True-True-True-False] 57.2210μs 30.6533μs 32.6229 KOps/s 32.7798 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-True-True-False-True] 8.6244ms 32.5877μs 30.6865 KOps/s 31.0269 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-True-True-False-False] 46.6510μs 18.5729μs 53.8418 KOps/s 54.3337 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-True-False-True-True] 0.1201ms 53.4237μs 18.7183 KOps/s 18.7519 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-False-True-False] 61.1220μs 33.5782μs 29.7813 KOps/s 29.8660 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-True-False-False-True] 67.1910μs 34.4959μs 28.9889 KOps/s 28.9534 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[False-True-False-False-False] 61.0610μs 21.6988μs 46.0855 KOps/s 47.6810 KOps/s $\color{#d91a1a}-3.35\%$
test_step_mdp_speed[False-False-True-True-True] 0.1116ms 56.4231μs 17.7232 KOps/s 17.8273 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-False-True-True-False] 90.3620μs 36.8407μs 27.1439 KOps/s 27.3459 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-False-True-False-True] 93.2920μs 34.7286μs 28.7947 KOps/s 29.0135 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-False-True-False-False] 49.4110μs 21.6892μs 46.1059 KOps/s 47.7387 KOps/s $\color{#d91a1a}-3.42\%$
test_step_mdp_speed[False-False-False-True-True] 90.2020μs 58.2725μs 17.1608 KOps/s 17.3358 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-False-False-True-False] 67.8720μs 38.7460μs 25.8091 KOps/s 25.8730 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-False-False-False-True] 99.4930μs 37.0978μs 26.9558 KOps/s 27.6129 KOps/s $\color{#d91a1a}-2.38\%$
test_step_mdp_speed[False-False-False-False-False] 53.8610μs 24.0598μs 41.5632 KOps/s 42.9743 KOps/s $\color{#d91a1a}-3.28\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8809s 0.7748s 1.2906 Ops/s 1.2868 Ops/s $\color{#35bf28}+0.29\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7385s 0.6404s 1.5615 Ops/s 1.5621 Ops/s $\color{#d91a1a}-0.04\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7802s 1.6986s 0.5887 Ops/s 0.5881 Ops/s $\color{#35bf28}+0.10\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.5582s 1.4776s 0.6768 Ops/s 0.6799 Ops/s $\color{#d91a1a}-0.47\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 2.0255s 1.9420s 0.5149 Ops/s 0.5147 Ops/s $\color{#35bf28}+0.04\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7978s 1.7247s 0.5798 Ops/s 0.5809 Ops/s $\color{#d91a1a}-0.18\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.7616s 4.6987s 0.2128 Ops/s 0.2164 Ops/s $\color{#d91a1a}-1.64\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5700s 4.4576s 0.2243 Ops/s 0.2249 Ops/s $\color{#d91a1a}-0.27\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0683s 1.9899s 0.5025 Ops/s 0.5081 Ops/s $\color{#d91a1a}-1.10\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.7815s 1.6922s 0.5910 Ops/s 0.5781 Ops/s $\color{#35bf28}+2.22\%$
test_values[generalized_advantage_estimate-True-True] 10.2965ms 10.1613ms 98.4129 Ops/s 99.0465 Ops/s $\color{#d91a1a}-0.64\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.8103ms 17.6431ms 56.6793 Ops/s 57.2329 Ops/s $\color{#d91a1a}-0.97\%$
test_values[td0_return_estimate-False-False] 0.2221ms 0.1278ms 7.8217 KOps/s 7.6299 KOps/s $\color{#35bf28}+2.51\%$
test_values[td1_return_estimate-False-False] 28.0456ms 27.7708ms 36.0090 Ops/s 36.2771 Ops/s $\color{#d91a1a}-0.74\%$
test_values[vec_td1_return_estimate-False-False] 18.1132ms 17.7069ms 56.4752 Ops/s 56.9979 Ops/s $\color{#d91a1a}-0.92\%$
test_values[td_lambda_return_estimate-True-False] 41.8580ms 41.1889ms 24.2784 Ops/s 24.5492 Ops/s $\color{#d91a1a}-1.10\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.8163ms 17.7688ms 56.2783 Ops/s 56.9240 Ops/s $\color{#d91a1a}-1.13\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.1851ms 9.0595ms 110.3820 Ops/s 112.3773 Ops/s $\color{#d91a1a}-1.78\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9918ms 1.5420ms 648.5084 Ops/s 656.2443 Ops/s $\color{#d91a1a}-1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4913ms 0.4177ms 2.3938 KOps/s 2.4011 KOps/s $\color{#d91a1a}-0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.7223ms 34.1422ms 29.2893 Ops/s 28.5788 Ops/s $\color{#35bf28}+2.49\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1316ms 1.7863ms 559.8236 Ops/s 570.2512 Ops/s $\color{#d91a1a}-1.83\%$
test_dqn_speed[False-None] 1.8174ms 1.4143ms 707.0813 Ops/s 711.8740 Ops/s $\color{#d91a1a}-0.67\%$
test_dqn_speed[False-backward] 1.9883ms 1.9467ms 513.7009 Ops/s 518.2739 Ops/s $\color{#d91a1a}-0.88\%$
test_dqn_speed[True-None] 0.9532ms 0.5458ms 1.8323 KOps/s 1.7898 KOps/s $\color{#35bf28}+2.38\%$
test_dqn_speed[True-backward] 1.0174ms 0.9916ms 1.0085 KOps/s 842.4081 Ops/s $\textbf{\color{#35bf28}+19.72\%}$
test_dqn_speed[reduce-overhead-None] 0.7103ms 0.5356ms 1.8670 KOps/s 1.8152 KOps/s $\color{#35bf28}+2.85\%$
test_dqn_speed[reduce-overhead-backward] 1.0100ms 0.9810ms 1.0193 KOps/s 899.8956 Ops/s $\textbf{\color{#35bf28}+13.27\%}$
test_ddpg_speed[False-None] 3.2045ms 2.8905ms 345.9643 Ops/s 343.1849 Ops/s $\color{#35bf28}+0.81\%$
test_ddpg_speed[False-backward] 4.1784ms 4.1154ms 242.9874 Ops/s 243.0054 Ops/s $-0.01\%$
test_ddpg_speed[True-None] 1.8013ms 1.3993ms 714.6199 Ops/s 699.2104 Ops/s $\color{#35bf28}+2.20\%$
test_ddpg_speed[True-backward] 2.4988ms 2.3985ms 416.9221 Ops/s 410.7611 Ops/s $\color{#35bf28}+1.50\%$
test_ddpg_speed[reduce-overhead-None] 1.5516ms 1.3898ms 719.5084 Ops/s 708.7745 Ops/s $\color{#35bf28}+1.51\%$
test_ddpg_speed[reduce-overhead-backward] 2.4217ms 2.3589ms 423.9197 Ops/s 412.2330 Ops/s $\color{#35bf28}+2.83\%$
test_sac_speed[False-None] 8.4583ms 8.0510ms 124.2077 Ops/s 124.6598 Ops/s $\color{#d91a1a}-0.36\%$
test_sac_speed[False-backward] 11.7271ms 11.3289ms 88.2695 Ops/s 88.4498 Ops/s $\color{#d91a1a}-0.20\%$
test_sac_speed[True-None] 2.5922ms 2.1601ms 462.9434 Ops/s 459.9551 Ops/s $\color{#35bf28}+0.65\%$
test_sac_speed[True-backward] 4.2152ms 4.0974ms 244.0548 Ops/s 244.2333 Ops/s $\color{#d91a1a}-0.07\%$
test_sac_speed[reduce-overhead-None] 2.5842ms 2.1607ms 462.8056 Ops/s 450.4822 Ops/s $\color{#35bf28}+2.74\%$
test_sac_speed[reduce-overhead-backward] 4.4577ms 4.0236ms 248.5309 Ops/s 233.4027 Ops/s $\textbf{\color{#35bf28}+6.48\%}$
test_redq_speed[False-None] 15.6131ms 10.8225ms 92.4000 Ops/s 93.6318 Ops/s $\color{#d91a1a}-1.32\%$
test_redq_speed[False-backward] 21.8119ms 18.2987ms 54.6488 Ops/s 54.5639 Ops/s $\color{#35bf28}+0.16\%$
test_redq_speed[True-None] 4.9527ms 4.4725ms 223.5864 Ops/s 214.6305 Ops/s $\color{#35bf28}+4.17\%$
test_redq_speed[True-backward] 10.5995ms 9.8921ms 101.0906 Ops/s 82.6854 Ops/s $\textbf{\color{#35bf28}+22.26\%}$
test_redq_speed[reduce-overhead-None] 4.9392ms 4.4688ms 223.7725 Ops/s 214.0289 Ops/s $\color{#35bf28}+4.55\%$
test_redq_speed[reduce-overhead-backward] 11.6864ms 10.4300ms 95.8769 Ops/s 98.5352 Ops/s $\color{#d91a1a}-2.70\%$
test_redq_deprec_speed[False-None] 12.4169ms 11.1983ms 89.2992 Ops/s 88.4664 Ops/s $\color{#35bf28}+0.94\%$
test_redq_deprec_speed[False-backward] 16.7001ms 15.9931ms 62.5271 Ops/s 62.2441 Ops/s $\color{#35bf28}+0.45\%$
test_redq_deprec_speed[True-None] 4.2411ms 3.7439ms 267.0978 Ops/s 270.9602 Ops/s $\color{#d91a1a}-1.43\%$
test_redq_deprec_speed[True-backward] 7.9549ms 7.7151ms 129.6160 Ops/s 123.8868 Ops/s $\color{#35bf28}+4.62\%$
test_redq_deprec_speed[reduce-overhead-None] 4.2407ms 3.7273ms 268.2891 Ops/s 269.9027 Ops/s $\color{#d91a1a}-0.60\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.0723ms 7.7596ms 128.8726 Ops/s 127.4351 Ops/s $\color{#35bf28}+1.13\%$
test_td3_speed[False-None] 8.3716ms 8.1279ms 123.0325 Ops/s 123.1964 Ops/s $\color{#d91a1a}-0.13\%$
test_td3_speed[False-backward] 11.8718ms 11.1093ms 90.0150 Ops/s 90.6413 Ops/s $\color{#d91a1a}-0.69\%$
test_td3_speed[True-None] 1.8905ms 1.8378ms 544.1411 Ops/s 533.9925 Ops/s $\color{#35bf28}+1.90\%$
test_td3_speed[True-backward] 3.7786ms 3.6728ms 272.2735 Ops/s 229.9486 Ops/s $\textbf{\color{#35bf28}+18.41\%}$
test_td3_speed[reduce-overhead-None] 1.8741ms 1.8299ms 546.4909 Ops/s 549.5973 Ops/s $\color{#d91a1a}-0.57\%$
test_td3_speed[reduce-overhead-backward] 3.8692ms 3.7476ms 266.8363 Ops/s 266.6794 Ops/s $\color{#35bf28}+0.06\%$
test_cql_speed[False-None] 30.5211ms 26.6727ms 37.4916 Ops/s 36.9804 Ops/s $\color{#35bf28}+1.38\%$
test_cql_speed[False-backward] 39.1724ms 35.8832ms 27.8682 Ops/s 27.8484 Ops/s $\color{#35bf28}+0.07\%$
test_cql_speed[True-None] 13.1267ms 12.6429ms 79.0956 Ops/s 77.2365 Ops/s $\color{#35bf28}+2.41\%$
test_cql_speed[True-backward] 18.9074ms 18.4925ms 54.0759 Ops/s 52.9439 Ops/s $\color{#35bf28}+2.14\%$
test_cql_speed[reduce-overhead-None] 13.2223ms 12.7040ms 78.7156 Ops/s 78.5826 Ops/s $\color{#35bf28}+0.17\%$
test_cql_speed[reduce-overhead-backward] 19.4240ms 18.7958ms 53.2034 Ops/s 53.6995 Ops/s $\color{#d91a1a}-0.92\%$
test_a2c_speed[False-None] 5.8228ms 5.5161ms 181.2869 Ops/s 180.1387 Ops/s $\color{#35bf28}+0.64\%$
test_a2c_speed[False-backward] 12.8339ms 12.1021ms 82.6306 Ops/s 83.7462 Ops/s $\color{#d91a1a}-1.33\%$
test_a2c_speed[True-None] 4.2006ms 3.8058ms 262.7601 Ops/s 262.5359 Ops/s $\color{#35bf28}+0.09\%$
test_a2c_speed[True-backward] 8.9281ms 8.5559ms 116.8789 Ops/s 113.0401 Ops/s $\color{#35bf28}+3.40\%$
test_a2c_speed[reduce-overhead-None] 4.1736ms 3.7641ms 265.6690 Ops/s 266.8302 Ops/s $\color{#d91a1a}-0.44\%$
test_a2c_speed[reduce-overhead-backward] 9.3072ms 8.9283ms 112.0035 Ops/s 112.3283 Ops/s $\color{#d91a1a}-0.29\%$
test_ppo_speed[False-None] 6.1894ms 5.9965ms 166.7633 Ops/s 166.1699 Ops/s $\color{#35bf28}+0.36\%$
test_ppo_speed[False-backward] 13.5578ms 12.8116ms 78.0545 Ops/s 78.3712 Ops/s $\color{#d91a1a}-0.40\%$
test_ppo_speed[True-None] 4.0650ms 3.6901ms 270.9979 Ops/s 274.7934 Ops/s $\color{#d91a1a}-1.38\%$
test_ppo_speed[True-backward] 8.8300ms 8.5516ms 116.9365 Ops/s 117.1479 Ops/s $\color{#d91a1a}-0.18\%$
test_ppo_speed[reduce-overhead-None] 3.8355ms 3.6495ms 274.0132 Ops/s 277.4217 Ops/s $\color{#d91a1a}-1.23\%$
test_ppo_speed[reduce-overhead-backward] 9.0817ms 8.7433ms 114.3728 Ops/s 112.9233 Ops/s $\color{#35bf28}+1.28\%$
test_reinforce_speed[False-None] 4.9817ms 4.6706ms 214.1053 Ops/s 210.0959 Ops/s $\color{#35bf28}+1.91\%$
test_reinforce_speed[False-backward] 7.8214ms 7.5373ms 132.6728 Ops/s 133.1311 Ops/s $\color{#d91a1a}-0.34\%$
test_reinforce_speed[True-None] 3.2706ms 2.9371ms 340.4664 Ops/s 322.0248 Ops/s $\textbf{\color{#35bf28}+5.73\%}$
test_reinforce_speed[True-backward] 8.0316ms 7.7641ms 128.7977 Ops/s 111.7321 Ops/s $\textbf{\color{#35bf28}+15.27\%}$
test_reinforce_speed[reduce-overhead-None] 3.2589ms 2.8925ms 345.7274 Ops/s 315.0380 Ops/s $\textbf{\color{#35bf28}+9.74\%}$
test_reinforce_speed[reduce-overhead-backward] 8.6280ms 8.0242ms 124.6230 Ops/s 116.8174 Ops/s $\textbf{\color{#35bf28}+6.68\%}$
test_iql_speed[False-None] 26.4926ms 20.5444ms 48.6751 Ops/s 48.8275 Ops/s $\color{#d91a1a}-0.31\%$
test_iql_speed[False-backward] 32.0629ms 30.6978ms 32.5757 Ops/s 32.3311 Ops/s $\color{#35bf28}+0.76\%$
test_iql_speed[True-None] 8.8691ms 8.6062ms 116.1953 Ops/s 113.2267 Ops/s $\color{#35bf28}+2.62\%$
test_iql_speed[True-backward] 17.5280ms 16.9411ms 59.0281 Ops/s 59.3277 Ops/s $\color{#d91a1a}-0.50\%$
test_iql_speed[reduce-overhead-None] 9.0774ms 8.6971ms 114.9812 Ops/s 112.7160 Ops/s $\color{#35bf28}+2.01\%$
test_iql_speed[reduce-overhead-backward] 17.8354ms 17.4652ms 57.2568 Ops/s 56.4751 Ops/s $\color{#35bf28}+1.38\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.2417ms 6.1432ms 162.7827 Ops/s 163.0762 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6426ms 0.3812ms 2.6230 KOps/s 2.7889 KOps/s $\textbf{\color{#d91a1a}-5.95\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7663ms 0.3436ms 2.9105 KOps/s 3.0013 KOps/s $\color{#d91a1a}-3.03\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1479ms 5.8844ms 169.9396 Ops/s 169.8016 Ops/s $\color{#35bf28}+0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1806ms 0.3039ms 3.2907 KOps/s 3.1517 KOps/s $\color{#35bf28}+4.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6540ms 0.3513ms 2.8464 KOps/s 3.4470 KOps/s $\textbf{\color{#d91a1a}-17.42\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6105ms 1.3711ms 729.3298 Ops/s 774.7139 Ops/s $\textbf{\color{#d91a1a}-5.86\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5879ms 1.2989ms 769.8594 Ops/s 829.8153 Ops/s $\textbf{\color{#d91a1a}-7.23\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.5352ms 6.1521ms 162.5452 Ops/s 166.8884 Ops/s $\color{#d91a1a}-2.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7655ms 0.4737ms 2.1110 KOps/s 2.2534 KOps/s $\textbf{\color{#d91a1a}-6.32\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7840ms 0.4827ms 2.0716 KOps/s 2.3525 KOps/s $\textbf{\color{#d91a1a}-11.94\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9841ms 5.8811ms 170.0355 Ops/s 168.4820 Ops/s $\color{#35bf28}+0.92\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7891ms 0.3288ms 3.0415 KOps/s 3.4233 KOps/s $\textbf{\color{#d91a1a}-11.15\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5828ms 0.3642ms 2.7461 KOps/s 3.4267 KOps/s $\textbf{\color{#d91a1a}-19.86\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1686ms 5.8325ms 171.4516 Ops/s 170.5878 Ops/s $\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0050ms 0.3675ms 2.7209 KOps/s 3.5478 KOps/s $\textbf{\color{#d91a1a}-23.31\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5653ms 0.3525ms 2.8370 KOps/s 3.7075 KOps/s $\textbf{\color{#d91a1a}-23.48\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0674ms 5.9313ms 168.5966 Ops/s 166.0514 Ops/s $\color{#35bf28}+1.53\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8639ms 0.5275ms 1.8957 KOps/s 2.2512 KOps/s $\textbf{\color{#d91a1a}-15.79\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.3473ms 0.5127ms 1.9504 KOps/s 1.9792 KOps/s $\color{#d91a1a}-1.46\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.5090ms 5.0753ms 197.0338 Ops/s 196.2256 Ops/s $\color{#35bf28}+0.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.0039ms 2.3554ms 424.5646 Ops/s 414.6205 Ops/s $\color{#35bf28}+2.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.9644ms 1.2178ms 821.1641 Ops/s 797.2176 Ops/s $\color{#35bf28}+3.00\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.6353s 17.7231ms 56.4234 Ops/s 52.6699 Ops/s $\textbf{\color{#35bf28}+7.13\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.7598ms 1.9508ms 512.6043 Ops/s 505.6322 Ops/s $\color{#35bf28}+1.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.2737ms 1.2265ms 815.3198 Ops/s 809.4917 Ops/s $\color{#35bf28}+0.72\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.5475ms 5.3169ms 188.0790 Ops/s 187.5586 Ops/s $\color{#35bf28}+0.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.8015ms 2.2661ms 441.2777 Ops/s 440.4695 Ops/s $\color{#35bf28}+0.18\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.5105ms 1.3700ms 729.9499 Ops/s 721.8493 Ops/s $\color{#35bf28}+1.12\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 36.1741ms 34.6266ms 28.8795 Ops/s 28.9966 Ops/s $\color{#d91a1a}-0.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 20.4457ms 18.6733ms 53.5524 Ops/s 56.4983 Ops/s $\textbf{\color{#d91a1a}-5.21\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 39.1199ms 35.7998ms 27.9331 Ops/s 28.0882 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.2690ms 18.3405ms 54.5241 Ops/s 55.3037 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 0.5749s 48.4106ms 20.6566 Ops/s 20.1392 Ops/s $\color{#35bf28}+2.57\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.4339ms 20.0197ms 49.9508 Ops/s 50.2837 Ops/s $\color{#d91a1a}-0.66\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens