[Refactor] Upgrade to torchcodec for video export

[vmoens]

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3540

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 2 Cancelled Jobs, 1 Pending

As of commit eed5ec0 with merge base ff77695 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark / GPU Pytest benchmark (gh)
  RuntimeError: Cannot sample from an empty storage.
• Generate documentation / build-docs (3.12, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 69f3c3ec8bbfda1924c2ed62042e900ebb085b684a50cc30bdda35ebde9d9827 /exec failed with exit code 2
• Libs Tests on Linux / unittests-sklearn (3.10, 12.8) / linux-job (gh)
  test/test_libs.py::TestOpenML::test_env[batch_size0-magic]
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_rb.py::TestStorages::test__rand_given_ndim_recompile
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_rb.py::TestStorages::test__rand_given_ndim_recompile
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_rb.py::TestStorages::test__rand_given_ndim_recompile
• Unit-tests on Linux / tests-cpu (3.13) / linux-job (gh)
  test/test_rb.py::TestStorages::test__rand_given_ndim_recompile
• Unit-tests on Linux / tests-olddeps (3.10, 11.8) / linux-job (gh)
  test/test_loggers.py::TestCSVLogger::test_log_video[mp4-steps1]
• Unit-tests on Linux / tests-optdeps (3.12, 13.0) / linux-job (gh)
  test/test_rb.py::TestStorages::test__rand_given_ndim_recompile

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark / CPU Pytest benchmark (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[NicolasHug]

write_video from torchvision was a self-contained helper that relied on pyav. If you want to avoid having to maintain multiple version of your own _write_video with different sets of dependencies for each, you could consider just vendoring write_video inside of torchRL

You'd have a single optional dependency on pyav instead of having an optional dep on both TV and TC, and you wouldn't have to worry about TV's versions either

[vmoens]

write_video from torchvision was a self-contained helper that relied on pyav. If you want to avoid having to maintain multiple version of your own _write_video with different sets of dependencies for each, you could consider just vendoring write_video inside of torchRL

Thanks!

We still use tv's transforms so tv dep is not a big worry. But the point of removing an intermediary makes total sense!

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 172. Improved: $\large\color{#35bf28}12$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	80.4499μs	79.5003μs	12.5786 KOps/s	12.2731 KOps/s	$\color{#35bf28}+2.49\%$
test_tensor_to_bytestream_speed[torch.save]	0.1451ms	0.1437ms	6.9597 KOps/s	7.0914 KOps/s	$\color{#d91a1a}-1.86\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1163s	0.1154s	8.6643 Ops/s	8.6357 Ops/s	$\color{#35bf28}+0.33\%$
test_tensor_to_bytestream_speed[numpy]	2.5085μs	2.5024μs	399.6194 KOps/s	388.9869 KOps/s	$\color{#35bf28}+2.73\%$
test_tensor_to_bytestream_speed[safetensors]	40.5241μs	40.2717μs	24.8313 KOps/s	25.8668 KOps/s	$\color{#d91a1a}-4.00\%$
test_simple	0.7910s	0.7895s	1.2666 Ops/s	1.2252 Ops/s	$\color{#35bf28}+3.38\%$
test_transformed	1.3849s	1.3819s	0.7236 Ops/s	0.7063 Ops/s	$\color{#35bf28}+2.46\%$
test_serial	2.3213s	2.3191s	0.4312 Ops/s	0.4172 Ops/s	$\color{#35bf28}+3.35\%$
test_parallel	1.9238s	1.8409s	0.5432 Ops/s	0.5476 Ops/s	$\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-True-True-True]	0.2954ms	42.5110μs	23.5233 KOps/s	23.5593 KOps/s	$\color{#d91a1a}-0.15\%$
test_step_mdp_speed[True-True-True-True-False]	75.7610μs	23.5008μs	42.5517 KOps/s	42.7545 KOps/s	$\color{#d91a1a}-0.47\%$
test_step_mdp_speed[True-True-True-False-True]	70.0110μs	23.8330μs	41.9586 KOps/s	41.3525 KOps/s	$\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-True-True-False-False]	42.5100μs	13.2121μs	75.6883 KOps/s	77.5357 KOps/s	$\color{#d91a1a}-2.38\%$
test_step_mdp_speed[True-True-False-True-True]	80.7110μs	45.9116μs	21.7810 KOps/s	22.1080 KOps/s	$\color{#d91a1a}-1.48\%$
test_step_mdp_speed[True-True-False-True-False]	68.0000μs	26.2525μs	38.0915 KOps/s	38.7233 KOps/s	$\color{#d91a1a}-1.63\%$
test_step_mdp_speed[True-True-False-False-True]	60.7700μs	26.5241μs	37.7015 KOps/s	37.4422 KOps/s	$\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-False-False-False]	56.8200μs	15.8364μs	63.1456 KOps/s	63.7102 KOps/s	$\color{#d91a1a}-0.89\%$
test_step_mdp_speed[True-False-True-True-True]	84.2710μs	48.6425μs	20.5581 KOps/s	20.7364 KOps/s	$\color{#d91a1a}-0.86\%$
test_step_mdp_speed[True-False-True-True-False]	97.0510μs	28.3891μs	35.2248 KOps/s	34.8324 KOps/s	$\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-False-True-False-True]	62.4010μs	26.8898μs	37.1888 KOps/s	37.1737 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-False-True-False-False]	46.2610μs	15.7731μs	63.3991 KOps/s	63.7573 KOps/s	$\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-False-False-True-True]	83.8400μs	50.9811μs	19.6151 KOps/s	19.8439 KOps/s	$\color{#d91a1a}-1.15\%$
test_step_mdp_speed[True-False-False-True-False]	64.3810μs	31.1286μs	32.1248 KOps/s	32.0667 KOps/s	$\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-False-False-True]	62.8400μs	28.9152μs	34.5839 KOps/s	34.0025 KOps/s	$\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-False-False-False-False]	66.6300μs	18.0485μs	55.4063 KOps/s	54.1393 KOps/s	$\color{#35bf28}+2.34\%$
test_step_mdp_speed[False-True-True-True-True]	0.1019ms	48.4855μs	20.6247 KOps/s	20.7152 KOps/s	$\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-True-True-True-False]	62.0910μs	28.8217μs	34.6961 KOps/s	34.6376 KOps/s	$\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-True-True-False-True]	2.3606ms	30.2659μs	33.0405 KOps/s	32.3812 KOps/s	$\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-True-True-False-False]	70.2610μs	17.4324μs	57.3646 KOps/s	57.9637 KOps/s	$\color{#d91a1a}-1.03\%$
test_step_mdp_speed[False-True-False-True-True]	83.5110μs	50.4541μs	19.8200 KOps/s	19.7366 KOps/s	$\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-True-False-True-False]	61.9410μs	31.3999μs	31.8473 KOps/s	32.1685 KOps/s	$\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-True-False-False-True]	66.2710μs	33.0065μs	30.2971 KOps/s	30.0314 KOps/s	$\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-True-False-False-False]	61.1200μs	20.0620μs	49.8454 KOps/s	50.5345 KOps/s	$\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-False-True-True-True]	88.7010μs	53.5375μs	18.6785 KOps/s	18.5836 KOps/s	$\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-True-True-False]	93.1310μs	33.5716μs	29.7870 KOps/s	29.2209 KOps/s	$\color{#35bf28}+1.94\%$
test_step_mdp_speed[False-False-True-False-True]	99.7310μs	33.0164μs	30.2880 KOps/s	30.1247 KOps/s	$\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-False-True-False-False]	54.9600μs	19.8586μs	50.3560 KOps/s	49.6922 KOps/s	$\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-False-False-True-True]	0.1071ms	54.7670μs	18.2592 KOps/s	17.9384 KOps/s	$\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-False-False-True-False]	69.0210μs	36.1937μs	27.6292 KOps/s	27.3277 KOps/s	$\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-False-False-True]	83.8010μs	34.9775μs	28.5898 KOps/s	28.2494 KOps/s	$\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-False-False-False-False]	71.4610μs	22.4932μs	44.4578 KOps/s	44.6795 KOps/s	$\color{#d91a1a}-0.50\%$
test_non_tensor_env_rollout_speed[1000-single-True]	0.8503s	0.7565s	1.3219 Ops/s	1.3169 Ops/s	$\color{#35bf28}+0.38\%$
test_non_tensor_env_rollout_speed[1000-single-False]	0.7039s	0.6087s	1.6429 Ops/s	1.6251 Ops/s	$\color{#35bf28}+1.10\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True]	1.7380s	1.6591s	0.6027 Ops/s	0.6031 Ops/s	$\color{#d91a1a}-0.07\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False]	1.5076s	1.4308s	0.6989 Ops/s	0.6958 Ops/s	$\color{#35bf28}+0.45\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True]	1.9899s	1.9122s	0.5229 Ops/s	0.5222 Ops/s	$\color{#35bf28}+0.15\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False]	1.7648s	1.6855s	0.5933 Ops/s	0.5922 Ops/s	$\color{#35bf28}+0.19\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True]	4.7758s	4.6568s	0.2147 Ops/s	0.2141 Ops/s	$\color{#35bf28}+0.31\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False]	4.6990s	4.4806s	0.2232 Ops/s	0.2216 Ops/s	$\color{#35bf28}+0.73\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True]	1.9765s	1.8976s	0.5270 Ops/s	0.5157 Ops/s	$\color{#35bf28}+2.19\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False]	1.8229s	1.6535s	0.6048 Ops/s	0.6135 Ops/s	$\color{#d91a1a}-1.42\%$
test_values[generalized_advantage_estimate-True-True]	21.4751ms	20.6633ms	48.3950 Ops/s	47.4457 Ops/s	$\color{#35bf28}+2.00\%$
test_values[vec_generalized_advantage_estimate-True-True]	0.1281s	3.4935ms	286.2474 Ops/s	262.1013 Ops/s	$\textbf{\color{#35bf28}+9.21\%}$
test_values[td0_return_estimate-False-False]	0.1092ms	85.7151μs	11.6666 KOps/s	11.4886 KOps/s	$\color{#35bf28}+1.55\%$
test_values[td1_return_estimate-False-False]	51.4019ms	49.3809ms	20.2507 Ops/s	19.7994 Ops/s	$\color{#35bf28}+2.28\%$
test_values[vec_td1_return_estimate-False-False]	1.3578ms	1.1037ms	906.0822 Ops/s	896.1862 Ops/s	$\color{#35bf28}+1.10\%$
test_values[td_lambda_return_estimate-True-False]	84.1615ms	81.0140ms	12.3435 Ops/s	12.1574 Ops/s	$\color{#35bf28}+1.53\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.3104ms	1.0989ms	910.0008 Ops/s	905.4713 Ops/s	$\color{#35bf28}+0.50\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	22.1045ms	21.9103ms	45.6405 Ops/s	47.8966 Ops/s	$\color{#d91a1a}-4.71\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.0740ms	0.7717ms	1.2958 KOps/s	1.2796 KOps/s	$\color{#35bf28}+1.27\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7283ms	0.6850ms	1.4598 KOps/s	1.4063 KOps/s	$\color{#35bf28}+3.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5690ms	1.5084ms	662.9667 Ops/s	661.4046 Ops/s	$\color{#35bf28}+0.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7510ms	0.7039ms	1.4207 KOps/s	1.4143 KOps/s	$\color{#35bf28}+0.45\%$
test_dqn_speed[False-None]	1.6672ms	1.5492ms	645.4828 Ops/s	634.6639 Ops/s	$\color{#35bf28}+1.70\%$
test_dqn_speed[False-backward]	2.4170ms	2.2371ms	447.0109 Ops/s	442.7901 Ops/s	$\color{#35bf28}+0.95\%$
test_dqn_speed[True-None]	0.7389ms	0.5937ms	1.6844 KOps/s	1.6934 KOps/s	$\color{#d91a1a}-0.53\%$
test_dqn_speed[True-backward]	1.3108ms	1.2555ms	796.5009 Ops/s	788.0376 Ops/s	$\color{#35bf28}+1.07\%$
test_dqn_speed[reduce-overhead-None]	0.6751ms	0.6067ms	1.6483 KOps/s	1.6002 KOps/s	$\color{#35bf28}+3.00\%$
test_ddpg_speed[False-None]	3.3484ms	2.9188ms	342.6085 Ops/s	341.9660 Ops/s	$\color{#35bf28}+0.19\%$
test_ddpg_speed[False-backward]	4.8912ms	4.4328ms	225.5911 Ops/s	223.7216 Ops/s	$\color{#35bf28}+0.84\%$
test_ddpg_speed[True-None]	1.4757ms	1.3967ms	715.9566 Ops/s	708.2588 Ops/s	$\color{#35bf28}+1.09\%$
test_ddpg_speed[True-backward]	2.7571ms	2.6505ms	377.2837 Ops/s	370.0794 Ops/s	$\color{#35bf28}+1.95\%$
test_ddpg_speed[reduce-overhead-None]	1.4868ms	1.4270ms	700.7656 Ops/s	693.9399 Ops/s	$\color{#35bf28}+0.98\%$
test_sac_speed[False-None]	8.9635ms	8.5426ms	117.0608 Ops/s	116.8936 Ops/s	$\color{#35bf28}+0.14\%$
test_sac_speed[False-backward]	12.4460ms	11.9199ms	83.8932 Ops/s	81.7610 Ops/s	$\color{#35bf28}+2.61\%$
test_sac_speed[True-None]	2.9499ms	1.9096ms	523.6624 Ops/s	518.7080 Ops/s	$\color{#35bf28}+0.96\%$
test_sac_speed[True-backward]	3.8491ms	3.7955ms	263.4692 Ops/s	260.0670 Ops/s	$\color{#35bf28}+1.31\%$
test_sac_speed[reduce-overhead-None]	16.1985ms	9.8025ms	102.0151 Ops/s	252.7298 Ops/s	$\textbf{\color{#d91a1a}-59.63\%}$
test_redq_deprec_speed[False-None]	10.2717ms	9.5598ms	104.6044 Ops/s	101.7351 Ops/s	$\color{#35bf28}+2.82\%$
test_redq_deprec_speed[False-backward]	13.5797ms	13.1570ms	76.0049 Ops/s	75.5343 Ops/s	$\color{#35bf28}+0.62\%$
test_redq_deprec_speed[True-None]	3.0475ms	2.6587ms	376.1284 Ops/s	377.8536 Ops/s	$\color{#d91a1a}-0.46\%$
test_redq_deprec_speed[True-backward]	4.8376ms	4.4984ms	222.3025 Ops/s	225.5665 Ops/s	$\color{#d91a1a}-1.45\%$
test_redq_deprec_speed[reduce-overhead-None]	14.8067ms	9.6794ms	103.3118 Ops/s	102.8430 Ops/s	$\color{#35bf28}+0.46\%$
test_td3_speed[False-None]	8.5824ms	8.4311ms	118.6083 Ops/s	118.1726 Ops/s	$\color{#35bf28}+0.37\%$
test_td3_speed[False-backward]	11.8388ms	11.2178ms	89.1439 Ops/s	90.5256 Ops/s	$\color{#d91a1a}-1.53\%$
test_td3_speed[True-None]	1.7166ms	1.6866ms	592.9214 Ops/s	580.1611 Ops/s	$\color{#35bf28}+2.20\%$
test_td3_speed[True-backward]	3.4409ms	3.3944ms	294.6046 Ops/s	288.5521 Ops/s	$\color{#35bf28}+2.10\%$
test_td3_speed[reduce-overhead-None]	60.9000ms	25.8099ms	38.7448 Ops/s	38.3862 Ops/s	$\color{#35bf28}+0.93\%$
test_cql_speed[False-None]	17.9684ms	17.6907ms	56.5270 Ops/s	55.6928 Ops/s	$\color{#35bf28}+1.50\%$
test_cql_speed[False-backward]	24.0287ms	23.6315ms	42.3163 Ops/s	41.5586 Ops/s	$\color{#35bf28}+1.82\%$
test_cql_speed[True-None]	3.6729ms	3.4361ms	291.0238 Ops/s	291.0121 Ops/s	$+0.00\%$
test_cql_speed[True-backward]	6.0776ms	5.6246ms	177.7905 Ops/s	169.5908 Ops/s	$\color{#35bf28}+4.84\%$
test_cql_speed[reduce-overhead-None]	17.7993ms	12.0814ms	82.7719 Ops/s	54.6180 Ops/s	$\textbf{\color{#35bf28}+51.55\%}$
test_a2c_speed[False-None]	4.1131ms	3.2928ms	303.6970 Ops/s	300.2058 Ops/s	$\color{#35bf28}+1.16\%$
test_a2c_speed[False-backward]	6.7818ms	6.3122ms	158.4231 Ops/s	150.6206 Ops/s	$\textbf{\color{#35bf28}+5.18\%}$
test_a2c_speed[True-None]	1.4372ms	1.3708ms	729.4770 Ops/s	723.0219 Ops/s	$\color{#35bf28}+0.89\%$
test_a2c_speed[True-backward]	3.2141ms	3.0484ms	328.0402 Ops/s	305.1701 Ops/s	$\textbf{\color{#35bf28}+7.49\%}$
test_a2c_speed[reduce-overhead-None]	1.0885ms	1.0290ms	971.8533 Ops/s	961.9360 Ops/s	$\color{#35bf28}+1.03\%$
test_ppo_speed[False-None]	4.1782ms	3.9549ms	252.8489 Ops/s	249.3735 Ops/s	$\color{#35bf28}+1.39\%$
test_ppo_speed[False-backward]	7.6574ms	7.2472ms	137.9852 Ops/s	133.4593 Ops/s	$\color{#35bf28}+3.39\%$
test_ppo_speed[True-None]	1.5937ms	1.5155ms	659.8532 Ops/s	660.3610 Ops/s	$\color{#d91a1a}-0.08\%$
test_ppo_speed[True-backward]	3.2445ms	3.2068ms	311.8373 Ops/s	303.7035 Ops/s	$\color{#35bf28}+2.68\%$
test_ppo_speed[reduce-overhead-None]	1.3146ms	1.1023ms	907.1929 Ops/s	904.4735 Ops/s	$\color{#35bf28}+0.30\%$
test_reinforce_speed[False-None]	2.9637ms	2.3308ms	429.0434 Ops/s	423.2411 Ops/s	$\color{#35bf28}+1.37\%$
test_reinforce_speed[False-backward]	3.4199ms	3.3818ms	295.7041 Ops/s	282.2973 Ops/s	$\color{#35bf28}+4.75\%$
test_reinforce_speed[True-None]	1.5212ms	1.3551ms	737.9660 Ops/s	737.5636 Ops/s	$\color{#35bf28}+0.05\%$
test_reinforce_speed[True-backward]	3.1144ms	3.0434ms	328.5817 Ops/s	311.7557 Ops/s	$\textbf{\color{#35bf28}+5.40\%}$
test_reinforce_speed[reduce-overhead-None]	16.0766ms	9.1158ms	109.7001 Ops/s	110.7725 Ops/s	$\color{#d91a1a}-0.97\%$
test_iql_speed[False-None]	10.4680ms	9.6646ms	103.4708 Ops/s	102.6608 Ops/s	$\color{#35bf28}+0.79\%$
test_iql_speed[False-backward]	14.2768ms	13.5490ms	73.8059 Ops/s	71.8371 Ops/s	$\color{#35bf28}+2.74\%$
test_iql_speed[True-None]	2.3503ms	2.2622ms	442.0533 Ops/s	435.1229 Ops/s	$\color{#35bf28}+1.59\%$
test_iql_speed[True-backward]	5.4017ms	4.9301ms	202.8348 Ops/s	201.8357 Ops/s	$\color{#35bf28}+0.50\%$
test_iql_speed[reduce-overhead-None]	0.7626s	13.0160ms	76.8284 Ops/s	99.3021 Ops/s	$\textbf{\color{#d91a1a}-22.63\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5976ms	6.0005ms	166.6524 Ops/s	164.5406 Ops/s	$\color{#35bf28}+1.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5433ms	0.3420ms	2.9241 KOps/s	3.4371 KOps/s	$\textbf{\color{#d91a1a}-14.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7133ms	0.3279ms	3.0499 KOps/s	3.7103 KOps/s	$\textbf{\color{#d91a1a}-17.80\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.4055ms	5.8440ms	171.1147 Ops/s	167.7064 Ops/s	$\color{#35bf28}+2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.5922ms	0.2799ms	3.5730 KOps/s	3.5108 KOps/s	$\color{#35bf28}+1.77\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4798ms	0.2690ms	3.7175 KOps/s	3.7530 KOps/s	$\color{#d91a1a}-0.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.7379ms	1.2861ms	777.5246 Ops/s	766.8038 Ops/s	$\color{#35bf28}+1.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.4202ms	1.1903ms	840.0969 Ops/s	827.0430 Ops/s	$\color{#35bf28}+1.58\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.2449ms	6.0380ms	165.6187 Ops/s	162.2827 Ops/s	$\color{#35bf28}+2.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1226ms	0.4854ms	2.0600 KOps/s	2.0435 KOps/s	$\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6322ms	0.4343ms	2.3027 KOps/s	2.3488 KOps/s	$\color{#d91a1a}-1.96\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.1086ms	5.8874ms	169.8557 Ops/s	165.1161 Ops/s	$\color{#35bf28}+2.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.0082ms	0.2864ms	3.4916 KOps/s	3.1664 KOps/s	$\textbf{\color{#35bf28}+10.27\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.4691ms	0.2654ms	3.7681 KOps/s	3.2880 KOps/s	$\textbf{\color{#35bf28}+14.60\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.2043ms	5.8442ms	171.1103 Ops/s	167.6332 Ops/s	$\color{#35bf28}+2.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9385ms	0.3569ms	2.8018 KOps/s	2.6620 KOps/s	$\textbf{\color{#35bf28}+5.25\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5019ms	0.2638ms	3.7901 KOps/s	2.8915 KOps/s	$\textbf{\color{#35bf28}+31.08\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1921ms	5.9909ms	166.9201 Ops/s	162.5502 Ops/s	$\color{#35bf28}+2.69\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9614ms	0.5142ms	1.9449 KOps/s	2.2588 KOps/s	$\textbf{\color{#d91a1a}-13.90\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6683ms	0.4988ms	2.0047 KOps/s	2.0955 KOps/s	$\color{#d91a1a}-4.33\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.6063ms	5.1269ms	195.0490 Ops/s	193.7982 Ops/s	$\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	10.5463ms	1.9630ms	509.4163 Ops/s	468.8869 Ops/s	$\textbf{\color{#35bf28}+8.64\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	8.5179ms	1.3320ms	750.7372 Ops/s	798.5397 Ops/s	$\textbf{\color{#d91a1a}-5.99\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	6.7715ms	5.1453ms	194.3507 Ops/s	44.7921 Ops/s	$\textbf{\color{#35bf28}+333.89\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	12.3035ms	2.0528ms	487.1354 Ops/s	498.7867 Ops/s	$\color{#d91a1a}-2.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.0682ms	0.9536ms	1.0486 KOps/s	978.8847 Ops/s	$\textbf{\color{#35bf28}+7.12\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.6881s	19.0522ms	52.4873 Ops/s	184.8586 Ops/s	$\textbf{\color{#d91a1a}-71.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	13.7138ms	2.2886ms	436.9556 Ops/s	465.7646 Ops/s	$\textbf{\color{#d91a1a}-6.19\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.1910ms	1.2109ms	825.8610 Ops/s	834.4042 Ops/s	$\color{#d91a1a}-1.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	39.8375ms	36.4607ms	27.4268 Ops/s	27.3816 Ops/s	$\color{#35bf28}+0.16\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	20.5602ms	18.6507ms	53.6173 Ops/s	52.4189 Ops/s	$\color{#35bf28}+2.29\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	40.7746ms	37.6756ms	26.5424 Ops/s	26.1700 Ops/s	$\color{#35bf28}+1.42\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	20.6902ms	18.8700ms	52.9942 Ops/s	53.1167 Ops/s	$\color{#d91a1a}-0.23\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	41.8250ms	39.4467ms	25.3506 Ops/s	24.9233 Ops/s	$\color{#35bf28}+1.71\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	21.7880ms	20.6445ms	48.4391 Ops/s	48.2100 Ops/s	$\color{#35bf28}+0.48\%$
test_storage_write_lazystack[50-img_shape0-small]	0.8684ms	0.2281ms	4.3845 KOps/s	4.4655 KOps/s	$\color{#d91a1a}-1.81\%$
test_storage_write_lazystack[100-img_shape1-atari]	1.8662ms	1.3953ms	716.7025 Ops/s	704.0582 Ops/s	$\color{#35bf28}+1.80\%$
test_storage_write_lazystack[100-img_shape2-large_img]	2.4612ms	2.3115ms	432.6219 Ops/s	435.9519 Ops/s	$\color{#d91a1a}-0.76\%$
test_storage_write_lazystack[200-img_shape3-large_batch]	3.4446ms	2.8708ms	348.3361 Ops/s	338.1843 Ops/s	$\color{#35bf28}+3.00\%$
test_storage_write_contiguous[50-img_shape0-small]	0.2599ms	0.1694ms	5.9018 KOps/s	6.0280 KOps/s	$\color{#d91a1a}-2.09\%$
test_storage_write_contiguous[100-img_shape1-atari]	0.3856ms	0.2391ms	4.1816 KOps/s	4.1349 KOps/s	$\color{#35bf28}+1.13\%$
test_storage_write_contiguous[100-img_shape2-large_img]	2.3281ms	1.7292ms	578.2985 Ops/s	570.0773 Ops/s	$\color{#35bf28}+1.44\%$
test_storage_write_contiguous[200-img_shape3-large_batch]	1.5010ms	1.3047ms	766.4464 Ops/s	749.9749 Ops/s	$\color{#35bf28}+2.20\%$
test_collector_stack_then_write[50-img_shape0-small]	1.3833ms	1.1579ms	863.5977 Ops/s	861.7347 Ops/s	$\color{#35bf28}+0.22\%$
test_collector_stack_then_write[100-img_shape1-atari]	7.6661ms	3.6995ms	270.3072 Ops/s	266.3996 Ops/s	$\color{#35bf28}+1.47\%$
test_collector_stack_then_write[100-img_shape2-large_img]	11.2106ms	5.8571ms	170.7324 Ops/s	170.3032 Ops/s	$\color{#35bf28}+0.25\%$
test_collector_stack_then_write[200-img_shape3-large_batch]	14.9021ms	7.1164ms	140.5200 Ops/s	140.9817 Ops/s	$\color{#d91a1a}-0.33\%$
test_collector_lazystack_then_write[50-img_shape0-small]	0.4241ms	0.2758ms	3.6262 KOps/s	3.5281 KOps/s	$\color{#35bf28}+2.78\%$
test_collector_lazystack_then_write[100-img_shape1-atari]	1.6894ms	1.5203ms	657.7656 Ops/s	655.6796 Ops/s	$\color{#35bf28}+0.32\%$
test_collector_lazystack_then_write[100-img_shape2-large_img]	2.5926ms	2.4495ms	408.2527 Ops/s	411.8939 Ops/s	$\color{#d91a1a}-0.88\%$
test_collector_lazystack_then_write[200-img_shape3-large_batch]	3.4300ms	3.0850ms	324.1486 Ops/s	314.5848 Ops/s	$\color{#35bf28}+3.04\%$
test_collector_without_rb[100-img_shape0-atari]	33.9002ms	32.9218ms	30.3750 Ops/s	29.8483 Ops/s	$\color{#35bf28}+1.76\%$
test_collector_without_rb[200-img_shape1-large_batch]	65.9279ms	64.6012ms	15.4796 Ops/s	15.2226 Ops/s	$\color{#35bf28}+1.69\%$
test_collector_with_rb[100-img_shape0-atari]	38.3944ms	37.4628ms	26.6931 Ops/s	26.1768 Ops/s	$\color{#35bf28}+1.97\%$
test_collector_with_rb[200-img_shape1-large_batch]	74.6524ms	74.0440ms	13.5055 Ops/s	13.2220 Ops/s	$\color{#35bf28}+2.14\%$
test_collector_without_rb_cuda[100-img_shape0-atari]	59.8574ms	56.9423ms	17.5617 Ops/s	17.5821 Ops/s	$\color{#d91a1a}-0.12\%$
test_collector_without_rb_cuda[200-img_shape1-large_batch]	0.1153s	0.1132s	8.8353 Ops/s	8.8165 Ops/s	$\color{#35bf28}+0.21\%$
test_collector_with_rb_cuda[100-img_shape0-atari]	59.9090ms	58.0007ms	17.2412 Ops/s	16.8847 Ops/s	$\color{#35bf28}+2.11\%$
test_collector_with_rb_cuda[200-img_shape1-large_batch]	0.1196s	0.1158s	8.6367 Ops/s	8.5117 Ops/s	$\color{#35bf28}+1.47\%$