[Feature] Make ProbabilisticActor compatible with Composite distributions

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2220

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unrelated Failures

As of commit 0c604e7 with merge base 3787a9e ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 00abb6bb74b55be0a92ee7367d04ad5dd1eebae9a18796a38be541a274e2d517 /exec failed with exit code 139
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 4fe5ec11ab9280f6a462ce1a36aee3a6b90240942fc9d4a2212c85cb53c9e9e8 /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1208s	60.4128ms	16.5528 Ops/s	17.1410 Ops/s	$\color{#d91a1a}-3.43\%$
test_sync	39.0389ms	31.4674ms	31.7789 Ops/s	31.1697 Ops/s	$\color{#35bf28}+1.95\%$
test_async	59.2461ms	29.6140ms	33.7678 Ops/s	35.9820 Ops/s	$\textbf{\color{#d91a1a}-6.15\%}$
test_simple	0.4580s	0.3940s	2.5378 Ops/s	2.6044 Ops/s	$\color{#d91a1a}-2.56\%$
test_transformed	0.5359s	0.5328s	1.8767 Ops/s	1.7639 Ops/s	$\textbf{\color{#35bf28}+6.40\%}$
test_serial	1.3241s	1.2668s	0.7894 Ops/s	0.7623 Ops/s	$\color{#35bf28}+3.54\%$
test_parallel	1.1440s	1.0817s	0.9245 Ops/s	0.9298 Ops/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-True-True-True-True]	0.1996ms	21.8022μs	45.8670 KOps/s	46.4636 KOps/s	$\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-True-True-True-False]	50.7350μs	13.1343μs	76.1367 KOps/s	75.9406 KOps/s	$\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-True-False-True]	51.1060μs	12.7895μs	78.1891 KOps/s	77.6131 KOps/s	$\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-True-False-False]	52.6690μs	7.7689μs	128.7178 KOps/s	128.1189 KOps/s	$\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-True-False-True-True]	57.6880μs	23.2292μs	43.0493 KOps/s	43.9340 KOps/s	$\color{#d91a1a}-2.01\%$
test_step_mdp_speed[True-True-False-True-False]	51.4970μs	14.6292μs	68.3563 KOps/s	68.5721 KOps/s	$\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-True-False-False-True]	41.5280μs	14.2224μs	70.3118 KOps/s	70.8934 KOps/s	$\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-True-False-False-False]	43.0410μs	9.1229μs	109.6144 KOps/s	111.7511 KOps/s	$\color{#d91a1a}-1.91\%$
test_step_mdp_speed[True-False-True-True-True]	59.1800μs	24.6642μs	40.5445 KOps/s	40.7595 KOps/s	$\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-False-True-True-False]	62.0760μs	15.9678μs	62.6262 KOps/s	63.9270 KOps/s	$\color{#d91a1a}-2.03\%$
test_step_mdp_speed[True-False-True-False-True]	45.3850μs	14.1810μs	70.5171 KOps/s	70.3476 KOps/s	$\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-False-True-False-False]	45.4850μs	9.0096μs	110.9932 KOps/s	109.6319 KOps/s	$\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-False-False-True-True]	62.8470μs	25.8042μs	38.7533 KOps/s	39.7456 KOps/s	$\color{#d91a1a}-2.50\%$
test_step_mdp_speed[True-False-False-True-False]	56.8470μs	17.0949μs	58.4968 KOps/s	58.4584 KOps/s	$\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-False-False-False-True]	62.7670μs	15.3189μs	65.2787 KOps/s	65.8038 KOps/s	$\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-False-False-False-False]	37.8510μs	10.1396μs	98.6232 KOps/s	98.2408 KOps/s	$\color{#35bf28}+0.39\%$
test_step_mdp_speed[False-True-True-True-True]	59.9630μs	24.6724μs	40.5312 KOps/s	41.3775 KOps/s	$\color{#d91a1a}-2.05\%$
test_step_mdp_speed[False-True-True-True-False]	50.4140μs	15.8413μs	63.1260 KOps/s	63.1297 KOps/s	$-0.01\%$
test_step_mdp_speed[False-True-True-False-True]	66.4550μs	16.4397μs	60.8285 KOps/s	61.5912 KOps/s	$\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-True-False-False]	50.3340μs	10.2266μs	97.7839 KOps/s	98.0596 KOps/s	$\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-True-False-True-True]	75.4510μs	25.8094μs	38.7456 KOps/s	39.0918 KOps/s	$\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-True-False-True-False]	42.5700μs	17.0817μs	58.5421 KOps/s	58.6418 KOps/s	$\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-True-False-False-True]	43.7120μs	17.6920μs	56.5228 KOps/s	57.2082 KOps/s	$\color{#d91a1a}-1.20\%$
test_step_mdp_speed[False-True-False-False-False]	37.3900μs	11.5506μs	86.5752 KOps/s	87.9441 KOps/s	$\color{#d91a1a}-1.56\%$
test_step_mdp_speed[False-False-True-True-True]	65.7030μs	27.1248μs	36.8667 KOps/s	37.2953 KOps/s	$\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-False-True-True-False]	50.4950μs	18.4632μs	54.1617 KOps/s	54.2735 KOps/s	$\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-False-True-False-True]	64.2300μs	17.5995μs	56.8196 KOps/s	58.1199 KOps/s	$\color{#d91a1a}-2.24\%$
test_step_mdp_speed[False-False-True-False-False]	53.2400μs	11.5040μs	86.9266 KOps/s	88.5161 KOps/s	$\color{#d91a1a}-1.80\%$
test_step_mdp_speed[False-False-False-True-True]	44.6430μs	28.6723μs	34.8769 KOps/s	35.5866 KOps/s	$\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-False-False-True-False]	44.5630μs	19.6457μs	50.9017 KOps/s	52.3181 KOps/s	$\color{#d91a1a}-2.71\%$
test_step_mdp_speed[False-False-False-False-True]	60.7130μs	18.7601μs	53.3045 KOps/s	54.7993 KOps/s	$\color{#d91a1a}-2.73\%$
test_step_mdp_speed[False-False-False-False-False]	38.0910μs	12.6421μs	79.1010 KOps/s	80.4149 KOps/s	$\color{#d91a1a}-1.63\%$
test_values[generalized_advantage_estimate-True-True]	9.9859ms	9.6262ms	103.8830 Ops/s	105.7405 Ops/s	$\color{#d91a1a}-1.76\%$
test_values[vec_generalized_advantage_estimate-True-True]	50.3621ms	35.6645ms	28.0391 Ops/s	28.3836 Ops/s	$\color{#d91a1a}-1.21\%$
test_values[td0_return_estimate-False-False]	0.2426ms	0.1787ms	5.5955 KOps/s	5.8626 KOps/s	$\color{#d91a1a}-4.56\%$
test_values[td1_return_estimate-False-False]	25.8171ms	23.7707ms	42.0686 Ops/s	41.6069 Ops/s	$\color{#35bf28}+1.11\%$
test_values[vec_td1_return_estimate-False-False]	37.9262ms	35.4332ms	28.2221 Ops/s	28.2708 Ops/s	$\color{#d91a1a}-0.17\%$
test_values[td_lambda_return_estimate-True-False]	35.7745ms	34.3421ms	29.1188 Ops/s	28.3258 Ops/s	$\color{#35bf28}+2.80\%$
test_values[vec_td_lambda_return_estimate-True-False]	38.5542ms	35.3193ms	28.3131 Ops/s	28.2918 Ops/s	$\color{#35bf28}+0.08\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.3434ms	8.2076ms	121.8386 Ops/s	115.8715 Ops/s	$\textbf{\color{#35bf28}+5.15\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.2362ms	1.8099ms	552.5144 Ops/s	488.3303 Ops/s	$\textbf{\color{#35bf28}+13.14\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4544ms	0.3515ms	2.8446 KOps/s	2.7458 KOps/s	$\color{#35bf28}+3.60\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	48.5960ms	45.9510ms	21.7623 Ops/s	22.3051 Ops/s	$\color{#d91a1a}-2.43\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.7558ms	3.0477ms	328.1145 Ops/s	321.3985 Ops/s	$\color{#35bf28}+2.09\%$
test_dqn_speed	1.5079ms	1.3618ms	734.3254 Ops/s	717.5998 Ops/s	$\color{#35bf28}+2.33\%$
test_ddpg_speed	3.4643ms	2.8720ms	348.1929 Ops/s	336.1410 Ops/s	$\color{#35bf28}+3.59\%$
test_sac_speed	10.0632ms	8.5702ms	116.6839 Ops/s	113.3896 Ops/s	$\color{#35bf28}+2.91\%$
test_redq_speed	15.0874ms	13.5657ms	73.7153 Ops/s	70.5450 Ops/s	$\color{#35bf28}+4.49\%$
test_redq_deprec_speed	22.4671ms	14.2169ms	70.3386 Ops/s	67.2040 Ops/s	$\color{#35bf28}+4.66\%$
test_td3_speed	8.8279ms	8.4822ms	117.8935 Ops/s	114.1373 Ops/s	$\color{#35bf28}+3.29\%$
test_cql_speed	39.6776ms	37.3588ms	26.7674 Ops/s	26.7660 Ops/s	$+0.01\%$
test_a2c_speed	8.8029ms	7.6324ms	131.0204 Ops/s	129.0036 Ops/s	$\color{#35bf28}+1.56\%$
test_ppo_speed	8.8209ms	7.8519ms	127.3574 Ops/s	126.4879 Ops/s	$\color{#35bf28}+0.69\%$
test_reinforce_speed	10.9492ms	6.8321ms	146.3680 Ops/s	147.5586 Ops/s	$\color{#d91a1a}-0.81\%$
test_iql_speed	39.5787ms	33.7253ms	29.6513 Ops/s	29.7755 Ops/s	$\color{#d91a1a}-0.42\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.2955ms	3.6742ms	272.1716 Ops/s	269.6716 Ops/s	$\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8564ms	0.5082ms	1.9678 KOps/s	1.9199 KOps/s	$\color{#35bf28}+2.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	1.0516ms	0.4811ms	2.0787 KOps/s	2.0442 KOps/s	$\color{#35bf28}+1.69\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.3884ms	3.6517ms	273.8470 Ops/s	268.3054 Ops/s	$\color{#35bf28}+2.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.0365ms	0.5067ms	1.9737 KOps/s	1.9432 KOps/s	$\color{#35bf28}+1.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.8008ms	0.4794ms	2.0858 KOps/s	2.0724 KOps/s	$\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.1507ms	1.7149ms	583.1131 Ops/s	574.9517 Ops/s	$\color{#35bf28}+1.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.7742ms	1.6187ms	617.7838 Ops/s	605.2823 Ops/s	$\color{#35bf28}+2.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.4299ms	3.8099ms	262.4762 Ops/s	262.9240 Ops/s	$\color{#d91a1a}-0.17\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1402ms	0.6296ms	1.5884 KOps/s	1.3616 KOps/s	$\textbf{\color{#35bf28}+16.66\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9675ms	0.6042ms	1.6551 KOps/s	1.6567 KOps/s	$\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.0671ms	3.7488ms	266.7510 Ops/s	268.7687 Ops/s	$\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1158ms	0.5066ms	1.9740 KOps/s	1.9146 KOps/s	$\color{#35bf28}+3.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7297ms	0.4867ms	2.0548 KOps/s	2.0219 KOps/s	$\color{#35bf28}+1.63\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.6069ms	3.7207ms	268.7698 Ops/s	271.5775 Ops/s	$\color{#d91a1a}-1.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7825ms	0.5009ms	1.9963 KOps/s	1.9538 KOps/s	$\color{#35bf28}+2.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.5986ms	0.4900ms	2.0410 KOps/s	2.0614 KOps/s	$\color{#d91a1a}-0.99\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.0010ms	3.7826ms	264.3709 Ops/s	263.0552 Ops/s	$\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.3135ms	0.6250ms	1.5999 KOps/s	1.5810 KOps/s	$\color{#35bf28}+1.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9034ms	0.6023ms	1.6603 KOps/s	1.6355 KOps/s	$\color{#35bf28}+1.52\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1145s	5.9599ms	167.7890 Ops/s	122.7383 Ops/s	$\textbf{\color{#35bf28}+36.70\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	15.3675ms	12.6548ms	79.0217 Ops/s	76.8412 Ops/s	$\color{#35bf28}+2.84\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.3654ms	1.0731ms	931.8412 Ops/s	843.6743 Ops/s	$\textbf{\color{#35bf28}+10.45\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1040s	7.6647ms	130.4683 Ops/s	176.9937 Ops/s	$\textbf{\color{#d91a1a}-26.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	15.9385ms	12.8509ms	77.8155 Ops/s	75.8089 Ops/s	$\color{#35bf28}+2.65\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.0652ms	1.1455ms	872.9572 Ops/s	922.2030 Ops/s	$\textbf{\color{#d91a1a}-5.34\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1059s	6.0771ms	164.5512 Ops/s	126.5785 Ops/s	$\textbf{\color{#35bf28}+30.00\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	15.7878ms	13.0126ms	76.8487 Ops/s	76.5253 Ops/s	$\color{#35bf28}+0.42\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.9548ms	1.2671ms	789.2263 Ops/s	809.5860 Ops/s	$\color{#d91a1a}-2.51\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1170s	0.1162s	8.6037 Ops/s	8.4691 Ops/s	$\color{#35bf28}+1.59\%$
test_sync	0.1016s	98.3283ms	10.1700 Ops/s	9.5062 Ops/s	$\textbf{\color{#35bf28}+6.98\%}$
test_async	0.1920s	77.7158ms	12.8674 Ops/s	12.1763 Ops/s	$\textbf{\color{#35bf28}+5.68\%}$
test_single_pixels	0.1279s	0.1276s	7.8359 Ops/s	7.7983 Ops/s	$\color{#35bf28}+0.48\%$
test_sync_pixels	85.4130ms	83.8345ms	11.9283 Ops/s	11.9400 Ops/s	$\color{#d91a1a}-0.10\%$
test_async_pixels	0.1590s	68.8200ms	14.5307 Ops/s	14.2992 Ops/s	$\color{#35bf28}+1.62\%$
test_simple	0.8829s	0.8263s	1.2102 Ops/s	1.2279 Ops/s	$\color{#d91a1a}-1.44\%$
test_transformed	1.1417s	1.0760s	0.9294 Ops/s	0.9221 Ops/s	$\color{#35bf28}+0.79\%$
test_serial	2.5263s	2.4626s	0.4061 Ops/s	0.3992 Ops/s	$\color{#35bf28}+1.73\%$
test_parallel	2.4165s	2.3555s	0.4245 Ops/s	0.4234 Ops/s	$\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-True-True-True-True]	0.1021ms	32.7181μs	30.5641 KOps/s	29.5154 KOps/s	$\color{#35bf28}+3.55\%$
test_step_mdp_speed[True-True-True-True-False]	44.6510μs	19.6053μs	51.0067 KOps/s	50.0836 KOps/s	$\color{#35bf28}+1.84\%$
test_step_mdp_speed[True-True-True-False-True]	45.5610μs	18.6205μs	53.7041 KOps/s	51.6027 KOps/s	$\color{#35bf28}+4.07\%$
test_step_mdp_speed[True-True-True-False-False]	28.0100μs	11.1417μs	89.7529 KOps/s	88.3282 KOps/s	$\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-True-False-True-True]	75.7210μs	34.8249μs	28.7151 KOps/s	28.2355 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-True-False-True-False]	37.1510μs	21.0873μs	47.4220 KOps/s	46.4546 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-False-False-True]	39.9100μs	20.3956μs	49.0302 KOps/s	47.3818 KOps/s	$\color{#35bf28}+3.48\%$
test_step_mdp_speed[True-True-False-False-False]	37.4310μs	12.8854μs	77.6073 KOps/s	75.6011 KOps/s	$\color{#35bf28}+2.65\%$
test_step_mdp_speed[True-False-True-True-True]	53.5210μs	36.8689μs	27.1231 KOps/s	26.7666 KOps/s	$\color{#35bf28}+1.33\%$
test_step_mdp_speed[True-False-True-True-False]	41.5710μs	23.3299μs	42.8635 KOps/s	42.5219 KOps/s	$\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-False-True-False-True]	39.0210μs	20.4898μs	48.8048 KOps/s	47.0993 KOps/s	$\color{#35bf28}+3.62\%$
test_step_mdp_speed[True-False-True-False-False]	28.4700μs	12.8808μs	77.6352 KOps/s	74.8457 KOps/s	$\color{#35bf28}+3.73\%$
test_step_mdp_speed[True-False-False-True-True]	64.1210μs	38.4551μs	26.0044 KOps/s	25.4579 KOps/s	$\color{#35bf28}+2.15\%$
test_step_mdp_speed[True-False-False-True-False]	65.8510μs	24.6669μs	40.5402 KOps/s	39.5279 KOps/s	$\color{#35bf28}+2.56\%$
test_step_mdp_speed[True-False-False-False-True]	57.6310μs	21.8853μs	45.6927 KOps/s	43.8585 KOps/s	$\color{#35bf28}+4.18\%$
test_step_mdp_speed[True-False-False-False-False]	35.9200μs	14.8287μs	67.4368 KOps/s	66.3519 KOps/s	$\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-True-True-True]	64.5900μs	36.5598μs	27.3524 KOps/s	26.5465 KOps/s	$\color{#35bf28}+3.04\%$
test_step_mdp_speed[False-True-True-True-False]	47.9900μs	23.5592μs	42.4463 KOps/s	41.8390 KOps/s	$\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-False-True]	52.4610μs	24.7983μs	40.3254 KOps/s	39.3469 KOps/s	$\color{#35bf28}+2.49\%$
test_step_mdp_speed[False-True-True-False-False]	39.0510μs	14.8971μs	67.1270 KOps/s	66.3003 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-False-True-True]	65.5700μs	38.6142μs	25.8972 KOps/s	25.3163 KOps/s	$\color{#35bf28}+2.29\%$
test_step_mdp_speed[False-True-False-True-False]	41.8910μs	25.3743μs	39.4100 KOps/s	39.3266 KOps/s	$\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-True-False-False-True]	56.0710μs	26.3263μs	37.9848 KOps/s	36.7506 KOps/s	$\color{#35bf28}+3.36\%$
test_step_mdp_speed[False-True-False-False-False]	35.5900μs	16.6956μs	59.8959 KOps/s	59.0481 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-False-True-True-True]	66.0210μs	40.5245μs	24.6765 KOps/s	24.2216 KOps/s	$\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-True-True-False]	44.3510μs	26.8984μs	37.1770 KOps/s	36.1313 KOps/s	$\color{#35bf28}+2.89\%$
test_step_mdp_speed[False-False-True-False-True]	41.5000μs	26.1797μs	38.1975 KOps/s	36.7624 KOps/s	$\color{#35bf28}+3.90\%$
test_step_mdp_speed[False-False-True-False-False]	34.1800μs	16.6407μs	60.0937 KOps/s	58.7284 KOps/s	$\color{#35bf28}+2.32\%$
test_step_mdp_speed[False-False-False-True-True]	72.9410μs	42.9450μs	23.2856 KOps/s	22.9873 KOps/s	$\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-False-False-True-False]	50.2900μs	29.0542μs	34.4185 KOps/s	33.5340 KOps/s	$\color{#35bf28}+2.64\%$
test_step_mdp_speed[False-False-False-False-True]	55.2810μs	28.0830μs	35.6088 KOps/s	34.5697 KOps/s	$\color{#35bf28}+3.01\%$
test_step_mdp_speed[False-False-False-False-False]	34.3100μs	18.4024μs	54.3406 KOps/s	53.1645 KOps/s	$\color{#35bf28}+2.21\%$
test_values[generalized_advantage_estimate-True-True]	25.8263ms	24.8723ms	40.2054 Ops/s	39.5725 Ops/s	$\color{#35bf28}+1.60\%$
test_values[vec_generalized_advantage_estimate-True-True]	89.3971ms	2.6920ms	371.4768 Ops/s	375.0360 Ops/s	$\color{#d91a1a}-0.95\%$
test_values[td0_return_estimate-False-False]	86.9510μs	66.3750μs	15.0659 KOps/s	15.0292 KOps/s	$\color{#35bf28}+0.24\%$
test_values[td1_return_estimate-False-False]	56.4160ms	55.8571ms	17.9028 Ops/s	17.7158 Ops/s	$\color{#35bf28}+1.06\%$
test_values[vec_td1_return_estimate-False-False]	1.2602ms	1.0866ms	920.2820 Ops/s	917.5687 Ops/s	$\color{#35bf28}+0.30\%$
test_values[td_lambda_return_estimate-True-False]	92.7590ms	89.3074ms	11.1973 Ops/s	11.2374 Ops/s	$\color{#d91a1a}-0.36\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.2053ms	1.0846ms	921.9941 Ops/s	918.6842 Ops/s	$\color{#35bf28}+0.36\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.2955ms	25.1117ms	39.8221 Ops/s	39.2408 Ops/s	$\color{#35bf28}+1.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9673ms	0.7263ms	1.3768 KOps/s	1.3701 KOps/s	$\color{#35bf28}+0.50\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7690ms	0.6731ms	1.4857 KOps/s	1.4717 KOps/s	$\color{#35bf28}+0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5719ms	1.4773ms	676.9184 Ops/s	676.8534 Ops/s	$+0.01\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7131ms	0.6900ms	1.4494 KOps/s	1.4366 KOps/s	$\color{#35bf28}+0.89\%$
test_dqn_speed	1.8402ms	1.4764ms	677.3335 Ops/s	681.3367 Ops/s	$\color{#d91a1a}-0.59\%$
test_ddpg_speed	3.2818ms	2.9782ms	335.7714 Ops/s	332.4097 Ops/s	$\color{#35bf28}+1.01\%$
test_sac_speed	8.7716ms	8.5420ms	117.0693 Ops/s	116.2632 Ops/s	$\color{#35bf28}+0.69\%$
test_redq_speed	12.7859ms	10.8040ms	92.5583 Ops/s	92.5222 Ops/s	$\color{#35bf28}+0.04\%$
test_redq_deprec_speed	11.8500ms	11.3902ms	87.7950 Ops/s	85.7499 Ops/s	$\color{#35bf28}+2.39\%$
test_td3_speed	8.5661ms	8.4959ms	117.7044 Ops/s	117.7325 Ops/s	$\color{#d91a1a}-0.02\%$
test_cql_speed	27.6626ms	26.1180ms	38.2878 Ops/s	38.2015 Ops/s	$\color{#35bf28}+0.23\%$
test_a2c_speed	5.8702ms	5.6051ms	178.4104 Ops/s	171.9770 Ops/s	$\color{#35bf28}+3.74\%$
test_ppo_speed	6.1634ms	5.9783ms	167.2715 Ops/s	162.3729 Ops/s	$\color{#35bf28}+3.02\%$
test_reinforce_speed	5.3025ms	4.5831ms	218.1939 Ops/s	210.9889 Ops/s	$\color{#35bf28}+3.41\%$
test_iql_speed	20.1228ms	19.5618ms	51.1200 Ops/s	49.8275 Ops/s	$\color{#35bf28}+2.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.1475ms	4.8417ms	206.5398 Ops/s	204.1792 Ops/s	$\color{#35bf28}+1.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.2980ms	0.6060ms	1.6502 KOps/s	1.6423 KOps/s	$\color{#35bf28}+0.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7807ms	0.5782ms	1.7294 KOps/s	1.7145 KOps/s	$\color{#35bf28}+0.87\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.0340ms	4.7993ms	208.3655 Ops/s	204.9362 Ops/s	$\color{#35bf28}+1.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3783ms	0.5981ms	1.6720 KOps/s	1.6562 KOps/s	$\color{#35bf28}+0.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7612ms	0.5696ms	1.7558 KOps/s	1.7205 KOps/s	$\color{#35bf28}+2.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	4.9557ms	2.1317ms	469.1180 Ops/s	460.8999 Ops/s	$\color{#35bf28}+1.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.2895ms	2.0093ms	497.6953 Ops/s	484.8079 Ops/s	$\color{#35bf28}+2.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0535ms	4.9750ms	201.0041 Ops/s	198.0499 Ops/s	$\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9979ms	0.7321ms	1.3659 KOps/s	1.3485 KOps/s	$\color{#35bf28}+1.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.8066ms	0.7051ms	1.4183 KOps/s	1.4079 KOps/s	$\color{#35bf28}+0.74\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.9331ms	4.8310ms	206.9944 Ops/s	201.7133 Ops/s	$\color{#35bf28}+2.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7134ms	0.6040ms	1.6555 KOps/s	1.6309 KOps/s	$\color{#35bf28}+1.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.5181ms	0.5808ms	1.7218 KOps/s	1.6942 KOps/s	$\color{#35bf28}+1.62\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.0545ms	4.8162ms	207.6306 Ops/s	203.3116 Ops/s	$\color{#35bf28}+2.12\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3780ms	0.5932ms	1.6857 KOps/s	1.6549 KOps/s	$\color{#35bf28}+1.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6856ms	0.5727ms	1.7460 KOps/s	1.7399 KOps/s	$\color{#35bf28}+0.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0166ms	4.9703ms	201.1933 Ops/s	197.0124 Ops/s	$\color{#35bf28}+2.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8498ms	0.7240ms	1.3812 KOps/s	1.3434 KOps/s	$\color{#35bf28}+2.81\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9483ms	0.7076ms	1.4133 KOps/s	1.3889 KOps/s	$\color{#35bf28}+1.76\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1231s	9.6043ms	104.1205 Ops/s	131.5250 Ops/s	$\textbf{\color{#d91a1a}-20.84\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.8657ms	16.4161ms	60.9160 Ops/s	58.4079 Ops/s	$\color{#35bf28}+4.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.2954ms	1.3093ms	763.7475 Ops/s	755.3734 Ops/s	$\color{#35bf28}+1.11\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1048s	7.1628ms	139.6106 Ops/s	139.2785 Ops/s	$\color{#35bf28}+0.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.9436ms	16.3696ms	61.0888 Ops/s	52.4642 Ops/s	$\textbf{\color{#35bf28}+16.44\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.4595ms	1.3265ms	753.8608 Ops/s	738.9639 Ops/s	$\color{#35bf28}+2.02\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1063s	9.3511ms	106.9390 Ops/s	135.9787 Ops/s	$\textbf{\color{#d91a1a}-21.36\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	19.0483ms	16.6635ms	60.0112 Ops/s	58.5012 Ops/s	$\color{#35bf28}+2.58\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.6094ms	1.5839ms	631.3661 Ops/s	664.1759 Ops/s	$\color{#d91a1a}-4.94\%$