Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] use_vmap=False for SAC #2392

Open
wants to merge 2 commits into
base: gh/vmoens/18/base
Choose a base branch
from
Open

[Feature] use_vmap=False for SAC #2392

wants to merge 2 commits into from

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 13, 2024 •
edited
Loading

@vmoens
Update
7827a41 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 13, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2392

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 4 Unrelated Failures

As of commit 3cc6253 with merge base 25e8bd2 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens mentioned this pull request Aug 13, 2024
Open
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 13, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 13, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.7878ms 60.1535ms 16.6241 Ops/s 16.9925 Ops/s $\color{#d91a1a}-2.17\%$
test_sync 50.7583ms 34.8206ms 28.7187 Ops/s 29.8522 Ops/s $\color{#d91a1a}-3.80\%$
test_async 69.9955ms 32.8126ms 30.4761 Ops/s 30.9886 Ops/s $\color{#d91a1a}-1.65\%$
test_simple 0.5242s 0.4383s 2.2813 Ops/s 2.3722 Ops/s $\color{#d91a1a}-3.83\%$
test_transformed 0.6718s 0.5935s 1.6849 Ops/s 1.7292 Ops/s $\color{#d91a1a}-2.56\%$
test_serial 1.3817s 1.2939s 0.7728 Ops/s 0.7933 Ops/s $\color{#d91a1a}-2.58\%$
test_parallel 1.2114s 1.1343s 0.8816 Ops/s 0.9005 Ops/s $\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-True-True-True-True] 0.1417ms 25.1061μs 39.8309 KOps/s 39.9618 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-True-True-True-False] 66.0130μs 14.3793μs 69.5445 KOps/s 70.0023 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-True-True-False-True] 52.1270μs 14.4263μs 69.3180 KOps/s 70.0192 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[True-True-True-False-False] 68.4780μs 8.2974μs 120.5190 KOps/s 121.6981 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-True-False-True-True] 0.1148ms 27.3099μs 36.6168 KOps/s 37.4347 KOps/s $\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-False-True-False] 36.3770μs 16.0304μs 62.3813 KOps/s 62.6521 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-True-False-False-True] 59.1900μs 15.9764μs 62.5925 KOps/s 63.0129 KOps/s $\color{#d91a1a}-0.67\%$
test_step_mdp_speed[True-True-False-False-False] 72.7560μs 9.8996μs 101.0137 KOps/s 102.4355 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-False-True-True-True] 74.5990μs 28.8897μs 34.6144 KOps/s 35.3213 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-False-True-True-False] 70.9520μs 17.7473μs 56.3467 KOps/s 56.7609 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[True-False-True-False-True] 77.6440μs 16.1366μs 61.9708 KOps/s 63.4491 KOps/s $\color{#d91a1a}-2.33\%$
test_step_mdp_speed[True-False-True-False-False] 50.7440μs 9.9930μs 100.0703 KOps/s 101.6781 KOps/s $\color{#d91a1a}-1.58\%$
test_step_mdp_speed[True-False-False-True-True] 0.1028ms 29.8887μs 33.4575 KOps/s 33.7688 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[True-False-False-True-False] 53.6900μs 19.1366μs 52.2559 KOps/s 52.3708 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[True-False-False-False-True] 67.4210μs 17.4465μs 57.3181 KOps/s 57.7448 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-False-False-False] 42.5790μs 11.3659μs 87.9826 KOps/s 88.8224 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-True-True-True] 90.8400μs 28.6792μs 34.8685 KOps/s 34.9879 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[False-True-True-True-False] 83.2450μs 17.6741μs 56.5799 KOps/s 57.1489 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-True-True-False-True] 61.7940μs 18.6406μs 53.6463 KOps/s 54.0436 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-True-True-False-False] 35.0860μs 11.0335μs 90.6327 KOps/s 90.9661 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-True-False-True-True] 82.7740μs 29.8621μs 33.4872 KOps/s 32.9755 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-True-False-True-False] 58.2580μs 19.2313μs 51.9985 KOps/s 52.6227 KOps/s $\color{#d91a1a}-1.19\%$
test_step_mdp_speed[False-True-False-False-True] 60.6930μs 20.0629μs 49.8432 KOps/s 52.4659 KOps/s $\color{#d91a1a}-5.00\%$
test_step_mdp_speed[False-True-False-False-False] 65.8830μs 12.6292μs 79.1816 KOps/s 80.8920 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-True-True-True] 3.1905ms 31.8537μs 31.3935 KOps/s 31.4770 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-True-True-False] 74.0380μs 20.9367μs 47.7630 KOps/s 48.8914 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[False-False-True-False-True] 82.3030μs 20.2492μs 49.3845 KOps/s 50.8685 KOps/s $\color{#d91a1a}-2.92\%$
test_step_mdp_speed[False-False-True-False-False] 34.6340μs 12.7546μs 78.4031 KOps/s 79.5786 KOps/s $\color{#d91a1a}-1.48\%$
test_step_mdp_speed[False-False-False-True-True] 90.9690μs 32.6937μs 30.5869 KOps/s 30.8044 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-False-False-True-False] 75.1790μs 22.2351μs 44.9740 KOps/s 45.5990 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[False-False-False-False-True] 47.4580μs 21.2996μs 46.9493 KOps/s 47.5407 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-False-False-False-False] 61.9350μs 13.9028μs 71.9282 KOps/s 72.5842 KOps/s $\color{#d91a1a}-0.90\%$
test_values[generalized_advantage_estimate-True-True] 12.2485ms 9.8677ms 101.3405 Ops/s 101.9258 Ops/s $\color{#d91a1a}-0.57\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.9890ms 33.5503ms 29.8060 Ops/s 27.8273 Ops/s $\textbf{\color{#35bf28}+7.11\%}$
test_values[td0_return_estimate-False-False] 0.2830ms 0.1888ms 5.2980 KOps/s 5.1904 KOps/s $\color{#35bf28}+2.07\%$
test_values[td1_return_estimate-False-False] 29.2322ms 24.5839ms 40.6770 Ops/s 40.5808 Ops/s $\color{#35bf28}+0.24\%$
test_values[vec_td1_return_estimate-False-False] 40.5454ms 33.8222ms 29.5664 Ops/s 27.6089 Ops/s $\textbf{\color{#35bf28}+7.09\%}$
test_values[td_lambda_return_estimate-True-False] 39.7914ms 35.1667ms 28.4360 Ops/s 27.7430 Ops/s $\color{#35bf28}+2.50\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.7780ms 33.5446ms 29.8111 Ops/s 27.3210 Ops/s $\textbf{\color{#35bf28}+9.11\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.6380ms 8.4940ms 117.7296 Ops/s 118.1262 Ops/s $\color{#d91a1a}-0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3093ms 1.9670ms 508.3837 Ops/s 519.7810 Ops/s $\color{#d91a1a}-2.19\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6068ms 0.3660ms 2.7324 KOps/s 2.7279 KOps/s $\color{#35bf28}+0.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 40.4662ms 39.3389ms 25.4202 Ops/s 22.5150 Ops/s $\textbf{\color{#35bf28}+12.90\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.7632ms 3.0795ms 324.7327 Ops/s 323.9983 Ops/s $\color{#35bf28}+0.23\%$
test_dqn_speed 6.2171ms 1.2804ms 781.0188 Ops/s 762.0434 Ops/s $\color{#35bf28}+2.49\%$
test_ddpg_speed 3.9031ms 2.7386ms 365.1509 Ops/s 364.2117 Ops/s $\color{#35bf28}+0.26\%$
test_sac_speed 9.1574ms 8.0574ms 124.1102 Ops/s 114.2814 Ops/s $\textbf{\color{#35bf28}+8.60\%}$
test_redq_speed 14.6833ms 12.7649ms 78.3397 Ops/s 76.4316 Ops/s $\color{#35bf28}+2.50\%$
test_redq_deprec_speed 14.7206ms 13.2170ms 75.6599 Ops/s 74.0518 Ops/s $\color{#35bf28}+2.17\%$
test_td3_speed 8.8404ms 8.0994ms 123.4652 Ops/s 117.5632 Ops/s $\textbf{\color{#35bf28}+5.02\%}$
test_cql_speed 40.5368ms 36.1958ms 27.6275 Ops/s 26.3618 Ops/s $\color{#35bf28}+4.80\%$
test_a2c_speed 9.1719ms 7.6622ms 130.5108 Ops/s 134.6258 Ops/s $\color{#d91a1a}-3.06\%$
test_ppo_speed 9.2365ms 8.0332ms 124.4839 Ops/s 129.3201 Ops/s $\color{#d91a1a}-3.74\%$
test_reinforce_speed 8.1695ms 6.7138ms 148.9471 Ops/s 152.6153 Ops/s $\color{#d91a1a}-2.40\%$
test_iql_speed 34.2321ms 32.6320ms 30.6448 Ops/s 30.7246 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7881ms 5.0549ms 197.8291 Ops/s 211.9783 Ops/s $\textbf{\color{#d91a1a}-6.67\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8284ms 0.5015ms 1.9939 KOps/s 2.0331 KOps/s $\color{#d91a1a}-1.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6329ms 0.4694ms 2.1304 KOps/s 2.1665 KOps/s $\color{#d91a1a}-1.66\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.4926ms 5.0592ms 197.6596 Ops/s 207.9582 Ops/s $\color{#d91a1a}-4.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4894ms 0.4941ms 2.0240 KOps/s 2.1168 KOps/s $\color{#d91a1a}-4.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0428ms 0.4721ms 2.1184 KOps/s 2.1452 KOps/s $\color{#d91a1a}-1.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9381ms 1.7124ms 583.9768 Ops/s 589.7073 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3245ms 1.6217ms 616.6196 Ops/s 616.2496 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3877ms 5.2452ms 190.6500 Ops/s 198.4175 Ops/s $\color{#d91a1a}-3.91\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1495s 0.7645ms 1.3080 KOps/s 1.5985 KOps/s $\textbf{\color{#d91a1a}-18.17\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1828ms 0.6160ms 1.6234 KOps/s 1.6661 KOps/s $\color{#d91a1a}-2.57\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.6719ms 5.1237ms 195.1727 Ops/s 201.2735 Ops/s $\color{#d91a1a}-3.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5988ms 0.4902ms 2.0401 KOps/s 1.9742 KOps/s $\color{#35bf28}+3.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.1737ms 0.4784ms 2.0904 KOps/s 2.1359 KOps/s $\color{#d91a1a}-2.13\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.6026ms 5.0366ms 198.5477 Ops/s 205.3551 Ops/s $\color{#d91a1a}-3.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0152ms 0.4856ms 2.0594 KOps/s 2.0903 KOps/s $\color{#d91a1a}-1.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7929ms 0.4724ms 2.1167 KOps/s 2.1818 KOps/s $\color{#d91a1a}-2.99\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5756ms 5.3168ms 188.0834 Ops/s 190.1709 Ops/s $\color{#d91a1a}-1.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3939ms 0.6413ms 1.5593 KOps/s 1.5989 KOps/s $\color{#d91a1a}-2.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9577ms 0.6186ms 1.6166 KOps/s 1.6811 KOps/s $\color{#d91a1a}-3.84\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1421s 6.7567ms 148.0020 Ops/s 155.1066 Ops/s $\color{#d91a1a}-4.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1286s 16.1248ms 62.0164 Ops/s 76.3865 Ops/s $\textbf{\color{#d91a1a}-18.81\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.1282ms 1.3267ms 753.7598 Ops/s 829.2883 Ops/s $\textbf{\color{#d91a1a}-9.11\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1223s 6.3815ms 156.7022 Ops/s 160.3427 Ops/s $\color{#d91a1a}-2.27\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.7467ms 13.4977ms 74.0868 Ops/s 64.5814 Ops/s $\textbf{\color{#35bf28}+14.72\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.9829ms 1.2610ms 793.0026 Ops/s 784.6904 Ops/s $\color{#35bf28}+1.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1234s 6.4953ms 153.9587 Ops/s 154.9892 Ops/s $\color{#d91a1a}-0.66\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.6823ms 13.8282ms 72.3158 Ops/s 70.7831 Ops/s $\color{#35bf28}+2.17\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.2198ms 1.5442ms 647.5765 Ops/s 746.3590 Ops/s $\textbf{\color{#d91a1a}-13.24\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 13, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}22$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1076s 0.1069s 9.3531 Ops/s 9.5007 Ops/s $\color{#d91a1a}-1.55\%$
test_sync 94.7679ms 91.7882ms 10.8947 Ops/s 10.7611 Ops/s $\color{#35bf28}+1.24\%$
test_async 0.1690s 88.4217ms 11.3094 Ops/s 11.3115 Ops/s $\color{#d91a1a}-0.02\%$
test_single_pixels 0.1194s 0.1181s 8.4698 Ops/s 8.5514 Ops/s $\color{#d91a1a}-0.95\%$
test_sync_pixels 79.2588ms 77.7089ms 12.8685 Ops/s 12.9604 Ops/s $\color{#d91a1a}-0.71\%$
test_async_pixels 0.1368s 71.6547ms 13.9558 Ops/s 14.1271 Ops/s $\color{#d91a1a}-1.21\%$
test_simple 0.8570s 0.7828s 1.2775 Ops/s 1.3082 Ops/s $\color{#d91a1a}-2.35\%$
test_transformed 1.0974s 1.0225s 0.9780 Ops/s 1.0205 Ops/s $\color{#d91a1a}-4.17\%$
test_serial 2.2623s 2.1869s 0.4573 Ops/s 0.4721 Ops/s $\color{#d91a1a}-3.14\%$
test_parallel 1.9813s 1.9036s 0.5253 Ops/s 0.5275 Ops/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-True-True-True-True] 0.1027ms 37.8786μs 26.4001 KOps/s 26.8851 KOps/s $\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-True-True-True-False] 54.2410μs 21.6496μs 46.1902 KOps/s 48.1876 KOps/s $\color{#d91a1a}-4.15\%$
test_step_mdp_speed[True-True-True-False-True] 89.6010μs 21.7886μs 45.8955 KOps/s 47.6228 KOps/s $\color{#d91a1a}-3.63\%$
test_step_mdp_speed[True-True-True-False-False] 28.2110μs 12.5488μs 79.6889 KOps/s 84.0973 KOps/s $\textbf{\color{#d91a1a}-5.24\%}$
test_step_mdp_speed[True-True-False-True-True] 79.2610μs 41.1921μs 24.2765 KOps/s 25.4371 KOps/s $\color{#d91a1a}-4.56\%$
test_step_mdp_speed[True-True-False-True-False] 43.1510μs 24.0555μs 41.5705 KOps/s 43.1007 KOps/s $\color{#d91a1a}-3.55\%$
test_step_mdp_speed[True-True-False-False-True] 58.2210μs 24.5075μs 40.8039 KOps/s 43.5419 KOps/s $\textbf{\color{#d91a1a}-6.29\%}$
test_step_mdp_speed[True-True-False-False-False] 38.4100μs 15.0271μs 66.5463 KOps/s 71.5631 KOps/s $\textbf{\color{#d91a1a}-7.01\%}$
test_step_mdp_speed[True-False-True-True-True] 71.1510μs 44.0248μs 22.7145 KOps/s 24.2779 KOps/s $\textbf{\color{#d91a1a}-6.44\%}$
test_step_mdp_speed[True-False-True-True-False] 42.5710μs 26.9799μs 37.0646 KOps/s 39.8486 KOps/s $\textbf{\color{#d91a1a}-6.99\%}$
test_step_mdp_speed[True-False-True-False-True] 43.4400μs 24.0450μs 41.5886 KOps/s 44.0832 KOps/s $\textbf{\color{#d91a1a}-5.66\%}$
test_step_mdp_speed[True-False-True-False-False] 36.2700μs 14.9643μs 66.8255 KOps/s 71.9522 KOps/s $\textbf{\color{#d91a1a}-7.13\%}$
test_step_mdp_speed[True-False-False-True-True] 86.1310μs 46.3061μs 21.5954 KOps/s 22.8187 KOps/s $\textbf{\color{#d91a1a}-5.36\%}$
test_step_mdp_speed[True-False-False-True-False] 49.9010μs 29.1424μs 34.3143 KOps/s 36.2750 KOps/s $\textbf{\color{#d91a1a}-5.41\%}$
test_step_mdp_speed[True-False-False-False-True] 56.6310μs 26.2338μs 38.1187 KOps/s 39.9284 KOps/s $\color{#d91a1a}-4.53\%$
test_step_mdp_speed[True-False-False-False-False] 41.9810μs 17.3456μs 57.6515 KOps/s 61.1451 KOps/s $\textbf{\color{#d91a1a}-5.71\%}$
test_step_mdp_speed[False-True-True-True-True] 81.0910μs 44.5416μs 22.4509 KOps/s 23.8031 KOps/s $\textbf{\color{#d91a1a}-5.68\%}$
test_step_mdp_speed[False-True-True-True-False] 64.0910μs 26.6148μs 37.5731 KOps/s 39.4380 KOps/s $\color{#d91a1a}-4.73\%$
test_step_mdp_speed[False-True-True-False-True] 51.9700μs 28.2204μs 35.4354 KOps/s 35.8242 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-True-True-False-False] 42.5110μs 16.7295μs 59.7747 KOps/s 62.7498 KOps/s $\color{#d91a1a}-4.74\%$
test_step_mdp_speed[False-True-False-True-True] 74.5010μs 46.2073μs 21.6416 KOps/s 23.0751 KOps/s $\textbf{\color{#d91a1a}-6.21\%}$
test_step_mdp_speed[False-True-False-True-False] 52.2800μs 29.2872μs 34.1446 KOps/s 36.7672 KOps/s $\textbf{\color{#d91a1a}-7.13\%}$
test_step_mdp_speed[False-True-False-False-True] 61.2610μs 30.5139μs 32.7720 KOps/s 33.5459 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[False-True-False-False-False] 35.9110μs 18.8398μs 53.0792 KOps/s 55.6771 KOps/s $\color{#d91a1a}-4.67\%$
test_step_mdp_speed[False-False-True-True-True] 3.8173ms 48.4947μs 20.6208 KOps/s 21.8044 KOps/s $\textbf{\color{#d91a1a}-5.43\%}$
test_step_mdp_speed[False-False-True-True-False] 51.1410μs 32.5418μs 30.7297 KOps/s 33.0945 KOps/s $\textbf{\color{#d91a1a}-7.15\%}$
test_step_mdp_speed[False-False-True-False-True] 52.9310μs 31.1079μs 32.1461 KOps/s 33.1751 KOps/s $\color{#d91a1a}-3.10\%$
test_step_mdp_speed[False-False-True-False-False] 42.6510μs 19.0574μs 52.4731 KOps/s 54.2566 KOps/s $\color{#d91a1a}-3.29\%$
test_step_mdp_speed[False-False-False-True-True] 87.1120μs 49.6883μs 20.1255 KOps/s 20.9595 KOps/s $\color{#d91a1a}-3.98\%$
test_step_mdp_speed[False-False-False-True-False] 66.1310μs 33.6520μs 29.7159 KOps/s 31.3940 KOps/s $\textbf{\color{#d91a1a}-5.35\%}$
test_step_mdp_speed[False-False-False-False-True] 47.9510μs 32.3544μs 30.9077 KOps/s 31.6368 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[False-False-False-False-False] 45.0500μs 22.1570μs 45.1325 KOps/s 49.5506 KOps/s $\textbf{\color{#d91a1a}-8.92\%}$
test_values[generalized_advantage_estimate-True-True] 26.5320ms 25.0530ms 39.9154 Ops/s 41.7926 Ops/s $\color{#d91a1a}-4.49\%$
test_values[vec_generalized_advantage_estimate-True-True] 96.9257ms 2.8382ms 352.3313 Ops/s 358.3892 Ops/s $\color{#d91a1a}-1.69\%$
test_values[td0_return_estimate-False-False] 91.3610μs 66.7339μs 14.9849 KOps/s 15.3149 KOps/s $\color{#d91a1a}-2.16\%$
test_values[td1_return_estimate-False-False] 59.5752ms 56.7445ms 17.6229 Ops/s 18.3537 Ops/s $\color{#d91a1a}-3.98\%$
test_values[vec_td1_return_estimate-False-False] 1.3837ms 1.0941ms 913.9726 Ops/s 922.3498 Ops/s $\color{#d91a1a}-0.91\%$
test_values[td_lambda_return_estimate-True-False] 93.6327ms 89.4694ms 11.1770 Ops/s 11.6249 Ops/s $\color{#d91a1a}-3.85\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4336ms 1.0994ms 909.6099 Ops/s 924.7199 Ops/s $\color{#d91a1a}-1.63\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.4899ms 25.2363ms 39.6254 Ops/s 40.9955 Ops/s $\color{#d91a1a}-3.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9870ms 0.7225ms 1.3840 KOps/s 1.4062 KOps/s $\color{#d91a1a}-1.58\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7656ms 0.6742ms 1.4833 KOps/s 1.4916 KOps/s $\color{#d91a1a}-0.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5627ms 1.4702ms 680.1945 Ops/s 682.2223 Ops/s $\color{#d91a1a}-0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7684ms 0.7101ms 1.4082 KOps/s 1.4705 KOps/s $\color{#d91a1a}-4.24\%$
test_dqn_speed 7.1214ms 1.4290ms 699.8040 Ops/s 738.9109 Ops/s $\textbf{\color{#d91a1a}-5.29\%}$
test_ddpg_speed 3.0385ms 2.7931ms 358.0198 Ops/s 365.2875 Ops/s $\color{#d91a1a}-1.99\%$
test_sac_speed 8.5136ms 8.0252ms 124.6072 Ops/s 127.3800 Ops/s $\color{#d91a1a}-2.18\%$
test_redq_speed 12.5275ms 10.4938ms 95.2943 Ops/s 99.5197 Ops/s $\color{#d91a1a}-4.25\%$
test_redq_deprec_speed 11.6345ms 11.2481ms 88.9037 Ops/s 91.7076 Ops/s $\color{#d91a1a}-3.06\%$
test_td3_speed 17.8677ms 8.1183ms 123.1784 Ops/s 127.6446 Ops/s $\color{#d91a1a}-3.50\%$
test_cql_speed 26.4991ms 25.8075ms 38.7484 Ops/s 39.8712 Ops/s $\color{#d91a1a}-2.82\%$
test_a2c_speed 5.9428ms 5.7290ms 174.5512 Ops/s 182.4591 Ops/s $\color{#d91a1a}-4.33\%$
test_ppo_speed 6.7677ms 6.0161ms 166.2219 Ops/s 171.9193 Ops/s $\color{#d91a1a}-3.31\%$
test_reinforce_speed 4.8363ms 4.6090ms 216.9645 Ops/s 224.7070 Ops/s $\color{#d91a1a}-3.45\%$
test_iql_speed 20.4829ms 19.9738ms 50.0656 Ops/s 52.9490 Ops/s $\textbf{\color{#d91a1a}-5.45\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.9755ms 6.7925ms 147.2218 Ops/s 150.6151 Ops/s $\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0696ms 0.5275ms 1.8956 KOps/s 1.9188 KOps/s $\color{#d91a1a}-1.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6774ms 0.5035ms 1.9862 KOps/s 2.0029 KOps/s $\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9495ms 6.5813ms 151.9445 Ops/s 151.8920 Ops/s $\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1094ms 0.5242ms 1.9076 KOps/s 1.9651 KOps/s $\color{#d91a1a}-2.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6756ms 0.5006ms 1.9975 KOps/s 2.0310 KOps/s $\color{#d91a1a}-1.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1656ms 2.0212ms 494.7564 Ops/s 505.9224 Ops/s $\color{#d91a1a}-2.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 6.0956ms 1.9324ms 517.4885 Ops/s 536.3232 Ops/s $\color{#d91a1a}-3.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0945ms 6.9090ms 144.7392 Ops/s 147.9414 Ops/s $\color{#d91a1a}-2.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3460ms 0.6792ms 1.4722 KOps/s 1.4916 KOps/s $\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8545ms 0.6552ms 1.5263 KOps/s 1.5398 KOps/s $\color{#d91a1a}-0.88\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8256ms 6.6562ms 150.2366 Ops/s 150.3503 Ops/s $\color{#d91a1a}-0.08\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5767ms 0.5307ms 1.8842 KOps/s 1.9139 KOps/s $\color{#d91a1a}-1.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6728ms 0.5074ms 1.9710 KOps/s 2.0268 KOps/s $\color{#d91a1a}-2.75\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8623ms 6.5716ms 152.1689 Ops/s 154.0336 Ops/s $\color{#d91a1a}-1.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6512ms 0.5189ms 1.9273 KOps/s 1.9541 KOps/s $\color{#d91a1a}-1.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6949ms 0.5046ms 1.9816 KOps/s 2.0153 KOps/s $\color{#d91a1a}-1.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0189ms 6.7878ms 147.3242 Ops/s 145.2202 Ops/s $\color{#35bf28}+1.45\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0899ms 0.6794ms 1.4719 KOps/s 1.4896 KOps/s $\color{#d91a1a}-1.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8010ms 0.6560ms 1.5245 KOps/s 1.5544 KOps/s $\color{#d91a1a}-1.93\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1452s 8.2282ms 121.5326 Ops/s 122.3615 Ops/s $\color{#d91a1a}-0.68\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.8285ms 16.7583ms 59.6719 Ops/s 59.5506 Ops/s $\color{#35bf28}+0.20\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.8707ms 1.5982ms 625.7098 Ops/s 694.9938 Ops/s $\textbf{\color{#d91a1a}-9.97\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1261s 7.8948ms 126.6662 Ops/s 96.3653 Ops/s $\textbf{\color{#35bf28}+31.44\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.8092ms 16.7379ms 59.7447 Ops/s 59.4704 Ops/s $\color{#35bf28}+0.46\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.8440ms 1.4201ms 704.1791 Ops/s 790.1302 Ops/s $\textbf{\color{#d91a1a}-10.88\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1273s 10.4475ms 95.7166 Ops/s 125.6179 Ops/s $\textbf{\color{#d91a1a}-23.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.6724ms 16.9620ms 58.9554 Ops/s 60.0517 Ops/s $\color{#d91a1a}-1.83\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.7799ms 1.6223ms 616.4237 Ops/s 612.2555 Ops/s $\color{#35bf28}+0.68\%$

@vmoens
Update
3cc6253 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot