Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix slice sampler end computation at the cursor place #2225

Merged
merged 4 commits into from
Jun 12, 2024
Merged

[BugFix] Fix slice sampler end computation at the cursor place #2225

merged 4 commits into from
Jun 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jun 11, 2024

Enables this code to run without error in all cases:

rb = ReplayBuffer(
storage=LazyTensorStorage(100, ndim=2), 
sampler=SliceSampler(strict_length=True, num_slices=1), batch_size=8, 
transform=Compose(
    UnsqueezeTransform(-2, in_keys=["step_count"], out_keys=["obs_cat0"]),
    CatFrames(N=4, dim=-2, in_keys=["obs_cat0"], done_key="done"),
    UnsqueezeTransform(-2, in_keys=["step_count"], out_keys=["obs_cat1"]),
    CatFrames(N=5, dim=-2, in_keys=["obs_cat1"], done_key="done")
)
)
env = SerialEnv(2, lambda: GymEnv("CartPole-v1", device=None).append_transform(StepCounter()))
r = env.rollout(120, break_when_any_done=False)
rb.extend(r)
print(rb._storage._last_cursor)
s, info = rb.sample(return_info=True)
assert (s["obs_cat0"].squeeze()[4:]!=0).all(), info

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jun 11, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2225

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit eaf9ef2 with merge base f613eef (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 11, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1100s 58.6732ms 17.0436 Ops/s 17.4875 Ops/s $\color{#d91a1a}-2.54\%$
test_sync 31.9678ms 30.9995ms 32.2586 Ops/s 30.4264 Ops/s $\textbf{\color{#35bf28}+6.02\%}$
test_async 57.8898ms 28.6807ms 34.8666 Ops/s 34.5737 Ops/s $\color{#35bf28}+0.85\%$
test_simple 0.4656s 0.4043s 2.4732 Ops/s 2.6120 Ops/s $\textbf{\color{#d91a1a}-5.31\%}$
test_transformed 0.6037s 0.5497s 1.8193 Ops/s 1.8543 Ops/s $\color{#d91a1a}-1.89\%$
test_serial 1.3154s 1.2581s 0.7948 Ops/s 0.7681 Ops/s $\color{#35bf28}+3.48\%$
test_parallel 1.1371s 1.0604s 0.9430 Ops/s 0.9375 Ops/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-True-True-True-True] 0.1796ms 21.9155μs 45.6299 KOps/s 46.3847 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[True-True-True-True-False] 43.9720μs 13.3208μs 75.0706 KOps/s 74.8732 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-True-False-True] 37.3390μs 12.9426μs 77.2645 KOps/s 77.8492 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-True-False-False] 32.1190μs 7.8464μs 127.4471 KOps/s 128.2882 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[True-True-False-True-True] 57.6380μs 23.0415μs 43.3999 KOps/s 43.2630 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[True-True-False-True-False] 43.9920μs 14.4832μs 69.0454 KOps/s 68.7975 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-True-False-False-True] 39.7840μs 14.0706μs 71.0703 KOps/s 70.5241 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[True-True-False-False-False] 31.5990μs 9.0554μs 110.4319 KOps/s 110.0740 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-False-True-True-True] 72.9360μs 24.4448μs 40.9085 KOps/s 40.8614 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-False-True-True-False] 42.6690μs 16.0344μs 62.3659 KOps/s 62.7608 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-False-True-False-True] 61.7550μs 13.8833μs 72.0288 KOps/s 70.9909 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-False-True-False-False] 33.9530μs 9.0386μs 110.6371 KOps/s 109.2877 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-False-False-True-True] 57.7080μs 25.4134μs 39.3493 KOps/s 38.6653 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[True-False-False-True-False] 51.4650μs 16.9649μs 58.9454 KOps/s 57.8183 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-False-False-False-True] 70.9030μs 15.0419μs 66.4808 KOps/s 65.3998 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-False-False-False-False] 39.5130μs 10.1498μs 98.5242 KOps/s 96.3010 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-True-True-True-True] 62.9680μs 24.4867μs 40.8384 KOps/s 40.5766 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-True-True-False] 38.0310μs 15.9586μs 62.6620 KOps/s 62.0885 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-True-True-False-True] 42.2590μs 16.4271μs 60.8750 KOps/s 60.8166 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-False-False] 31.9100μs 10.1776μs 98.2545 KOps/s 95.8099 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-True-False-True-True] 78.0560μs 25.3232μs 39.4895 KOps/s 39.4883 KOps/s $+0.00\%$
test_step_mdp_speed[False-True-False-True-False] 41.9080μs 17.0247μs 58.7381 KOps/s 58.6751 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[False-True-False-False-True] 44.2420μs 17.3815μs 57.5326 KOps/s 57.0580 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-False-False-False] 42.2490μs 11.4333μs 87.4637 KOps/s 85.9966 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-False-True-True-True] 55.7330μs 26.7540μs 37.3776 KOps/s 36.9649 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-False-True-True-False] 70.0100μs 18.2704μs 54.7335 KOps/s 54.3706 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-False-True] 43.5810μs 17.5219μs 57.0715 KOps/s 57.0788 KOps/s $\color{#d91a1a}-0.01\%$
test_step_mdp_speed[False-False-True-False-False] 37.6400μs 11.5078μs 86.8972 KOps/s 86.0738 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-False-True-True] 39.5140μs 28.1209μs 35.5608 KOps/s 35.1689 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-False-False-True-False] 44.7840μs 19.3666μs 51.6354 KOps/s 51.8535 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-False-False-False-True] 51.0550μs 18.4928μs 54.0752 KOps/s 54.4853 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-False-False-False-False] 40.6360μs 12.5957μs 79.3919 KOps/s 79.9640 KOps/s $\color{#d91a1a}-0.72\%$
test_values[generalized_advantage_estimate-True-True] 9.8453ms 9.1331ms 109.4919 Ops/s 104.9476 Ops/s $\color{#35bf28}+4.33\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.5980ms 33.6183ms 29.7457 Ops/s 28.1883 Ops/s $\textbf{\color{#35bf28}+5.53\%}$
test_values[td0_return_estimate-False-False] 0.2234ms 0.1680ms 5.9532 KOps/s 6.0744 KOps/s $\color{#d91a1a}-2.00\%$
test_values[td1_return_estimate-False-False] 25.9824ms 23.0606ms 43.3640 Ops/s 43.1916 Ops/s $\color{#35bf28}+0.40\%$
test_values[vec_td1_return_estimate-False-False] 35.6820ms 33.6384ms 29.7279 Ops/s 28.2073 Ops/s $\textbf{\color{#35bf28}+5.39\%}$
test_values[td_lambda_return_estimate-True-False] 37.9636ms 33.1852ms 30.1339 Ops/s 29.8207 Ops/s $\color{#35bf28}+1.05\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.2873ms 33.4484ms 29.8968 Ops/s 28.1367 Ops/s $\textbf{\color{#35bf28}+6.26\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.8803ms 8.0192ms 124.7003 Ops/s 122.8795 Ops/s $\color{#35bf28}+1.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 4.8251ms 2.0440ms 489.2422 Ops/s 553.0120 Ops/s $\textbf{\color{#d91a1a}-11.53\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4417ms 0.3540ms 2.8247 KOps/s 2.8053 KOps/s $\color{#35bf28}+0.69\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.3112ms 45.9829ms 21.7472 Ops/s 20.5132 Ops/s $\textbf{\color{#35bf28}+6.02\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7230ms 3.0346ms 329.5331 Ops/s 324.2552 Ops/s $\color{#35bf28}+1.63\%$
test_dqn_speed 1.6010ms 1.3241ms 755.2573 Ops/s 724.7545 Ops/s $\color{#35bf28}+4.21\%$
test_ddpg_speed 3.6603ms 2.8505ms 350.8157 Ops/s 346.2159 Ops/s $\color{#35bf28}+1.33\%$
test_sac_speed 10.4109ms 8.4279ms 118.6529 Ops/s 117.0730 Ops/s $\color{#35bf28}+1.35\%$
test_redq_speed 15.1175ms 13.7209ms 72.8815 Ops/s 72.2563 Ops/s $\color{#35bf28}+0.87\%$
test_redq_deprec_speed 15.5767ms 13.6597ms 73.2080 Ops/s 72.9184 Ops/s $\color{#35bf28}+0.40\%$
test_td3_speed 16.5874ms 8.3797ms 119.3362 Ops/s 114.7912 Ops/s $\color{#35bf28}+3.96\%$
test_cql_speed 37.0202ms 36.4623ms 27.4256 Ops/s 26.4311 Ops/s $\color{#35bf28}+3.76\%$
test_a2c_speed 8.8146ms 7.4180ms 134.8072 Ops/s 129.0040 Ops/s $\color{#35bf28}+4.50\%$
test_ppo_speed 8.5039ms 7.6721ms 130.3429 Ops/s 125.6883 Ops/s $\color{#35bf28}+3.70\%$
test_reinforce_speed 7.8127ms 6.6249ms 150.9454 Ops/s 147.3620 Ops/s $\color{#35bf28}+2.43\%$
test_iql_speed 34.9054ms 33.2376ms 30.0864 Ops/s 30.1223 Ops/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.6480ms 3.6458ms 274.2892 Ops/s 283.9772 Ops/s $\color{#d91a1a}-3.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2326ms 0.5108ms 1.9576 KOps/s 2.0088 KOps/s $\color{#d91a1a}-2.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6746ms 0.4741ms 2.1092 KOps/s 1.9945 KOps/s $\textbf{\color{#35bf28}+5.76\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9777ms 3.5372ms 282.7096 Ops/s 288.0734 Ops/s $\color{#d91a1a}-1.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0737ms 0.4880ms 2.0490 KOps/s 2.0410 KOps/s $\color{#35bf28}+0.39\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7608ms 0.4674ms 2.1393 KOps/s 2.1712 KOps/s $\color{#d91a1a}-1.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.5002ms 1.6896ms 591.8603 Ops/s 592.6038 Ops/s $\color{#d91a1a}-0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.5585ms 1.6033ms 623.7321 Ops/s 624.3166 Ops/s $\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.1384ms 3.5988ms 277.8706 Ops/s 278.2570 Ops/s $\color{#d91a1a}-0.14\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1082s 0.6914ms 1.4463 KOps/s 1.4111 KOps/s $\color{#35bf28}+2.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0669ms 0.6200ms 1.6128 KOps/s 1.6820 KOps/s $\color{#d91a1a}-4.12\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7963ms 3.5883ms 278.6832 Ops/s 270.9187 Ops/s $\color{#35bf28}+2.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8161ms 0.5030ms 1.9881 KOps/s 2.0212 KOps/s $\color{#d91a1a}-1.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6168ms 0.4712ms 2.1224 KOps/s 2.1090 KOps/s $\color{#35bf28}+0.63\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1295ms 3.4373ms 290.9237 Ops/s 275.8840 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0242ms 0.4890ms 2.0449 KOps/s 2.0110 KOps/s $\color{#35bf28}+1.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7507ms 0.4697ms 2.1292 KOps/s 2.1270 KOps/s $\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4767ms 3.5759ms 279.6491 Ops/s 271.0329 Ops/s $\color{#35bf28}+3.18\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7252ms 0.6088ms 1.6426 KOps/s 1.6232 KOps/s $\color{#35bf28}+1.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8500ms 0.5979ms 1.6725 KOps/s 1.7124 KOps/s $\color{#d91a1a}-2.33\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1265s 8.3105ms 120.3304 Ops/s 165.8255 Ops/s $\textbf{\color{#d91a1a}-27.44\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.7190ms 12.4153ms 80.5459 Ops/s 67.5266 Ops/s $\textbf{\color{#35bf28}+19.28\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5316ms 1.0395ms 961.9597 Ops/s 961.0530 Ops/s $\color{#35bf28}+0.09\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1106s 5.8387ms 171.2704 Ops/s 174.0567 Ops/s $\color{#d91a1a}-1.60\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.2775ms 12.5089ms 79.9428 Ops/s 78.4946 Ops/s $\color{#35bf28}+1.85\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.5393ms 1.0393ms 962.2156 Ops/s 883.0124 Ops/s $\textbf{\color{#35bf28}+8.97\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1088s 5.9305ms 168.6207 Ops/s 167.6914 Ops/s $\color{#35bf28}+0.55\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1177s 14.6402ms 68.3049 Ops/s 67.2519 Ops/s $\color{#35bf28}+1.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9067ms 1.1903ms 840.1240 Ops/s 768.7295 Ops/s $\textbf{\color{#35bf28}+9.29\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 11, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1174s 0.1170s 8.5503 Ops/s 8.3275 Ops/s $\color{#35bf28}+2.67\%$
test_sync 0.1048s 0.1039s 9.6281 Ops/s 9.7085 Ops/s $\color{#d91a1a}-0.83\%$
test_async 0.1993s 0.1007s 9.9261 Ops/s 10.1494 Ops/s $\color{#d91a1a}-2.20\%$
test_single_pixels 0.1285s 0.1283s 7.7939 Ops/s 7.9676 Ops/s $\color{#d91a1a}-2.18\%$
test_sync_pixels 84.8250ms 80.9513ms 12.3531 Ops/s 12.3683 Ops/s $\color{#d91a1a}-0.12\%$
test_async_pixels 0.1588s 69.0241ms 14.4877 Ops/s 14.5651 Ops/s $\color{#d91a1a}-0.53\%$
test_simple 0.8020s 0.8010s 1.2484 Ops/s 1.2472 Ops/s $\color{#35bf28}+0.10\%$
test_transformed 1.0684s 1.0652s 0.9388 Ops/s 0.9490 Ops/s $\color{#d91a1a}-1.07\%$
test_serial 2.5546s 2.4860s 0.4023 Ops/s 0.4125 Ops/s $\color{#d91a1a}-2.49\%$
test_parallel 2.3819s 2.3344s 0.4284 Ops/s 0.4288 Ops/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-True-True-True-True] 93.0920μs 33.2061μs 30.1150 KOps/s 30.8427 KOps/s $\color{#d91a1a}-2.36\%$
test_step_mdp_speed[True-True-True-True-False] 61.1210μs 19.5556μs 51.1363 KOps/s 50.8376 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-True-True-False-True] 85.5910μs 18.7120μs 53.4416 KOps/s 53.5868 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-True-True-False-False] 52.9100μs 11.1927μs 89.3440 KOps/s 89.2390 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-False-True-True] 68.8810μs 34.8776μs 28.6717 KOps/s 29.0822 KOps/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[True-True-False-True-False] 48.7810μs 21.5056μs 46.4994 KOps/s 46.3735 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-True-False-False-True] 54.9200μs 20.4978μs 48.7857 KOps/s 48.4469 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-True-False-False-False] 43.7800μs 13.1059μs 76.3016 KOps/s 76.0362 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-False-True-True-True] 82.8810μs 36.9452μs 27.0671 KOps/s 27.3842 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-False-True-True-False] 56.9710μs 23.5340μs 42.4917 KOps/s 42.3424 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-False-True-False-True] 48.4610μs 20.6276μs 48.4787 KOps/s 48.6993 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-True-False-False] 45.4710μs 13.1344μs 76.1361 KOps/s 76.4193 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-False-False-True-True] 71.2310μs 38.4265μs 26.0237 KOps/s 26.0661 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-False-False-True-False] 54.5610μs 25.4110μs 39.3530 KOps/s 39.4080 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-False-False-False-True] 58.5700μs 22.3082μs 44.8265 KOps/s 45.1158 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[True-False-False-False-False] 54.6910μs 15.0030μs 66.6534 KOps/s 66.4502 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-True-True-True-True] 69.4510μs 37.1345μs 26.9292 KOps/s 27.5732 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-True-True-True-False] 48.3910μs 23.5284μs 42.5019 KOps/s 42.2891 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-True-True-False-True] 53.8110μs 24.6671μs 40.5398 KOps/s 40.3190 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-True-True-False-False] 46.7810μs 14.7520μs 67.7875 KOps/s 65.8768 KOps/s $\color{#35bf28}+2.90\%$
test_step_mdp_speed[False-True-False-True-True] 74.7610μs 38.5922μs 25.9120 KOps/s 26.4553 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[False-True-False-True-False] 53.2700μs 25.4488μs 39.2946 KOps/s 39.5572 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-True-False-False-True] 60.7210μs 26.6400μs 37.5375 KOps/s 37.7258 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-True-False-False-False] 38.7710μs 16.7259μs 59.7876 KOps/s 59.5334 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-True-True-True] 73.0310μs 40.7710μs 24.5272 KOps/s 24.6312 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-False-True-True-False] 59.1410μs 27.3350μs 36.5832 KOps/s 36.7544 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-False-True-False-True] 58.5610μs 26.7911μs 37.3258 KOps/s 37.3926 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-False-True-False-False] 39.9710μs 16.7623μs 59.6576 KOps/s 58.6721 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-False-False-True-True] 81.7210μs 43.4757μs 23.0014 KOps/s 23.5998 KOps/s $\color{#d91a1a}-2.54\%$
test_step_mdp_speed[False-False-False-True-False] 68.7510μs 29.3635μs 34.0559 KOps/s 34.3424 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-False-False-False-True] 68.0910μs 28.3441μs 35.2808 KOps/s 35.5106 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[False-False-False-False-False] 74.3410μs 18.5306μs 53.9649 KOps/s 53.8326 KOps/s $\color{#35bf28}+0.25\%$
test_values[generalized_advantage_estimate-True-True] 26.4705ms 25.4903ms 39.2306 Ops/s 42.2272 Ops/s $\textbf{\color{#d91a1a}-7.10\%}$
test_values[vec_generalized_advantage_estimate-True-True] 91.6816ms 2.7256ms 366.8962 Ops/s 370.4662 Ops/s $\color{#d91a1a}-0.96\%$
test_values[td0_return_estimate-False-False] 91.4820μs 64.7736μs 15.4384 KOps/s 15.5207 KOps/s $\color{#d91a1a}-0.53\%$
test_values[td1_return_estimate-False-False] 57.9975ms 54.9990ms 18.1822 Ops/s 18.6909 Ops/s $\color{#d91a1a}-2.72\%$
test_values[vec_td1_return_estimate-False-False] 1.3399ms 1.0745ms 930.6248 Ops/s 936.4206 Ops/s $\color{#d91a1a}-0.62\%$
test_values[td_lambda_return_estimate-True-False] 91.5727ms 88.2609ms 11.3301 Ops/s 11.6948 Ops/s $\color{#d91a1a}-3.12\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3990ms 1.0748ms 930.3704 Ops/s 930.5607 Ops/s $\color{#d91a1a}-0.02\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 26.3924ms 26.0826ms 38.3397 Ops/s 42.1672 Ops/s $\textbf{\color{#d91a1a}-9.08\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9930ms 0.7478ms 1.3373 KOps/s 1.4131 KOps/s $\textbf{\color{#d91a1a}-5.36\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7485ms 0.6574ms 1.5212 KOps/s 1.5391 KOps/s $\color{#d91a1a}-1.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5424ms 1.4602ms 684.8353 Ops/s 687.9262 Ops/s $\color{#d91a1a}-0.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7358ms 0.6746ms 1.4825 KOps/s 1.4982 KOps/s $\color{#d91a1a}-1.05\%$
test_dqn_speed 2.2884ms 1.4884ms 671.8643 Ops/s 699.8841 Ops/s $\color{#d91a1a}-4.00\%$
test_ddpg_speed 3.2076ms 3.0314ms 329.8800 Ops/s 332.1989 Ops/s $\color{#d91a1a}-0.70\%$
test_sac_speed 9.1307ms 8.7658ms 114.0797 Ops/s 117.8104 Ops/s $\color{#d91a1a}-3.17\%$
test_redq_speed 14.1490ms 11.1615ms 89.5937 Ops/s 93.1362 Ops/s $\color{#d91a1a}-3.80\%$
test_redq_deprec_speed 12.4334ms 11.9773ms 83.4911 Ops/s 83.6837 Ops/s $\color{#d91a1a}-0.23\%$
test_td3_speed 8.8113ms 8.5862ms 116.4656 Ops/s 118.5041 Ops/s $\color{#d91a1a}-1.72\%$
test_cql_speed 29.5955ms 26.4931ms 37.7457 Ops/s 38.1679 Ops/s $\color{#d91a1a}-1.11\%$
test_a2c_speed 5.8103ms 5.6676ms 176.4415 Ops/s 175.0470 Ops/s $\color{#35bf28}+0.80\%$
test_ppo_speed 6.2039ms 5.9945ms 166.8187 Ops/s 164.3663 Ops/s $\color{#35bf28}+1.49\%$
test_reinforce_speed 4.9493ms 4.6381ms 215.6070 Ops/s 211.6826 Ops/s $\color{#35bf28}+1.85\%$
test_iql_speed 20.6981ms 19.9878ms 50.0306 Ops/s 49.2573 Ops/s $\color{#35bf28}+1.57\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.0549ms 4.8647ms 205.5637 Ops/s 209.7583 Ops/s $\color{#d91a1a}-2.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7289ms 0.6008ms 1.6645 KOps/s 1.6819 KOps/s $\color{#d91a1a}-1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8076ms 0.5787ms 1.7281 KOps/s 1.7422 KOps/s $\color{#d91a1a}-0.81\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.0282ms 4.8116ms 207.8327 Ops/s 211.2381 Ops/s $\color{#d91a1a}-1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7013ms 0.5948ms 1.6812 KOps/s 1.6997 KOps/s $\color{#d91a1a}-1.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6775ms 0.5703ms 1.7535 KOps/s 1.4770 KOps/s $\textbf{\color{#35bf28}+18.72\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2935ms 2.1415ms 466.9656 Ops/s 475.0392 Ops/s $\color{#d91a1a}-1.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2346ms 2.0349ms 491.4181 Ops/s 498.7752 Ops/s $\color{#d91a1a}-1.48\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.0409ms 4.9508ms 201.9879 Ops/s 204.9605 Ops/s $\color{#d91a1a}-1.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8403ms 0.7335ms 1.3633 KOps/s 1.3838 KOps/s $\color{#d91a1a}-1.48\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6609ms 0.7124ms 1.4036 KOps/s 1.4312 KOps/s $\color{#d91a1a}-1.93\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.9106ms 4.8323ms 206.9426 Ops/s 210.5046 Ops/s $\color{#d91a1a}-1.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7062ms 0.6150ms 1.6261 KOps/s 1.6802 KOps/s $\color{#d91a1a}-3.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.4558ms 0.5888ms 1.6984 KOps/s 1.7321 KOps/s $\color{#d91a1a}-1.95\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.0281ms 4.8111ms 207.8522 Ops/s 210.4638 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3643ms 0.6007ms 1.6648 KOps/s 1.6884 KOps/s $\color{#d91a1a}-1.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7547ms 0.5760ms 1.7362 KOps/s 1.7581 KOps/s $\color{#d91a1a}-1.25\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1088ms 4.9413ms 202.3740 Ops/s 203.5674 Ops/s $\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8354ms 0.7319ms 1.3662 KOps/s 1.3755 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8449ms 0.7073ms 1.4138 KOps/s 1.4044 KOps/s $\color{#35bf28}+0.67\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1166s 9.5004ms 105.2586 Ops/s 132.3098 Ops/s $\textbf{\color{#d91a1a}-20.45\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.3666ms 16.8499ms 59.3477 Ops/s 60.2861 Ops/s $\color{#d91a1a}-1.56\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3691ms 1.3361ms 748.4674 Ops/s 734.8903 Ops/s $\color{#35bf28}+1.85\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1053s 7.1817ms 139.2424 Ops/s 140.3009 Ops/s $\color{#d91a1a}-0.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.1849ms 16.8203ms 59.4520 Ops/s 61.5705 Ops/s $\color{#d91a1a}-3.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.3749ms 1.3366ms 748.1604 Ops/s 688.9337 Ops/s $\textbf{\color{#35bf28}+8.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1063s 9.3792ms 106.6189 Ops/s 107.0856 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.5205ms 16.9335ms 59.0546 Ops/s 62.3638 Ops/s $\textbf{\color{#d91a1a}-5.31\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.6435ms 1.6133ms 619.8427 Ops/s 663.0822 Ops/s $\textbf{\color{#d91a1a}-6.52\%}$

@vmoens vmoens added the bug Something isn't working label Jun 12, 2024
@vmoens vmoens merged commit 47a1005 into main Jun 12, 2024
54 of 58 checks passed
@vmoens vmoens deleted the fix-set-truncated branch June 12, 2024 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot