Upload docs for preview on HUD

[DanilBaibak]

Description

Added a step to the doc workflow to upload docs for preview on HUD.

Motivation and Context

Previously I enabled Dr. CI for RL repo. To make link for doc preview work, we need to upload the docs to the correct S3 bucket.

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1682

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (5 Unrelated Failures)

As of commit e7a57b5 with merge base eebef29 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Continuous Benchmark (PR) / GPU Pytest benchmark (linux.4xlarge.nvidia.gpu, 3.8) (gh)

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Continuous Benchmark / GPU Pytest benchmark (linux.4xlarge.nvidia.gpu, 3.8) (gh)
• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.11) / linux-job (gh)
• Unit-tests on Windows GPU / unittests / windows-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}15$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	63.9761ms	63.1037ms	15.8469 Ops/s	15.8007 Ops/s	$\color{#35bf28}+0.29\%$
test_sync	47.6161ms	35.3716ms	28.2713 Ops/s	30.4560 Ops/s	$\textbf{\color{#d91a1a}-7.17\%}$
test_async	63.9280ms	33.2145ms	30.1073 Ops/s	32.6989 Ops/s	$\textbf{\color{#d91a1a}-7.93\%}$
test_simple	0.5231s	0.4713s	2.1217 Ops/s	2.3265 Ops/s	$\textbf{\color{#d91a1a}-8.80\%}$
test_transformed	0.6690s	0.6136s	1.6297 Ops/s	1.6970 Ops/s	$\color{#d91a1a}-3.96\%$
test_serial	1.3269s	1.2698s	0.7875 Ops/s	0.7935 Ops/s	$\color{#d91a1a}-0.75\%$
test_parallel	1.3227s	1.2535s	0.7978 Ops/s	0.8111 Ops/s	$\color{#d91a1a}-1.65\%$
test_step_mdp_speed[True-True-True-True-True]	64.5700μs	25.3396μs	39.4639 KOps/s	40.6808 KOps/s	$\color{#d91a1a}-2.99\%$
test_step_mdp_speed[True-True-True-True-False]	52.4070μs	15.1613μs	65.9574 KOps/s	67.7261 KOps/s	$\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-True-True-False-True]	41.5670μs	15.4225μs	64.8404 KOps/s	66.1020 KOps/s	$\color{#d91a1a}-1.91\%$
test_step_mdp_speed[True-True-True-False-False]	35.7760μs	9.2570μs	108.0261 KOps/s	111.4716 KOps/s	$\color{#d91a1a}-3.09\%$
test_step_mdp_speed[True-True-False-True-True]	57.2560μs	26.6168μs	37.5703 KOps/s	38.0998 KOps/s	$\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-True-False-True-False]	84.2760μs	16.6887μs	59.9209 KOps/s	61.1902 KOps/s	$\color{#d91a1a}-2.07\%$
test_step_mdp_speed[True-True-False-False-True]	45.7350μs	17.0778μs	58.5554 KOps/s	59.5749 KOps/s	$\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-True-False-False-False]	37.1290μs	10.9416μs	91.3941 KOps/s	95.3462 KOps/s	$\color{#d91a1a}-4.15\%$
test_step_mdp_speed[True-False-True-True-True]	81.4780μs	28.2317μs	35.4211 KOps/s	35.9084 KOps/s	$\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-True-True-False]	53.1490μs	18.5607μs	53.8773 KOps/s	56.3387 KOps/s	$\color{#d91a1a}-4.37\%$
test_step_mdp_speed[True-False-True-False-True]	64.8500μs	17.2225μs	58.0637 KOps/s	59.8807 KOps/s	$\color{#d91a1a}-3.03\%$
test_step_mdp_speed[True-False-True-False-False]	44.9140μs	10.9539μs	91.2917 KOps/s	95.5842 KOps/s	$\color{#d91a1a}-4.49\%$
test_step_mdp_speed[True-False-False-True-True]	84.0460μs	29.8735μs	33.4744 KOps/s	34.4635 KOps/s	$\color{#d91a1a}-2.87\%$
test_step_mdp_speed[True-False-False-True-False]	43.5510μs	19.9222μs	50.1952 KOps/s	52.3395 KOps/s	$\color{#d91a1a}-4.10\%$
test_step_mdp_speed[True-False-False-False-True]	74.7090μs	18.3899μs	54.3778 KOps/s	55.9586 KOps/s	$\color{#d91a1a}-2.82\%$
test_step_mdp_speed[True-False-False-False-False]	43.3400μs	12.3279μs	81.1167 KOps/s	85.1099 KOps/s	$\color{#d91a1a}-4.69\%$
test_step_mdp_speed[False-True-True-True-True]	87.1320μs	28.5426μs	35.0354 KOps/s	35.8060 KOps/s	$\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-True-True-True-False]	47.5880μs	18.2578μs	54.7711 KOps/s	54.9814 KOps/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-True-True-False-True]	81.0010μs	20.1785μs	49.5577 KOps/s	52.3229 KOps/s	$\textbf{\color{#d91a1a}-5.28\%}$
test_step_mdp_speed[False-True-True-False-False]	65.6020μs	12.3282μs	81.1151 KOps/s	85.1330 KOps/s	$\color{#d91a1a}-4.72\%$
test_step_mdp_speed[False-True-False-True-True]	59.4610μs	30.0624μs	33.2642 KOps/s	34.4940 KOps/s	$\color{#d91a1a}-3.57\%$
test_step_mdp_speed[False-True-False-True-False]	68.2870μs	19.6008μs	51.0184 KOps/s	52.0224 KOps/s	$\color{#d91a1a}-1.93\%$
test_step_mdp_speed[False-True-False-False-True]	59.7400μs	21.0748μs	47.4501 KOps/s	48.7890 KOps/s	$\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-True-False-False-False]	66.2730μs	13.6228μs	73.4063 KOps/s	75.2288 KOps/s	$\color{#d91a1a}-2.42\%$
test_step_mdp_speed[False-False-True-True-True]	62.1950μs	31.5073μs	31.7387 KOps/s	32.8619 KOps/s	$\color{#d91a1a}-3.42\%$
test_step_mdp_speed[False-False-True-True-False]	73.1660μs	20.9935μs	47.6338 KOps/s	49.1744 KOps/s	$\color{#d91a1a}-3.13\%$
test_step_mdp_speed[False-False-True-False-True]	53.5490μs	21.6359μs	46.2195 KOps/s	48.7750 KOps/s	$\textbf{\color{#d91a1a}-5.24\%}$
test_step_mdp_speed[False-False-True-False-False]	66.8840μs	13.8969μs	71.9584 KOps/s	76.6959 KOps/s	$\textbf{\color{#d91a1a}-6.18\%}$
test_step_mdp_speed[False-False-False-True-True]	89.5670μs	32.9476μs	30.3512 KOps/s	31.5519 KOps/s	$\color{#d91a1a}-3.81\%$
test_step_mdp_speed[False-False-False-True-False]	76.9730μs	22.8263μs	43.8090 KOps/s	45.8695 KOps/s	$\color{#d91a1a}-4.49\%$
test_step_mdp_speed[False-False-False-False-True]	75.1500μs	22.0520μs	45.3474 KOps/s	45.3156 KOps/s	$\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-False-False-False-False]	54.3410μs	15.0461μs	66.4624 KOps/s	70.3229 KOps/s	$\textbf{\color{#d91a1a}-5.49\%}$
test_values[generalized_advantage_estimate-True-True]	12.4446ms	12.1429ms	82.3525 Ops/s	83.1884 Ops/s	$\color{#d91a1a}-1.00\%$
test_values[vec_generalized_advantage_estimate-True-True]	37.0301ms	26.6432ms	37.5331 Ops/s	36.9368 Ops/s	$\color{#35bf28}+1.61\%$
test_values[td0_return_estimate-False-False]	0.2825ms	0.1928ms	5.1856 KOps/s	5.3100 KOps/s	$\color{#d91a1a}-2.34\%$
test_values[td1_return_estimate-False-False]	27.1253ms	25.9421ms	38.5474 Ops/s	38.8132 Ops/s	$\color{#d91a1a}-0.68\%$
test_values[vec_td1_return_estimate-False-False]	34.8416ms	26.5666ms	37.6413 Ops/s	36.9126 Ops/s	$\color{#35bf28}+1.97\%$
test_values[td_lambda_return_estimate-True-False]	89.8161ms	37.3461ms	26.7766 Ops/s	27.4935 Ops/s	$\color{#d91a1a}-2.61\%$
test_values[vec_td_lambda_return_estimate-True-False]	35.0333ms	26.8374ms	37.2614 Ops/s	37.0194 Ops/s	$\color{#35bf28}+0.65\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.3395ms	8.1124ms	123.2685 Ops/s	125.9351 Ops/s	$\color{#d91a1a}-2.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	10.8652ms	1.9748ms	506.3768 Ops/s	510.6610 Ops/s	$\color{#d91a1a}-0.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5466ms	0.4209ms	2.3759 KOps/s	2.3335 KOps/s	$\color{#35bf28}+1.82\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	41.9487ms	40.0076ms	24.9953 Ops/s	24.6962 Ops/s	$\color{#35bf28}+1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	11.1351ms	2.5327ms	394.8295 Ops/s	397.5862 Ops/s	$\color{#d91a1a}-0.69\%$
test_dqn_speed	15.7797ms	1.7229ms	580.4263 Ops/s	592.3406 Ops/s	$\color{#d91a1a}-2.01\%$
test_ddpg_speed	28.6872ms	3.5761ms	279.6377 Ops/s	325.7994 Ops/s	$\textbf{\color{#d91a1a}-14.17\%}$
test_sac_speed	18.4482ms	8.7975ms	113.6691 Ops/s	115.5920 Ops/s	$\color{#d91a1a}-1.66\%$
test_redq_speed	0.1097s	19.5130ms	51.2478 Ops/s	59.6378 Ops/s	$\textbf{\color{#d91a1a}-14.07\%}$
test_redq_deprec_speed	24.3207ms	15.7731ms	63.3989 Ops/s	66.0264 Ops/s	$\color{#d91a1a}-3.98\%$
test_td3_speed	10.1293ms	8.8966ms	112.4021 Ops/s	110.8267 Ops/s	$\color{#35bf28}+1.42\%$
test_cql_speed	45.4273ms	37.8285ms	26.4351 Ops/s	26.8630 Ops/s	$\color{#d91a1a}-1.59\%$
test_a2c_speed	17.3192ms	8.6236ms	115.9607 Ops/s	119.3286 Ops/s	$\color{#d91a1a}-2.82\%$
test_ppo_speed	17.0827ms	9.0961ms	109.9366 Ops/s	107.7574 Ops/s	$\color{#35bf28}+2.02\%$
test_reinforce_speed	16.2961ms	7.7192ms	129.5466 Ops/s	134.8304 Ops/s	$\color{#d91a1a}-3.92\%$
test_iql_speed	42.0204ms	33.9509ms	29.4543 Ops/s	30.1160 Ops/s	$\color{#d91a1a}-2.20\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.7307ms	1.9956ms	501.0983 Ops/s	496.9989 Ops/s	$\color{#35bf28}+0.82\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	4.0680ms	2.1529ms	464.4968 Ops/s	461.6829 Ops/s	$\color{#35bf28}+0.61\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.7128ms	2.1446ms	466.2784 Ops/s	453.3263 Ops/s	$\color{#35bf28}+2.86\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.5943ms	2.0166ms	495.8743 Ops/s	471.6147 Ops/s	$\textbf{\color{#35bf28}+5.14\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1399s	2.4938ms	400.9869 Ops/s	454.4106 Ops/s	$\textbf{\color{#d91a1a}-11.76\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.2834ms	2.1236ms	470.9094 Ops/s	479.5186 Ops/s	$\color{#d91a1a}-1.80\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.7886ms	1.9793ms	505.2269 Ops/s	498.1144 Ops/s	$\color{#35bf28}+1.43\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.3058ms	2.0972ms	476.8292 Ops/s	458.8747 Ops/s	$\color{#35bf28}+3.91\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.1716s	2.4739ms	404.2215 Ops/s	457.1447 Ops/s	$\textbf{\color{#d91a1a}-11.58\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.2432ms	1.8977ms	526.9479 Ops/s	486.9164 Ops/s	$\textbf{\color{#35bf28}+8.22\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	4.0827ms	2.1887ms	456.8846 Ops/s	454.0767 Ops/s	$\color{#35bf28}+0.62\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.0507ms	2.1769ms	459.3604 Ops/s	460.0224 Ops/s	$\color{#d91a1a}-0.14\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.0744ms	1.9666ms	508.4851 Ops/s	480.6108 Ops/s	$\textbf{\color{#35bf28}+5.80\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1341s	2.4738ms	404.2283 Ops/s	452.6798 Ops/s	$\textbf{\color{#d91a1a}-10.70\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.3911ms	2.1464ms	465.8972 Ops/s	404.4408 Ops/s	$\textbf{\color{#35bf28}+15.20\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.2679ms	1.9787ms	505.3872 Ops/s	492.1833 Ops/s	$\color{#35bf28}+2.68\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.6489ms	2.1544ms	464.1658 Ops/s	450.9270 Ops/s	$\color{#35bf28}+2.94\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.1415s	2.4981ms	400.3092 Ops/s	452.6925 Ops/s	$\textbf{\color{#d91a1a}-11.57\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.3010s	24.6429ms	40.5797 Ops/s	38.7674 Ops/s	$\color{#35bf28}+4.67\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1366s	20.7876ms	48.1057 Ops/s	47.9517 Ops/s	$\color{#35bf28}+0.32\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1310s	22.5661ms	44.3143 Ops/s	43.1210 Ops/s	$\color{#35bf28}+2.77\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1274s	22.0678ms	45.3148 Ops/s	48.1024 Ops/s	$\textbf{\color{#d91a1a}-5.80\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1257s	19.8988ms	50.2544 Ops/s	43.3541 Ops/s	$\textbf{\color{#35bf28}+15.92\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1276s	21.7283ms	46.0229 Ops/s	48.2418 Ops/s	$\color{#d91a1a}-4.60\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1259s	19.8359ms	50.4137 Ops/s	42.9927 Ops/s	$\textbf{\color{#35bf28}+17.26\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1262s	21.7650ms	45.9453 Ops/s	48.8827 Ops/s	$\textbf{\color{#d91a1a}-6.01\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1241s	19.5452ms	51.1634 Ops/s	43.5310 Ops/s	$\textbf{\color{#35bf28}+17.53\%}$

[vmoens]

LGTM thanks for the help with this!