[Doc] Update pendulum and rnn tutos

[vmoens]

Copies the corrected tutorials from pytorch/tutorials#2653 and pytorch/tutorials#2643

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1691

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 4 Unrelated Failures

As of commit ca7aa4f with merge base 4ab5b10 ():

NEW FAILURES - The following jobs have failed:

• Lint / c-source / linux-job (gh)
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
• Unit-tests on Windows CPU / unittests / windows-job (gh)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Continuous Benchmark (PR) / GPU Pytest benchmark (linux.4xlarge.nvidia.gpu, 3.8) (gh)
• Unit-tests on Linux CPU / tests (3.11) / linux-job (gh)
• Unit-tests on Linux GPU / tests (3.8, 12.1) / linux-job (gh)

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	65.7019ms	63.4958ms	15.7491 Ops/s	15.1960 Ops/s	$\color{#35bf28}+3.64\%$
test_sync	36.0177ms	34.0892ms	29.3348 Ops/s	28.9970 Ops/s	$\color{#35bf28}+1.16\%$
test_async	56.2800ms	32.4368ms	30.8292 Ops/s	30.7501 Ops/s	$\color{#35bf28}+0.26\%$
test_simple	0.5062s	0.4615s	2.1671 Ops/s	2.2728 Ops/s	$\color{#d91a1a}-4.65\%$
test_transformed	0.6559s	0.6043s	1.6547 Ops/s	1.6228 Ops/s	$\color{#35bf28}+1.97\%$
test_serial	1.3252s	1.2659s	0.7900 Ops/s	0.7846 Ops/s	$\color{#35bf28}+0.68\%$
test_parallel	1.2984s	1.2462s	0.8024 Ops/s	0.7956 Ops/s	$\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-True-True-True]	0.1868ms	25.0364μs	39.9419 KOps/s	40.4176 KOps/s	$\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-True-True-True-False]	49.9240μs	14.6696μs	68.1683 KOps/s	66.8525 KOps/s	$\color{#35bf28}+1.97\%$
test_step_mdp_speed[True-True-True-False-True]	50.1430μs	15.3181μs	65.2822 KOps/s	64.6256 KOps/s	$\color{#35bf28}+1.02\%$
test_step_mdp_speed[True-True-True-False-False]	51.9080μs	8.9568μs	111.6469 KOps/s	109.3488 KOps/s	$\color{#35bf28}+2.10\%$
test_step_mdp_speed[True-True-False-True-True]	51.8970μs	26.2255μs	38.1308 KOps/s	38.0692 KOps/s	$\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-True-False-True-False]	68.9190μs	16.1067μs	62.0859 KOps/s	61.0529 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-True-False-False-True]	57.6680μs	16.5288μs	60.5004 KOps/s	59.4294 KOps/s	$\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-True-False-False-False]	51.8070μs	10.2846μs	97.2326 KOps/s	95.6453 KOps/s	$\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-False-True-True-True]	68.2570μs	27.5556μs	36.2903 KOps/s	36.1024 KOps/s	$\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-False-True-True-False]	50.5440μs	17.5176μs	57.0854 KOps/s	55.7166 KOps/s	$\color{#35bf28}+2.46\%$
test_step_mdp_speed[True-False-True-False-True]	49.6620μs	16.6672μs	59.9982 KOps/s	59.6967 KOps/s	$\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-False-True-False-False]	47.6590μs	10.3331μs	96.7763 KOps/s	95.4293 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-False-False-True-True]	64.8810μs	28.8045μs	34.7168 KOps/s	34.7037 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-False-False-True-False]	71.2230μs	18.7604μs	53.3039 KOps/s	52.5174 KOps/s	$\color{#35bf28}+1.50\%$
test_step_mdp_speed[True-False-False-False-True]	67.4270μs	17.8731μs	55.9500 KOps/s	56.1015 KOps/s	$\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-False-False-False-False]	42.9610μs	11.5509μs	86.5734 KOps/s	83.8570 KOps/s	$\color{#35bf28}+3.24\%$
test_step_mdp_speed[False-True-True-True-True]	79.7200μs	27.7254μs	36.0680 KOps/s	36.1041 KOps/s	$\color{#d91a1a}-0.10\%$
test_step_mdp_speed[False-True-True-True-False]	49.1920μs	17.7688μs	56.2784 KOps/s	56.4761 KOps/s	$\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-True-True-False-True]	78.6270μs	19.2928μs	51.8328 KOps/s	51.9171 KOps/s	$\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-True-True-False-False]	43.4220μs	11.6726μs	85.6708 KOps/s	83.8148 KOps/s	$\color{#35bf28}+2.21\%$
test_step_mdp_speed[False-True-False-True-True]	0.1006ms	28.9568μs	34.5342 KOps/s	34.5821 KOps/s	$\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-True-False-True-False]	48.8510μs	19.0301μs	52.5482 KOps/s	51.9350 KOps/s	$\color{#35bf28}+1.18\%$
test_step_mdp_speed[False-True-False-False-True]	75.4410μs	20.6422μs	48.4443 KOps/s	48.5443 KOps/s	$\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-True-False-False-False]	64.5820μs	13.0463μs	76.6501 KOps/s	74.4293 KOps/s	$\color{#35bf28}+2.98\%$
test_step_mdp_speed[False-False-True-True-True]	93.0920μs	30.1428μs	33.1754 KOps/s	32.9789 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-True-True-False]	78.1450μs	20.3658μs	49.1019 KOps/s	48.6039 KOps/s	$\color{#35bf28}+1.02\%$
test_step_mdp_speed[False-False-True-False-True]	78.8980μs	20.6312μs	48.4703 KOps/s	48.6410 KOps/s	$\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-False-True-False-False]	29.9260μs	13.0735μs	76.4907 KOps/s	74.6211 KOps/s	$\color{#35bf28}+2.51\%$
test_step_mdp_speed[False-False-False-True-True]	0.1108ms	31.3190μs	31.9295 KOps/s	31.7182 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-False-True-False]	61.2750μs	21.3362μs	46.8687 KOps/s	45.8595 KOps/s	$\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-False-False-False-True]	77.3070μs	21.3219μs	46.9002 KOps/s	45.7198 KOps/s	$\color{#35bf28}+2.58\%$
test_step_mdp_speed[False-False-False-False-False]	46.2560μs	14.1998μs	70.4235 KOps/s	68.7108 KOps/s	$\color{#35bf28}+2.49\%$
test_values[generalized_advantage_estimate-True-True]	12.3315ms	12.0682ms	82.8623 Ops/s	81.9445 Ops/s	$\color{#35bf28}+1.12\%$
test_values[vec_generalized_advantage_estimate-True-True]	35.8550ms	27.5239ms	36.3321 Ops/s	37.5213 Ops/s	$\color{#d91a1a}-3.17\%$
test_values[td0_return_estimate-False-False]	0.2592ms	0.1837ms	5.4429 KOps/s	5.3863 KOps/s	$\color{#35bf28}+1.05\%$
test_values[td1_return_estimate-False-False]	27.5309ms	25.7829ms	38.7854 Ops/s	38.1427 Ops/s	$\color{#35bf28}+1.68\%$
test_values[vec_td1_return_estimate-False-False]	35.6781ms	27.4773ms	36.3936 Ops/s	37.6380 Ops/s	$\color{#d91a1a}-3.31\%$
test_values[td_lambda_return_estimate-True-False]	41.2282ms	35.9259ms	27.8351 Ops/s	27.4514 Ops/s	$\color{#35bf28}+1.40\%$
test_values[vec_td_lambda_return_estimate-True-False]	36.1763ms	27.4850ms	36.3835 Ops/s	37.3347 Ops/s	$\color{#d91a1a}-2.55\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.9038ms	8.0101ms	124.8417 Ops/s	122.5934 Ops/s	$\color{#35bf28}+1.83\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	10.7858ms	1.9576ms	510.8237 Ops/s	505.9263 Ops/s	$\color{#35bf28}+0.97\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	25.5483ms	0.4508ms	2.2185 KOps/s	2.3028 KOps/s	$\color{#d91a1a}-3.66\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	47.0532ms	39.6777ms	25.2031 Ops/s	24.5716 Ops/s	$\color{#35bf28}+2.57\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	11.0748ms	2.4892ms	401.7365 Ops/s	404.2879 Ops/s	$\color{#d91a1a}-0.63\%$
test_dqn_speed	10.0730ms	1.7209ms	581.1001 Ops/s	582.5443 Ops/s	$\color{#d91a1a}-0.25\%$
test_ddpg_speed	11.7566ms	3.1587ms	316.5826 Ops/s	289.0408 Ops/s	$\textbf{\color{#35bf28}+9.53\%}$
test_sac_speed	21.6625ms	8.8774ms	112.6457 Ops/s	112.2885 Ops/s	$\color{#35bf28}+0.32\%$
test_redq_speed	25.3180ms	16.9054ms	59.1528 Ops/s	59.5242 Ops/s	$\color{#d91a1a}-0.62\%$
test_redq_deprec_speed	24.3022ms	15.0114ms	66.6158 Ops/s	65.7688 Ops/s	$\color{#35bf28}+1.29\%$
test_td3_speed	16.8884ms	9.0961ms	109.9368 Ops/s	110.0553 Ops/s	$\color{#d91a1a}-0.11\%$
test_cql_speed	48.8614ms	37.2837ms	26.8214 Ops/s	26.4573 Ops/s	$\color{#35bf28}+1.38\%$
test_a2c_speed	16.9028ms	8.7662ms	114.0745 Ops/s	115.1920 Ops/s	$\color{#d91a1a}-0.97\%$
test_ppo_speed	17.5998ms	9.0803ms	110.1286 Ops/s	106.4956 Ops/s	$\color{#35bf28}+3.41\%$
test_reinforce_speed	18.7224ms	7.8594ms	127.2367 Ops/s	127.5767 Ops/s	$\color{#d91a1a}-0.27\%$
test_iql_speed	42.0578ms	33.6293ms	29.7360 Ops/s	29.1007 Ops/s	$\color{#35bf28}+2.18\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.9300ms	1.9925ms	501.8811 Ops/s	496.7214 Ops/s	$\color{#35bf28}+1.04\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	4.1985ms	2.1159ms	472.6098 Ops/s	471.3733 Ops/s	$\color{#35bf28}+0.26\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.2762ms	2.1630ms	462.3160 Ops/s	463.3864 Ops/s	$\color{#d91a1a}-0.23\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.0543ms	2.0273ms	493.2649 Ops/s	500.9529 Ops/s	$\color{#d91a1a}-1.53\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1309s	2.4665ms	405.4255 Ops/s	472.5746 Ops/s	$\textbf{\color{#d91a1a}-14.21\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.6169ms	2.1221ms	471.2393 Ops/s	462.1736 Ops/s	$\color{#35bf28}+1.96\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.4521ms	1.9984ms	500.4044 Ops/s	502.4149 Ops/s	$\color{#d91a1a}-0.40\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.2387ms	2.1790ms	458.9244 Ops/s	465.4778 Ops/s	$\color{#d91a1a}-1.41\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.2812ms	2.1543ms	464.1809 Ops/s	463.0462 Ops/s	$\color{#35bf28}+0.25\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.4264ms	1.9838ms	504.0850 Ops/s	497.1831 Ops/s	$\color{#35bf28}+1.39\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	4.3558ms	2.1257ms	470.4364 Ops/s	403.6094 Ops/s	$\textbf{\color{#35bf28}+16.56\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.3444ms	2.1347ms	468.4603 Ops/s	451.3274 Ops/s	$\color{#35bf28}+3.80\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.9313ms	2.0066ms	498.3602 Ops/s	497.3850 Ops/s	$\color{#35bf28}+0.20\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.4848ms	2.1561ms	463.7941 Ops/s	462.9689 Ops/s	$\color{#35bf28}+0.18\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	5.0825ms	2.2035ms	453.8220 Ops/s	388.3890 Ops/s	$\textbf{\color{#35bf28}+16.85\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.9764ms	1.9826ms	504.3908 Ops/s	507.2889 Ops/s	$\color{#d91a1a}-0.57\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	4.4718ms	2.2106ms	452.3711 Ops/s	466.3623 Ops/s	$\color{#d91a1a}-3.00\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.7873ms	2.2133ms	451.8166 Ops/s	463.8479 Ops/s	$\color{#d91a1a}-2.59\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.2960s	24.4142ms	40.9598 Ops/s	40.1839 Ops/s	$\color{#35bf28}+1.93\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1382s	23.1025ms	43.2853 Ops/s	49.9713 Ops/s	$\textbf{\color{#d91a1a}-13.38\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1358s	20.6801ms	48.3557 Ops/s	44.8472 Ops/s	$\textbf{\color{#35bf28}+7.82\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1571s	24.2580ms	41.2234 Ops/s	49.5188 Ops/s	$\textbf{\color{#d91a1a}-16.75\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1360s	20.8208ms	48.0289 Ops/s	44.1570 Ops/s	$\textbf{\color{#35bf28}+8.77\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1372s	23.3480ms	42.8302 Ops/s	48.4698 Ops/s	$\textbf{\color{#d91a1a}-11.64\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1407s	21.0012ms	47.6164 Ops/s	44.9907 Ops/s	$\textbf{\color{#35bf28}+5.84\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1407s	23.5413ms	42.4786 Ops/s	49.5364 Ops/s	$\textbf{\color{#d91a1a}-14.25\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1523s	21.2874ms	46.9761 Ops/s	44.1060 Ops/s	$\textbf{\color{#35bf28}+6.51\%}$