[Minor] Remove gymnasium num_envs warnings

[vmoens]

Description

Describe your changes in detail.

Motivation and Context

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1690

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures, 8 Unrelated Failures

As of commit 3a405c4 with merge base 4ab5b10 ():

NEW FAILURES - The following jobs have failed:

• Lint / python-source-and-configs / linux-job (gh)
• Olddeps Tests on Linux / unittests (3.8, 11.6) / linux-job (gh)
• Optdeps Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.10) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.11) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.9) / linux-job (gh)
• Unit-tests on Linux GPU / tests (3.8, 12.1) / linux-job (gh)
• Unit-tests on Windows CPU / unittests / windows-job (gh)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Continuous Benchmark (PR) / GPU Pytest benchmark (linux.4xlarge.nvidia.gpu, 3.8) (gh)
• Unit-tests on Linux CPU / tests (3.8) / linux-job (gh)
• Unit-tests on Linux GPU, latest stable release / tests (3.8, 11.8) / linux-job (gh)
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
• Unit-tests on Windows GPU / unittests / windows-job (gh)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Continuous Benchmark / GPU Pytest benchmark (linux.4xlarge.nvidia.gpu, 3.8) (gh)
• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	64.0635ms	61.7468ms	16.1952 Ops/s	15.7019 Ops/s	$\color{#35bf28}+3.14\%$
test_sync	41.3604ms	34.1395ms	29.2916 Ops/s	28.7809 Ops/s	$\color{#35bf28}+1.77\%$
test_async	68.8711ms	33.2476ms	30.0774 Ops/s	31.4020 Ops/s	$\color{#d91a1a}-4.22\%$
test_simple	0.4926s	0.4477s	2.2339 Ops/s	2.2891 Ops/s	$\color{#d91a1a}-2.41\%$
test_transformed	0.6526s	0.5984s	1.6711 Ops/s	1.6770 Ops/s	$\color{#d91a1a}-0.35\%$
test_serial	1.2915s	1.2419s	0.8052 Ops/s	0.8029 Ops/s	$\color{#35bf28}+0.29\%$
test_parallel	1.2974s	1.2484s	0.8011 Ops/s	0.8076 Ops/s	$\color{#d91a1a}-0.81\%$
test_step_mdp_speed[True-True-True-True-True]	0.1760ms	25.0807μs	39.8713 KOps/s	41.0292 KOps/s	$\color{#d91a1a}-2.82\%$
test_step_mdp_speed[True-True-True-True-False]	69.6180μs	14.6428μs	68.2929 KOps/s	68.0348 KOps/s	$\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-True-False-True]	75.2900μs	15.1407μs	66.0470 KOps/s	64.9420 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-True-True-False-False]	37.7000μs	9.0523μs	110.4697 KOps/s	111.5465 KOps/s	$\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-True-False-True-True]	95.5180μs	26.5766μs	37.6271 KOps/s	38.0660 KOps/s	$\color{#d91a1a}-1.15\%$
test_step_mdp_speed[True-True-False-True-False]	45.5050μs	16.3534μs	61.1492 KOps/s	61.5462 KOps/s	$\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-True-False-False-True]	64.3500μs	16.5671μs	60.3607 KOps/s	59.4731 KOps/s	$\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-True-False-False-False]	44.1320μs	10.6239μs	94.1278 KOps/s	95.7954 KOps/s	$\color{#d91a1a}-1.74\%$
test_step_mdp_speed[True-False-True-True-True]	76.9930μs	27.5117μs	36.3482 KOps/s	36.2036 KOps/s	$\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-False-True-True-False]	59.6010μs	18.1013μs	55.2448 KOps/s	56.8103 KOps/s	$\color{#d91a1a}-2.76\%$
test_step_mdp_speed[True-False-True-False-True]	66.1030μs	16.4545μs	60.7736 KOps/s	59.8195 KOps/s	$\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-False-True-False-False]	53.0090μs	10.6472μs	93.9211 KOps/s	96.2060 KOps/s	$\color{#d91a1a}-2.38\%$
test_step_mdp_speed[True-False-False-True-True]	76.6230μs	28.9263μs	34.5706 KOps/s	32.9135 KOps/s	$\textbf{\color{#35bf28}+5.03\%}$
test_step_mdp_speed[True-False-False-True-False]	71.5840μs	19.2771μs	51.8751 KOps/s	52.5438 KOps/s	$\color{#d91a1a}-1.27\%$
test_step_mdp_speed[True-False-False-False-True]	70.4710μs	17.9175μs	55.8115 KOps/s	55.2804 KOps/s	$\color{#35bf28}+0.96\%$
test_step_mdp_speed[True-False-False-False-False]	38.0510μs	11.6979μs	85.4856 KOps/s	85.4536 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-True-True-True-True]	84.1370μs	27.8409μs	35.9184 KOps/s	35.7644 KOps/s	$\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-True-True-True-False]	62.4150μs	17.7683μs	56.2799 KOps/s	56.4516 KOps/s	$\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-True-True-False-True]	52.5980μs	19.5307μs	51.2016 KOps/s	51.6930 KOps/s	$\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-True-False-False]	56.0540μs	11.7815μs	84.8787 KOps/s	83.4777 KOps/s	$\color{#35bf28}+1.68\%$
test_step_mdp_speed[False-True-False-True-True]	96.6290μs	28.9567μs	34.5344 KOps/s	34.2117 KOps/s	$\color{#35bf28}+0.94\%$
test_step_mdp_speed[False-True-False-True-False]	84.0360μs	19.1013μs	52.3524 KOps/s	51.8718 KOps/s	$\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-True-False-False-True]	60.7730μs	21.0180μs	47.5784 KOps/s	47.8094 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-True-False-False-False]	56.4460μs	13.1234μs	76.1999 KOps/s	74.3667 KOps/s	$\color{#35bf28}+2.47\%$
test_step_mdp_speed[False-False-True-True-True]	69.3490μs	30.3334μs	32.9669 KOps/s	32.4346 KOps/s	$\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-False-True-True-False]	71.2930μs	20.6610μs	48.4003 KOps/s	48.7885 KOps/s	$\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-True-False-True]	65.8520μs	21.0282μs	47.5552 KOps/s	47.6132 KOps/s	$\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-False-True-False-False]	69.9110μs	13.4890μs	74.1343 KOps/s	75.2191 KOps/s	$\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-False-False-True-True]	71.5230μs	31.7551μs	31.4910 KOps/s	31.1024 KOps/s	$\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-True-False]	71.6840μs	22.0403μs	45.3714 KOps/s	45.7451 KOps/s	$\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-False-False-False-True]	72.4450μs	21.8153μs	45.8394 KOps/s	44.9392 KOps/s	$\color{#35bf28}+2.00\%$
test_step_mdp_speed[False-False-False-False-False]	48.6110μs	14.5255μs	68.8443 KOps/s	68.8446 KOps/s	$-0.00\%$
test_values[generalized_advantage_estimate-True-True]	13.9858ms	12.0125ms	83.2469 Ops/s	84.4065 Ops/s	$\color{#d91a1a}-1.37\%$
test_values[vec_generalized_advantage_estimate-True-True]	35.8706ms	26.6308ms	37.5504 Ops/s	37.8034 Ops/s	$\color{#d91a1a}-0.67\%$
test_values[td0_return_estimate-False-False]	0.3140ms	0.1787ms	5.5946 KOps/s	5.5698 KOps/s	$\color{#35bf28}+0.45\%$
test_values[td1_return_estimate-False-False]	26.8307ms	25.7157ms	38.8867 Ops/s	39.3842 Ops/s	$\color{#d91a1a}-1.26\%$
test_values[vec_td1_return_estimate-False-False]	37.3344ms	26.8253ms	37.2782 Ops/s	38.0122 Ops/s	$\color{#d91a1a}-1.93\%$
test_values[td_lambda_return_estimate-True-False]	39.4771ms	36.0060ms	27.7732 Ops/s	27.8174 Ops/s	$\color{#d91a1a}-0.16\%$
test_values[vec_td_lambda_return_estimate-True-False]	36.1817ms	26.7402ms	37.3969 Ops/s	36.9966 Ops/s	$\color{#35bf28}+1.08\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.1054ms	7.9350ms	126.0243 Ops/s	127.0115 Ops/s	$\color{#d91a1a}-0.78\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	13.3249ms	1.9173ms	521.5599 Ops/s	513.8268 Ops/s	$\color{#35bf28}+1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5735ms	0.4309ms	2.3208 KOps/s	2.3171 KOps/s	$\color{#35bf28}+0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	49.3555ms	41.3147ms	24.2045 Ops/s	24.2960 Ops/s	$\color{#d91a1a}-0.38\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	13.0694ms	2.5055ms	399.1291 Ops/s	394.5530 Ops/s	$\color{#35bf28}+1.16\%$
test_dqn_speed	12.0538ms	1.6917ms	591.1046 Ops/s	578.7343 Ops/s	$\color{#35bf28}+2.14\%$
test_ddpg_speed	13.1903ms	3.0816ms	324.5102 Ops/s	316.5068 Ops/s	$\color{#35bf28}+2.53\%$
test_sac_speed	18.4514ms	8.9853ms	111.2925 Ops/s	112.8586 Ops/s	$\color{#d91a1a}-1.39\%$
test_redq_speed	26.9466ms	16.6114ms	60.1998 Ops/s	61.4477 Ops/s	$\color{#d91a1a}-2.03\%$
test_redq_deprec_speed	25.9721ms	14.4811ms	69.0553 Ops/s	68.8908 Ops/s	$\color{#35bf28}+0.24\%$
test_td3_speed	10.0874ms	8.8186ms	113.3962 Ops/s	112.9422 Ops/s	$\color{#35bf28}+0.40\%$
test_cql_speed	45.6393ms	36.4340ms	27.4469 Ops/s	26.9940 Ops/s	$\color{#35bf28}+1.68\%$
test_a2c_speed	18.4878ms	8.0707ms	123.9053 Ops/s	114.2622 Ops/s	$\textbf{\color{#35bf28}+8.44\%}$
test_ppo_speed	9.5831ms	8.3220ms	120.1637 Ops/s	110.6400 Ops/s	$\textbf{\color{#35bf28}+8.61\%}$
test_reinforce_speed	19.0270ms	7.3800ms	135.5017 Ops/s	131.5319 Ops/s	$\color{#35bf28}+3.02\%$
test_iql_speed	43.8011ms	33.7946ms	29.5906 Ops/s	29.8916 Ops/s	$\color{#d91a1a}-1.01\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.3042ms	1.8782ms	532.4115 Ops/s	537.1219 Ops/s	$\color{#d91a1a}-0.88\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.8709ms	2.0661ms	483.9970 Ops/s	497.2126 Ops/s	$\color{#d91a1a}-2.66\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.3884ms	2.0256ms	493.6728 Ops/s	500.9021 Ops/s	$\color{#d91a1a}-1.44\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.8090ms	1.9025ms	525.6269 Ops/s	526.5573 Ops/s	$\color{#d91a1a}-0.18\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1218s	2.2868ms	437.2953 Ops/s	491.9313 Ops/s	$\textbf{\color{#d91a1a}-11.11\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.3367ms	2.0248ms	493.8664 Ops/s	486.0818 Ops/s	$\color{#35bf28}+1.60\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.9057ms	1.8880ms	529.6504 Ops/s	513.4726 Ops/s	$\color{#35bf28}+3.15\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.1899ms	2.0331ms	491.8660 Ops/s	481.4304 Ops/s	$\color{#35bf28}+2.17\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.0707ms	2.1070ms	474.5978 Ops/s	492.0634 Ops/s	$\color{#d91a1a}-3.55\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.6894ms	1.8770ms	532.7536 Ops/s	519.8225 Ops/s	$\color{#35bf28}+2.49\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.2325ms	2.0154ms	496.1773 Ops/s	430.0433 Ops/s	$\textbf{\color{#35bf28}+15.38\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	2.9993ms	2.0453ms	488.9243 Ops/s	483.7782 Ops/s	$\color{#35bf28}+1.06\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.8606ms	1.9638ms	509.2256 Ops/s	510.2267 Ops/s	$\color{#d91a1a}-0.20\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.6356ms	2.0576ms	486.0126 Ops/s	492.8729 Ops/s	$\color{#d91a1a}-1.39\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.1158ms	2.0541ms	486.8197 Ops/s	424.6115 Ops/s	$\textbf{\color{#35bf28}+14.65\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.3876ms	1.9302ms	518.0789 Ops/s	534.8007 Ops/s	$\color{#d91a1a}-3.13\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.4470ms	2.0733ms	482.3335 Ops/s	494.2978 Ops/s	$\color{#d91a1a}-2.42\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.4725ms	2.1010ms	475.9539 Ops/s	492.5102 Ops/s	$\color{#d91a1a}-3.36\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.2752s	22.7706ms	43.9164 Ops/s	41.9221 Ops/s	$\color{#35bf28}+4.76\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1278s	21.9334ms	45.5927 Ops/s	50.7059 Ops/s	$\textbf{\color{#d91a1a}-10.08\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1332s	20.0439ms	49.8905 Ops/s	46.6513 Ops/s	$\textbf{\color{#35bf28}+6.94\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1501s	23.0568ms	43.3711 Ops/s	51.3723 Ops/s	$\textbf{\color{#d91a1a}-15.57\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1366s	19.6886ms	50.7908 Ops/s	46.4854 Ops/s	$\textbf{\color{#35bf28}+9.26\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1207s	21.6142ms	46.2659 Ops/s	51.1187 Ops/s	$\textbf{\color{#d91a1a}-9.49\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1313s	19.8568ms	50.3606 Ops/s	45.8770 Ops/s	$\textbf{\color{#35bf28}+9.77\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1316s	22.0492ms	45.3531 Ops/s	45.9027 Ops/s	$\color{#d91a1a}-1.20\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1314s	21.9983ms	45.4581 Ops/s	51.1709 Ops/s	$\textbf{\color{#d91a1a}-11.16\%}$