[Feature] MCTS Scoring functions

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] MCTS policy #2359
• -> [Feature] MCTS Scoring functions #2358
• [Feature] TensorSpec.enumerate() #2354
• [Feature] MCTSForest #2307
• [Feature] TensorDictMap #2306
• [Feature] TensorDictMap Query module #2305
• [Feature] TensorDictMap hashing functions #2304

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2358

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 18 New Failures, 2 Cancelled Jobs, 7 Unrelated Failures

As of commit ed88a63 with merge base 35a1c5b ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 4.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 4.
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t fb726a7ee77b075bb825e11341a502cc9aaea94e00fbb76c3251b55ad7dadeb9 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 100229222d2974b27e6ca325fb203f1ed772801b5e62ced231a5535b391f4af3 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 1cfd2afbaa23d264244fd4f42fc48d83cbbd7f6f7dcdaf4a49e6a8eb1be4e5c2 /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 0fd602a45e7c9e3621c9d4b7bb4c76122364d133cd56937766d517913325093f /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh)
  torchrl/modules/mcts/scores.py:14:1: F401 'torch.nn' imported but unused
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 67c286ec80b45c2b8c2d4e3430728140b8cea233bc9166a24c9ebfef1340f5c8 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t cf90bd6e846a85cdadcbc7e88684bcdd72e20b62400ef0bcd31a2cbfbf3b796f /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  RuntimeError: Command docker exec -t 357261a4f7ea38428b05b0c0df6610d49df07c2d83f7fbaf8251d46deeff91ed /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  RuntimeError: Command docker exec -t 62659af8d5a0fbe44b0f694239b82a2867e5fa9589bf0742cb5c3e22773477a4 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 5734811b4f4e64685ae22c7e485d82d48f13be85593164d4f1ef92eb8b9aabe3 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  RuntimeError: Command docker exec -t 8a32cd6100989b88aacab0a1d35b6d6fbdcbb7f206b3f3c72222dc1bff8ee1b5 /exec failed with exit code 1
• Unit-tests on Linux / tests-gpu (3.11, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t e73bcd37e548c915985b118157fb4462292a8fab1382d96f0978fae3ee942824 /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t ee85d5e0a5f5e52f3e1dfc40b53bdc8481ef31cb431ee7b137de9d990c90788c /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 6c4f3068a6df8ea2d6338270207ded27f0c251b249061d4085bc7d5959bf485d /exec failed with exit code 1
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  RuntimeError: Command docker exec -t cb9e3cb3025c353d6d1f4bfa4f543492508daa443c11e243178cdecbe22a37e1 /exec failed with exit code 1
• Wheels / build-wheel-windows (3.11, 3.11) (gh)
  Process completed with exit code 1.

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Wheels / build-wheel-windows (3.10, 3.10.3) (gh)
• Wheels / build-wheel-windows (3.12, 3.12) (gh)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cpu (gh) (similar failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda11_8 (gh) (similar failure)
  Unable to find any artifacts for the associated workflow
• Wheels / build-wheel-windows (3.8, 3.8) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.9, 3.9) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Unit-tests on Windows / unittests-cpu / windows-job (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.