[V1][VLM] Proper memory profiling for image language models

[ywang96]

This PR adds memory profiling for image language models in V1 so that peak memory usage is correctly measured to avoid potential CUDA OOM.

A few things to note:

1. We still use the dummy multi_modal_data for memory profiling. This not only allows us to reuse the existing code, but is also developer-friendly since defining a dummy PIL image is much more intuitive than defining dummy pixel values. In order to do so, mm_input_mapper is added to model runner only for profiling purpose.

2. This current version only works for image (because of limitation on mm_input_mapper). The memory profiling section has supported profiling for other single non-text modality models, but may require some additional design to be extended to mixed non-text modality profiling.

3. Encoder compute budget and cache size are added to scheduler config, but they remain hardcoded and should be optimized in a later PR.

Verified with VLLM_USE_V1=1 VLLM_ENABLE_V1_MULTIPROCESSING=1 python3 mmmu_bench.py --model OpenGVLab/InternVL2_5-8B --trust-remote-code --gpu-memory-utilization 0.99 from #11196

Main branch:

...
INFO 12-15 10:18:11 uniproc_executor.py:64] # GPU blocks: 31207
...
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 322.00 MiB. GPU 0 has a total capacity of 79.10 GiB of which 249.88 MiB is free. 

This PR:

...
INFO 12-15 10:16:26 uniproc_executor.py:64] # GPU blocks: 30973
...
Request throughput: 8.13 req/s
Total generated tokens: 39100
Token generation rate: 317.89 tok/s

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[DarkLight1337]

LGTM now!

[WoosukKwon]

Thanks for the PR! Left some minor comments.

[ywang96]

@WoosukKwon I've addressed your comments via 1720f68 - Let me know if there's any other concern!

[mgoin]

LGTM thanks!