lm-evaluation-harness dependency conflicts

[penfever]

After pip install using the current TOML, and running the following command --

python -m eval.eval \
            --model hf \
            --tasks "leaderboard" \
            --model_args \"pretrained=meta-llama/Llama-3.1-8B-Instruct,dtype=bfloat16\" \
            --batch_size auto \
            --output_path logs

The run fails (after successfully completing all evaluation datapoints) because of a dependency conflict in the LaTeX parser.

2024-12-21:00:47:07,159 ERROR    [utils.py:180] LaTeX parsing requires the antlr4 Python package, provided by pip (antlr4-python3-runtime) or conda (antlr-python-runtime), version 4.11
2024-12-21:00:47:07,160 ERROR    [eval.py:213] Error in pretrain evaluation: LaTeX parsing requires the antlr4 Python package, provided by pip (antlr4-python3-runtime) or conda (antlr-python-runtime), version 4.11

Upgrading 'antlr4-python3-runtime' introduces a new dependency conflict --

omegaconf 2.2.3 requires antlr4-python3-runtime==4.9.*, but you have antlr4-python3-runtime 4.11.0 which is incompatible.

Upon upgrading omegaconf, another new dependency conflict appears --

sagemaker 2.237.1 requires omegaconf<2.3,>=2.2, but you have omegaconf 2.3.0 which is incompatible.

SUGGESTION: make Cloud & Storage dependencies optional.