Dependency conflicts

[penfever]

Suggesting a few small changes (these were necessary to install in my environment)

[sedrick-keh-tri]

Thanks for raising this! I'm curious what is the error message you're seeing.

For reference, I'm able to run evalchemy on a fresh install with a Dockerfile like this (simplified for clarity).

ENV PATH="/opt/ml/code:${PATH}"
ENV PYTHONPATH="/opt/ml/code/eval:${PYTHONPATH}"

RUN git clone https://github.com/mlfoundations/evalchemy.git /opt/ml/code/evalchemy/
RUN cp -r /opt/ml/code/evalchemy/* /opt/ml/code/
WORKDIR /opt/ml/code
RUN pip install -e ".[eval]"
RUN pip install -e eval/chat_benchmarks/alpaca_eval
RUN python -m nltk.downloader punkt
RUN python -m nltk.downloader punkt_tab

[penfever]

Hi there! As a point of reference, my cluster uses Singularity containers. The error message for MT-Bench was ``` ERROR: Exception: Traceback (most recent call last): File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper status = run_func(*args) ^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper return func(self, options, args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 377, in run requirement_set = resolver.resolve( ^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve result = self._result = resolver.resolve( ^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve failure_causes = self._attempt_to_pin_criterion(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion criteria = self._get_updated_criteria(candidate) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 229, in _get_updated_criteria for requirement in self._p.get_dependencies(candidate=candidate): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 244, in get_dependencies return [r for r in candidate.iter_dependencies(with_requires) if r is not None] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 239, in iter_dependencies for r in requires: File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py", line 221, in iter_dependencies req = Requirement(req_string) ^^^^^^^^^^^^^^^^^^^^^^^ File "/ext3/miniconda3/lib/python3.12/site-packages/pip/_vendor/packaging/requirements.py", line 113, in __init__ raise InvalidRequirement("Invalid URL given") pip._vendor.packaging.requirements.InvalidRequirement: Invalid URL given ``` And the error message for faiss-cpu was a `failed to build wheel` error for that particular version.

…

On Fri, Dec 20, 2024 at 9:52 AM sedrick-keh-tri ***@***.***> wrote: Thanks for raising this! I'm curious what is the error message you're seeing. For reference, I'm able to run evalchemy on a fresh install with a Dockerfile like this (simplified for clarity). ENV PATH="/opt/ml/code:${PATH}"ENV PYTHONPATH="/opt/ml/code/eval:${PYTHONPATH}" RUN git clone https://github.com/mlfoundations/evalchemy.git <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mlfoundations_evalchemy.git&d=DwQFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=cRVZDAsVf8YMy7ncQBvRsw&m=4T2V8lVpftpDQ935y0eXAvqU7H7cm2vD7d3rQTGgyWQOLytfbPFJlbArEV1d11dU&s=xZEy-xNwJJ-FhX7tWw7tcu26CLpH4Cg9JUlMGa8MKss&e=> /opt/ml/code/evalchemy/RUN cp -r /opt/ml/code/evalchemy/* /opt/ml/code/WORKDIR /opt/ml/codeRUN pip install -e ".[eval]"RUN pip install -e eval/chat_benchmarks/alpaca_evalRUN python -m nltk.downloader punktRUN python -m nltk.downloader punkt_tab — Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mlfoundations_evalchemy_pull_44-23issuecomment-2D2557159980&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=cRVZDAsVf8YMy7ncQBvRsw&m=4T2V8lVpftpDQ935y0eXAvqU7H7cm2vD7d3rQTGgyWQOLytfbPFJlbArEV1d11dU&s=noHh2B9Qo3nDYk9lDl36oXr7f0D9c8xgjBRFe-jaoCY&e=>, or unsubscribe <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AL4G7QFWWBVTDQLUPNGLSG32GQVL7AVCNFSM6AAAAABT7HLSZGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJXGE2TSOJYGA&d=DwMFaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=cRVZDAsVf8YMy7ncQBvRsw&m=4T2V8lVpftpDQ935y0eXAvqU7H7cm2vD7d3rQTGgyWQOLytfbPFJlbArEV1d11dU&s=xOjyiAkaPVn1xgs_B5TM__1_WbLxvpO50Zv8f8d1k68&e=> . You are receiving this because you authored the thread.Message ID: ***@***.***>