Code

Improve HF device handling#359

Code

Merged

rmitsch merged 10 commits intomainexplosion/spacy-llm:mainfrom fix/hf-device-handlingexplosion/spacy-llm:fix/hf-device-handlingCopy head branch name to clipboardNov 13, 2023

+138-34

Commits

Commits on Nov 9, 2023

• Consider device/device_map conflicts. Always move inputs to model device for models using AutoModelForCausalLM.

  rmitsch

  committed
  f688e6c

Commits on Nov 10, 2023

• Add test for device conflict checks.

  rmitsch

  committed
  139d2b3

• Remove accelerate dependency.

  rmitsch

  committed
  ad95834

• Add torch_dtype handling.

  rmitsch

  committed
  9f3cca3

Commits on Nov 13, 2023

• Fix torch_dtype check.

  rmitsch

  committed
  05cffab

• Fix default dtype.

  rmitsch

  committed
  161d14a

• Empty cache in HF tests.

  rmitsch

  committed
  af64c71

• Fix OpenLLaMa default config bug. Skip Mistral test due to lack of GPU memory.

  rmitsch

  committed
  d6e4c24

• Update spacy_llm/tests/models/test_hf.py

  Show description for 565fbef
  

  rmitsch

  and

  svlandeg

  authored
  565fbef

• Change device check sequence.

  rmitsch

  committed
  17a6402