Error: cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'

[ArijitSinghEDA]

Facing the mentioned error when trying the following:

from llama_index.embeddings.huggingface import HuggingFaceEmbedding

embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

This is the complete traceback:

Traceback (most recent call last):
  File "/code/app/llama_hf.py", line 105, in index_data
    embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
  File "/usr/local/lib/python3.10/site-packages/llama_index/embeddings/huggingface/base.py", line 82, in __init__
    model = AutoModel.from_pretrained(
  File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 565, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 387, in _get_model_class
    supported_models = model_mapping[type(config)]
  File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 740, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 754, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
  File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 698, in getattribute_from_module
    if hasattr(module, attr):
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1354, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1366, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'

Versions used

llama-index==0.10.22
transformers==4.37.0
huggingface-hub==0.20.3

[dosubot[bot]]

The error "cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub'" is likely due to a version mismatch between the transformers and huggingface-hub libraries. The function split_torch_state_dict_into_shards was introduced in huggingface-hub version 0.11.0.

To resolve this issue, ensure that you are using compatible versions of these libraries. Based on your provided versions, huggingface-hub==0.17.2 should be compatible with transformers==4.37.0. However, you might want to update huggingface-hub to a newer version to ensure compatibility.

Update your huggingface-hub to a newer version, such as 0.24.0, in your pyproject.toml or directly via pip:

pip install --upgrade huggingface-hub==0.24.0

After updating, your pyproject.toml should look like this:

[tool.poetry.dependencies]
python = ">=3.8.1,<4.0"
llama-index = "0.10.22"
huggingface-hub = "^0.24.0"
transformers = "^4.37.0"

Then, run poetry update to apply the changes if you are using Poetry. This should resolve the import error ^[1].

To continue talking to Dosu, mention @dosu.

[ArijitSinghEDA]

@dosu I apologize for the misinformation earlier. The huggingface-hub version is 0.20.3. Should this error still occur?

[dosubot[bot]]

The error you're encountering is due to the split_torch_state_dict_into_shards function not being available in huggingface-hub version 0.20.3. This function is included starting from version 0.23.0.

To resolve this issue, update the huggingface-hub library to version 0.23.0 or later:

pip install --upgrade huggingface-hub

This should include the required function and resolve the import error ^[1].

To continue talking to Dosu, mention @dosu.

[heyomy]

@dosu I have the upgraded huggingface-hub and still getting the same error.
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
cannot import name 'split_torch_state_dict_into_shards' from 'huggingface_hub' (../../anaconda3/lib/python3.10/site-packages/huggingface_hub/init.py)

[mausch]

In my case I had to pin "accelerate" to 0.31.0
accelerate 0.32.1 triggers this error.

[JiafengDu]

how do I pin "accelerate" using miniconda?

[IIIskiplikIII]

@JiafengDu in this case you need to install the specific version for the package like:

pip install accelerate==0.31.0

That worked for me, so thanks to @mausch