Error: unable to load weights from PyTorch

[elissasoroj]

Apologies if you have already encountered this issue, but I wasn't able to find it in the Discussions.

I successfully installed khoj_assistant once on my Mac, and I believe it was working properly. I then uninstalled it and re-installed it later. Upon re-install, I keep getting the following error message. I did try making a separate environment for khoj and installing everything from scratch, but the same issue occurs when I run the comment khoj --no-gui:

Traceback (most recent call last):
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict
    return torch.load(checkpoint_file, map_location="cpu")
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/torch/serialization.py", line 777, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/torch/serialization.py", line 282, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 419, in load_state_dict
    if f.read(7) == "version":
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/elissa/miniconda3/envs/khoj_env/bin/khoj", line 8, in <module>
    sys.exit(run())
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/main.py", line 70, in run
    configure_server(args, required=False)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/configure.py", line 49, in configure_server
    state.model = configure_search(state.model, state.config, args.regenerate)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/configure.py", line 114, in configure_search
    model.markdown_search = text_search.setup(
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/search_type/text_search.py", line 173, in setup
    bi_encoder, cross_encoder, top_k = initialize_model(search_config)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/search_type/text_search.py", line 40, in initialize_model
    bi_encoder = load_model(
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/utils/helpers.py", line 68, in load_model
    model = model_type_class(get_absolute_path(model_path), device=device)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 97, in __init__
    modules = self._load_auto_model(model_path)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 806, in _load_auto_model
    transformer_model = Transformer(model_name_or_path)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 29, in __init__
    self._load_model(model_name_or_path, config, cache_dir)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 49, in _load_model
    self.auto_model = AutoModel.from_pretrained(model_name_or_path, config=config, cache_dir=cache_dir)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 471, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2429, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 431, in load_state_dict
    raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/Users/elissa/.khoj/search/asymmetric/sentence-transformers_multi-qa-MiniLM-L6-cos-v1/pytorch_model.bin' at '/Users/elissa/.khoj/search/asymmetric/sentence-transformers_multi-qa-MiniLM-L6-cos-v1/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

Thanks for any guidance!
~Elissa

[debanjum]

@elissasoroj, it's unfortunate that you can't set it up a second time after getting it to work previously.

I'll need some more details to investigate. Can you share the following:

1. Which Khoj version are you on? (you can get that by running khoj --version)
2. How the search-type section in your ~/.khoj/khoj.yml file looks?
3. The name of files under ~/.khoj/search/asymmetric directory (i.e ls -l ~/.khoj/search/asymmetric)
4. Did you use pip or conda to install khoj? Or are you just installing khoj via pip but in a (mini)conda environment?
5. The more verbose output from khoj failing using khoj --no-gui --verbose

It's strange that khoj can't find the multi_qa sentence transformer model (or is seeing corrupted forms of it on disk).
Were you on a stable internet when you were trying to setup khoj?

I did try making a separate environment for khoj and installing everything from scratch

Did you delete your ~/.khoj folder, before re-installing khoj? That'd be a good idea to setup khoj from scratch. See khoj uninstall doc for details

[elissasoroj]

So I went ahead and deleted the ~/.khoj folder and tried un-installing and re-installing khoj inside my (mini)conda environment using pip - and it worked! Thank you so much for the prompt reply, and I am sorry it was such a small thing.

~Elissa

In case it's helpful for later, here are the extra details you asked for:

1. Khoj version is 0.5.0
2. search-type of the ~/.khoj/khoj.yml file:

search-type:
  asymmetric:
    cross-encoder: cross-encoder/ms-marco-MiniLM-L-6-v2
    encoder: sentence-transformers/multi-qa-MiniLM-L6-cos-v1
    encoder-type: null
    model-directory: ~/.khoj/search/asymmetric
  image:
    encoder: sentence-transformers/clip-ViT-B-32
    encoder-type: null
    model-directory: ~/.khoj/search/image
  symmetric:
    cross-encoder: cross-encoder/ms-marco-MiniLM-L-6-v2
    encoder: sentence-transformers/all-MiniLM-L6-v2
    encoder-type: null
    model-directory: ~/.khoj/search/symmetric

3. There is only another directory in ~/.khoj/search/asymmetric, which has the following files in it:

4. Ah, great question. I used pip to install khoj, but inside a (mini)conda environment.
5. Unfortunately the verbose output seems to be exactly the same:

Traceback (most recent call last):
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict
    return torch.load(checkpoint_file, map_location="cpu")
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/torch/serialization.py", line 777, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/torch/serialization.py", line 282, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 419, in load_state_dict
    if f.read(7) == "version":
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/elissa/miniconda3/envs/khoj_env/bin/khoj", line 8, in <module>
    sys.exit(run())
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/main.py", line 70, in run
    configure_server(args, required=False)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/configure.py", line 49, in configure_server
    state.model = configure_search(state.model, state.config, args.regenerate)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/configure.py", line 114, in configure_search
    model.markdown_search = text_search.setup(
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/search_type/text_search.py", line 173, in setup
    bi_encoder, cross_encoder, top_k = initialize_model(search_config)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/search_type/text_search.py", line 40, in initialize_model
    bi_encoder = load_model(
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/khoj/utils/helpers.py", line 68, in load_model
    model = model_type_class(get_absolute_path(model_path), device=device)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 97, in __init__
    modules = self._load_auto_model(model_path)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 806, in _load_auto_model
    transformer_model = Transformer(model_name_or_path)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 29, in __init__
    self._load_model(model_name_or_path, config, cache_dir)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 49, in _load_model
    self.auto_model = AutoModel.from_pretrained(model_name_or_path, config=config, cache_dir=cache_dir)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 471, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2429, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
  File "/Users/elissa/miniconda3/envs/khoj_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 431, in load_state_dict
    raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/Users/elissa/.khoj/search/asymmetric/sentence-transformers_multi-qa-MiniLM-L6-cos-v1/pytorch_model.bin' at '/Users/elissa/.khoj/search/asymmetric/sentence-transformers_multi-qa-MiniLM-L6-cos-v1/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

[debanjum]

So I went ahead and deleted the ~/.khoj folder and tried un-installing and re-installing khoj inside my (mini)conda environment using pip - and it worked! Thank you so much for the prompt reply, and I am sorry it was such a small thing.

Great to know that fixed it for you! Seems like the model got corrupted during download. And it's good that your shared this as other folks may hit similar issues and be wondering how to resolve it.

Also thanks for still sharing the details! The contents of ls -al for ~/.khoj/search/asymmetric/ and ~/.khoj/search/asymmetric/sentence-transformers_multi-qa-MiniLM-L6-cos-v1/ seem to indicate that there was an issue in downloading the asymmetric search models as well (as the cross-encoder model isn't seen in the asymmetric directory and the multi-qa model doesn't have all the required files in it)