Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resource punkt_tab not found. #27

Closed
adripo opened this issue Oct 10, 2024 · 5 comments · May be fixed by #28
Closed

Resource punkt_tab not found. #27

adripo opened this issue Oct 10, 2024 · 5 comments · May be fixed by #28

Comments

@adripo
Copy link

adripo commented Oct 10, 2024 •
edited
Loading

Error:


LookupError:
**********************************************************************
  Resource punkt_tab not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt_tab')

  For more information see: https://www.nltk.org/data.html

  Attempted to load tokenizers/punkt_tab/english/

  Searched in:
    - 'C:\\Users\\XXX/nltk_data'
    - 'C:\\Users\\XXX\\.conda\\envs\\local_file_organizer\\nltk_data'
    - 'C:\\Users\\XXX\\.conda\\envs\\local_file_organizer\\share\\nltk_data'
    - 'C:\\Users\\XXX\\.conda\\envs\\local_file_organizer\\lib\\nltk_data'
    - 'C:\\Users\\XXX\\AppData\\Roaming\\nltk_data'
    - 'C:\\nltk_data'
    - 'D:\\nltk_data'
    - 'E:\\nltk_data'

It gives the error just after analyzing a png photo.
@adripo adripo mentioned this issue Oct 10, 2024
Open
@MShirazAhmad
Copy link

Same Issue

@adripo
Copy link
Author

adripo commented Oct 11, 2024

@MShirazAhmad I fixed the issue in pr #28. You can modify the files locally until it is accepted

@QiuYannnn
Copy link
Owner

when I execute the full project, punkt_tab fails wirely, even though it works fine when executed separately. There's a bug in the package for MacOS version. So, I replaced punkt_tab with punkt in the early stage of the project. I will fix the bug when working on the Window System. Appreciate it.

@leonpawelzik
Copy link

Still not working for me, unfortunately. What do I have to change?

@galeka
Copy link

galeka commented Nov 10, 2024 •
edited
Loading

Sort by Date and Type success, but error when short By Content

Traceback (most recent call last):
  File "/Users/xxx/Local-File-Organizer/main.py", line 337, in <module>
    main()
  File "/Users/xxx/Local-File-Organizer/main.py", line 251, in main
    data_images = process_image_files(image_files, image_inference, text_inference, silent=silent_mode, log_file=log_file)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxxLocal-File-Organizer/image_data_processing.py", line 59, in process_image_files
    data = process_single_image(image_path, image_inference, text_inference, silent=silent, log_file=log_file)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Local-File-Organizer/image_data_processing.py", line 36, in process_single_image
    foldername, filename, description = generate_image_metadata(image_path, progress, task_id, image_inference, text_inference)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Local-File-Organizer/image_data_processing.py", line 71, in generate_image_metadata
    description_generator = image_inference._chat(description_prompt, image_path)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/local_file_organizer/lib/python3.12/site-packages/nexa/gguf/nexa_inference_vlm.py", line 332, in _chat
    return self.model.create_chat_completion(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/local_file_organizer/lib/python3.12/site-packages/nexa/gguf/llama/llama.py", line 1912, in create_chat_completion
    return handler(
           ^^^^^^^^
  File "/opt/miniconda3/envs/local_file_organizer/lib/python3.12/site-packages/nexa/gguf/llama/llama_chat_format.py", line 2842, in __call__
    if llama.n_tokens + embed.contents.n_image_pos > llama.n_ctx():
                        ^^^^^^^^^^^^^^
ValueError: NULL pointer access

======================

Same error, what should I do?

 LookupError: 
**********************************************************************
  Resource punkt_tab not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt_tab')
  
  For more information see: https://www.nltk.org/data.html

  Attempted to load tokenizers/punkt_tab/english/

  Searched in:
    - '/Users/xxx/nltk_data'
    - '/opt/miniconda3/envs/local_file_organizer/nltk_data'
    - '/opt/miniconda3/envs/local_file_organizer/share/nltk_data'
    - '/opt/miniconda3/envs/local_file_organizer/lib/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
**********************************************************************

edited, solve by installing manually
python -m nltk.downloader all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: download punkt_tab adripo/Local-File-Organizer
5 participants
@adripo @MShirazAhmad @galeka @leonpawelzik @QiuYannnn