An error occurred: module 'mediapipe.python._framework_bindings.model_ckpt_util' has no attribute 'GenerateGpuTfLite'

[akau16]

I want to convert phi-2 model to MediaPipe format model by above link:
https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference#convert-model

But when I run it I encount the exception's error "An error occurred: module 'mediapipe.python._framework_bindings.model_ckpt_util' has no attribute 'GenerateGpuTfLite'", above is all my code:

import mediapipe as mp
from mediapipe.tasks.python.genai import converter

INPUT_CKPT = './content/phi-2/model*.safetensors'
CKPT_FORMAT = 'safetensors'
MODEL_TYPE = 'PHI_2'
BACKEND = 'cpu'
OUTPUT_DIR = './content/intermediate/phi-2/'
VOCAB_MODEL_FILE = './content/phi-2/tokenizer*.json'
OUTPUT_TFLITE_FILE = './content/converted_models/phi2_gpu.bin'

try:
config = converter.ConversionConfig(
input_ckpt=INPUT_CKPT,
ckpt_format=CKPT_FORMAT,
model_type=MODEL_TYPE,
backend=BACKEND,
output_dir=OUTPUT_DIR,
combine_file_only=False,
vocab_model_file=VOCAB_MODEL_FILE,
output_tflite_file=OUTPUT_TFLITE_FILE,
)

print(config)

converter.convert_checkpoint(config)

print(f"Model converted and saved to phi2_gpu.bin")
except Exception as e:
print(f"An error occurred: {e}")

Anybody can help me?

python: 3.9.19
mediapipe: 0.10.14
torch: 2.4.0+cu118
pip: 24.2

[kuaashish]

Hi @akau16,

Could you please follow this Colab example to convert the model to TFLite and let us know if you still encounter the same error?

Thank you!!

[akau16]

Hi @kuaashish:

Did you mean to 'use' Colab example to cover the model? I use it and it is ok to conver the model, but I hope to conver it in my local code then I meet this problem.

Thank you!

[kuaashish]

Hi @akau16,

Glad to hear that. Did you use the same example code to convert the model locally? If you are encountering an error with the same code, could you please let us know the OS you are using?

Thank you!!

[akau16]

Hi @kuaashish:
I download Colab example and run it on my device, I meet same problem "mediapipe.python._framework_bindings.model_ckpt_util' has no attribute 'GenerateGpuTfLite"

My Device OS is Win11 22H2
python: 3.9.19

If I run it in other Python Env.(like 3.10.14) it will show another error message "OSError: [WinError 126] 找不到指定的模組。 Error loading "C:\Users\0807061\AppData\Local\miniconda3\envs\colab_mediapipe\lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies."

but I check the file 'fbgemm.dll' is in that path.

Thank you!

[kuaashish]

Hi @schmidt-sebastian,

We believe this issue is a genuine bug specific to our OS environment. We have tested our Colab example for converting models to TFLite and found it works correctly on Colab, regular Linux, and macOS. However, we are encountering issues on Windows. Unfortunately, we do not currently have a suitable environment to reproduce this problem. Could you please investigate this issue, as well? There is a similar report on GitHub issue number #5481.

Thank you!!