New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

怎么用npu跑stable diffusion？ #112

Open

lxil520 opened this issue Nov 6, 2023 · 1 comment

lxil520 commented Nov 6, 2023

1.现在的sd底模都是.safetensors格式的权重模型怎么转成量化后的.rknn模型
2.如何将rknn与sd协调做到用npu做到文生图的效果

zifeng-radxa commented Mar 12, 2024

参考思路：（实际没验证过）

diffusers 利用pipeline 加载 .safetensors
分离 text_encoder, unet, vae_encoder, vae_decoder 这四个模型出来
分别将他们trace 成 pt模型或者转换为onnx
将得到的pt 或者 onnx 利用rknn-toolkit2 转化为.rknn
精度，性能分析
编写sd整体代码，涉及推理部分用rknn-toolkit2-lite做python推理，或用C推理

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment