Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在Paddle 3.0 下无法加载导出的Pir模型进行推理 #3902

Open
Liyulingyue opened this issue Nov 22, 2024 · 1 comment
Open

在Paddle 3.0 下无法加载导出的Pir模型进行推理 #3902

Liyulingyue opened this issue Nov 22, 2024 · 1 comment

Comments

@Liyulingyue
Copy link
Contributor

复现流程:

  1. git clone https://github.com/PaddlePaddle/PaddleSpeech.git
  2. 修改 PaddleSpeech/paddlespeech/s2t/exps/deepspeech2/model.py 函数 setup_model(self)(在末尾部分)为
    def setup_model(self):
        super().setup_model()

        # 如果存在新IR,以新IR格式加载
        if os.path.exists(self.args.export_path + ".json"):
          deepspeech_config = inference.Config(
              self.args.export_path + ".json",
              self.args.export_path + ".pdiparams")
        else:
          deepspeech_config = inference.Config(
              self.args.export_path + ".pdmodel",
              self.args.export_path + ".pdiparams")

        if (os.environ['CUDA_VISIBLE_DEVICES'].strip() != ''):
            deepspeech_config.enable_use_gpu(100, 0)
            deepspeech_config.enable_memory_optim()
        deepspeech_predictor = inference.create_predictor(deepspeech_config)
        self.predictor = deepspeech_predictor
  1. 以源码形式安装paddlespeech 和 paddlepaddle
  2. cd PaddleSpeech/examples/aishell/asr0
  3. 运行如下命令(默认的数据集处理可能比较耗时,另行获取链接下载数据放在指定路径更快一点)
source path.sh
source ${MAIN_ROOT}/utils/parse_options.sh
bash ./local/data.sh
wget https://paddlespeech.bj.bcebos.com/s2t/aishell/asr0/asr0_deepspeech2_offline_aishell_ckpt_1.0.1.model.tar.gz
tar xzvf asr0_deepspeech2_offline_aishell_ckpt_1.0.1.model.tar.gz
source path.sh
./local/export.sh conf/deepspeech2.yaml exp/deepspeech2/checkpoints/avg_10 exp/deepspeech2/newir/avg_10.jit
CUDA_VISIBLE_DEVICES=0 ./local/test_export.sh conf/deepspeech2.yaml conf/tuning/decode.yaml exp/deepspeech2/newir/avg_10.jit

运行环境为Aistudio框架开发环境,Paddle 3.0+Python 3.8+CUDA 12.0

报错如下:
974FC61F3A4334C63DD230FD7DCDF990

@zxcd
Copy link
Collaborator

zxcd commented Nov 25, 2024

GLOG_v=5 FLAGS_call_stack_level=5 FLAGS_enable_pir_api=1 日志如下:
log_glog5_stack5_part1.txt
log_glog5_stack5_part2.txt

目前将self.predictor.clear_intermediate_tensor()删除后能够成功跑通。

由于该函数的用法为释放中间 Tensor,虽然 GPU/CPU 都有对应的显存复用机制,逻辑上不需要手动进行释放,但后续仍需关注该解法是否会引起显存泄露。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants