Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the exported SAM2 encoder and decoder only support inference images? #17

Open
Aimol-l opened this issue Aug 2, 2024 · 4 comments

Comments

@Aimol-l
Copy link

Aimol-l commented Aug 2, 2024

I plan to conduct video inference, but it seems that having only encoder and decoder is not enough.
Perhaps it includes:Image encoder,Memory attention. Prompt encoder and mask decoder,Memory encoder,Memory bank.

屏幕截图_20240802_152457

I'm a bit confused.😕

@vietanhdev
Copy link
Owner

Hi @Aimol-l ,
There is no inference code at the moment. Please use the official repo from Meta.
Thank you!

@MELANCHOLY828
Copy link

我打算做视频推理,但是好像只有encoder和decoder还不够, 可能还要包括:Imagecoder,Memory Attention,Prompt Encoder和Mask Decoder,Memory Encoder,Memory Bank。

在我们_20240802_152457

我有点困惑。😕

I also want to use it for video inference. Does the exported SAM2 encoder and decoder support it?

@Aimol-l
Copy link
Author

Aimol-l commented Aug 16, 2024

我打算做视频推理,但是好像只有encoder和decoder还不够, 可能还要包括:Imagecoder,Memory Attention,Prompt Encoder和Mask Decoder,Memory Encoder,Memory Bank。
在我们_20240802_152457
我有点困惑。😕

I also want to use it for video inference. Does the exported SAM2 encoder and decoder support it?

I am try to export Memory_attention module,but failed. this is my code :https://github.com/Aimol-l/SAM2Export.git
so,I'm stuck。
this is my note graph.
屏幕截图_20240816_141328

@MELANCHOLY828
Copy link

我打算做视频推理,但是好像只有encoder和decoder还不够, 可能还要包括:Imagecoder,Memory Attention,Prompt Encoder和Mask Decoder,Memory Encoder,Memory Bank。
在我们_20240802_152457
我有点困惑。😕

I also want to use it for video inference. Does the exported SAM2 encoder and decoder support it?

I am try to export Memory_attention module,but failed. this is my code :https://github.com/Aimol-l/SAM2Export.git so,I'm stuck。 this is my note graph. 屏幕截图_20240816_141328

你好,请问图片预测的运行成功了吗,我报错
Traceback (most recent call last):
File "", line 5, in
File "e:\anaconda\envs\sam2\lib\site-packages\imread_from_url\imread_from_url.py", line 52, in imread_from_url
image = cv2.cvtColor(np.array(image, dtype=np.uint8), cv2.COLOR_RGB2BGR)
cv2.error: OpenCV(4.10.0) 👎 error: (-5:Bad argument) in function 'cvtColor'

Overload resolution failed:

  • src is not a numpy array, neither a scalar
  • Expected Ptrcv::UMat for argument 'src'
    不知道是不是版本问题

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants