New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Finetuning Projection Layer or Not #139

Open

ziqipang opened this issue Jul 24, 2024 · 0 comments

ziqipang commented Jul 24, 2024

Hi Authors,

Thank you for your great paper! We are working on some follow-ups but are confused by an inconsistency between the paper and your code:

In the paper, you mentioned that the projection layer is trainable.
But the code indicates that the projection layer is frozen (

LISA/train_ds.py

Line 169 in dbe026a

for p in model.get_model().mm_projector.parameters():

).

Please let us know if I have made mistakes in my understanding or you have some new considerations for implementations. Thank you so much!

Best,

Ziqi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment