Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

example: add patch dropout to open_clip #11

Open
wants to merge 1 commit into
base: example_open_clip
Choose a base branch
from

Conversation

iejMac
Copy link
Owner

@iejMac iejMac commented Dec 4, 2022

@iejMac
Copy link
Owner Author

iejMac commented Dec 4, 2022

openai can you summarize this PR?

@github-actions
Copy link

github-actions bot commented Dec 4, 2022

This pull request adds a patch dropout function to the CLIP Vision model, which is a transformer model used for vision-language tasks. The patch dropout function helps improve model performance by randomly dropping visual tokens prior to the transformer layer. Additionally, the init_values parameter is set to 1e-5, and a LS init value of None is added to the CLIPVisionCfg class. Possible improvements could include adding more efficient pooling layers such as AdaptiveAvgPool2d, and adding a new parameter to the patch dropout function to increase the amount of tokens dropped.

@iejMac iejMac changed the title pr example: add patch dropout to open_clip Dec 4, 2022
@iejMac iejMac added the example label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant