Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add mllm_mapper #400

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Add mllm_mapper #400

wants to merge 13 commits into from

Conversation

Qirui-jiao
Copy link
Collaborator

Use multimodal large language models (MLLMs) for image-text question answering tasks.
Hyperparameters:

  • max_new_tokens: the maximum number of new tokens generated by the model.
  • sampling_params: sampling hyperparameters for text generation.

@Qirui-jiao Qirui-jiao added enhancement New feature or request dj:multimodal issues/PRs about multimodal data processing dj:op issues/PRs about some specific OPs labels Aug 22, 2024
@Qirui-jiao Qirui-jiao closed this Aug 22, 2024
@Qirui-jiao Qirui-jiao reopened this Aug 22, 2024
drcege
drcege reviewed Aug 23, 2024
View reviewed changes
configs/config_all.yaml Outdated Show resolved Hide resolved
data_juicer/ops/mapper/mllm_mapper.py Outdated Show resolved Hide resolved
data_juicer/ops/mapper/mllm_mapper.py Outdated Show resolved Hide resolved
data_juicer/ops/mapper/mllm_mapper.py Outdated Show resolved Hide resolved
configs/config_all.yaml Show resolved Hide resolved
data_juicer/ops/mapper/mllm_mapper.py Outdated Show resolved Hide resolved
zhijianma
zhijianma approved these changes Aug 27, 2024
View reviewed changes
configs/config_all.yaml Outdated Show resolved Hide resolved
data_juicer/ops/mapper/mllm_mapper.py Show resolved Hide resolved
tests/ops/mapper/test_mllm_mapper.py Outdated Show resolved Hide resolved
data_juicer/ops/mapper/mllm_mapper.py Show resolved Hide resolved
zhijianma
zhijianma approved these changes Aug 27, 2024
View reviewed changes
data_juicer/ops/mapper/mllm_mapper.py Outdated Show resolved Hide resolved
@zhijianma zhijianma self-requested a review August 27, 2024 05:47
drcege
drcege approved these changes Sep 12, 2024
View reviewed changes
Copy link
Collaborator

@drcege drcege left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall acceptable; we can further optimize the details later.

@drcege
Copy link
Collaborator

drcege commented Sep 12, 2024

@Qirui-jiao Thanks for your contribution! Please carefully resolve conflicts and ensure correct OP counting.

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 3, 2024

This PR is marked as stale because there has been no activity for 21 days. Remove stale label or add new comments or this PR will be closed in 3 day.

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 7, 2024

Close this stale PR.

@github-actions github-actions bot closed this Oct 7, 2024
@yxdyc yxdyc reopened this Oct 9, 2024
@github-actions github-actions bot removed the stale-pr label Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@drcege drcege drcege approved these changes

@HYLcool HYLcool Awaiting requested review from HYLcool

@BeachWang BeachWang Awaiting requested review from BeachWang

@yxdyc yxdyc Awaiting requested review from yxdyc

@zhijianma zhijianma Awaiting requested review from zhijianma

Assignees
No one assigned
Labels
dj:multimodal issues/PRs about multimodal data processing dj:op issues/PRs about some specific OPs enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Qirui-jiao @drcege @zhijianma @yxdyc