Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sandbox bench experiment workflow #364

Merged
merged 76 commits into from
Aug 2, 2024
Merged

sandbox bench experiment workflow #364

merged 76 commits into from
Aug 2, 2024

Conversation

BeachWang
Copy link
Collaborator

  1. Add workflow demo for sandbox bench experiments.
  2. Update corresponding docs.

BeachWang and others added 30 commits March 21, 2024 18:02
@BeachWang
FVD and ISV for video eval
28f43d5
@BeachWang
Merge branch 'main' of github.com:alibaba/data-juicer into dev/fvd_eval
dbc1e0a
@BeachWang
restore tools init
40e69e7
@BeachWang
restore tools init
8744767
@BeachWang
pre-commit done
6ae9b1b
@BeachWang
add FID KID IS PR and PRV metrics
173f0ce
@BeachWang
add KVD metric
809775f
@BeachWang
fix doc
381cf1c
@BeachWang
allow relative path
535748c
@BeachWang
fix sample 50000 image
f2cbadb
Merge branch 'main' of github.com:alibaba/data-juicer into dev/fvd_eval
4915deb
@BeachWang
fvd sandbox
4ec9efe
@BeachWang
fvd sandbox test done
84a5290
@BeachWang
precommit done
c11e045
@BeachWang
easyanimate train and infer in sandbox
9a36d7d
@BeachWang
merge main
68d83cb
@BeachWang
divide dataset pipline
280907e
@BeachWang
merge fix_sandbox_pipline
95ef2b7
@BeachWang
fix data num for each partition
0a897c1
@BeachWang
Merge branch 'dev/fix_sandbox_pipline' of github.com:alibaba/data-jui…
87051ce 
…cer into dev/easyanimate_for_sandbox
@BeachWang
pre-commit done
651e946
@BeachWang
Merge branch 'dev/fix_sandbox_pipline' of github.com:alibaba/data-jui…
f6364d4 
…cer into dev/easyanimate_for_sandbox
@BeachWang
test sandbox for videos done
a7e8f79
@BeachWang
fix executor
bfe8091
@BeachWang
fix executor
d41a01a
@BeachWang
check datalen
fd4bea0
@BeachWang
sort data for partition
e4d3ecd
@BeachWang
sort data for partition
c935adc
@BeachWang
fix video_aspect_ratio_filter
846af68
@BeachWang
Merge branch 'dev/fix_video_filter_for_bench' into dev/easyanimate_fo…
a4af5dd 
…r_sandbox
@BeachWang BeachWang added documentation Improvements or additions to documentation enhancement New feature or request dj:multimodal issues/PRs about multimodal data processing labels Jul 24, 2024
@BeachWang BeachWang added this to the DJ-SORA milestone Jul 24, 2024
@BeachWang BeachWang self-assigned this Jul 24, 2024
yxdyc
yxdyc reviewed Jul 26, 2024
View reviewed changes
Copy link
Collaborator

@yxdyc yxdyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specific contents:

  • integrate EasyAnimate
  • integrate V-Bench
  • add two ops for select op
  • add data recipe for Data-Juicer-T2V (Vbench top1 model)
  • enhance & refactor the sandbox according to our paper

yxdyc
yxdyc reviewed Jul 26, 2024
View reviewed changes
Copy link
Collaborator

@yxdyc yxdyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the third-party codes (Easy-Animate), we may use a git submodule to make the dj repo lighter?

@yxdyc yxdyc merged commit 4d8c521 into main Aug 2, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yxdyc yxdyc yxdyc approved these changes

@drcege drcege Awaiting requested review from drcege

@HYLcool HYLcool Awaiting requested review from HYLcool

Assignees

@BeachWang BeachWang

Labels
dj:multimodal issues/PRs about multimodal data processing documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Milestone
DJ-SORA
Development

Successfully merging this pull request may close these issues.
2 participants
@BeachWang @yxdyc