Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce stop_at_shortest in sample and round_robin #76

Merged
merged 35 commits into from
Oct 5, 2023
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
c137b46
Add tensor.h to utils
najielhachem Sep 7, 2023
204ffa7
Add sample_data_source class
najielhachem Sep 7, 2023
f544f9e
Add sample method to data_pipeline
najielhachem Sep 7, 2023
f99e870
Add sample binding to data_pipeline
najielhachem Sep 7, 2023
07a9d65
Add test_sample
najielhachem Sep 7, 2023
894dbc1
Fix build errors
najielhachem Sep 7, 2023
8ce33db
fix python lint errors
najielhachem Sep 7, 2023
134c585
Add empty lines at end of c++ files
najielhachem Sep 8, 2023
08a3a65
remove \n from runtime error regex match
najielhachem Sep 8, 2023
8869aef
Merge remote-tracking branch 'origin/main' into feature_sampling_data…
najielhachem Sep 28, 2023
a75141d
Use float32 instead of float
najielhachem Sep 28, 2023
8b06f85
Resolve nit comments
najielhachem Sep 28, 2023
a10b5b4
Remove set_seed from sampler
najielhachem Sep 28, 2023
0773e89
Fix document error
najielhachem Sep 28, 2023
1662c11
Handle generator state out of sample operator
najielhachem Sep 29, 2023
9805525
Add circular data source
najielhachem Sep 28, 2023
bd642cd
Merge branch 'main' into up_sampling
najielhachem Oct 2, 2023
4d774dd
Use circular data_soruce in round_robin
najielhachem Oct 2, 2023
73e348f
fix circular data source
najielhachem Oct 2, 2023
a24baa0
Add stop_on_shortest flag
najielhachem Oct 2, 2023
97b6aca
Add stop_at_shortest flag to round_robin
najielhachem Oct 2, 2023
563ba05
Add stop_at_shortest round_robin test
najielhachem Oct 3, 2023
b0d8e54
Fix circular data source stop_at_shortest flag
najielhachem Oct 3, 2023
6abfb53
Use circular data_source in sample op
najielhachem Oct 3, 2023
48f6cb5
Add up_sampling test
najielhachem Oct 3, 2023
7164e1f
Rename circular_data_source to multi_data_source
najielhachem Oct 3, 2023
cdc35f0
Improve sample and round_robin doc
najielhachem Oct 3, 2023
394b9bb
lint code
najielhachem Oct 3, 2023
74202b8
Merge remote-tracking branch 'origin/main' into up_sampling
najielhachem Oct 3, 2023
006bf59
skip test that uses new API
najielhachem Oct 3, 2023
218f69b
Merge remote-tracking branch 'origin/main' into up_sampling
najielhachem Oct 5, 2023
996801e
nit changes
najielhachem Oct 5, 2023
c535d02
rename multi_data_source to composite_data_source
najielhachem Oct 5, 2023
a6e5a58
use std::exchange instead of =
najielhachem Oct 5, 2023
e2dcab6
resolve last nit comment
najielhachem Oct 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add empty lines at end of c++ files
najielhachem committed Sep 8, 2023
commit 134c5857c829ad1aa44faef82091ac664a15adba
2 changes: 1 addition & 1 deletion fairseq2n/src/fairseq2n/data/sample_data_source.cc
Original file line number Diff line number Diff line change
@@ -80,4 +80,4 @@ sample_data_source::next_index()
return static_cast<std::size_t>(result);
}

} // namespace fairseq2::detail
} // namespace fairseq2::detail
2 changes: 1 addition & 1 deletion fairseq2n/src/fairseq2n/data/sample_data_source.h
Original file line number Diff line number Diff line change
@@ -48,4 +48,4 @@ class sample_data_source final : public data_source {
at::Tensor weights_;
};

} // namespace fairseq2::detail
} // namespace fairseq2::detail
2 changes: 1 addition & 1 deletion fairseq2n/src/fairseq2n/utils/tensor.h
Original file line number Diff line number Diff line change
@@ -28,4 +28,4 @@ make_tensor_from_vector(
);
}

} // namespace fairseq2::detail
} // namespace fairseq2::detail