Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add HumanEvalPack, QuixBugs, Python Bugs, Parity #120

Merged
merged 243 commits into from
Aug 15, 2023
Merged

Add HumanEvalPack, QuixBugs, Python Bugs, Parity #120

merged 243 commits into from
Aug 15, 2023

Conversation

Muennighoff
Copy link
Contributor

@Muennighoff Muennighoff commented Jul 26, 2023 •
edited
Loading

Old PR: #47

@Muennighoff Muennighoff changed the title [WIP] Add HumanEvalPack, QuixBugs, Python Bugs, Parity Jul 29, 2023
Muennighoff
Muennighoff commented Jul 29, 2023
View reviewed changes
README.md Show resolved Hide resolved
@Muennighoff Muennighoff mentioned this pull request Jul 26, 2023
Closed
loubnabnl
loubnabnl reviewed Aug 7, 2023
View reviewed changes
Copy link
Collaborator

@loubnabnl loubnabnl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these benchamrks and for all the new features 🚀 It looks good

README.md Show resolved Hide resolved
lm_eval/evaluator.py Outdated Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
main.py Show resolved Hide resolved
lm_eval/utils.py Show resolved Hide resolved
lm_eval/generation.py Outdated Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
lm_eval/utils.py Outdated Show resolved Hide resolved
docs/README.md Show resolved Hide resolved
loubnabnl
loubnabnl approved these changes Aug 7, 2023
View reviewed changes
main.py Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
main.py Outdated Show resolved Hide resolved
@loubnabnl
Copy link
Collaborator

Looks good! feel free to merge

@Muennighoff Muennighoff merged commit d5a2c40 into main Aug 15, 2023
1 check passed
phuonglvh pushed a commit to phuonglvh/bigcode-evaluation-harness that referenced this pull request Nov 15, 2024
@Muennighoff
Merge pull request bigcode-project#120 from bigcode-project/octopack
badac48 
Add HumanEvalPack, QuixBugs, Python Bugs, Parity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@loubnabnl loubnabnl loubnabnl approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Muennighoff @loubnabnl @iCSawyer @SivilTaram @mitya52 @terryyz