Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

143 Open 513 Closed
143 Open 513 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[MIEB] migrate results from tmp repo to results repo
#1823 opened Jan 16, 2025 by isaac-chung
[v2] Remove LANG_MAPPING
#1822 opened Jan 16, 2025 by Samoed
Leaderboard: jasper_en_vision_language_v1 has NaN on MSMARCO
#1816 opened Jan 15, 2025 by x-tabdeveloping
1
Leaderboard: What metric should we sort on? leaderboard issues related to the leaderboard
#1813 opened Jan 15, 2025 by Samoed
11
Leaderboard(Scandinavian) : Missing most results on DKHate bug Something isn't working leaderboard issues related to the leaderboard
#1808 opened Jan 15, 2025 by x-tabdeveloping
@x-tabdeveloping
6
Missing ModelMeta for Chinese models good first issue Good for newcomers help wanted Extra attention is needed leaderboard issues related to the leaderboard
#1803 opened Jan 14, 2025 by x-tabdeveloping
10
[MIEB] Complete all model meta based on v2.0.0 schema
#1800 opened Jan 14, 2025 by isaac-chung
[MIEB] Fill out all task description stats metadata mieb The image extension of MTEB
#1799 opened Jan 14, 2025 by isaac-chung
1
Leaderboard: Missing results for C-MTEB
#1797 opened Jan 14, 2025 by x-tabdeveloping
22
[MIEB] performance of VOC2007 mieb The image extension of MTEB
#1792 opened Jan 13, 2025 by gowitheflow-1998
8
Merge v2.0.0: Overview issue
#1791 opened Jan 13, 2025 by KennethEnevoldsen
3 of 20 tasks
[v2] Refactor to Use Enums Instead of Literals
#1784 opened Jan 13, 2025 by sam-hey
8
Overwrite results when task revision changes enhancement New feature or request
#1782 opened Jan 13, 2025 by gaohongkui
1
Integrate Gene-MTEB new benchmark Issues related to adding a new benchmark
#1781 opened Jan 13, 2025 by Muennighoff
[v2] Updated ModelMeta parameters
#1779 opened Jan 12, 2025 by Samoed
1
Missing HF dataset: HausaNLP/afrisenti-lid-data
#1777 opened Jan 12, 2025 by isaac-chung
4
Update cross-encoders to use model_prompts
#1776 opened Jan 12, 2025 by orionw
[v2] remove save_suffix from AbsTask
#1770 opened Jan 11, 2025 by KennethEnevoldsen
Integrate vdr-2b-multilingual mieb The image extension of MTEB new dataset Issues related to adding a new task or dataset new model Questions related to adding a new model to the benchmark
#1769 opened Jan 11, 2025 by Muennighoff
[v2] convert category from length descriptor to modality in task metadata
#1767 opened Jan 11, 2025 by KennethEnevoldsen
7
[v2] Getting password prompt for task and tests
#1766 opened Jan 11, 2025 by sam-hey
8
Missing results for CQDupstack*Retrieval leaderboard issues related to the leaderboard
#1763 opened Jan 11, 2025 by KennethEnevoldsen
@Muennighoff
Defining zero-shot for MTEB leaderboard issues related to the leaderboard
#1760 opened Jan 11, 2025 by KennethEnevoldsen
2
Leaderboard: Share this benchmark does not work for custom benchmarks enhancement New feature or request leaderboard issues related to the leaderboard
#1755 opened Jan 10, 2025 by Muennighoff
3
Leaderboard: SFR-Embedding results don't match between old and new
#1754 opened Jan 10, 2025 by Muennighoff
4
ProTip! Type g i on any issue or pull request to go back to the issue listing page.