Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

113 Open 439 Closed
113 Open 439 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

GermanDPR Dataset Causes Cross-Encoder Failure Due to Unexpected dict
#1609 opened Dec 17, 2024 by sam-hey
MSMARCO eval split in MTEB English (classic) benchmark
#1608 opened Dec 17, 2024 by aashka-trivedi
Normalize image encoders interface to the text encoder interface
#1606 opened Dec 17, 2024 by KennethEnevoldsen
leaderboard v2 - add autoupdate
#1605 opened Dec 16, 2024 by KennethEnevoldsen
Investigate performance discrepancies in gte-Qwen and NV-embed models
#1600 opened Dec 16, 2024 by isaac-chung
2
Add Support for Index-Based Retrieval for ColBERT enhancement New feature or request
#1593 opened Dec 14, 2024 by sam-hey
Integrate UniBench mieb The image extension of MTEB new-dataset
#1590 opened Dec 14, 2024 by Samoed
Integrate ChemTEB
#1585 opened Dec 13, 2024 by Muennighoff
1
[MIEB] Update VLM2Vec model
#1582 opened Dec 12, 2024 by XMHZZ2018
1
Unable to reproduce the metrics of BAAI/bge-reranker-large on the T2Reranking dataset.
#1581 opened Dec 12, 2024 by HuDi2018
4
We should be able to filter GGUF models enhancement New feature or request leaderboard issues related to the leaderboard
#1579 opened Dec 12, 2024 by KennethEnevoldsen
3
Leaderboard 2.0: Missing results
#1571 opened Dec 9, 2024 by x-tabdeveloping 100+
10
Track contamination in new leaderboard leaderboard issues related to the leaderboard
#1556 opened Dec 5, 2024 by Muennighoff
1 of 4 tasks
7
New Italian tasks
#1553 opened Dec 4, 2024 by MattiaSangermano
3
MIRACLRetrieval results are missing for most models. leaderboard issues related to the leaderboard
#1550 opened Dec 4, 2024 by Samoed
17
Add bge-m3 dense, sparse, multi-vector
#1529 opened Nov 29, 2024 by yjoonjang
2
[MIEB] Add new multimodal retrieval tasks mieb The image extension of MTEB
#1523 opened Nov 28, 2024 by izhx
8
leaderboard 2.0: Performance on subtasks (Bright retrieval) leaderboard issues related to the leaderboard
#1517 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard v2.0: disagreement in scores between v2.0 and v1.0 leaderboard issues related to the leaderboard
#1516 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard 2.0: Add missing models leaderboard issues related to the leaderboard
#1515 opened Nov 27, 2024 by KennethEnevoldsen
15
leaderboard 2.0: Some task <> benchmark associations seem wrong leaderboard issues related to the leaderboard
#1514 opened Nov 27, 2024 by KennethEnevoldsen
Remove nan scores on leaderboard leaderboard issues related to the leaderboard
#1509 opened Nov 27, 2024 by KennethEnevoldsen
Add mixedbread-ai/mxbai-embed-xsmall-v1 and mixedbread-ai/mxbai-embed-2d-large-v1
#1502 opened Nov 26, 2024 by KennethEnevoldsen
Add voyage-3 and voyage-3-lite new-model Questions related to adding a new model to the benchmark
#1501 opened Nov 26, 2024 by KennethEnevoldsen
Investigate/Fix ClimateFever
#1498 opened Nov 25, 2024 by Muennighoff
ProTip! Add no:assignee to see everything that’s not assigned.