Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Gujarati news classification #493

Conversation

SaitejaUtpala
Copy link
Contributor

@SaitejaUtpala SaitejaUtpala commented Apr 22, 2024

Checklist for adding MMTEB dataset

Reason for dataset addition:

  • I have tested that the dataset runs with the mteb package.
  • I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.
    • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
    • intfloat/multilingual-e5-small
  • I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
  • I have considered the size of the dataset and reduced it if it is too big (2048 examples is typically large enough for most tasks)
  • I have filled out the metadata object in the dataset file (find documentation on it here).
  • Run tests locally to make sure nothing is broken using make test.
  • Run the formatter to format the code using make lint.
  • I have added points for my submission to the points folder using the PR number as the filename (e.g. 438.jsonl).

Copy link
Collaborator

@isaac-chung isaac-chung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!
Seems like Gujarati is new to Classification, so feel free to add 2 + 4 = 6 points for you and 2 points to me for the review.

@SaitejaUtpala
Copy link
Contributor Author

SaitejaUtpala commented Apr 23, 2024

Looks great! Seems like Gujarati is new to Classification, so feel free to add 2 + 4 = 6 points for you and 2 points to me for the review.

@isaac-chung Thanks for the review. Done!

@isaac-chung isaac-chung enabled auto-merge (squash) April 23, 2024 05:53
@isaac-chung isaac-chung merged commit 3a59a4d into embeddings-benchmark:main Apr 23, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants