Add Indic STS benchmark dataset

[jaygala24]

Checklist for adding MMTEB dataset

Reason for dataset addition:

• I have tested that the dataset runs with the mteb package.
• I have run the following models on the task (adding the results to the pr). These can be run using the mteb run -m {model_name} -t {task_name} command.
  • sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
  • intfloat/multilingual-e5-small
• I have checked that the performance is neither trivial (both models gain close to perfect scores) nor random (both models gain close to random scores).
• If the dataset is too big (e.g. >2048 examples), considering using self.stratified_subsampling() under dataset_transform()
• I have filled out the metadata object in the dataset file (find documentation on it here).
• Run tests locally to make sure nothing is broken using make test.
• Run the formatter to format the code using make lint.
• I have added points for my submission to the points folder using the PR number as the filename (e.g. 438.jsonl).

[jaygala24]

Add Semantic Textual Similarity benchmark between English and 12 high-resource Indic languages.

Team:

• Diganta Mista (@digantamisra98) - MILA
• Jay Gala (@jaygala24) - AI4Bharat

[KennethEnevoldsen]

Looks great! Feel free to add the points as well. I only have one pointer related to size

[jaygala24]

Points Summary:

This is the first dataset for Indic languages in the STS task category. Hence, all the 12 Indic languages are new ones for this task category.

• This dataset = 2 pts
• Any new language not covered previously in STS task = 4 pts x 12 languages

Total points = 2 + 4 x 12 = 50 pts

[KennethEnevoldsen]

Thanks for downsampling. Will merge this PR in