Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add minishlab models (model2vec) #1503

Closed
KennethEnevoldsen opened this issue Nov 26, 2024 · 3 comments
Closed

Add minishlab models (model2vec) #1503

KennethEnevoldsen opened this issue Nov 26, 2024 · 3 comments
Labels
good first issue Good for newcomers new-model Questions related to adding a new model to the benchmark

Comments

@KennethEnevoldsen
Copy link
Contributor

KennethEnevoldsen commented Nov 26, 2024

We have these in the results repo, but we do not have an implementation in the repo. These are also very good integration test models due to their very small size and low number of dependencies.

'minishlab/M2V_base_glove',
'minishlab/M2V_base_glove_subword',
'minishlab/M2V_base_output',
'minishlab/potion-base-2M',
'minishlab/potion-base-4M',
'minishlab/potion-base-8M',

You can find the documentation on custom models here and you can find an example of an implementation here

@KennethEnevoldsen KennethEnevoldsen added good first issue Good for newcomers new-model Questions related to adding a new model to the benchmark labels Nov 26, 2024
@KennethEnevoldsen
Copy link
Contributor Author

KennethEnevoldsen commented Nov 27, 2024

@HadiaSamil will take a stab at this problem here

A good step by step approach:

is to

  1. implement to model and have it run on MTEB (using some task)
  2. once that works integrate into the mteb package

@KennethEnevoldsen
Copy link
Contributor Author

related to #1515

@KennethEnevoldsen KennethEnevoldsen changed the title Add minishlab models Add minishlab models (model2vec) Dec 4, 2024
@x-tabdeveloping
Copy link
Collaborator

These were added a week and a half ago

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers new-model Questions related to adding a new model to the benchmark
Projects
None yet
Development

No branches or pull requests

2 participants