[Model Request]: Model2Vec models

[Pringled]

What happened?

It would be great if Model2Vec models could be supported, e.g. https://huggingface.co/minishlab/potion-base-8M. They include ONNX files and are extremely fast (since the embeddings are static), small, and high performant.

What Python version are you on? e.g. python --version

Python3.10

Version

0.2.7 (Latest)

What os are you seeing the problem on?

No response

Relevant stack traces and/or logs

No response

[joein]

Hey @Pringled

Sounds good, do you think you could submit a PR with a couple of the most prosperous models?

[Pringled]

Hey @joein , great!

I just had a look and sadly the ONNX models don't work out of the box since they do not have the same expected inputs from the tokenizer. However, ONNX models in general don't add any benefits for these models as they are static embeddings. I was wondering if you are open to the idea of a static_embedding.py option that follows the same structure als the normal TextEmbedding type but uses model2vec instead of ONNX. I think it would be a good fit for fastembed as these embeddings are extremely fast (e.g. ~50x faster than MiniLM and ~500x faster than bge-base), while still having good performance. There are also very few dependencies in the package (numpy is the biggest one).

However, I understand if this is not something you want to introduce. In any case, if you are interested in it, I would be willing to write some integration code and make a PR, let me know!

[joein]

We would rather prefer to keep it onnx only
What inputs does it have? Maybe we could adjust the interface or introduce a separate class

[Pringled]

That would be great if that's possible! The inputs are input_ids and offsets (as the forward pass is simple an embeddingbag). These are the forward pass and tokenize functions for the onnx models. So I guess to make it work, the onnx_embed function would have to be altered slightly, or a variant for static embeddings would have to be introduced?