You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am currently mostly working with the ghcr.io/huggingface/text-embeddings-inference:cpu-1.2 Docker image on MacOS. Currently, I am only trying to find out which reranker models with a context size larger than 512 tokens are supported by TEI, but apart from the two BAAI ones mentioned in the readme, I haven't been able to run any other reranker models from Huggingface. jinaai/jina-reranker-v1-tiny-en for example does not work:
reranker-1 | Error: The `--pooling` arg is not set and we could not find a pooling configuration (`1_Pooling/config.json`) for this model.
reranker-1 |
reranker-1 | Caused by:
reranker-1 | No such file or directory (os error 2)
reranker-1 exited with code 1
Apparently, this is because TEI treats it as an embedding model in this case. According to the readme, BERT models are supported, but the Jina reranker's description says:
JinaBERT itself is a unique variant of the BERT architecture
Would that already tell me that it is not supported?
Additionally, the BAAI models are listed with a revision in the readme, but I have found no mention of revisions on the model page on Huggingface. How do I find the correct revision to use?
I would appreciate if documentation could be added how to find out if a model is supported. Additionally, showing a more meaningful error message when a model is not supported would help, if at all possible.
Information
Docker
The CLI directly
Tasks
An officially supported command
My own modifications
Reproduction
Docker-compose for jinaai/jina-reranker-v1-tiny-en that fails.
Show a clear error message why the model is not supported.
Provide a list with supported models containing more than just the two BAAI/bge-reranker-large and BAAI/bge-reranker-base.
The text was updated successfully, but these errors were encountered:
Models that are not officialy supported by Transformers are always hard to support and parse because they don't need to respect our config files.
For example, Jina is notorious for saying in their config that their architecture is Bert, whereas that's absolutely NOT the case (they add alibi and other modifications) making it very hard for us to parse their configs accurately. In the case of this specific model, they also ommit to say that's it's a classifier in the config. I will add a small change on TEI's side to hardcode this one but there is not much we can do since Huggingface doesn't own this model.
I have found no mention of revisions on the model page on Huggingface. How do I find the correct revision to use?
It can be important to pin revisions if you want to be sure that you always serve the correct version of a model. It can also be important if you want to serve your own model and want to A/B test between two different trainings.
Hi @OlivierDehaene, thank you for the clarification!
If you don't mind this follow-up question: How do I know which models are officially supported? Say I am looking for all supported rerankers, is there a list or some other way to find out?
System Info
I am currently mostly working with the ghcr.io/huggingface/text-embeddings-inference:cpu-1.2 Docker image on MacOS. Currently, I am only trying to find out which reranker models with a context size larger than 512 tokens are supported by TEI, but apart from the two BAAI ones mentioned in the readme, I haven't been able to run any other reranker models from Huggingface. jinaai/jina-reranker-v1-tiny-en for example does not work:
Apparently, this is because TEI treats it as an embedding model in this case. According to the readme, BERT models are supported, but the Jina reranker's description says:
Additionally, the BAAI models are listed with a revision in the readme, but I have found no mention of revisions on the model page on Huggingface. How do I find the correct revision to use?
I would appreciate if documentation could be added how to find out if a model is supported. Additionally, showing a more meaningful error message when a model is not supported would help, if at all possible.
Information
Tasks
Reproduction
Docker-compose for jinaai/jina-reranker-v1-tiny-en that fails.
Expected behavior
Show a clear error message why the model is not supported.
Provide a list with supported models containing more than just the two BAAI/bge-reranker-large and BAAI/bge-reranker-base.
The text was updated successfully, but these errors were encountered: