How to set the works for the bge-reranker-v2-m3 model? #401
Unanswered
kennard520
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When I deploy the bge-reranker-v2-m3 model on the TEI I discover it doesn't support multiple threads, How to make it support mutiple threads? bellows are my configuration
- MODEL_ID=BAAI/bge-reranker-v2-m3 - TOKENIZATION_WORKERS=10 - MAX_BATCH_REQUESTS=10 - MAX_CLIENT_BATCH_SIZE=500 - POOLING=mean - HOSTNAME=0.0.0.0Beta Was this translation helpful? Give feedback.
All reactions