MAX_BATCH_TOKENS parameter #367
Unanswered
maioranisimone
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
hello everyone, I would like to ask where I can find the maximum value to assign to max_batch_tokens parameter. I read that text-embeddings-inference cannot find this value automatically, where is this value declared in the models present on Hugging Face?
Thanks in advance.
Beta Was this translation helpful? Give feedback.
All reactions