POST http://localhost:3002/api/chat net::ERR_EMPTY_RESPONSE (index.mjs:345)

[VladSolyankin]

Hi, i was building a project using Next Js + Vercel AI Sdk for it and it was working fine until now. I'm getting same error while trying to send message in chat.

route.ts for chat/api is basically copy-paste of the tutorial one

[VladSolyankin]

Also, it can be something to do with that, i guess:

[MaxLeiter]

This is an error from huggingface, not related to the AI SDK.

[VladSolyankin]

Thanks. By the way, can you recommed working model, because this one was in docs and i'm really not familiar with ai stuff and hugging face, so i appreciate any help in finding other models working like this one.

[DayakarH]

Hey try using the meta-llama/Meta-Llama-3-8B-Instruct model. This is the only popular model that I came across that we can use for free with the Inference API and is serving requests without throwing a 500 error.

[jonasfroeller]

Hey try using the meta-llama/Meta-Llama-3-8B-Instruct model. This is the only popular model that I came across that we can use for free with the Inference API and is serving requests without throwing a 500 error.

Mistral-7B-Instruct-v0.1 works too. OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 worked the last few months tho. Sadly not anymore.