Something err when stream is not true in requst body

[GuanJdoJ]

I run a local service according to this document.

It is OK when "stream": true is set in request body. But if you don't set the stream, it will fail.

Success:

curl --location 'http://localhost:8001/proxy/v1/chat/completions' \
--header 'Authorization: Bearer $BRAINTRUST_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "user",
            "content": "What is your model?"
        }
    ],
    "stream": true,
    "max_tokens": 150,
    "seed": 1
}'

Failed:

curl --location 'http://localhost:8001/proxy/v1/chat/completions' \
--header 'Authorization: $BRAINTRUST_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "user",
            "content": "What is your model?"
        }
    ],
    "max_tokens": 150,
    "seed": 1
}'

But call https://braintrustproxy.com will be OK. Why?

curl --location 'https://braintrustproxy.com/proxy/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "What is a proxy?"
      }
    ],
    "seed": 1
  }' \
  -H "Authorization: Bearer $OPENAI_API_KEY"

[ankrgyl]

What is the error that you see?

[GuanJdoJ]

No specific error reported. I use postman. It shows "Error: Decompression failed" in console.

[ankrgyl]

Got it. It may be something related to a compression header being forwarded incorrectly. If you need a short term workaround, I would recommend running the Cloudflare proxy (apis/cloudflare) locally (via npx wrangler dev). It's more tested than the node one.

[GuanJdoJ]

I will try cloudflare. But I still want to implement it in node. I want to make sure that both the openai eventstream and json results can be processed in @braintrust/proxy. I would really appreciate your support.😊

[GuanJdoJ]

I found the reason why non-stream requests fail. In openai, the response header of a non-stream openai request contains content-encoding: br. It will be set in reponse header of node request. Something went wrong in http. I guess http cant br a br.
I filed a PR #65

[ankrgyl]

Thanks @GuanJdoJ. Will follow up on that PR.