[Bug?]: Deploy to Vercel breaks all server functions (500 errors with cryptic messages)

[DrakenKor]

What's not working?

Hi,
My RedwoodJS project, which was running fine previously, now breaks on deployment in the live environment.

All backend functions are throwing 500 errors, with cryptic messages. Everything is working same as before locally (100% fine). These functions are event/timer driven as well as user triggered, all are broken.

My intuition says that the way Vercel is building Redwoodjs is changed, and this change is breaking the site

Not sure if it's related to Vercel changing the deployment for RedwoodJS:

#10022

Currently my project, which serves a client, is down and I've tried the following fixes on my end (No resolution):

Upgrade Redwood from ~7.6 to latest 7.7.2
Add a vercel.json with some basic configuration:

Errors:

Complete error from backend function:

Unhandled Rejection: SyntaxError: Unexpected end of JSON input
at parse (<anonymous>)
at JSON.parse (/var/task/node_modules/core-js/modules/esnext.json.parse.js:249:56)
at IncomingMessage.<anonymous> (/opt/rust/nodejs.js:1:12751)
at IncomingMessage.emit (node:events:519:28)
at endReadableNT (node:internal/streams/readable:1696:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Node.js process exited with exit status: 128. The logs above can help with debugging the issue.

How do we reproduce the bug?

Deploy redwood 7.7.2 to Vercel, with atleast 1 backend function apart from graphql.

What's your environment? (If it applies)

My local environment
System:
    OS: macOS 14.5
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.10.0 - /var/folders/r5/mjyksjsd78b81bj1lyrtd0980000gn/T/yarn--1720509206850-0.9503137294015356/node
    Yarn: 1.22.17 - /var/folders/r5/mjyksjsd78b81bj1lyrtd0980000gn/T/yarn--1720509206850-0.9503137294015356/yarn
  Databases:
    SQLite: 3.43.2 - /usr/bin/sqlite3
  Browsers:
    Chrome: 126.0.6478.127
    Safari: 17.5
  npmPackages:
    @redwoodjs/auth-dbauth-setup: 7.7.2 => 7.7.2 
    @redwoodjs/core: 7.7.2 => 7.7.2 
  redwood.toml:
    [web]
    title = "Rin Kim Law"
    port = 8910
    apiUrl = "/api"
    includeEnvironmentVariables = [
      'AS_CLIENT_ID',
      'APP_URL',
      'ENVIRONMENT'
      # Add any ENV vars that should be available to the web side to this array
      # See https://redwoodjs.com/docs/environment-variables#web
    ]
    [api]
    port = 8911
    [browser]
    open = true
    [notifications]
    versionUpdates = ["latest"]

Update

I just noticed that Vercel now have a 'comments middleware' that applies to all routes, and it's an edge function. This could be the culprit:

Are you interested in working on this?

• I'm interested in working on this

[RinKimLaw]

Hi does anyone else have any issues with Vercel deployed Redwood? I've deleted my project off Vercel. Given my experience I wouldn't recommend Vercel as a deploy target for RedwoodJS.

[dthyresson]

Hi @DrakenKor and @RinKimLaw I was working with Vercel to improve the deployment and have them fix the build issue where functions didn't get configured for memory or duration and I think their cli update vercel/vercel#11776 got released before we could let you and other Vercel users know.

I have a draft for the docs change: #10891

We intended to have fixes for #10022 and #1664 with vercel/vercel#11776.

Documents support for Vercel API endpoint (ie, functions) configuration via export const config syntax.

I'll try a few builds and deploys as well but, could you try:

1. not using vercel.json
2. confirmed the functions the new upcoming docs

By default, API requests in Vercel have a timeout limit of 15 seconds, but can be configured to be up to 90 seconds. Pro and other plans allow for longer [duration](https://vercel.com/docs/functions/runtimes#max-duration) and larger [memory-size limits](https://vercel.com/docs/functions/runtimes#memory-size-limits).

To change the `maxDuration` or `memory` per function, export a `config` with the settings you want applied in your function. For example:

```ts
import type { APIGatewayEvent, Context } from 'aws-lambda'

import { logger } from 'src/lib/logger'

export const config = {
  maxDuration: 30,
  memory: 512,
}

export const handler = async (event: APIGatewayEvent, _context: Context) => {
  logger.info(`${event.httpMethod} ${event.path}: vercel function`)

  return {
    statusCode: 200,
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      data: 'vercel function',
    }),

Thanks and appreciate patience as we wanted to make the RW + Vercel experience better.

[dthyresson]

@DrakenKor Could you give me some more information on:

I just noticed that Vercel now have a 'comments middleware' that applies to all routes, and it's an edge function. This could be the culprit.

I used your https://github.com/DrakenKor/vercel-redwood-tester to reproduce and shared with Vercel to try it out.

If you could update that repo with how you have setup and edge function, I can try that and then get in touch with Vercel to see what might be happening.

I'll try with an example like:

export const config = {
  runtime: 'edge',
}

export default function handler(_req: Request) {
  return new Response('Hello World')
}

[dthyresson]

@DrakenKor and @RinKimLaw I'm trying to reproduce and also wanted to make sure your Vercel cli is the latest:

vercel-redwood-tester main % vercel --version
Vercel CLI 34.3.1
34.3.1

If not, please npm i -g vercel@latest.

I did deploy that tester app and can call the function and get a response:

As @DrakenKor does, after adding a hello edge function, I see the middleware:

I tried to call a hello function in edge as they documented but I probably didn't get the response correct:

export const config = {
  runtime: 'edge',
}

export default function handler(_req: Request) {
  return new Response('Hello World')
}

I'll see what the edge function should be instead.

[RinKimLaw]

This is the error I see as well:

I didn't have any edge functions configured, but vercel deployed one of my functions at the edge anyway.

[RinKimLaw]

Hi,
I've redeployed our site with a new config on each backend function that has the runtime value nodejs20.x:

I can confirm the system is now fully functional again, and that the config maxDuration & memory is being honoured.

So I can confirm that this is a bug with RedwoodJS backend functions deployed as edge functions on Vercel. Any edge function will throw 502/500 and if a backend function is wrapped with a middleware edge function, that'll also throw 502/500.

My issue is resolved but if I can be of any further assistance please let me know.

[dthyresson]

@RinKimLaw or @DrakenKor

I can confirm the system is now fully functional again, and that the config maxDuration & memory is being honoured.

Great to hear!

So I can confirm that this is a bug with RedwoodJS backend functions deployed as edge functions on Vercel. Any edge function will throw 502/500 and if a backend function is wrapped with a middleware edge function, that'll also throw 502/500.

Can you share more details about this? Am a bit confused if you have edge functions or not.

Actually, maybe we should pair on it to resolve faster -- then I can also communicate back to Vercel any issues more clearly.

Could you find some time next week perhaps here: https://savvycal.com/dt/redwoodjs

Thanks!

[RinKimLaw]

@dthyresson

To summarise:
The issue arose for us when some of our backend functions were automatically deployed as edge functions by Vercel. We had no edge function config (Or any other config) in our Redwoodjs project and the edge function config is some default config by Vercel. I noticed that there was a middleware edge function and this wrapped all the backend functions, and all the backend functions were throwing 502/500 errors. Then you gave your suggestion on how to configure the functions for Vercel using export const config = {} syntax and I was finally able to configure all my backend functions as node20.x functions, and the errors stopped. The middleware function disappeared.

I know that NextJS has a middleware edge function deployed by default, and I think Vercel's default deploy for RedwoodJS introduced that middleware similar to NextJS. I think when they changed the config method for RedwoodJS, they broke the default deployment config.

[DrakenKor]

@dthyresson I've booked a time for you on (I assume) your Monday evening (My Tuesday morning)