Is it possible to have multiple instances of the prefect server running with a shared db instance?

[calebhskim]

We are currently hosting Prefect ourselves in GCP with Cloud Run v2 and Cloud SQL. We have a single Cloud Run service running the prefect server with max CPU and memory set along with a single Cloud SQL postgres15 instance. We have a single Cloud Run v2 worker and a cloud run v2 work pool. We are curious if it is possible to have multiple cloud run instances of the prefect server communicate with a single shared db instance. @zzstoatzz mentioned that it should be possible in principle but there "are some in-memory stuff that [we] might have to tinker with" https://github.com/prefecthq/prefect/blob/main/src/prefect/settings/models/server/events.py#L125-L133.

Has anyone tried this before?

[desertaxle]

Hey @calebhskim! Most features of Prefect will work if you run multiple servers against a single Postgres DB instance. The exception is some events and task-scheduling features will not work correctly because they use an in-memory messaging system that isn't shared between server instances.

It is technically possible to use an external message broker via the server.events.messaging_broker setting, but it would require implementing a broker module that follows this protocol. We'd like to implement a message broker that uses Redis in the future, which will make this a lot easier.

[calebhskim]

Thanks for the reply @desertaxle! Can you clarify what events and task-scheduling specifically might not work? Are you referring to scheduled flows/tasks and event driven flow runs (https://docs.prefect.io/3.0/automate/add-schedules)? Just want to verify before we start testing with multiple server instances.

[desertaxle]

Scheduling flow runs will work with multiple servers. When I say task-scheduling, I'm referring to running task workers and running tasks with task.delay or task.async_apply. Event-driven flow runs won't work because the message broker is not being shared between server instances.