Problem with some tickets not registering, even after paying

[Tiddons]

Describe the bug
I'm part of a student organisation which organises several events throughout the year with ~150 attendees. Using this platform has rarely resulted in issues for the students. Recently we launched the sale of a bigger event with other student organisations (650 tickets). These tickets tend to sell out within 5 minutes so it didn't surprise us that the site went down twice during the sale. After this concluded we got several mails from students that they never received their QR-codes even though they paid and arrived on the /succes page.
Checking their id's gives no results in the reservation page nor do they appear to be in the cancelled list. Are there concrete steps we can undertake to resolve this and/or prevent this from happening in the future? I noticed that 19 tickets are stuck on "pending" on the homepage of the event (which is give or take the amount of complaints we got), even though I can't find these in the reservations tab. Hopefully that gives a clue on what happened here. I'll put the screenshots down below. Thanks in advance.

Version: Alf.io v.2.0-M5 using docker-compose

Screenshots

[syjer]

hi @Tiddons , as a confirmation, which version of alf.io are you using?

[syjer]

also, @Tiddons , we may need logs: from the current information we can't really debug the issue. For privacy, you can send them to [email protected] .

And also, we have some performance improvement on the main branch that will be released soon, so this may help you handle a little better the onslaught.

[Tiddons]

hi @Tiddons , as a confirmation, which version of alf.io are you using?

Forgot to mention this. We're running Alf.io v.2.0-M5.
I'll send the logs to the provided mail.

[syjer]

As you are using docker compose, I would guess you are running alf.io and postgresql on a single server ?

[Tiddons]

As you are using docker compose, I would guess you are running alf.io and postgresql on a single server ?

Correct, we're using postgres 16 on the same machine. I also mailed the logs just now.

[syjer]

from a first look at the log:

• about the mails: it seems some users provided a wrong email address domain ( Caused by: jakarta.mail.internet.AddressException: Domain contains illegal character )
• the application was not able to open db connections (which is kinda expected if the whole setup is underprovisioned)
• in the logs we noticed some error also on the database side (deadlocks), this is not really good too, most likely related to the underprovisioning of the resources.

Currently it's hard to understand the others issues.

[Tiddons]

Under check-in we found the "pending" tickets. There was no wrong info in them and I confirmed with mollie that the payment was successful. There was one instance of a student who entered a wrong email but we have since corrected this.

So we still don't know why this really happened? I can imagine that for such sales we need to scale our deployment but 19 tickets isn't insignificant and seeing that the system did register them in some way I feel like there should be a way to solve this.
I provided some screenshots to illustrate what i mean.

[syjer]

The finalizing status for a pending reservation may be a good hint for where the error could have been from.

Are you using javascript extensions in your alfio instance?

[Tiddons]

No, we're running vanilla alf.io so that should eliminate some factors

[syjer]

From the log we also noticed some network error (DNS issues) so this is also a sign that even the network was overwhelmed (some kind of rate limiting?).

By the way, we are analyzing in how we can at least make it easier for processing the reservations stuck in the finalizing phase.