Capture an error when a transaction is rejected due to being invalid

[the21st]

Problem Statement

Whenever a transaction grows too large, it gets discarded due to being Invalid, since it exceeded the size limit. Currently there's no mechanism that would inform us engineers that this happened.

Solution Brainstorm

Sentry could automatically capture an error in such case, so that we engineers know about this invalid transaction and can fix it.

Product Area

Performance

[getsantry]

Assigning to @getsentry/support for routing ⏲️

[getsantry]

Routing to @getsentry/product-owners-performance for triage ⏲️

[bcoe]

Hello @the21st, what SDK are you using that you're running into these size limits with?

[the21st]

@bcoe we're using the Python SDK. We're running into limits when e.g. we fire a lot of SQL queries (think 400+) within a single transaction. These queries are auto-instrumented as spans by the Sentry SDK automatically.

[sentrivana]

Hey @the21st, I'm wondering whether this pops up in your logs? We should be logging an error in the SDK if we receive a reply with a non-200 status code when sending an event to Sentry. (I believe the server would reply with a 400 in case the payload is too big.) That'd be one place to look.

You might also be able to see some stats on dropped transactions on the Stats page in the UI.

[the21st]

@sentrivana what should I search for in our logs? I'll take a look.

As for the stats page - yes, I see it there, but that cannot be alerted on, and doesn't give us almost any debugging info for that dropped transaction.

[szokeasaurusrex]

what should I search for in our logs? I'll take a look.

@the21st logs from the Sentry SDK should be prefixed with [sentry].

[the21st]

@szokeasaurusrex thanks! I can see in Sentry stats that there were 5 transactions dropped due to a network error in the last hour but in our logs there are no entries with [sentry] in them.

Edit: I also looked for "Invalid" as a reason for dropping, found 1, searched our logs for that time period, and also found nothing.

[szokeasaurusrex]

I can see in Sentry stats that there were 5 transactions dropped

Where exactly are you seeing this information @the21st? A link to the exact page and/or a screenshot of the information would be helpful

[the21st]

@szokeasaurusrex on the stats page: https://invert.sentry.io/stats/?clientDiscard=true&dataCategory=transactions&environment=prod&project=4507972815028224&statsPeriod=7d . I can send a screenshot in a private channel if there is some.

[sl0thentr0py]

@the21st apologies for the back and forth.

So yes, it is true that currently we only expose information on things such as invalid transactions, size limits, network errors, etc only on that Stats page. See here for information on all the various types of stats we collect.

There are reasons we wouldn't want to automatically create errors for each of these because this would mean you would have to pay us (via quota usage) for things that are wrong not in your, i.e. userland code but something within our SDK and our servers. Storing errors/transactions is costly for us (and hence you) so we only do it for actual user problems and collect statistics for our internal problems.

However, I do agree that it makes sense to somehow surface a spike based on those statistics somewhere in the product UX. I will talk internally about this and get back to you.