feat(dynamic-sampling): adapt docs to new dynamic sampling logic #11886
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
2 Skipped Deployments
|
There was a problem hiding this comment.
Is there a place where you're planning to describe the newly introduced organization and project options? I also noticed that the biases don't describe how their behavior is different in the two modes. The most notable differences are:
- Boost low volume projects is disabled in manual mode
- Boost low volume transactions uses the sample rate from the project option
- All other biases compute factors that are independent of the sample rate
- Recalibration switches to project level
| @@ -13,19 +13,20 @@ A sample rate is a number in the interval `[0.0, 1.0]` that will determine the l | |||
|
|
|||
| ## The Concept of Fidelity | |||
|
|
|||
| At the core of Dynamic Sampling there is the concept of **fidelity**, which translates to an overall **target sample rate** that should be applied across all transactions of an organization. | |||
| At the core of Dynamic Sampling there is the concept of **fidelity**, which translates to an overall **target sample rate** that should be applied across all events of an organization. | |||
There was a problem hiding this comment.
Today, it applies to some of our tracing data but not all. DS applies to spans and transaction events, but not to error and message events from error monitoring. What further complicates this is that technically errors are a part of the trace, and as such the trace definition you deliver below is absolutely correct.
Note that also in many parts of the code base "Event" refers to a base type for the different kind of data collected through error monitoring (error, security, message), whereas spans have a different root type. Transaction events used to share that same base type for historical reasons, but over time were considered something else until we started considering their deprecation.
| - **Automatic mode** dynamically manages the target sample rate for each project based on the target sample rate for the organization, prioritizing lower volume projects to increase visibility. | ||
| - **Manual mode** allows the user to set static target sample rates on a per-project basis that serve as the baseline sample rate before applying the dynamic biases outlined below. Target sample rates are not adjusted by the system. |
There was a problem hiding this comment.
Should we mention the in-code names here in contrast to the marketing term?
There was a problem hiding this comment.
Yeah good point - I was a bit worried about the redundancy, but I suppose pointing out the two namings is a valid way of adding context for anyone who will work on this.
| # ... | ||
| ] | ||
| ``` | ||
| This bias is used to prioritize traces that are coming from a new release. The goal is to increase the sample rate in the time window that occurs between the creation of a release and its adoption by users. _The identification of a new release is done in the `event_manager` defined [here](https://github.com/getsentry/sentry/blob/master/src/sentry/event_manager.py#L937-L937)._ |
There was a problem hiding this comment.
The line numbers are likely to move frequently. What would you prefer to make this a stable reference?
- Pin a current revision
- Reference some name from code
- Move the code to a more stable place and reference the file
|
|
||
| The latest release bias uses a decaying rule to interpolate between a starting sample rate and an ending sample rate over a time window that is statically defined for each platform (the list of time to adoptions is define [here](https://github.com/getsentry/sentry/blob/master/src/sentry/dynamic_sampling/rules/helpers/time_to_adoptions.py#L26-L26). For example, Android has a bigger time window than Javascript because on average Android apps take more time to get adopted by users. | ||
| In order to rebalance transactions, the system computes the counts of the transactions for each project and runs an algorithm that, given the sample rate of the organization and the counts of each transaction, computes a new sample rate for each transaction assuming an ideal distribution of the counts. |
There was a problem hiding this comment.
In a recent conversation, you brought up that it is surprising how the algorithm tries to achieve an equal sample count for each "class" (as it is called in code). This could be considered an implementation detail, but I think it's worth writing this down as the desired behavior of the system at this time.
|
|
||
| <Alert title="💡 Note" level="info"> | ||
|
|
||
| Dynamic Sampling currently operates on either spans or transactions, based on the feature flag `dynamic-sampling-spans`. The logic between the two event types is similar, so most of this documentation is kept at a generic level and important differences are pointed out explicitly. |
There was a problem hiding this comment.
The feature flag will be tied to what is the primary stats and billing metric according to the organization's plan. The default going forward is spans.
| ## Sequencing | ||
|
|
||
| Dynamic Sampling occurs at the edge of our ingestion pipeline, precisely in [Relay](https://github.com/getsentry/relay). | ||
|
|
||
| When transaction events arrive, in a simplified model, they go through the following steps (some of which won't apply if you self-host Sentry): | ||
| When events arrive, in a simplified model, they go through the following steps (some of which won't apply if you self-host Sentry): |
There was a problem hiding this comment.
Reading through it again, all of these apply in a self-hosted environment by now. We can remove this remark.
| When events arrive, in a simplified model, they go through the following steps (some of which won't apply if you self-host Sentry): | |
| When events arrive, in a simplified model, they go through the following steps: |
|
|
||
| Dynamic Sampling further reduces how many transactions get stored internally. **While many-to-most graphs and numbers in Sentry are based on total transactions**, accessing spans and tags requires stored transactions. The sample rates apply on top of each other. | ||
| Dynamic Sampling further reduces how many events get stored internally. **While most graphs and numbers in Sentry are based on total events**, accessing spans and tags requires stored events. The sample rates apply on top of each other. |
There was a problem hiding this comment.
| Dynamic Sampling further reduces how many events get stored internally. **While most graphs and numbers in Sentry are based on total events**, accessing spans and tags requires stored events. The sample rates apply on top of each other. | |
| Dynamic Sampling further reduces how many events get stored internally. **While most graphs and numbers in Sentry are based on metrics**, accessing spans and tags requires stored events. The sample rates apply on top of each other. |
Should we already foreshadow extrapolation? In that case, everything is based on stored.
Update the documentation for dynamic sampling to reflect the new status quo:
transactionfor the event description, we useeventsto encompass both transactions and spans.