-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fine-grained sampling rate per checkpoint #156
Labels
Comments
After discussing with @trieloff, we decided to stick to the 1. alternative instead for now |
ramboz
added a commit
that referenced
this issue
Aug 19, 2024
…ific use cases This introduces a new global variable, `window.RUM_SAMPLING_RATE`, that can be set before loading the library to increase the sampling rate for specific use cases that require more data collected for short-term reporting. For instance: - when running an experiment in a 2-week time-frame and achieve statistical significance even with low traffic - when running short-lived marketing campaign and wanting to collect enough data over a single weekend ## Usage For instance: ```js window.SAMPLE_PAGEVIEWS_AT_RATE = 'high'; ``` Or in an HTML context: ```html <head> <meta name="experiment" content="Foo"/> <meta name="experiment-variants" content="/bar,/baz"/> <script>window.SAMPLE_PAGEVIEWS_AT_RATE = document.head.querySelector('meta[name="experiment"]') ? 'high' : null</script> <script src="/scripts/aem.js" type="module"></script> <script src="/scripts/scripts.js" type="module"></script> <link rel="stylesheet" href="/styles/styles.css"> </head> ``` ## Related Issues Fix #156
🎉 This issue has been resolved in version 2.3.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
In the context of running experiments and hoping to gain fast convergence on the winner, we've seen over the last year that the default 1/100 sampling rate is not enough to reach statistical significance in 2 weeks for most customers. We've lowered the sampling to 1/10 since the start of the year on pages that run an experiment but this triggered a few issues:
weight
after its initialization, which leads to 2.top
,load
, possibly others). So we end up with some sessions missing those 2 events and break the integrity of the events lifecycle for that sessionDescribe the solution you'd like
Ideally, the
sampelRUM
object would expose an API to change the sampling rate for a specific checkpoint, so only that 1 checkpoint (or a few likeexperiment
&convert
) is (are) impacted, and not the others.Something along the lines:
and/or
The 2nd approach doesn't create tight coupling between the 2 events, and the experiments can decide to increase sampling for conversion so they align on the page that needed it without impacting pages that just have conversion with no experiments.
Describe alternatives you've considered
head.html
, likewindow.RUM_SAMPLING_RATE = 10
, beforeaem.js
/lib-franklin.js
is loaded so we can adjust the default sampling before the 1st events are triggered.experiment
checkpoint is firedexperiment
works tightly coupled withconvert
, we actually also need to wait for the 1st conversion to happen otherwise we can't compute the winnerThe text was updated successfully, but these errors were encountered: