Site-wide Donate banner A/B testing demo #12927

danielfmiranda · 2024-09-27T00:47:49Z

Description

The current Donate banner, implemented as a snippet, cannot be A/B tested because our A/B testing package only supports testing at the Page level, not for snippets or fields that aren’t directly tied to a page.

This PR enables site-wide A/B testing for the DonateBanner by implementing the following changes:

Added a new donate_banner field to the Homepage model and updated the homepage template to render the banner when applicable.
- Since the Homepage is a Page model, this allows us to run A/B tests on different donate banners.
Updated the BasePage model with a new get_donate_banner() method and modified its template, base.html, to use this method:
- The method retrieves the Donate banner from the Homepage.
- If an active A/B test is running on the Homepage, the method checks if the user has already been assigned a test variant by looking for a specific cookie.
- If no cookie is found, the user is added as a new participant in the test using the add_participant() method, which assigns them to either the "control" or "variant" group.
- Based on their assigned group, the method selects the appropriate version of the donate banner to display. Additionally, a cookie is set to ensure the user continues to see the same version on subsequent visits.
Introduced a custom Wagtail A/B test event called DonateBannerLinkClick and updated the Donate banner's JavaScript to trigger this event whenever the banner's CTA button is clicked.

Notes

Here is a list of all pages that inherit from the BasePage class. If the donate banner is set on the homepage, it will render on the following pages (including during A/B tests):

AppInstallPage
ArticlePage
BanneredCampaignPage
BlogIndexPage
BlogPage
BuyersGuideArticlePage
BuyersGuideCampaignPage
BuyersGuideEditorialContentIndexPage
BuyersGuidePage
CampaignIndexPage
CampaignPage
ConsumerCreepometerPage
DearInternetPage
GeneralProductPage
IndexPage
InitiativesPage
MiniSiteNameSpace
ModularPage
OpportunityPage
ParticipatePage2
PrimaryPage
ProductPage
PublicationPage
RCCAuthorsIndexPage
RCCDetailPage
RCCLandingPage
RCCLibraryPage
ResearchAuthorsIndexPage
ResearchDetailPage
ResearchLandingPage
ResearchLibraryPage
Styleguide
YoutubeRegrets2021Page
YoutubeRegrets2022Page
YoutubeRegretsPage
YoutubeRegretsReporterPage

Screenshots

CMS:

Steps to test A/B Testing

Visit https://foundation-s-donate-ban-wzxauo.herokuapp.com/cms/pages/46/edit/
Log in with credentials: admin2/admin2
Select a new donate banner for the "donate banner" field
Select "save and create AB test"
On the A/B test page, select "Donate Banner Link Click" on the "what is the goal?" dropdown. Since the "donate banner link click" event is global, you do not need to choose a page for the green "Choose a page" button.
Click "save and run test"
You can now test that a visit to the site will serve you a variant of the donate banner, and that the AB test dashboard is accurately tracking variants served and link clicks 👍

…vent

danielfmiranda · 2024-09-27T20:04:19Z

network-api/networkapi/templates/pages/base.html

+            {% if donate_banner %}
+                {% include "fragments/donate_banner.html" with banner=donate_banner.localized %}


Instead of displaying the donate banner set in the site’s settings, we are now rendering the banner provided to the context by the get_donate_banner() method.

danielfmiranda · 2024-09-27T20:05:22Z

network-api/networkapi/wagtailpages/templates/wagtailpages/homepage.html

+{% block donate_banner %}
+    {% if page.donate_banner %}
+        {% include "fragments/donate_banner.html" with banner=page.donate_banner.localized %}
+    {% endif %}
+{% endblock %}


Updating the homepage template to render the donate banner it has set for it's donate_banner field.

danielfmiranda · 2024-09-27T20:07:44Z

network-api/networkapi/wagtailpages/wagtail_hooks.py

+class DonateBannerLinkClick(BaseEvent):
+    name = "Donate Banner Link Click"
+    requires_page = False  # Set to False to create a "Global" event type that could be reached on any page
+
+
+@hooks.register("register_ab_testing_event_types")
+def register_donate_banner_link_click_event_type():
+    return {
+        "donate-banner-link-click": DonateBannerLinkClick,
+    }


For more details on adding custom Wagtail A/B testing events, refer to the Wagtail A/B testing docs.

danielfmiranda · 2024-09-27T20:18:09Z

network-api/networkapi/wagtailpages/pagemodels/base.py

+        # Check for the cookie specific to this A/B test.
+        test_cookie_name = f"wagtail-ab-testing_{active_ab_test.id}_version"
+        test_version = request.COOKIES.get(test_cookie_name)


In the wagtail-ab-testing source code, the cookie name is defined as "wagtail-ab-testing_{ab_test.id}_version".

For reference, see the relevant source code here.

danielfmiranda · 2024-09-27T20:30:43Z

network-api/networkapi/wagtailpages/pagemodels/base.py

+
+        # If no cookie is found, add user as a participant in test, and set a cookie for their assigned version.
+        if not test_version:
+            test_version = active_ab_test.add_participant()


The add_participant() method adds the user as a participant to the A/B test, and returns their assigned version of either "control" or "variant".

For reference, see the relevant source code here.

danielfmiranda · 2024-09-27T20:41:04Z

network-api/networkapi/wagtailpages/pagemodels/base.py

@@ -33,6 +36,37 @@ class BasePage(FoundationMetadataPageMixin, FoundationNavigationPageMixin, Page)
    class Meta:
        abstract = True

+    def get_donate_banner(self, request):


TODO: If we continue forward with this implementation, we would need to check if the user has "do not track" enabled.

We might also want to check if the AB test includes the donate banner. If not, we can skip through all of this.

danielfmiranda · 2024-09-27T20:57:48Z

network-api/networkapi/templates/fragments/donate_banner.html

@@ -26,7 +26,7 @@
                            {{ banner.subtitle }}
                        </p>
                        <div class="tw-w-full tw-pt-6">
-                            <a class="{{ btn_class }}" href="{{ banner.cta_link }}">
+                            <a id="banner-cta-button" class="{{ btn_class }}" href="{{ banner.cta_link }}">


ID added for JS event tracking purposes

danielfmiranda added 5 commits September 26, 2024 17:33

add donate_banner field to homepage

71d4a4d

add custom DonateBannerLinkClick A/B test event

403d4ac

update donate banner template and JS to account for new A/B testing e…

96b7742

…vent

add get_donate_banner method to BasePage model

d8b6ac8

update templates to account for new donate_banner field and method

6eac9bf

danielfmiranda changed the title ~~Donate banner ab testing demo~~ [WIP] - Donate banner sitewide ab testing demo Sep 27, 2024

danielfmiranda commented Sep 27, 2024

View reviewed changes

danielfmiranda changed the title ~~[WIP] - Donate banner sitewide ab testing demo~~ Site-wide Donate banner A/B testing demo Sep 27, 2024

danielfmiranda commented Sep 27, 2024

View reviewed changes

danielfmiranda added 2 commits September 27, 2024 14:06

updated help text and panel format

5579069

removed redundant donate banner panel

5d5a5b8

danielfmiranda marked this pull request as ready for review September 27, 2024 21:10

danielfmiranda requested review from mmmavis, robdivincenzo, dlopezvsr and ramram-mf September 27, 2024 21:10

danielfmiranda temporarily deployed to foundation-s-donate-ban-wzxauo September 27, 2024 21:15 Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Site-wide Donate banner A/B testing demo #12927

Site-wide Donate banner A/B testing demo #12927

danielfmiranda commented Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024

danielfmiranda Sep 27, 2024

danielfmiranda Sep 27, 2024

danielfmiranda Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024

		{% if donate_banner %}
		{% include "fragments/donate_banner.html" with banner=donate_banner.localized %}

Site-wide Donate banner A/B testing demo #12927

Are you sure you want to change the base?

Site-wide Donate banner A/B testing demo #12927

Conversation

danielfmiranda commented Sep 27, 2024 • edited Loading

Description

Notes

Screenshots

CMS:

Steps to test A/B Testing

danielfmiranda Sep 27, 2024

Choose a reason for hiding this comment

danielfmiranda Sep 27, 2024

Choose a reason for hiding this comment

danielfmiranda Sep 27, 2024

Choose a reason for hiding this comment

danielfmiranda Sep 27, 2024 • edited Loading

Choose a reason for hiding this comment

danielfmiranda Sep 27, 2024 • edited Loading

Choose a reason for hiding this comment

danielfmiranda Sep 27, 2024 • edited Loading

Choose a reason for hiding this comment

danielfmiranda Sep 27, 2024

Choose a reason for hiding this comment

danielfmiranda commented Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024 •

edited

Loading

danielfmiranda Sep 27, 2024 •

edited

Loading