Skip to content

Roll variant headlines out to 25% of users - #16626

Closed
Fweddi wants to merge 4 commits into
mainfrom
fp/headline-test-rollout-stage-1
Closed

Fweddi wants to merge 4 commits into
mainfrom
fp/headline-test-rollout-stage-1

Conversation

@Fweddi

@Fweddi Fweddi commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Begin rolling out variant headlines rendered through the platform test fronts-and-curation-editorial-test (renamed from fronts-and-curation-editorial-headline-test to future-proof for when we test articles using same test): #16554.

The end goal is a 100% 50/50 test, we are reaching this threshold in stages to avoid overloading the cache.

@Fweddi
Fweddi requested a review from a team as a code owner August 27, 2026 14:23
@github-actions

github-actions Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Comment thread ab-testing/config/abTests.ts Outdated
@Fweddi Fweddi changed the title Roll variant headline out to 5% of users Roll variant B headlines out to 5% of users Aug 27, 2026
@Fweddi Fweddi linked an issue Aug 27, 2026 that may be closed by this pull request
@Fweddi Fweddi added the feature Departmental tracking: work on a new feature label Aug 27, 2026
@Fweddi
Fweddi marked this pull request as draft August 27, 2026 14:26
@github-actions

github-actions Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Comment thread ab-testing/config/abTests.ts Outdated
Comment thread ab-testing/config/abTests.ts Outdated
Comment thread ab-testing/config/abTests.ts Outdated
@github-actions

github-actions Bot commented Sep 1, 2026 •

Copy link
Copy Markdown

🚀 Image pushed to AWS ECR

Image digest: sha256:90e84b335d0a7686bd3860873d0e11e5d499135de47c0c81771c8d1d8e9d5967

🐛 Run the image locally

The following can be used to run the image locally:

# Refer to image using the immutable digest. Find alternatives below.
IMAGE_IDENTIFIER="@sha256:90e84b335d0a7686bd3860873d0e11e5d499135de47c0c81771c8d1d8e9d5967"

# Refer to image using branch tag
# IMAGE_IDENTIFIER=":branch-fp-headline-test-rollout-stage-1"

# Refer to image using build tag
# IMAGE_IDENTIFIER=":build-30677"

# Refer to image via the GitHub commit SHA tag
# IMAGE_IDENTIFIER=":sha-727ac9d306ac2ec98f44301d9da15323e07d0bf5"

# Set environment variables for the AWS CLI
AWS_PROFILE="<A_PROFILE_FROM_JANUS>"
AWS_DEFAULT_REGION="eu-west-1"

IMAGE_ACCOUNT_ID=$(aws ssm get-parameter --name /organisation/accounts/deployTools --query "Parameter.Value" --output text)
REGISTRY="${IMAGE_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com"
IMAGE="${REGISTRY}/guardian/dotcom-rendering${IMAGE_IDENTIFIER}"

# Login to AWS ECR https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html
aws ecr get-login-password | docker login --username AWS --password-stdin $REGISTRY

# Pull the image
docker pull $IMAGE

# Run the image. You'll likely need to set additional flags. See https://docs.docker.com/reference/cli/docker/container/run.
docker run $IMAGE

From guardian/actions-publish-image.

@Fweddi
Fweddi force-pushed the fp/headline-test-rollout-stage-1 branch from 0d78e37 to 727ac9d Compare September 2, 2026 08:19
@Fweddi
Fweddi marked this pull request as ready for review September 2, 2026 08:19
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Hello 👋! When you're ready to run Chromatic, please apply the run_chromatic label to this PR.

You will need to reapply the label each time you want to run Chromatic.

Click here to see the Chromatic project.

{
name: "fronts-and-curation-editorial-headline-test",
description: "Allow editorial headline A/B tests to run on web",
name: "fronts-and-curation-editorial-test",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we roll out article A/B tests, might we want to roll this back down and up to avoid the same load on cache? 🤔

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep this is a good point! We should ticket this up to make sure that we do this but also that we let editorial know we'll need a day's downtime on headline testing

@abeddow91 abeddow91 changed the title Roll variant B headlines out to 5% of users Roll variant B headlines out to 25% of users Sep 2, 2026
@Fweddi Fweddi changed the title Roll variant B headlines out to 25% of users Roll variant headlines out to 25% of users Sep 3, 2026
@Fweddi Fweddi closed this Sep 3, 2026
@Fweddi
Fweddi deleted the fp/headline-test-rollout-stage-1 branch September 3, 2026 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Do not merge ⚠️ feature Departmental tracking: work on a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set Platform AB Test to 50/50

2 participants