Skip to content

feat(experimentation): choose the rollout when completing an experiment - #9021

Open
Zaimwa9 wants to merge 3 commits into
feat/experiment-complete-rollout-decisionfrom
feat/experiment-complete-modal
Open

Zaimwa9 wants to merge 3 commits into
feat/experiment-complete-rollout-decisionfrom
feat/experiment-complete-modal

Conversation

@Zaimwa9

@Zaimwa9 Zaimwa9 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Contributes to https://linear.app/flagsmith/issue/ENG-663/what-happens-to-rollout-when-an-experiment-ends

How did you test this code?

Unit tests, plus a local end-to-end run against #9018 in light and dark mode (each decision, one and several variants).

No results: control pre-selected Variant chosen Single variant

@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
flagsmith-frontend-preview Ready Ready Preview Nov 19, 2026 4:31pm UTC
flagsmith-frontend-staging Ready Ready Preview Nov 19, 2026 4:31pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
docs Ignored Ignored Preview Nov 19, 2026 4:31pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 97a21aa6-4492-4ab0-8b62-c7d68673c95e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@linear-code

linear-code Bot commented Nov 19, 2026

Copy link
Copy Markdown

ENG-663

@Zaimwa9
Zaimwa9 force-pushed the feat/experiment-complete-modal branch from 63a1e16 to 490464c Compare November 19, 2026 16:22
@Zaimwa9
Zaimwa9 changed the base branch from main to feat/experiment-complete-rollout-decision November 19, 2026 16:22
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request labels Nov 19, 2026
@Zaimwa9
Zaimwa9 marked this pull request as ready for review November 19, 2026 16:27
@Zaimwa9
Zaimwa9 requested a review from a team as a code owner November 19, 2026 16:27
@Zaimwa9
Zaimwa9 requested review from talissoncosta and removed request for a team November 19, 2026 16:27
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request labels Nov 19, 2026
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-api-test:pr-9021 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith:pr-9021 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-private-cloud:pr-9021 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-e2e:pr-9021 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api:pr-9021 Finished ✅ Results ✅
ghcr.io/flagsmith/flagsmith-frontend:pr-9021 Finished ✅ Results ✅

@codecov

codecov Bot commented Nov 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.93%. Comparing base (48f525e) to head (490464c).

Additional details and impacted files
@@                            Coverage Diff                             @@
##           feat/experiment-complete-rollout-decision    #9021   +/-   ##
==========================================================================
  Coverage                                      98.93%   98.93%           
==========================================================================
  Files                                           1714     1714           
  Lines                                          72685    72685           
==========================================================================
  Hits                                           71908    71908           
  Misses                                           777      777           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 issues found across 18 files

Confidence score: 3/5

  • In CompleteExperimentModal.tsx, a failed Bayesian-results request can be treated as no results and silently roll out control during irreversible completion. Handle the results error separately and block completion.
  • In CompleteExperimentModal.tsx, an environment request failure leaves the modal loading forever with no error or retry path. Render an error state when the environment cannot load.
  • In VariantChoiceRow.tsx, long or unbroken variant values can widen the completion UI instead of wrapping. Add a wrapping or min-width constraint to the value element.
  • In SelectableCard.tsx, a disabled card’s footer control can still receive keyboard focus. Make the footer inert when the card is disabled.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="frontend/web/components/base/SelectableCard/SelectableCard.tsx">

<violation number="1" location="frontend/web/components/base/SelectableCard/SelectableCard.tsx:80">
P3: The footer slot drops numeric zero because `footer` is checked for truthiness. Preserve valid ReactNode values such as `0` when deciding whether to render the footer wrapper.</violation>

<violation number="2" location="frontend/web/components/base/SelectableCard/SelectableCard.tsx:80">
P2: A disabled card can still expose an interactive footer control to keyboard users because `disabled` only reaches the body button and `pointer-events: none` does not prevent sequential focus. Mark the footer inert when `disabled` is true.</violation>
</file>

<file name="frontend/web/components/experiments/CompleteExperimentModal/VariantChoiceRow.tsx">

<violation number="1" location="frontend/web/components/experiments/CompleteExperimentModal/VariantChoiceRow.tsx:13">
P2: Long or unbroken variant values can overflow this flex row and widen the completion UI instead of wrapping. Add a wrapping/min-width constraint to the value element.</violation>
</file>

<file name="frontend/web/components/experiments/CompleteExperimentModal/CompleteExperimentModal.tsx">

<violation number="1" location="frontend/web/components/experiments/CompleteExperimentModal/CompleteExperimentModal.tsx:34">
P1: A failed Bayesian-results request is treated as a successful no-results response, so the modal can silently roll out control during an irreversible completion. Handle the results error separately and block completion with an error state.</violation>

<violation number="2" location="frontend/web/components/experiments/CompleteExperimentModal/CompleteExperimentModal.tsx:36">
P2: A failed environment request leaves this modal showing a loader forever with no error or retry path. Destructure the query error state and render an error state when the environment cannot load.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

}) => {
const params = { environmentId, experimentId }
const { data: experiment, isError } = useGetExperimentQuery(params)
const { data: bayesianResults, isLoading: isLoadingResults } =

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: A failed Bayesian-results request is treated as a successful no-results response, so the modal can silently roll out control during an irreversible completion. Handle the results error separately and block completion with an error state.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At frontend/web/components/experiments/CompleteExperimentModal/CompleteExperimentModal.tsx, line 34:

<comment>A failed Bayesian-results request is treated as a successful no-results response, so the modal can silently roll out control during an irreversible completion. Handle the results error separately and block completion with an error state.</comment>

<file context>
@@ -0,0 +1,144 @@
+}) => {
+  const params = { environmentId, experimentId }
+  const { data: experiment, isError } = useGetExperimentQuery(params)
+  const { data: bayesianResults, isLoading: isLoadingResults } =
+    useGetExperimentBayesianResultsQuery(params)
+  const { data: environment } = useGetEnvironmentQuery({ id: environmentId })
</file context>

const VariantChoiceRow: FC<VariantChoiceRowProps> = ({ choice }) => (
<div className='d-flex align-items-center gap-2'>
<VariantName colour={choice.colour} name={choice.name} />
<code>{choice.value}</code>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Long or unbroken variant values can overflow this flex row and widen the completion UI instead of wrapping. Add a wrapping/min-width constraint to the value element.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At frontend/web/components/experiments/CompleteExperimentModal/VariantChoiceRow.tsx, line 13:

<comment>Long or unbroken variant values can overflow this flex row and widen the completion UI instead of wrapping. Add a wrapping/min-width constraint to the value element.</comment>

<file context>
@@ -0,0 +1,22 @@
+const VariantChoiceRow: FC<VariantChoiceRowProps> = ({ choice }) => (
+  <div className='d-flex align-items-center gap-2'>
+    <VariantName colour={choice.colour} name={choice.name} />
+    <code>{choice.value}</code>
+    {choice.chanceToWin !== null && (
+      <span className='ms-auto text-secondary fs-caption'>
</file context>

const { data: experiment, isError } = useGetExperimentQuery(params)
const { data: bayesianResults, isLoading: isLoadingResults } =
useGetExperimentBayesianResultsQuery(params)
const { data: environment } = useGetEnvironmentQuery({ id: environmentId })

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A failed environment request leaves this modal showing a loader forever with no error or retry path. Destructure the query error state and render an error state when the environment cannot load.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At frontend/web/components/experiments/CompleteExperimentModal/CompleteExperimentModal.tsx, line 36:

<comment>A failed environment request leaves this modal showing a loader forever with no error or retry path. Destructure the query error state and render an error state when the environment cannot load.</comment>

<file context>
@@ -0,0 +1,144 @@
+  const { data: experiment, isError } = useGetExperimentQuery(params)
+  const { data: bayesianResults, isLoading: isLoadingResults } =
+    useGetExperimentBayesianResultsQuery(params)
+  const { data: environment } = useGetEnvironmentQuery({ id: environmentId })
+  const [completeExperiment, { isLoading: isCompleting }] =
+    useCompleteExperimentMutation()
</file context>

)}
</BareButton>
</BareButton>
{footer && <div className='px-3 pb-3'>{footer}</div>}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A disabled card can still expose an interactive footer control to keyboard users because disabled only reaches the body button and pointer-events: none does not prevent sequential focus. Mark the footer inert when disabled is true.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At frontend/web/components/base/SelectableCard/SelectableCard.tsx, line 80:

<comment>A disabled card can still expose an interactive footer control to keyboard users because `disabled` only reaches the body button and `pointer-events: none` does not prevent sequential focus. Mark the footer inert when `disabled` is true.</comment>

<file context>
@@ -43,34 +46,39 @@ const SelectableCard: FC<SelectableCardProps> = ({
-      )}
-    </BareButton>
+      </BareButton>
+      {footer && <div className='px-3 pb-3'>{footer}</div>}
+    </div>
   )
</file context>
Suggested change
{footer && <div className='px-3 pb-3'>{footer}</div>}
{footer && (
<div className='px-3 pb-3' inert={disabled || undefined}>
{footer}
</div>
)}

)}
</BareButton>
</BareButton>
{footer && <div className='px-3 pb-3'>{footer}</div>}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The footer slot drops numeric zero because footer is checked for truthiness. Preserve valid ReactNode values such as 0 when deciding whether to render the footer wrapper.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At frontend/web/components/base/SelectableCard/SelectableCard.tsx, line 80:

<comment>The footer slot drops numeric zero because `footer` is checked for truthiness. Preserve valid ReactNode values such as `0` when deciding whether to render the footer wrapper.</comment>

<file context>
@@ -43,34 +46,39 @@ const SelectableCard: FC<SelectableCardProps> = ({
-      )}
-    </BareButton>
+      </BareButton>
+      {footer && <div className='px-3 pb-3'>{footer}</div>}
+    </div>
   )
</file context>
Suggested change
{footer && <div className='px-3 pb-3'>{footer}</div>}
{(footer || footer === 0) && <div className='px-3 pb-3'>{footer}</div>}

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor
❌ private-cloud · depot-ubuntu-latest-16 — run #23481 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

failed  1 failed
passed  1 passed

Details

stats  2 tests across 2 suites
duration  56.9 seconds
commit  58c6f98
info  📦 Artifacts: View test results and HTML report
🔄 Run: #23481 (attempt 1)

Failed tests

firefox › tests/environment-permission-test.pw.ts › Environment Permission Tests › Environment-level permissions control access to features, identities, and segments @enterprise

🗂️ Previous results
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #23481 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  47.1 seconds
commit  58c6f98
info  🔄 Run: #23481 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #23481 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  22 passed
skipped  1 skipped

Details

stats  23 tests across 14 suites
duration  1 minute, 16 seconds
commit  58c6f98
info  🔄 Run: #23481 (attempt 1)

Skipped tests

firefox › tests/onboarding-tests.pw.ts › Onboarding › New user connects via the single-page onboarding flow @oss

✅ oss · depot-ubuntu-latest-arm-16 — run #23481 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  36.5 seconds
commit  58c6f98
info  🔄 Run: #23481 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #23477 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  36.1 seconds
commit  490464c
info  🔄 Run: #23477 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-16 — run #23477 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  33.2 seconds
commit  490464c
info  🔄 Run: #23477 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #23477 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  34.3 seconds
commit  490464c
info  🔄 Run: #23477 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #23477 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  35 seconds
commit  490464c
info  🔄 Run: #23477 (attempt 1)

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 18 files

Confidence score: 5/5

  • utils.test.ts does not cover the turn-flag-off completion path, so regressions in its payload could go unnoticed. Add a disable case.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="frontend/web/components/experiments/CompleteExperimentModal/__tests__/utils.test.ts">

<violation number="1" location="frontend/web/components/experiments/CompleteExperimentModal/__tests__/utils.test.ts:77">
P2: The “turn flag off” completion path has no test here. Add a `disable` case so regressions in the payload for this user-visible decision are caught.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment on lines +77 to +84
it.each<[RolloutSelection, object]>([
[{ decision: 'control', variantId: 10 }, { rollout_decision: 'control' }],
[
{ decision: 'variant', variantId: 10 },
{ rollout_decision: 'variant', rollout_variant: 10 },
],
])('%o', (selection, expected) => {
expect(buildCompleteExperimentBody(selection)).toEqual(expected)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The “turn flag off” completion path has no test here. Add a disable case so regressions in the payload for this user-visible decision are caught.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At frontend/web/components/experiments/CompleteExperimentModal/__tests__/utils.test.ts, line 77:

<comment>The “turn flag off” completion path has no test here. Add a `disable` case so regressions in the payload for this user-visible decision are caught.</comment>

<file context>
@@ -0,0 +1,86 @@
+})
+
+describe('buildCompleteExperimentBody', () => {
+  it.each<[RolloutSelection, object]>([
+    [{ decision: 'control', variantId: 10 }, { rollout_decision: 'control' }],
+    [
</file context>
Suggested change
it.each<[RolloutSelection, object]>([
[{ decision: 'control', variantId: 10 }, { rollout_decision: 'control' }],
[
{ decision: 'variant', variantId: 10 },
{ rollout_decision: 'variant', rollout_variant: 10 },
],
])('%o', (selection, expected) => {
expect(buildCompleteExperimentBody(selection)).toEqual(expected)
it.each<[RolloutSelection, object]>([
[{ decision: 'control', variantId: 10 }, { rollout_decision: 'control' }],
[
{ decision: 'variant', variantId: 10 },
{ rollout_decision: 'variant', rollout_variant: 10 },
],
[{ decision: 'disable', variantId: 10 }, { rollout_decision: 'disable' }],
])('%o', (selection, expected) =>
expect(buildCompleteExperimentBody(selection)).toEqual(expected),
)

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Visual Regression

24 screenshots compared. See report for details.
View full report

This branch was successfully deployed

2 active deployments
Preview – flagsmith-frontend-preview — 58c6f98a Deployed Nov 19, 2026 by vercel[bot]
Preview – flagsmith-frontend-staging — 58c6f98a Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request front-end Issue related to the React Front End Dashboard

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant