Skip to content

feat: Spec getExperimentFlag under local evaluation - #21

Merged
gagantrivedi merged 2 commits into
mainfrom
feat/local-evaluation-experiment-flag
Nov 10, 2026
Merged

gagantrivedi merged 2 commits into
mainfrom
feat/local-evaluation-experiment-flag

Conversation

@gagantrivedi

Copy link
Copy Markdown
Member

Specs that getExperimentFlag records exposures under local evaluation, as it does under remote evaluation. Since Flagsmith/flagsmith#8532 the environment document carries each feature state's experiment, so a server-side SDK can tell whether an identity is enrolled without asking Flagsmith.

Changes

  • features/local__get_experiment_flag.feature (@local-evaluation @events): an enrolled identity is exposed to its variant; an identity a 0% rollout does not enroll, and one outside any experiment, are not.
  • New client step Create the SDK client with local evaluation and events enabled, taking the same arguments as Create the SDK client with events enabled. The runner asks for it when a scenario needs both; until now it silently dropped local evaluation.
  • a running experiment on feature … with value … takes an optional to N% of identities, defaulting to every identity as before.

Identity overrides aren't covered: self-hosted Flagsmith doesn't put them in the environment document.

Verified

Ran the whole spec against the Python SDK with Flagsmith/flagsmith-python-client#276 and the new client step: 12 passed. Against the SDK's main, the enrolled scenario fails with no exposure delivered.

Add a client step creating a client with both local evaluation and
events, which the runner asks for when a scenario needs both; until now
it dropped local evaluation. Let a running experiment enroll a given
percentage of identities, so that a scenario can run one enrolling none.
@gagantrivedi
gagantrivedi marked this pull request as ready for review November 5, 2026 08:42
@gagantrivedi
gagantrivedi requested review from a team as code owners November 5, 2026 08:42
@gagantrivedi
gagantrivedi requested review from khvn26 and kyle-ssg and removed request for a team November 5, 2026 08:42
@Zaimwa9
Zaimwa9 requested review from Zaimwa9 and removed request for khvn26 and kyle-ssg November 5, 2026 08:53
Comment thread runner/sdk_spec/flagsmith_admin.py Resolved
gagantrivedi added a commit to Flagsmith/flagsmith-go-client that referenced this pull request Nov 6, 2026
Since Flagsmith/flagsmith#8532, the environment document carries
`metadata.experiment` on feature states. The SDK dropped it when mapping
the document for the engine, so under local evaluation `Flag.Experiment`
was always nil and `GetExperimentFlag` never recorded an exposure.

### Changes
- `FeatureStateModel` reads `metadata.experiment`; the engine's
`FeatureMetadata` gains `Experiment`.
- The mapper copies it for environment and segment feature states.
Identity overrides carry none, as on the API.
- Flags built from evaluation results set `Experiment`.
- Locally evaluated environment flags carry no experiment, matching the
API, which only sends it for identity flags.
- Bump `engine-test-data` to v3.13.0 for its experiment cases
(Flagsmith/engine-test-data#64).
- README and doc comments no longer say exposures need remote
evaluation.

### Tests
`GetExperimentFlag` under local evaluation: an enrolled identity records
one exposure with its variant and experiment ID; an identity outside the
rollout, one with an identity override, and a feature with no experiment
record none. Environment flags carry no experiment.

Also ran Flagsmith/sdk-spec#21 against this branch: 12 passed.
Comment thread runner/tests/conftest.py Resolved
An experiment now states both its rollout and the share of enrolled identities
served the variant, so a scenario can enrol an identity into control. Event
expectations can match the scenario's experiment id with "<experiment>".

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go imo!

@gagantrivedi
gagantrivedi merged commit c4ed6b4 into main Nov 10, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants