Skip to content

feat: Record exposures under local evaluation - #276

Merged
gagantrivedi merged 7 commits into
mainfrom
feat/local-evaluation-experiment-exposure
Nov 6, 2026
Merged

gagantrivedi merged 7 commits into
mainfrom
feat/local-evaluation-experiment-exposure

Conversation

@gagantrivedi

Copy link
Copy Markdown
Member

Since Flagsmith/flagsmith#8532, the environment document carries metadata.experiment on feature states. The SDK dropped it when mapping the document for the engine, so under local evaluation Flag.experiment was always None and get_experiment_flag never recorded an exposure.

Changes

  • FeatureStateModel gains metadata; FeatureMetadata gains experiment.
  • The mapper copies metadata.experiment for environment and segment feature states. Identity overrides carry none, as on the API.
  • Flag.from_evaluation_result sets experiment. The engine already returns the winning feature context's metadata, so no engine change is needed (flagsmith-flag-engine 11.1.1).
  • Locally evaluated environment flags carry no experiment, matching the API, which only sends it for identity flags.

Tests

get_experiment_flag under local evaluation: an enrolled identity records one exposure with its variant and experiment ID; an identity outside the rollout, one with an identity override, and a feature with no experiment record none.

Follow-up

Bump the engine-test-data submodule once Flagsmith/engine-test-data#64 is released.

Carry each feature state's experiment metadata from the environment
document into the engine context, and set it on flags built from
evaluation results, so get_experiment_flag records exposures under
local evaluation as it does under remote evaluation.

Environment flags carry no experiment, matching the API, which only
sends experiment metadata for identity flags.
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Explain Change →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 30 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: f3cd9c00-e5d5-439a-b250-776f33318cd6
Commits

Reviewing files that changed from the base of the PR and between 0b39bc8 and a6e721b.

Files ignored due to path filters: 1
  • poetry.lock is excluded by !**/*.lock
Files selected for processing: 3
  • pyproject.toml
  • tests/engine_tests/engine-test-data
  • tests/test_engine_test_data.py

No actionable comments were generated in the recent review.

Walkthrough

The change adds experiment metadata to feature-state and feature-context types. The mapper and flag conversion carry experiment details into flags. Local evaluation removes experiment metadata from environment-level flags. Added tests cover experiment metadata mapping, exposure tracking, identity overrides, and flags without experiments.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: Minimal · previous review 0b39b

The change carries experiment metadata into locally evaluated identity flags while excluding it from environment flags. No actionable merge-blocking issue remains; merge after normal checks.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 406a3009-9641-4009-9597-2c74d0451833
Commits

Reviewing files that changed from the base of the PR and between 344b9f7 and 0b39bc8.

Files selected for processing: 9
  • flagsmith/api/types.py
  • flagsmith/local_evaluation.py
  • flagsmith/mappers.py
  • flagsmith/models.py
  • flagsmith/types.py
  • tests/conftest.py
  • tests/test_flagsmith.py
  • tests/test_mappers.py
  • tests/test_models.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@codspeed

codspeed Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 15 untouched benchmarks


Comparing feat/local-evaluation-experiment-exposure (a6e721b) with main (344b9f7)

Open in CodSpeed

@gagantrivedi
gagantrivedi marked this pull request as ready for review November 4, 2026 08:49
@gagantrivedi
gagantrivedi requested a review from a team as a code owner November 4, 2026 08:49
@gagantrivedi
gagantrivedi requested review from emyller and removed request for a team November 4, 2026 08:49

@Zaimwa9 Zaimwa9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@gagantrivedi
gagantrivedi merged commit 4a25d50 into main Nov 6, 2026
9 checks passed
@gagantrivedi
gagantrivedi deleted the feat/local-evaluation-experiment-exposure branch November 6, 2026 03:14
gagantrivedi pushed a commit that referenced this pull request Nov 6, 2026
🤖 I have created a release *beep* *boop*
---


##
[6.5.0](v6.4.0...v6.5.0)
(2026-11-06)


### Features

* Record exposures under local evaluation
([#276](#276))
([4a25d50](4a25d50))


### Dependency Updates

* update dependency flagsmith-flag-engine to v11.1.2
([#277](#277))
([dbf4554](dbf4554))


### Other

* Let Renovate keep updating spec bumps the webhook handler commits to
([#279](#279))
([f47856d](f47856d))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
gagantrivedi added a commit to Flagsmith/flagsmith-java-client that referenced this pull request Nov 6, 2026
Part of Flagsmith/flagsmith-private#325. Java counterpart of
Flagsmith/flagsmith-python-client#276, Flagsmith/flagsmith-go-client#246
and Flagsmith/flagsmith-nodejs-client#318.

`getExperimentFlag` only recorded exposures under remote evaluation. The
environment document carries `metadata.experiment` on feature states,
but local evaluation dropped it, along with the variant.

## Changes

- Map the multivariate option `key` into the evaluation context, and
carry `variant` from the evaluation result onto `Flag`. Locally
evaluated identity flags now report their variant.
- Map the feature state's `metadata.experiment` into the context's
feature metadata, and carry it onto identity flags.
- Environment flags carry no experiment: it is removed once, when the
environment evaluation context is built, so `getEnvironmentFlags` does
no extra work per call.
- `getExperimentFlag` records no exposure when the flag has no variant,
as the other SDKs do.

## Tests

- Local evaluation: an enrolled identity records one exposure with the
variant and `experiment_id`; outside the rollout, an identity override,
and a feature without an experiment record none.
- Environment flags carry no experiment under local evaluation.
- Remote: an enrolled flag without a variant records no exposure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants