Repository navigation
feat: Add the file-based override source - #529
Merged
Merged
Conversation
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
September 28, 2026 20:29
042fce2 to
0e55650
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
September 28, 2026 20:29
021778c to
2074b74
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
September 30, 2026 20:31
0e55650 to
c79f244
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
September 30, 2026 20:31
2074b74 to
1950d45
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 1, 2026 23:43
c79f244 to
352d360
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 1, 2026 23:43
1950d45 to
3e47045
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 3, 2026 01:18
352d360 to
9f4108c
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 3, 2026 01:18
3e47045 to
cfb781a
Compare
Member
Author
|
bugbot review |
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 13, 2026 21:02
9f4108c to
051cf7d
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 13, 2026 21:02
cfb781a to
ea4dc19
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 13, 2026 22:40
051cf7d to
0124739
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 13, 2026 22:40
ea4dc19 to
3de11ab
Compare
Member
Author
|
bugbot review |
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 16, 2026 21:34
0124739 to
224677f
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 16, 2026 21:34
bb5a108 to
5e42ebc
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 28, 2026 22:17
224677f to
2b471ae
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 28, 2026 22:17
5e42ebc to
314dea2
Compare
Member
Author
|
bugbot review |
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 28, 2026 22:40
2b471ae to
c5a69c9
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 28, 2026 22:40
314dea2 to
4bace41
Compare
Member
Author
|
bugbot review |
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 29, 2026 16:48
c5a69c9 to
70ada89
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 29, 2026 16:48
99b851f to
fac9ec8
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
October 29, 2026 17:16
70ada89 to
cad8f0b
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 29, 2026 17:16
fac9ec8 to
0415ec1
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
November 4, 2026 22:55
1cba56a to
d6b022b
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
November 4, 2026 22:55
28c2e59 to
a174ad2
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
November 4, 2026 23:09
d6b022b to
a52ba14
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
November 4, 2026 23:09
a174ad2 to
723eed3
Compare
keelerm84
approved these changes
Nov 5, 2026
kinyoklion
added a commit
that referenced
this pull request
Nov 6, 2026
<!-- TERMAG_PR_BODY_BEGIN -->
## Summary
Adds `ldclient.impl.integrations.files.filedata`, the file reading,
parsing, and merging logic that the file-based override source described
by the OVERRIDE specification is built on. The specification requires
ordered multi-file merging with duplicate keys handling, a reload that
keeps the last good data across a malformed edit and retries, debouncing
of change notifications, a polling change detector as well as a watching
one, and correct handling of a configured file that does not exist yet.
This module provides those pieces; the override source itself is added
in a later PR of the stack.
**The existing file data sources are not changed.** The FDv1 file update
processor (`Files.new_data_source`) and the FDv2 file initializer and
synchronizer (`Files.new_data_source_v2`, `datasystem.file_ds_builder`)
keep their current implementation and behavior in every respect: the
`flagValues` expansion (an on flag with fallthrough variation 0 and
version 1, evaluated with a `FALLTHROUGH` reason), the version fallback
that stamps version 1 on entries without one, the failure messages and
their levels, the FDv2 synchronizer's `OFF` state on a failed initial
load and its `INVALID_DATA` error kind, and the polling and watching
rules. A new test module, `test_file_data_sources_pinned_behavior.py`,
pins each of those behaviors so the override feature stays purely
additive. The baseline test files for the existing sources are
unmodified.
What the new module provides for the override source:
- Document parsing. The file name chooses the parser: a name that ends
in `.yaml` or `.yml`, in any case (including a file named just `.yaml`),
is YAML, which needs `pyyaml`, and any other file is JSON, parsed by the
standard JSON parser even when `pyyaml` is installed. The content is
never inspected to choose a parser. Files must be UTF-8; a leading byte
order mark is ignored, and UTF-16, UTF-32 or invalid UTF-8 fails to
parse. The format decides whether an empty file is valid: an empty or
white-space JSON file is not a document and fails to parse (`the file
has no content, which is not a JSON document`), as does JSON that is not
an object, such as `null`; YAML that is empty, only comments, `---`, or
`null` has no entries. A parse error names the format the name chose.
Flag and segment definitions are decoded into the model classes while
the file is read, so an invalid definition fails that load. The map key
is the key of a flag or segment definition for every purpose: the
definition's own key is set to it, and duplicates are detected by it, so
evaluations report the flag by that key. A definition may omit its
version, which is filled in as 1.
- Merging. Files are combined in the configured order. The duplicate
keys handling is `fail` (the load fails) or `ignore` (the first file's
entry is kept). The result records how many entries each file supplied.
- `Reloader`. Serializes reloads, debounces change signals with a settle
window that each signal extends, keeps the last good data by not
applying a failed load, retries a failed load after a bounded delay so a
file observed mid-write recovers without another notification, reports
an identical failure once, skips an application whose files are
byte-identical to the last applied ones, and applies a success after a
failure even when the contents did not change. A configured file that
does not exist can contribute no data instead of failing the load. A
load reads, parses and classifies every configured file, even after one
fails, then fails with the first failure (logging the others at Debug
level), then holds files back (below), then merges. The skip-unchanged
digest covers each path, whether its file exists, and its bytes, since
the same bytes in a file of another name can be in another format.
Closing does not wait for an in-flight reload. An exception raised by
`apply` or `on_error` is reported like any other load failure and
retried, and nothing from that load is remembered.
- In-place rewrites. Applications and editors rewrite files in place:
they truncate the file, then write the new content, and a load in
between finds it empty. A file whose content is in effect and that a
load finds empty, white space only, or parsed to nothing (YAML with only
a comment or `---`) is held back: the load keeps the current data, logs
at Debug level and reports nothing, and the other files' changes wait
too, since a load applies all files or none. `{}` is a document and is
applied at once. The hold works as follows:
- The file is held until it has stayed so, unwritten, for the retry
delay. Its state (modification time and size, read through symbolic
links) is compared only with itself, never with the clock, and a change
starts the delay over.
- The clock is read after the file's own read, so a slow read of another
file does not change its delay.
- The hold asks for a retry when the first held file is due, plus a
tenth of the retry delay; the write's change signal starts a load
sooner.
- Each file is held for at most five retry delays after its first hold,
so a file kept written while empty, or files emptied in turn, are
applied in the end. A file's bound starts over only at a load that holds
nothing back, whatever that load's outcome.
- Content, even content that cannot be parsed, or a missing file ends a
file's empty state; a failed read keeps it.
- A file whose content is not in effect (a placeholder that was always
empty, a file whose deletion was applied and that is then created empty,
a file whose emptiness was applied) is never held, and with retries off
nothing is held. A file that is deleted and created empty again before a
load applies the deletion, in the same debounce window or while another
file fails or is held, still has its content in effect, so it is held
like a file rewritten in place.
- After the delay, empty YAML is applied with no entries, and empty JSON
fails to parse and keeps the data.
- `Poller`. Compares modification time and size on an interval, so a
same-size rewrite and a same-time size change are both detected, as are
files that appear or disappear.
- `Watcher`. Uses the `watchdog` package. It watches the directory of
each file so an absent file is picked up when it appears, matches the
destination of a move event so a file written by rename is detected,
retries a directory that does not exist yet, and reacts only to
notifications that can change a file's content or presence (the watchdog
inotify mask also reports a file being opened or read). A failed
observer start is retried every second, and a watched directory that is
deleted is watched again once it exists, with one change signaled. A
configured file that is a symbolic link is followed: the directory of
the link's target is watched for the target's path, and the links are
resolved again after every reported change, after a retry sets up
watches, and when a directory that holds a target is deleted, as a
Kubernetes ConfigMap update does. Watching does not follow a link
further along the chain that is pointed elsewhere while its old target
stays, a configured directory that is itself a link and is pointed
elsewhere, or a configured directory that is renamed and replaced; the
documentation points such deployments, like a `current` link to a
release directory, to polling, which is the default. A link that cannot
be resolved, for example because its target's directory does not exist
yet or was deleted, is resolved again on the retry interval until its
target is watched, and one change is then signaled. A relative link is
joined to the real path of the directory that holds it, so a `..` in it
is right when a directory on the way is itself a link.
Not changed: a read that sees a YAML file cut off part way can still
parse, with fewer entries or a different value, and is applied until the
next reload; the override source's docs recommend replacing a YAML file
in one operation.
Tests cover parsing (format by name, empty and non-object JSON, empty
and null YAML, byte order marks and other encodings), merging, loading,
the reloader (initial load, failure retention, debounce coalescing and
window extension, retry with and without further signals,
identical-failure and first-failure reporting, skip-unchanged and
recovery, serialization of concurrent reloads, close semantics, worker
thread lifecycle), the hold (on an injected clock, with a one-hour retry
delay so no real retry fires: the exact delay, restarts on a state
change and through a symbolic link, quick successive rewrites, files
emptied together and in turn, the per-file bound and when it starts
over, failures before and after a held file, slow reads, the retry each
hold asks for; and on the real clock, that the retries fire), the
poller, and the watcher (including symbolic links to files in the same
and in another directory, a link that is replaced, a ConfigMap-style
update, a relative link in a directory reached through another link, and
a target directory that appears late or is recreated), plus the pinned
behavior of the existing sources. The reloader tests rewrite their files
in place with a 30 ms pause, through `ldclient/testing/file_util.py`.
The in-place rewrite handling was reviewed in eight rounds as #560 (now
closed, its changes moved here and into #529): multi-agent and
adversarial reviews of each round, including a randomized end-to-end
stress of about 384 rewrites with no empty snapshot. The final mutation
check ran 59 mutants against `test_filedata.py` and
`test_file_override_source.py`: 54 fail a test, and the 5 survivors are
equivalent in this code (listed in #560). Both test files ran 10 times
in a row with no failure.
Python CI runs for this PR. The PRs above it in the stack get no CI;
they were checked locally.
[SDK-3250](https://launchdarkly.atlassian.net/browse/SDK-3250)
[SDK-3250]:
https://launchdarkly.atlassian.net/browse/SDK-3250?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
<!-- TERMAG_PR_BODY_END -->
<!-- termag: box=i-0575d1c54853058d7 project=Overrides -->
<sub>Opened from termag</sub>
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Overview**
> Introduces **`ldclient.impl.integrations.files.filedata`**, shared
logic for reading JSON/YAML flag/segment files, merging them in order
(`fail` vs `ignore` on duplicate keys), and keeping data fresh via
**`Reloader`** (debounced reloads, failure retention, retries,
skip-unchanged digests, and **hold-back** when an in-effect file is seen
empty mid in-place rewrite), plus **`Poller`** and **`Watcher`**
(watchdog, symlinks, ConfigMap-style dirs) change detectors.
>
> Adds **`ldclient/testing/file_util.py`** for simulated in-place
rewrites and large test suites: **`test_filedata.py`** for the new
module and **`test_file_data_sources_pinned_behavior.py`** to lock
FDv1/FDv2 behavior without changing those implementations in this diff.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
4c11d73. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
kinyoklion
marked this pull request as ready for review
November 6, 2026 17:43
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
November 6, 2026 17:53
a52ba14 to
36ceb3f
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
November 6, 2026 17:53
723eed3 to
26d9877
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 36ceb3f. Configure here.
3 of 4 tasks
Carries the override-affected marking on the evaluation event input, so the event processor keys on the marking alone and never reads the evaluation reason. An evaluation marked as override-affected produces no individual feature event and no debug event, even when the flag requests them, and it is counted in summary events like any other evaluation. The marking is part of the summary counter key, so override-affected and ordinary evaluations of the same flag, variation, and version accumulate into separate counters, and a counter that aggregates marked evaluations carries the overrideAffected marker, present only when true, like the unknown marker. The evaluator passes each prerequisite record's own marking to its event, so a marked prerequisite record produces no individual event while an unaffected prerequisite inside a marked evaluation is recorded as usual. The client passes the top-level marking to the evaluation event, keeps the marking on the reason when a migration evaluation replaces it with a wrong-type error, and presents an override-affected flag in the all-flags state with trackEvents and trackReason false and no debugEventsUntilDate, so a consumer bootstrapped from the state sends no individual events for it. The async client mirrors these changes. The OVERRIDE specification vector runner now also checks the marking each evaluation contributes to its summary counter.
…ons after the loop closes When an evaluation raises and the flag definition came from the override layer, the EXCEPTION reason carries the override indicator, the default event record is marked so no individual event is produced, and the all-flags state presents the flag with tracking off. The async data system drops a flag change notification that arrives after the event loop closed instead of raising on the reload thread.
Adds ldclient.integrations.overrides.FileOverrideSourceBuilder, the file-based override source described by the OVERRIDE specification. The source reads one or more JSON or YAML files in the file data source document format, with optional flags, flagValues, and segments members, and supplies each successful load to the SDK's override sink as a full snapshot. Files are combined in the configured order. The duplicate keys handling is fail by default, which rejects the reload and keeps the previously loaded overrides, or ignore, which keeps the first file's entry. A configured file that does not exist contributes no overrides, so a file can be created later and deleting a file removes its overrides. A file that exists but cannot be read or parsed fails that reload, the last good overrides stay in effect, the failure is logged, and the load is retried after a bounded delay and on the next detected change. Change detection is one of two modes: polling, the default, examines the files once per second by default with a one second minimum, and watching reacts to file system notifications through the watchdog package. Watching without the watchdog package and a builder with no paths are construction errors. The initial load completes during client construction. Every applied change is logged at Info level with the overrides in effect and what each configured file supplied.
…l of the file override source Change detection is set up before the initial load, so an edit made while the files are first read is picked up. Close stops the reloader even when the change detector fails to close. The builder rejects a poll interval that is not a number or not finite.
…ntries as duplicate keys A path listed more than once would make every entry of that file a duplicate of itself: under the default fail handling the load failed, and under ignore the first listing won by accident. The builder now drops the repeats, keeps the first occurrence in its place in the order, and logs a warning once at build time. This is the behavior the Node SDK has.
…t on Windows The builder dropped repeated paths by exact string match, so on Windows a path listed twice in different case survived as two paths, and every key of that file failed as a duplicate of itself under the default fail handling. Paths are now compared with os.path.normcase, which ignores case on Windows and compares exactly elsewhere. The first spelling keeps its place in the order.
… SDKs A flagValues entry becomes a flag without client-side availability, so an all-flags state for client-side SDKs leaves it out even when the flag it overrides is client-side. A full definition with clientSide is the way to override such a flag. A test pins both.
The override source keeps the overrides in effect while it finds a file empty during an in-place rewrite. A YAML file without the pyyaml package is now a configuration error when the source is built, since the file name chooses the format and such a file can never load. The builder docs say that the file name decides the format, that files must be UTF-8, that files can be rewritten in place, how to remove a file's overrides, that an empty JSON file is not a valid document, and that a YAML file is best replaced in one operation because a partly written one can still parse. End-to-end tests rewrite the files in place, slowly, in watching and polling modes, for JSON and YAML, and check that neither the source nor a client evaluating the flag ever goes without the override.
Watching follows edits to the configured files and to the targets of configured links, including a ConfigMap update, but not a link further along the path or a configured directory that is pointed elsewhere or replaced. The change detection documentation now says so and points such deployments to polling.
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
November 10, 2026 22:24
36ceb3f to
9a582aa
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-events
branch
from
November 10, 2026 22:24
26d9877 to
1588f31
Compare
Base automatically changed from
rlamb/overrides-python-events
to
feat/overrides
November 10, 2026 23:08
kinyoklion
added a commit
that referenced
this pull request
Nov 10, 2026
<!-- TERMAG_PR_BODY_BEGIN --> **Requirements** - [x] I have added test coverage for new or changed functionality - [x] I have followed the repository's [pull request submission guidelines](../blob/main/CONTRIBUTING.md#submitting-pull-requests) - [ ] I have validated my changes against all supported platform versions - [x] Sync/async parity: matching changes made to `async_*` siblings (or N/A). N/A: `LDClient` and `AsyncLDClient` use the same `FileOverrideSourceBuilder`. **Related issues** - Bugbot finding on #529: #529 (comment) **Describe the solution you've provided** `FileOverrideSourceBuilder` accepted one path as a `str`, or a list of paths. A single `pathlib.Path`, or any other `os.PathLike`, was passed to `list()`. A path object is not iterable, so the constructor raised `TypeError: 'PosixPath' object is not iterable`. The Bugbot finding describes a different failure, a path split into its components that fails every reload, but that cannot happen. The constructor raises before a client exists. The constructor now treats a `str` or an `os.PathLike` as a single path, and converts every path with `os.fspath`. A path that does not name a string path raises `TypeError` from the constructor, with a message that names the value. That includes `bytes` (given alone, in a list, or named by a path object), an `int`, and `None`. `bytes` and `bytearray` count as single values, so a bare `bytes` path is named whole instead of being read as a sequence of integers. The type hint is `Sequence` rather than `List`, so that type checkers accept a list of path objects but not a set, whose order is not fixed. The docstring says that paths may be path objects. Tests cover: - a single path object; - a list of path objects, which already worked; - a path object whose string form differs from its path, as with `os.DirEntry`; - a source that loads a file given as a path object; - the rejection of each of the values above. Each of six broken variants of the constructor fails at least one of these tests, including the original code and a variant that converts with `str()` instead of `os.fspath`. **Describe alternatives you've considered** Leaving the builder to accept only strings, as its type hint said. `pathlib` is the usual way to build paths, a list of path objects already worked, and the error did not say what was wrong. **Additional context** This PR is stacked on #530, the top of the overrides stack, so that the approved PRs below it do not change. Retarget it as the PRs below it merge. The review of this change found three existing gaps that this PR leaves alone: - An empty path resolves to the working directory, and its reads keep failing. - In polling mode, a path that contains a NUL character makes the source fail to start, so the client runs without overrides. In watching mode, the reload fails and keeps retrying. - The file data sources treat a tuple of paths differently from a list. Testing: `mypy ldclient`, `isort --check` and `pycodestyle` pass locally. The test suite passes (2,111 tests). It was run without the Redis, DynamoDB and Consul store integration tests, which need servers that were not running. CI runs only for PRs based on `main` or `feat/**`, so it will not run on this PR until it is retargeted. <!-- TERMAG_PR_BODY_END --> <!-- termag: box=i-0575d1c54853058d7 project=Overrides --> <sub>Opened from termag</sub> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Fixes a constructor bug where a **single** `pathlib.Path` (or other `os.PathLike`) was passed to `list()`, causing `TypeError: 'PosixPath' object is not iterable`. > > `FileOverrideSourceBuilder` now wraps a lone `str`, `bytes`/`bytearray`, or `PathLike` as one path, normalizes every entry with `os.fspath()` (so objects like `os.DirEntry` use their filesystem path, not `str()`), and stores string paths. Invalid inputs (`bytes` paths, ints, `None`, path objects that do not resolve to a `str`) fail fast with an explicit `TypeError`. Type hints use `Sequence` plus `PathLike` so callers can pass path objects while unordered collections like `set` are discouraged. > > New tests cover single/list path objects, `__fspath__` behavior, invalid types, and loading overrides from a path object. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 154b5b8. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
This is the fifth step of the flag overrides port described by the OVERRIDE specification. It is based on the events branch because the phases are stacked; retarget to
feat/overridesonce that branch merges.ldclient.integrations.overrides.FileOverrideSourceBuilderis the public entry point. It is passed todatasystem.ConfigBuilder.overrides(...)and builds an override source that reads one or more local files in the file data source document format (optionalflags,flagValues, andsegmentsmembers, JSON or YAML) and supplies each successful load to the SDK's override sink as a full snapshot.flagValuesentries expand into full flag definitions, so the layer holds only full entities; as the specification describes for a value-only override, the expansion is a flag that is on and serves its single variation as the fallthrough, so the evaluation reason kind isFALLTHROUGH. The existing file data sources keep their current behavior, including their ownflagValuesexpansion.Behavior:
.yamlor.ymlis YAML, which needs thepyyamlpackage, and any other file is JSON. Files must be UTF-8. A YAML file configured withoutpyyamlis a construction error, since it could never load. An empty JSON file is not a valid document, nor is JSON that is not an object, such asnull; an empty YAML file has no entries.duplicate_keys_handlingisfailby default, which rejects the reload and keeps the previously loaded overrides, orignore, which keeps the first configured file's entry.{}also removes a file's overrides.change_detectionis one of two alternatives.polling(the default) compares modification time and size on an interval, one second by default with a one second minimum; a lower interval is raised to the minimum with a warning, and a value that is not a finite number, or longer than the runtime can wait, is a construction error.watchinguses thewatchdogpackage and is a construction error when the package is not installed. Notifications are debounced so a burst from one edit produces one reload.start, which the data system calls during client construction, so an override present at startup takes effect from the first evaluation.Flag overrides in effect: 2 flags, 1 segment (/etc/ld/a.json: 2 flags, 1 segment; /etc/ld/b.json: absent), orFlag overrides: none in effect (...).change_detectionorduplicate_keys_handlingvalues raiseValueErrorwhen set. This is stricter than the Go builder, which treats an unrecognized duplicate keys handling asfail. A path listed more than once is logged at warning level and the repeats are ignored; the first occurrence keeps its place in the order. Paths are compared the way the platform compares file names (os.path.normcase): without regard to case on Windows, exactly elsewhere.The module is added to the API reference (
docs/api-integrations.rst) and every public docstring carries the experimental note. The public docs say what an application needs (the format rule, UTF-8, in-place rewrites, how to remove overrides, replacing YAML in one operation); the hold's mechanics are described in the internal docstrings and here.Tests cover the builder validation and defaults (including a YAML file without
pyyaml), synchronous initial load, YAML, multi-file order and duplicate handling, absent files appearing and disappearing, the Info log lines, both change detection modes, last-good retention across a malformed edit in both modes, the automatic retry with no change signal, close semantics, and an end-to-end run throughLDClientwhere a file is added, changed, and cleared with{}while the client never receives LaunchDarkly data. In-place rewrite tests, for JSON and YAML in watching and polling modes, rewrite the file slowly (truncate, pause, write) and check that the source never supplies a snapshot without the override and that a client evaluating the flag throughout serves the old value and then the new one, never the default; they also cover successive rewrites at the source's own timing, an emptied JSON file keeping the overrides, and an emptied YAML file removing them after the retry delay.The in-place rewrite handling was reviewed in eight rounds as #560 (now closed, its changes moved here and into #525). The final mutation check ran 59 mutants against
test_filedata.pyandtest_file_override_source.py: 54 fail a test, and the 5 survivors are equivalent in this code. Both test files ran 10 times in a row with no failure, and the harness'sflag overridessuite passes against the sync and async test services at the top of the stack.SDK-3250
Opened from termag
Note
Overview
Adds an experimental file-based flag override source so operators can force flag/segment values from local JSON or YAML files (same format as the file data source) via
FileOverrideSourceBuilder, wired throughdatasystem.ConfigBuilder.overrides(...).The source loads an initial snapshot synchronously at client startup, merges multiple files with configurable duplicate-key handling, and reloads on change using polling (default) or optional watchdog watching. It tolerates missing files, parse failures (keeps last good overrides with retry), and in-place rewrites without briefly clearing overrides. Applied changes are logged at Info level. API docs for
ldclient.integrations.overridesare added; behavior is covered by extensive integration tests including end-to-endLDClientscenarios.Reviewed by Cursor Bugbot for commit 9a582aa. Bugbot is set up for automated code reviews on this repo. Configure here.