feat(client-contract): add the contract the DFO clients follow by jm-rivera · Pull Request #11 · DevelopmentFinanceObservatory/spec-kit

jm-rivera · 2026-10-21T15:13:34Z

Stacks on #10, whose registry rows map the datasets refusals the recorded scenarios carry.

This adds dfo-spec-kit-client-contract under python/client-contract, the shared rules, scoped to client capabilities, and the data that pins them. README.md states each rule: how answers, choices and gaps read, how fetches, long builds, polls and retries behave, how the key resolves, which error each refusal raises and which trace id it carries, and what a dataset ref holds. The package data holds 118 scenarios (31 recorded against dfo-apis at 635943e03 and 87 synthetic edits of recorded ones), the OpenAPI document captured from that same app, strict-fields.json with the 29 required and 8 enum members a client acts on, actions.json with each refusal code's next step, client-codes.json, warning-codes.json, proxy-codes.json, the dataset-ref schema, and the key-lookup vectors under rules/. The Python and TypeScript clients in DevelopmentFinanceObservatory/sdk#1 and DevelopmentFinanceObservatory/sdk#2 fetch this data at their pinned commit, generate code from it and replay every scenario. The API gate validates recorded exchanges against the current schemas and registry; scheduled live SDK journeys check deployed behavior.

tools/capture_client_contract.py builds the served document (openapi), records the scenarios against a running app (record), and with record --offline derives every outcome again from the recorded exchanges. tools/validate_client_contract.py derives each scenario's expected outcome from the rules and replays it under the recorded base URL and under a second origin, so a client that hard-codes the host fails. CI runs the validator and the tools' tests. Cache rules apply to clients implementing disk caching, and synchronous Python platform DNS may exceed the documented timeout bounds. Atomic content-addressed artifact writes preserve byte identity; concurrent manifest writes can lose cache availability.

Clients treat malformed optional fields as absent, pace refused polls using Retry-After, share the build retry budget, and attribute errors to the response's trace id. An assumptions item reads only when its kind, binding_ref and text are strings, and an operations item only when its verb and phrase are strings, as the served Assumption and Operation schemas require. A read item holds its schema's members alone. The resubmit echo posts exactly the items read, built from the members read (choose-echo-drops-malformed-items). The item rule checks JSON types alone, so the server still checks the echo against its value constraints, such as the verb enum and the length of phrase. A candidate is chosen by index or by label, and a label that matches no candidate or several, or a boolean part or option, raises choice_invalid with no request. An integer is a JSON number with no fractional part that is not a boolean, as JSON Schema reads it, since a JavaScript client cannot tell 1.0 from 1. A rebuild during ref recovery must answer the ref's spec_id, as the snapshot step must, or the client raises contract_violation (fetch-ref-expired-rebuild-other-spec). A ref's definition holds the schema_version, spec_id and identity the served CanonicalDefinition requires and no other member, and four rejected examples under refs/ pin it. The format takes any integer schema_version, and a manifest definition without one raises contract_violation (fetch-spec-manifest-without-schema-version), so a client never writes a ref it would reject. The README states the download rule for a browser, which cannot read a manual redirect: the TypeScript client follows it and checks that the final URL is on the API's origin or the storage host. The scenario format names no runtime, so that client's own tests check this.

The capture script and the validator read and write a binary body only through a bare file name in the scenario's directory, and refuse an absolute path, a name that leaves that directory, or a symlink before touching the file. Both share one stored_version function for the ref recovery order.

Testing: the validator passes (101 action rows, 17 client codes, 29 required and 8 enum pointers, 118 scenarios). The tools' 502 tests and the package's 78 tests pass. record --offline at this head rewrites all 118 scenarios with no byte changed. Both SDK clients replay every scenario at this head (DevelopmentFinanceObservatory/sdk#5). The composed API schema and registry checks passed at 1a320c5 with #66, #67, and #68 present, before the six new synthetic scenarios.

dfo-spec-kit-client-contract (import dfo_client_contract) holds what the DFO Python and TypeScript clients generate from and what dfo-apis checks its app against: the served OpenAPI document, the strict fields clients act on, the next step for every active refusal code, the scenario and DatasetRef record formats, and rule vectors for the credentials file, key format and cache directory. tools/validate_client_contract.py checks the package data against the code registry and the served document.

tools/capture_client_contract.py captures the dfo-apis OpenAPI document in process and records scenarios against a base URL, redacting signed download queries. 14 scenarios are recorded on dev and 11 are derived from them: retries, proxy refusals, an unknown part status and enum value, a renamed strict field, and place choices in one and two parts.

The validator replays each record's steps over its exchanges and fails when expect differs from the outcome the bodies and actions.json imply: error class, gap kinds, option and candidate counts, and a dataset's members, complete and missing. The capture tool imports the same rules. The validator also tests a strict member's nullability with the schema validator itself, refuses an absolute URL or Location header that keeps its query, and lets a synthetic record name its parent's body file. The capture refuses untracked files and an environment out of step with the lock. door_not_implemented now reports and method_not_allowed upgrades.

Nineteen scenarios, four recorded on dev and fifteen derived from recorded parents, cover a timed-out and an unsupported part, an unknown answer kind, every choice refusal, a choice named by a mention, a several-part candidates choice, a candidates part left alone beside a chosen one, a placeholder for a part with no restatement, and the SearchWarning rule. An unchosen candidates part posts its candidate set in the server's order. The validator checks every Search request a record holds against the input the rules derive, and pins the part column of a stacked dataset.

client-codes.json lists each code a client raises when no problem document carries one, with the error class it raises. The codes are disjoint from actions.json and the codes registry, and every scenario error that no refusal in its exchanges carries names one of its class. record --offline re-derives a recorded scenario's expect from its own exchanges.

…rios Ten scenarios cover a frame-layout build, a 504 twice, an unresolvable member, a strict data warning, a digest mismatch, a resubmit answering another spec, a frame fetch of several readings, and an expired ref that rebuilds the same version, lands on a newer one, or cannot be rebuilt. Six more are defined and wait for a served document that declares the build's 202 or the file selector. The validator derives poll waits, resubmissions, the operation id and the newest version, and a refusal with no known code maps 408 to DfoTimeoutError and 410 to NotFoundError.

The server's scope replaces the spec's entity binding whole, so a client starts from the spec's bound roles, replaces the roles the caller names, drops a role bound to null, and posts the whole map. The validator checks every build's input.scope against that rule, and two dev recordings rebind and drop the origin of a two-role spec.

The identity entry a client posts to name a built spec carries the spec's entity_groups for each role the build binds to the same entities as the spec's definition. A window or facets narrowing keeps every group, and a role the narrowing rebound or dropped loses its group, so the sentence names "the 44 Least Developed Countries" where the group still holds and lists recipients where it does not. validate_client_contract.py derives the kept groups from the held spec and checks every posted identity entry against them. Two new recordings cover a narrowed build and a rebound origin, bringing the contract to 58 scenarios.

Every recorded scenario and the served OpenAPI document now come from one dfo-apis commit, 635943e03, which composes dev with the search choice options, answer statements and datasets client contract branches. The served document lists the four deployed surfaces: Search, Datasets, Graph and Presentation. The contract holds 67 scenarios, 29 recorded and 38 synthetic. The long builds record against the composed app: mint-async-build, operation-resume and their synthetic children, and the expired file redirected through the file selector. Signed download URLs read as storage.googleapis.com URLs with the signature redacted, since a production download is cross-origin; origin.base_url states the base URL the client was given. The rules the recording pins: - Every candidate pick, in a one-part answer too, posts the reading's identity to Search as a dataset_resubmit entry echoing reading_assumptions[spec_id] as its assumptions, and the picked answer and its sentence come from Search's response. - A client asks for the newest stored version with GET /datasets/{id}/versions?limit=1; only not-found reads as none. - A mint takes only a layout. - A required member that admits null must be present. - capability_unavailable keeps the fix_request action, pinned by fetch-spec-capability-unavailable.

The validator now refuses what a client replay would reject. A repeat of a refused request counts as a retry only while the refusal is retryable and the budget allows: two retries, one for a build cut off by a 504. A POST record carries its body, and every build's posted body is compared whole with the spec it builds. A poll names the operation the client holds, a download follows the files route's Location exactly, and synthetic exchanges go through the same problem code check as recorded ones. served/openapi.json must hold the Search, Datasets, Graph and Presentation surfaces and nothing else. A recorded scenario holds only exchanges the server made: the capture refuses to write one with a canned answer, and no synthetic derivation can make a request. choose-option-beside-definite and fetch-ref-newest-version are recorded at 635943e03, and fetch-ref-expired-not-reproducible derives from the latter through a new newest_version step. The README states every rule the validator derives, including the retry, warning, long-build and choice refusal rules. 68 scenarios, 31 recorded and 37 synthetic.

…hapes Outcomes gain retry_waits, the seconds before each retry, and warnings, the codes API responses issued. A retry waits Retry-After, else 0.5 s doubling to 8 s, and a record that ends with retries left is unfit. The files route follows the ordinary retry rules; a failed signed fetch gets one more round, and a second network failure raises as itself. A relative Location resolves against the files route. New tables: proxy-codes.json for the website proxy's error codes and warning-codes.json for the warnings a client issues itself, each with its schema. client-codes.json rows state retryable, and key_empty joins them. The validator compares the whole Search body, checks that recorded commits match served/SOURCE.json, refuses any absolute URL off the storage host, and reads facets: {} and entities: {} as no narrowing. New scenarios pin a signed URL refused twice, a relative Location, a failed operation with no error, an unreadable versions page, a 504 carrying Retry-After, a Sunset-only warning, a part status that is not a string, a definition naming another spec_id, an option without a data_query, a Search response with no JSON body, and an empty narrowing. Steps no longer take timeout; the README states the clients' timeouts. 79 scenarios, 31 recorded and 48 synthetic.

…usal and choice rules A manifest artifact's url must sit on the API's origin, relative or absolute; any other origin, the storage host's included, raises contract_violation before any request. Only a files route's Location may sit on the storage host. The strict check walks down from the body's root through strict members alone, so a spec nested where no rule reads it raises nothing. Spec.resolved, SpecDefinition.identity and DefinitionModel.identity join strict-fields.json, which now holds 28 required pointers. A resolution with empty parts raises contract_violation at /parts. A build's POST shares the two retries, and after a 504 cut-off it is posted once more only: 504 then 503 raises the 503. A refused poll changes no pacing, and a non-retryable one raises at once. A problem document without a code keeps its retryable and trace_id. A 401 resolves the key again. request() refuses a path that is not a path on the base URL. A one-part option whose data_query holds other than one part raises ContractError in a one-part answer too, and the validator derives it. 22 new synthetic scenarios pin these rules, the header warnings, the separate and operations echo, the unsupported placeholder and three choice_invalid cases. 101 scenarios, 31 recorded and 70 synthetic.

Every error outcome carries trace_id. An error that describes a response carries that response's trace id: the body's trace_id when the body is an object holding one, else the x-dfo-trace-id header, else null. Refusals, failed operations, VersionUnavailableError, unexpected_response, contract_violation on a body, answer_declined, answer_incomplete and a strict SearchWarning or DataWarning describe a response. An error that describes the caller's input, such as choice_invalid, invalid_argument or a key-lookup reason, and one raised on a storage-host response carry null. The capture tool keeps x-dfo-trace-id on recorded responses and writes trace_id on every error outcome. search-refused-without-code pins the body over the header, search-unauthorized the header on a bodiless refusal, answer-timed-out an incomplete answer taking the Search response's header, and choose-refused-shared-label a choice_invalid that stays null beside one.

…nd pin malformed members Every recorded scenario and served/openapi.json are recorded again at dfo-apis 635943e03, now keeping x-dfo-trace-id; the capture refuses an API response without it unless the response is Cache-Control: public, which the server sends untraced. The served document is unchanged. The validator replays each scenario under its recorded base URL and under https://replay.example.org with the recorded origin replaced in bodies and headers, so a record that holds only under its own host fails. A refused poll waits its Retry-After whatever its status, else the ordinary pacing. The trace id is a body's top-level trace_id, else the header; diagnostics.trace_id is not read. A signed fetch answering anything but 200 fails its round, and a files route answering a 2xx other than 200 or 302 raises unexpected_response. A strict member of the wrong JSON type raises contract_violation, and a non-strict list member of another type reads as absent. A ref member the snapshot manifest does not list takes its rendering from the ref. The README states the stacking, part column, variable_text and label rules, and the errors that carry no trace id. 108 scenarios, 31 recorded and 77 synthetic.

… and pin the files route and 429 polls The validator reads every non-strict member through one set of typed accessors, so a member or item of another JSON type reads as absent and no malformed body crashes it. A strict list's items must be objects, and one of another type raises contract_violation at its index pointer. A files route answers 302, which the client follows, or 200, whose body is the file; any other status but a refusal raises unexpected_response. A 429 poll whose problem document says retryable false raises at once. A definite part whose variable_text is absent or empty adds nothing to its member's parts. The validator reports a recorded API response without x-dfo-trace-id unless it is Cache-Control: public, and a body trace_id must be a non-empty string. New scenarios pin a 307 and a 200 from the files route, a non-retryable 429 poll and folded parts without names. 112 scenarios, 31 recorded and 81 synthetic.

…s, and report a crashing derivation against its record kept_entity_groups reads entity_groups, the identity and its entities through read_object and compares only the string refs a role binds, so a malformed binding reads as absent. An exception while deriving one record's outcomes is reported against that record, and the other records are still checked. A body trace_id that is empty or not a string reads as absent, and a test pins it. The scenario schema's headers description names the cache-control header that excuses a shared-cacheable response from x-dfo-trace-id.

…rose Split clause-joining semicolons and mid-sentence colons into sentences and state facts positively in the package README, the rules README, tools/README.md and the capture and validate tools. Every rule keeps its cases and codes; the tools keep their help text. No code changes.

Describe the API gate as schema and registry validation of recorded exchanges. Scope disk-cache rules to clients implementing that capability and state the synchronous platform DNS timeout exception.

Distinguish content-addressed byte entries from replaceable manifests. Concurrent manifest writes can reduce cache availability while artifact digest verification continues to protect data identity.

…covery gaps Capture and the validator refuse a symlinked or out-of-directory body file before reading or writing it. A ref's `definition` requires the `schema_version` and `identity` the served `CanonicalDefinition` needs, with three rejected examples. The resubmit echo posts the object items of `operations` and `assumptions` alone, a candidate may be chosen by its label as well as its index, and a rebuild that answers another `spec_id` raises `contract_violation`. The README states the browser redirect rule the TypeScript client follows. Five synthetic scenarios pin the new rules (117 in all).

An `assumptions` or `reading_assumptions` item now reads only when its `kind`, `binding_ref` and `text` are strings, and an `operations` item only when its `verb` and `phrase` are strings, matching the served `Assumption` and `Operation` schemas. A read item holds its schema's members alone, with wrong-typed optional members absent. The resubmit echo posts exactly the items read, built from the members read, so a client never posts back an item or member the server's schema refuses. `choose-echo-drops-malformed-items` pins a wrong-typed required member, wrong-typed optional members, a filtered `refs` list, an unknown member and an assumptions list that posts `[]`.

JSON Schema's `integer` admits `2020.0`, and a JavaScript client cannot tell it from `2020` after parsing, so an operation's `target_year` now reads when it is a number with no fractional part other than a boolean, as that integer. `2020.0` reads and echoes as `2020`; `2020.5` reads as absent.

A manifest definition's `schema_version` is now a strict required member, so a client refuses a manifest without one (pinned by `fetch-spec-manifest-without-schema-version`) and never writes a ref it would then reject. The ref format takes any integer version and leaves it to the server, and its definition admits no member CanonicalDefinition does not (`ref-definition-extra-member-rejected`). The README states that the echo's item rule checks JSON types alone and the server still checks its value constraints, that an operation that does not read gives no "Not done" hint, and that a boolean option is neither an index nor a label. The oracle quotes hint phrases keeping non-ASCII characters, as both clients do, and refuses a boolean option. The capture and the validator share one `stored_version` decision for ref recovery. `BODY_FILE_NAME` is compiled from the scenario schema, and a refused body file is checked once, on read, with a message naming it.

…boolean part An integer is a JSON number with no fractional part that is not a boolean, as JSON Schema reads it, since a JavaScript client cannot tell `1.0` from `1`. Strict integer members, a manifest definition's `schema_version` among them, and a ref's `definition.schema_version` admit `1.0` and refuse `1.5` and `true`. A choice that names its part by a boolean raises `choice_invalid`, as a boolean option does.

jm-rivera added this pull request to stack #12 October 21, 2026 15:13

jm-rivera mentioned this pull request Oct 21, 2026

test(contract): check the app against the client contract, and bind Search's registry endpoints DevelopmentFinanceObservatory/apis#72

Merged

jm-rivera marked this pull request as ready for review October 23, 2026 06:31

Base automatically changed from feat/datasets-selector-mappings to dev October 23, 2026 11:15

jm-rivera added 25 commits October 23, 2026 13:15

docs(contract): clarify conformance, capabilities and timeout scope

8ed610a

Describe the API gate as schema and registry validation of recorded exchanges. Scope disk-cache rules to clients implementing that capability and state the synchronous platform DNS timeout exception.

docs(contract): scope concurrent cache write guarantees

c1a9966

Distinguish content-addressed byte entries from replaceable manifests. Concurrent manifest writes can reduce cache availability while artifact digest verification continues to protect data identity.

fix(contract): describe choose-candidate-by-label without a fetch

091bebe

jm-rivera force-pushed the feat/client-contract branch from ef454f0 to 7ab92c3 Compare October 23, 2026 11:15

jm-rivera merged commit 95a15cf into dev Oct 23, 2026
19 checks passed

jm-rivera mentioned this pull request Oct 25, 2026

release: spec-kit dev to main #22

Merged

Navigation Menu

Repository navigation

feat(client-contract): add the contract the DFO clients follow - #11

feat(client-contract): add the contract the DFO clients follow#11
jm-rivera merged 25 commits into
devfrom
feat/client-contract

jm-rivera commented Oct 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jm-rivera commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jm-rivera commented Oct 21, 2026 •

edited

Loading