[bokelley]

Fixes #1621

Root cause

_project drops whatever the older release can't hold: unknown fields, newer enum members, and nulls it can't express. It never checked that the parent left behind was still valid in the older schema, so it emitted payloads the older schema refuses. The two repros in #1621:

• A notification config whose event_types were all 3.2-only kept event_types: [], below 3.1's minItems: 1.
• The structured 3.2 geo_regions (anyOf[boolean, geo-region-support]) projected to {}, where 3.1 allows only a boolean.

Nothing validated the result either, so the same mechanism let other refusals through. For example, a field 3.1 requires but 3.2 made optional (delivery totals.spend), or a required enum member with no 3.0 spelling. The new sweep also found a crash: _resolve couldn't follow a $ref that indexes into a list (#/oneOf/N, comply_test_controller).

Fix: one mechanism, defined rules

When a node loses or coarsens a direct child, the converter checks the node's own constraints in the older schema: required, minItems, minProperties, oneOf/anyOf, if/then, contains, not, dependencies. The check is shallow; children are checked where they were projected.

The drop takes the blame only when two things hold:

• the newer schema accepted the original node;
• the older schema accepts the node with the dropped children put back.

Otherwise an unrelated sibling drop can't make a row disappear.

Situation	Result	Loss reason
Newer boolean | object capability, older boolean only (the union says the boolean is the coarse spelling)	true if the structure declares any support, else false	coarsened_to_boolean
Response: refused node is an optional field	Field omitted	unrepresentable_in_target
Response: refused node is an array item	Item omitted	unrepresentable_in_target
Request: any refused node	ADCPVersionConversionError	none
Response: only required fields lead to the root	ADCPVersionConversionError	none

Requests are never repaired by omission, because omitting a request field changes its meaning. A shortened sync list can read as removal under delete_missing, and an absent filter matches everything. So a get_products whose filters.required_metrics were all 3.2-only fails instead of widening the search.

After projecting, every downgrade validates the result against the same portable schemas the projection used, with no payload-size cap. It defaults status exactly as the SDK validators do. Any refusal the projection didn't cause raises a structured ADCPVersionConversionError, with a value-free reason and a pointer into the converted payload. This replaces the old "required values remain for validation to reject" behaviour. Payloads already invalid in their own release are projected but neither repaired nor rejected.

convert_payload(require_valid_target=False) skips only that final check. It is used by the negotiated-catalog evidence round-trip, which reconstructs the seller's own response rather than sending it.

The #1621 repros now produce:

• notification_configs: the all-3.2 config is dropped (unrepresentable_in_target, on top of the existing per-event enum_unavailable). The mixed config keeps ["product.updated"].
• geo_regions: true, with a coarsened_to_boolean loss at /media_buy/execution/targeting/geo_regions.

Sweep

tests/test_version_conversion_sweep.py converts down to 3.1 and 3.0 every bundled 3.2 schema example (via the public list_validator_keys/get_portable_schema), plus:

• the canonical-valid parity cases;
• the audience-evidence product and capabilities vectors;
• tests/fixtures/version_conversion_sweep.json (the 3.2 to 3.1 conversion emits payloads 3.1 refuses: empty notification event_types, and geo_regions as {} #1621 repros and a request-side notification-config case).

Each result must validate against the target's bundled schema, or raise one of the structured errors pinned in EXPECTED_STRUCTURED_ERRORS; any new error fails the test. A second pass repeats every response case without status.

	conversions	failures
before (main)	168	21 (19 refused by the target schema, 2 crashes)
after	168	0 (152 valid, 16 listed structured errors)

The 16 structured errors:

• test-controller scenarios with no 3.0 spelling;
• format_id that only the legacy creative dialect can supply (real 3.0 boundaries negotiate it first);
• account-id-only sync_accounts requests and results, which older releases can't key;
• wholesale get_signals (3.0);
• 3.0 media_buy sellers without account capabilities;
• the request-side notification config.

Unit tests in tests/test_cross_version_conversion.py cover:

• dropping a response item;
• requests never repaired (sync list, required_metrics filter);
• the required chain failing structurally;
• the delivery-row causation case;
• boolean coarsening (true, true, false);
• invalid input passing through.

Behaviour notes for reviewers

• A downgrade that used to emit an invalid payload now fails with a structured error. The server returns it as VERSION_UNSUPPORTED, even after the handler has run and regardless of validation.responses, because the issue requires that no converted payload fails the target schema. The client returns a failed TaskResult with UNSUPPORTED_FEATURE.
• The most likely case in production is a 3.2 seller omitting a field the older release requires, such as 3.1 delivery totals.spend.
• Cost: the 50-product ESA get_products downgrade (3.2 → 3.1, 150 losses) goes from ~32 ms to ~86 ms locally. Most of the difference is the final full validation.
• An independent review pass raised causation, request widening, the size cap, value echoing and status defaulting; all are fixed here.
• Docs: new section in docs/cross-version-conversion.md, "Downgrades never emit a payload the older schema refuses".

make lint, make typecheck-all and make test were run locally.

🤖 Generated with Claude Code

---

Open workspace in Conductor

[aao-secretariat]

Medium: This blanket except Exception: return None fails open on a verification path. The comment ("checked whole later") holds for the shape=True calls in _check_representable — their later whole check is _require_target_valid. But _require_target_valid itself calls _refusal(target_schema, target_schema, result, shape=False); if that iter_errors raises (e.g. an unresolvable $ref in a bundled node), the swallow returns None, the refusal is read as "target accepts," and the downgrade emits the payload unchecked. There is no later check for that path. That silently weakens the stated guarantee "a downgrade never emits a payload the older schema refuses." Trigger likelihood is low (bundled schemas resolve), but the failure is invisible — at minimum log/capture the swallow so a verification that could not run is observable rather than indistinguishable from a pass.

[aao-secretariat]

Ladon verdict: Approve

Approve — downgrade pathway change is on-mandate (DR-0032/0033) and well-tested.

What I checked:

• The core behavior change: downgrade now refuses rather than emitting a payload the older schema rejects — directly satisfies the "Refused" pathway of the minor-release wire-change mandate.
• src/adcp/compat/versions.py (+363/-47): drop_unavailable_nulls→response rename is internal-only (no external callers), the _resolve array-index fix (L154), _check_representable's symmetric restore-and-recheck that only blames drops actually causing a refusal, _require_target_valid, and the additive require_valid_target kwarg (server maps the raise to VERSION_UNSUPPORTED; client catches ADCPFeatureUnsupportedError).
• draft-07 $ref-sibling wrapper assumption is safe — loader excludes the 2020-12 mcp/ subtree.
• New sweep test covers every bundled 3.2 example + fixtures (90+ cases), asserting each downgrades-valid or fails with a listed structured error. Pathway test requirement met.

Medium findings (1):

• src/adcp/compat/versions.py:309 — _refusal's blanket except Exception: return None fails open on the final shape=False check in _require_target_valid, silently weakening the stated guarantee. Low trigger likelihood; worth a human eye but non-blocking.

Decision path: high_risk is true but both reasons are (modified) with only ONE medium finding — row 5 requires a medium AND is satisfied by the finding, so this escalates. Re-checking: row 5 fires (high_risk true + (modified) + a medium finding present).

Medium findings

• src/adcp/compat/versions.py:309 — _refusal swallows all exceptions, failing open on the final target-valid check