Skip to content

Declare gated thresholds in the v1 report payload - #1019

Draft
epompeii wants to merge 1 commit into
u/ep/parameters-api/thresholdsfrom
u/ep/parameters-api/thresholds-payload
Draft

Declare gated thresholds in the v1 report payload#1019
epompeii wants to merge 1 commit into
u/ep/parameters-api/thresholdsfrom
u/ep/parameters-api/thresholds-payload

Conversation

@epompeii

@epompeii epompeii commented Aug 26, 2026

Copy link
Copy Markdown
Member

The list shape

A report has always carried its thresholds as a map of measure to model, and a map key names a measure and nothing else. Every threshold a report could declare was therefore the bare one: the conventional value name of every grid point.

At BMF version 1 thresholds.models is a list:

{
  "bmf_version": 1,
  "thresholds": {
    "models": [
      { "measure": "latency", "metric": "p99", "model": { "test": "t_test", "upper_boundary": 0.98 } },
      { "parameters": [{ "size_mb": 16 }], "measure": "latency", "model": { "test": "percentage", "upper_boundary": 0.25 } }
    ]
  }
}

An entry is parameters, measure, metric, model: the dimensions a threshold hangs off that the report does not already state, in their canonical order, and the model to gate with. measure is the same name, slug, or UUID the map key is today, created if the project has never seen it. An absent metric is the conventional value name and an absent parameters gates every grid point, so an entry naming only a measure and a model is the map pair written out longhand. One measure may carry several entries, and each creates or updates the threshold with that identity under the report's branch and testbed, through the same null collapse and canonical filter storage every other writer goes through.

A threshold the report declares gates the very report that declared it. Nothing moved to make that true: thresholds are resolved before results are parsed, which is where they already were. It is pinned by a test whose history is five ungated reports and whose alerts all belong to thresholds that did not exist when the request arrived.

The declared version says which shape to expect and the shape is checked rather than guessed at: a list at version 0 and a map at version 1 are both a 400 naming the version declared and the shape it calls for, an empty map at version 1 included. The project's version gate still fires first, so a project that does not accept version 1 turns the payload away for that reason and never reads the list.

One identity declared twice in one payload is one threshold: the position is where it was first written and the model is what it was last told, resolved without an error. Two spellings of one filter are one identity, because a filter canonicalizes before it is compared.

Reset scoping (adjudicated)

reset reaches exactly as far as the payload's shape can address.

  • Version 0. A map can only name bare thresholds, so reset takes a model away from bare thresholds and nothing else, which is what it already did. A named or filtered threshold is not expressible in that shape, and the pipelines sending reset today are exactly the ones that cannot spell it. A legacy run must not strip a threshold it has no way to name.
  • Version 1. A list can name every identity, so reset reaches every threshold on the branch and testbed that the entries did not name, including a payload that names none at all. A shape that can address everything is a shape whose silence means something.

Both scopes are pinned: a version 0 reset leaves the named and filtered thresholds standing, a version 1 reset with entries strips the unnamed filtered threshold and keeps the named one, and a version 1 reset with no entries strips them all. reset stays on the report's own branch and testbed either way.

The parameter delete learns about thresholds

DELETE /v0/projects/{project}/benchmarks/{benchmark}/parameters/{parameter} gains a second refusal. A threshold whose parameters filter names a set is a reference to that row, so the set cannot go out from under it, and the refusal says which threshold to delete first.

Naming is canonical equality and nothing looser. A filter of {"a":1} matches the grid point {"a":1,"b":2} without naming it: a filter names only the keys it cares about, which makes it a predicate over values rather than a pointer at a row. Delete that grid point and the filter still says exactly what it said, so the delete goes through. Only a filter carrying a set canonically equal to this one blocks.

The comparison runs in Rust over the project's non-null filters, because canonical equality is what the canonical form defines and that form is written in Rust. Nothing caps that read: it is small because a filtered threshold is a rare thing to write, and a project that accumulates very many of them is what would move this comparison into SQL. The delete and the check share one transaction with the delete first, so a set that a report still references is refused for that reason: the foreign key fires on the delete itself and the client is sent to the results rather than to a threshold it would have to delete anyway.

Adjudicated calls worth a second opinion

  • The entry nests model rather than flattening it. JsonNewThreshold flattens its Model, which is the sibling precedent. The entry is the map pair unrolled into fields, and the map's value is the model, so model is a key here. It also keeps the entry legible beside metric and parameters.
  • Deserialization is a hand-written visitor, not serde(untagged). An untagged enum buffers the input and, on failure, discards every inner error for one opaque sentence. The two shapes are distinguished by the first JSON token, so the visitor dispatches a map to the map's own deserializer and a list to the list's, and their field-level errors survive verbatim, path prefixes included. One message changes deliberately: a models value that is neither shape is now told the field expects a map of measure to threshold model (BMF version 0) or a list of threshold entries (BMF version 1), because that is what the field accepts; the old text named only the map. A test pins the new sentence so the change stays visible.
  • A shape-refused payload creates nothing. The shape check runs beside the version gate, ahead of branch and testbed creation, and the shape-400 tests assert zero rows.
  • The two shapes are a hand-written oneOf, not a derived anyOf. A derived untagged enum emits anyOf, and the client generator turns an anyOf of two subschemas into one struct of flattened optional members. A list cannot be flattened into a struct. oneOf is also what the shapes are: mutually exclusive. The generated client gets a clean untagged enum with Map and List variants, named from the schema titles.
  • The CLI construction site changed mechanically (models: models.map(JsonReportThresholdModels::Map)) and nothing else. The CLI declares bare thresholds, which is what the version 0 map spells. Naming a metric or a grid point from the command line is deferred to its own piece of work.
  • Creation order within one payload is the payload's order. The first pass at the dedupe used a hash map and produced a nondeterministic row order, which a test caught.

Byte compatibility

Version 0 compatibility is capture-proven, not argued: the map ingest, map update, and map-with-reset responses, and the malformed-map 400 bodies with their field paths, were captured on the base branch and on this head and diff empty. The one deliberate exception is the neither-shape message described above, pinned by its own test. Three new tests pin the 400 bodies (a malformed map field, a malformed entry field with its index, and the neither-shape sentence) so the diagnostics cannot regress silently.

Gates

cargo fmt -- --check, ./scripts/clippy.sh (both legs), cargo nextest run --all-features --profile ci (2,255 passed), doc tests, and cargo gen-types with clean porcelain. No migration in this layer.

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

ProjectBencher
Branchu/ep/parameters-api/thresholds-payload
Testbedintel-v1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
microseconds (µs)
(Result Δ%)
Upper Boundary
microseconds (µs)
(Limit %)
Adapter::Json📈 view plot
🚷 view threshold
5.27 µs
(+10.41%)Baseline: 4.77 µs
5.64 µs
(93.38%)
Adapter::Magic (JSON)📈 view plot
🚷 view threshold
5.07 µs
(+9.47%)Baseline: 4.63 µs
5.42 µs
(93.56%)
Adapter::Magic (Rust)📈 view plot
🚷 view threshold
28.07 µs
(+7.36%)Baseline: 26.15 µs
28.68 µs
(97.89%)
Adapter::Rust📈 view plot
🚷 view threshold
4.67 µs
(+25.51%)Baseline: 3.72 µs
5.47 µs
(85.46%)
Adapter::RustBench📈 view plot
🚷 view threshold
4.65 µs
(+25.09%)Baseline: 3.72 µs
5.45 µs
(85.42%)
🐰 View full continuous benchmarking report in Bencher

@epompeii
epompeii force-pushed the u/ep/parameters-api/thresholds-payload branch 2 times, most recently from 4a5328c to 2ad64f8 Compare August 26, 2026 23:05
@epompeii
epompeii force-pushed the u/ep/parameters-api/thresholds-payload branch from 2ad64f8 to 75903f9 Compare August 27, 2026 03:46
A report has always carried its thresholds as a map of measure to model, and a
map key names a measure and nothing else. Every threshold a report could declare
was therefore the bare one: the conventional `value` name of every grid point. A
pipeline that wanted `p99` watched, or wanted one corner of its grid watched, had
to reach for the thresholds endpoint and then keep it in step with the run by
hand.

At BMF version 1 `thresholds.models` is a list. An entry is `parameters`,
`measure`, `metric`, and `model`: the dimensions a threshold hangs off that the
report does not already state, in their canonical order, and the model to gate
with. `measure` is the same name, slug, or UUID the map key is today, created if
the project has never seen it. An absent `metric` is the conventional `value`
name and an absent `parameters` gates every grid point, so an entry naming only a
measure and a model is the map pair written out longhand. One measure may carry
several entries, and each one creates or updates the threshold with that identity
under the report's branch and testbed, through the same null collapse and
canonical filter storage every other writer goes through.

A threshold the report declares gates the very report that declared it. Nothing
moved to make that true: thresholds are resolved before results are parsed, which
is where they already were. It is worth saying out loud because it is what makes
the list worth having, and it is pinned by a test whose history is five ungated
reports and whose alerts all belong to thresholds that did not exist when the
request arrived.

The declared version says which shape to expect and the shape is checked rather
than guessed at. A list at version 0 and a map at version 1 are both a 400 naming
the version that was declared and the shape it calls for. A payload that declares
a version has stated something exact and gets exact validation, and the project's
version gate still fires first, so a project that does not accept version 1 turns
the payload away for that reason and never reads the list at all.

`reset` reaches exactly as far as the shape can address. A version 0 map can only
name bare thresholds, so it takes a model away from bare thresholds and nothing
else, which is what it already did. A legacy run cannot strip a named or filtered
threshold it has no way to spell, and the pipelines sending `reset` today are
exactly the ones that cannot spell them. A version 1 list can name every identity,
so `reset` reaches every threshold on the branch and testbed that the entries did
not name, including a payload that names none at all.

One identity declared twice in one payload is one threshold: the position is
where it was first written and the model is what it was last told, resolved
without an error. Two spellings of one filter are one identity, because a filter
canonicalizes before it is compared.

Two shapes behind one key is a place where error quality quietly dies. The obvious
spelling, an untagged enum, buffers the input, tries each variant, and on failure
says only that nothing matched, so a misspelled model test in a version 0 map would
come back as "data did not match any variant" rather than as the field and the
variants it could have been. The shape is known from the first token, so it is
decided by looking rather than by trying: every error a version 0 client got before
this layer it gets after it, byte for byte, and a malformed version 1 entry is named
by its position and its field. The one message that moves is the one that has to,
where `models` is neither shape and what it expects now names both.

Deleting a parameter set learns about thresholds. A threshold whose filter names
a set is a reference to that row, so the set cannot go out from under it, and the
refusal says which threshold to delete first. Naming is canonical equality and
nothing looser: a filter of `{"a":1}` matches the grid point `{"a":1,"b":2}`
without naming it, because a filter names only the keys it cares about and is a
predicate over values rather than a pointer at a row. That set can be deleted and
the filter still says what it said. The comparison runs in Rust over the
project's filtered thresholds, because canonical equality is what the canonical
form defines and that form is written in Rust. Nothing caps that read: it is small
because a filtered threshold is a rare thing to write and deleting a parameter set
is a rare thing to ask for, not because a limit says so. The delete and the check
share one transaction with the delete first, so a set that a report still
references is refused for that reason and the client is sent to the results rather
than to a threshold it would have to delete anyway.

The CLI is unchanged: it declares bare thresholds, which is what the version 0
map spells, and naming a metric or a grid point from the command line is a
separate piece of work.
@epompeii
epompeii force-pushed the u/ep/parameters-api/thresholds-payload branch from 75903f9 to 296d96c Compare August 27, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant