Skip to content

Commit 9bba446

Browse files
committed
Name the alert's grid point and read boundaries off the base tables
An alert said which benchmark and which measure it fired on, and a benchmark now has grid points, so two grid points of one benchmark under one threshold raised two alerts that read identically. `JsonAlert` gains `parameter`, the grid point the alert fired on, between the benchmark and the metric, because that is the order the dimensions run in. Every surface that returns an alert carries it: the alerts list, the alert detail, and the alerts a report response embeds. The perf response's slim alert reference is unchanged; the line it hangs off already names its grid point. Both alert readers move off the `metric_boundary` view and onto the base tables. An alert names its boundary, the boundary names its metric row, and that row names the report it landed in and the grid point it was measured under, so each hop is an identifier seek off the hop before it. The metric triple comes from the `value` row the boundary was computed for plus its `lower_value` and `upper_value` siblings, the same assembly the metric endpoint does, now shared rather than written twice. The move is what a later layer needs. The view carries at most one boundary per metric row, and a metric row is about to be allowed several, so a reader that reached its boundary through the view would fan out one alert into one per boundary. No reader joins a boundary through the view any more. Every field an alert carried before carries exactly what it carried before, `metric` included: it is still required, and it is still the triple built around the row the boundary was computed for. The alert responses are pinned against that shape by fixture, key set and values both. The view itself stays, and the migration that holds its column list unchanged stays pinned to it. Its Rust model goes, because nothing reads it now, and the migration test reads the view directly rather than through an endpoint that no longer does.
1 parent ae947aa commit 9bba446

11 files changed

Lines changed: 771 additions & 252 deletions

File tree

lib/api_projects/src/metrics.rs

Lines changed: 2 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ use bencher_schema::model::spec::SpecId;
88
use bencher_schema::{
99
actor_conn,
1010
context::{ApiContext, DbConnection},
11-
error::{issue_error, resource_not_found_err, with_auth_hint},
11+
error::{resource_not_found_err, with_auth_hint},
1212
model::{
1313
project::{
1414
ProjectId, QueryProject,
@@ -288,33 +288,7 @@ fn metric_triple(
288288
return Ok(None);
289289
}
290290

291-
// The bounds are the addressed row's siblings under the same report benchmark and
292-
// the same measure, so this rides `index_metric_report_benchmark_measure_name`.
293-
let bounds = schema::metric::table
294-
.filter(schema::metric::report_benchmark_id.eq(query_metric.report_benchmark_id))
295-
.filter(schema::metric::measure_id.eq(query_metric.measure_id))
296-
.filter(schema::metric::name.eq_any([MetricName::lower_value(), MetricName::upper_value()]))
297-
.select((schema::metric::name, schema::metric::value))
298-
.load::<(MetricName, f64)>(conn)
299-
.map_err(|e| {
300-
let message = format!(
301-
"Failed to query the bounds for metric ({metric_uuid})",
302-
metric_uuid = query_metric.uuid
303-
);
304-
issue_error("Failed to query metric bounds", &message, e)
305-
})?;
306-
307-
let bound = |bound: &MetricName| {
308-
bounds
309-
.iter()
310-
.find_map(|(name, value)| (name == bound).then_some((*value).into()))
311-
};
312-
Ok(Some(JsonMetricTriple {
313-
uuid: query_metric.uuid,
314-
value: query_metric.value.into(),
315-
lower_value: bound(&MetricName::lower_value()),
316-
upper_value: bound(&MetricName::upper_value()),
317-
}))
291+
query_metric.triple(conn).map(Some)
318292
}
319293

320294
fn metric_query_json(

lib/api_projects/tests/metric_migration.rs

Lines changed: 134 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,16 @@
1010
//!
1111
//! A migration that rebuilds the metric table is only safe if it is invisible from
1212
//! the outside. That is not a claim to assert, it is a thing to demonstrate: these
13-
//! tests seed a database in the shape the migration finds, capture the raw bytes of
14-
//! every response that reads a metric, run the migration, and capture them again.
15-
//! The two captures must be equal byte for byte.
13+
//! tests seed a database in the shape the migration finds, capture what the outside
14+
//! could see, run the migration, and capture it again. The two captures must be
15+
//! equal byte for byte.
1616
//!
17-
//! The same binary serves both captures because every reader captured here goes
18-
//! through the `metric_boundary` view, whose column list the migration holds
19-
//! unchanged.
17+
//! What both captures can read is the `metric_boundary` view, which the migration
18+
//! holds unchanged and which is defined on either shape of the table. No endpoint
19+
//! reads it any more: every reader now drives on the named `metric` rows, and those
20+
//! rows do not exist before the migration, so no response has a pre-migration form
21+
//! to compare. The view is therefore captured directly, and the responses that read
22+
//! the named rows are anchored on the first migrated capture instead.
2023
2124
use bencher_api_tests::{
2225
TestServer,
@@ -51,25 +54,25 @@ struct Fixture {
5154
metric_uuids: Vec<MetricUuid>,
5255
}
5356

54-
/// The raw bytes of every response that reads a metric through the
55-
/// `metric_boundary` view, which is the view this migration holds unchanged.
57+
/// Every row of the `metric_boundary` view, rendered, which is the artifact this
58+
/// migration holds unchanged.
5659
///
57-
/// The report, perf, and metric responses are deliberately absent. All three are
58-
/// built from the named `metric` rows rather than from the view, and those rows do
59-
/// not exist before the migration, so none of them has a pre-migration form to
60-
/// compare. None is stable across a down and up round trip either, because the
61-
/// migration mints a fresh uuid for every bound row it recreates and all three now
62-
/// echo those uuids. That is not a regression: a server runs its migrations before
63-
/// it serves, and the `value` row, which every reader of the old shape saw, keeps
64-
/// its identity. What the perf response owes older clients is pinned where that
65-
/// response lives, by the byte compatibility test in `perf.rs`; the metric endpoint
66-
/// is pinned the same way in `metrics.rs`, and what it owes this migration is that
67-
/// a down and up round trip leaves it unmoved, which
60+
/// The report, perf, alert, and metric responses are deliberately absent. All four
61+
/// are built from the named `metric` rows, and those rows do not exist before the
62+
/// migration, so none of them has a pre-migration form to compare. None is stable
63+
/// across a down and up round trip either, because the migration mints a fresh uuid
64+
/// for every bound row it recreates and all four now echo or read those rows. That
65+
/// is not a regression: a server runs its migrations before it serves, and the
66+
/// `value` row, which every reader of the old shape saw, keeps its identity. What
67+
/// the perf response owes older clients is pinned where that response lives, by the
68+
/// byte compatibility test in `perf.rs`; the metric endpoint is pinned the same way
69+
/// in `metrics.rs` and the alert responses in `parameters.rs`. What they owe this
70+
/// migration is that a down and up round trip leaves them unmoved, which
6871
/// [`migration_down_and_up_round_trips_the_metric_triple`] asserts against
69-
/// [`capture_metrics`].
72+
/// [`capture_metrics`] and [`capture_alerts`].
7073
#[derive(Debug, PartialEq, Eq)]
7174
struct Captured {
72-
alerts: String,
75+
view: String,
7376
}
7477

7578
/// The fixture rows, chosen to reach every branch of the explosion and the pivot.
@@ -111,24 +114,23 @@ const FIXTURE_METRICS: [LegacyMetric; 4] = [
111114
/// table hands back the same numbering either way.
112115
const FIXTURE_METRIC_IDS: [i32; 4] = [10, 20, 30, 40];
113116

114-
// Every response that reads a metric through the view is byte identical across the
115-
// migration.
117+
// The view every reader used to go through is byte identical across the migration.
116118
#[tokio::test]
117119
async fn responses_are_byte_identical_across_the_migration() {
118120
let server = TestServer::new().await;
119121
let fixture = seed_legacy_project(&server, "equiv").await;
120122

121-
let before = capture(&server, &fixture).await;
123+
let before = capture(&server);
122124
assert!(
123-
before.alerts.contains("40.5"),
124-
"the fixture reaches the alert response before anything is compared"
125+
before.view.contains("40.5"),
126+
"the fixture reaches the view before anything is compared"
125127
);
126128
apply_migration(&server);
127-
let after = capture(&server, &fixture).await;
129+
let after = capture(&server);
128130

129131
assert_eq!(
130-
before.alerts, after.alerts,
131-
"the alerts response changed across the migration"
132+
before.view, after.view,
133+
"the metric boundary view changed across the migration"
132134
);
133135

134136
// The metric endpoint reads the named rows, so the migration is what gives it a
@@ -348,41 +350,51 @@ async fn migration_down_and_up_round_trips_the_metric_triple() {
348350
let server = TestServer::new().await;
349351
let fixture = seed_legacy_project(&server, "roundtrip").await;
350352

351-
let before = capture(&server, &fixture).await;
353+
let before = capture(&server);
352354
apply_migration(&server);
353-
let after_first = capture(&server, &fixture).await;
355+
let after_first = capture(&server);
354356
let metrics_first = capture_metrics(&server, &fixture).await;
357+
let alerts_first = capture_alerts(&server, &fixture).await;
355358
assert_eq!(
356359
before, after_first,
357-
"applying the migration leaves every response unchanged"
360+
"applying the migration leaves the view unchanged"
358361
);
359362

360363
let mut conn = server.db_conn();
361364
revert_migration(&mut conn);
362365
drop(conn);
363366

364-
let after_revert = capture(&server, &fixture).await;
367+
let after_revert = capture(&server);
365368
assert_eq!(
366369
before, after_revert,
367-
"reverting the migration leaves every response unchanged"
370+
"reverting the migration leaves the view unchanged"
368371
);
369372

370373
apply_migration(&server);
371-
let after_round_trip = capture(&server, &fixture).await;
374+
let after_round_trip = capture(&server);
372375
assert_eq!(
373376
before, after_round_trip,
374-
"a down and up round trip leaves every response unchanged"
377+
"a down and up round trip leaves the view unchanged"
375378
);
376379

377-
// The metric endpoint has no response to capture in the legacy shape, so its
378-
// anchor is the first migrated capture rather than the pre-migration one. The
379-
// bound rows the round trip recreates carry fresh uuids, and the metric endpoint
380-
// never echoes a bound row's uuid, so the response is stable across it.
380+
// The metric endpoint and the alert responses have nothing to capture in the
381+
// legacy shape, so their anchor is the first migrated capture rather than the
382+
// pre-migration one. The bound rows the round trip recreates carry fresh uuids,
383+
// and neither reader echoes a bound row's uuid, so both are stable across it.
381384
assert_eq!(
382385
metrics_first,
383386
capture_metrics(&server, &fixture).await,
384387
"a down and up round trip leaves every metric response unchanged"
385388
);
389+
assert!(
390+
alerts_first.contains("40.5"),
391+
"the fixture reaches the alert response before anything is compared"
392+
);
393+
assert_eq!(
394+
alerts_first,
395+
capture_alerts(&server, &fixture).await,
396+
"a down and up round trip leaves the alerts response unchanged"
397+
);
386398

387399
// The responses can only be trusted if the schema they were served from is the
388400
// schema the code compiles against, so the round trip also has to land back on it.
@@ -1178,21 +1190,98 @@ fn seed_threshold_and_alert(
11781190
///
11791191
/// The bytes are compared, not parsed values: parsing and re-serializing would
11801192
/// hide exactly the float formatting drift this migration has to rule out.
1181-
async fn capture(server: &TestServer, fixture: &Fixture) -> Captured {
1193+
fn capture(server: &TestServer) -> Captured {
1194+
let mut conn = server.db_conn();
1195+
let view = diesel::sql_query(VIEW_ROWS_SQL)
1196+
.load::<ViewRow>(&mut conn)
1197+
.expect("Failed to read the metric boundary view")
1198+
.into_iter()
1199+
.map(
1200+
|ViewRow {
1201+
metric_id,
1202+
metric_uuid,
1203+
report_benchmark_id,
1204+
measure_id,
1205+
value,
1206+
lower_value,
1207+
upper_value,
1208+
boundary_id,
1209+
boundary_uuid,
1210+
threshold_id,
1211+
model_id,
1212+
baseline,
1213+
lower_limit,
1214+
upper_limit,
1215+
}| {
1216+
format!(
1217+
"{metric_id}|{metric_uuid}|{report_benchmark_id}|{measure_id}|{value:?}|{lower_value:?}|{upper_value:?}|{boundary_id:?}|{boundary_uuid:?}|{threshold_id:?}|{model_id:?}|{baseline:?}|{lower_limit:?}|{upper_limit:?}"
1218+
)
1219+
},
1220+
)
1221+
.collect::<Vec<_>>()
1222+
.join("\n");
1223+
Captured { view }
1224+
}
1225+
1226+
/// Every row of the `metric_boundary` view, in a stable order.
1227+
const VIEW_ROWS_SQL: &str = "SELECT * FROM metric_boundary ORDER BY metric_id";
1228+
1229+
/// One row of the `metric_boundary` view.
1230+
///
1231+
/// The floats are read as floats and rendered with Rust's shortest round trip
1232+
/// formatting, which is what makes formatting drift visible: it is exact, and it
1233+
/// prints the sign of a negative zero that `==` cannot see. The render destructures
1234+
/// the row, so a column added to the view is a build error here rather than a
1235+
/// silent omission.
1236+
#[derive(diesel::QueryableByName)]
1237+
struct ViewRow {
1238+
#[diesel(sql_type = Integer)]
1239+
metric_id: i32,
1240+
#[diesel(sql_type = Text)]
1241+
metric_uuid: String,
1242+
#[diesel(sql_type = Integer)]
1243+
report_benchmark_id: i32,
1244+
#[diesel(sql_type = Integer)]
1245+
measure_id: i32,
1246+
#[diesel(sql_type = Double)]
1247+
value: f64,
1248+
#[diesel(sql_type = Nullable<Double>)]
1249+
lower_value: Option<f64>,
1250+
#[diesel(sql_type = Nullable<Double>)]
1251+
upper_value: Option<f64>,
1252+
#[diesel(sql_type = Nullable<Integer>)]
1253+
boundary_id: Option<i32>,
1254+
#[diesel(sql_type = Nullable<Text>)]
1255+
boundary_uuid: Option<String>,
1256+
#[diesel(sql_type = Nullable<Integer>)]
1257+
threshold_id: Option<i32>,
1258+
#[diesel(sql_type = Nullable<Integer>)]
1259+
model_id: Option<i32>,
1260+
#[diesel(sql_type = Nullable<Double>)]
1261+
baseline: Option<f64>,
1262+
#[diesel(sql_type = Nullable<Double>)]
1263+
lower_limit: Option<f64>,
1264+
#[diesel(sql_type = Nullable<Double>)]
1265+
upper_limit: Option<f64>,
1266+
}
1267+
1268+
/// Capture the raw bytes of the alerts endpoint.
1269+
///
1270+
/// Separate from [`capture`] because the alert responses read the named `metric`
1271+
/// rows, so they only have a response to capture once the migration has made them.
1272+
async fn capture_alerts(server: &TestServer, fixture: &Fixture) -> String {
11821273
let Fixture {
11831274
project_slug,
11841275
token,
11851276
metric_uuids: _,
11861277
} = fixture;
11871278

1188-
let alerts = get(
1279+
get(
11891280
server,
11901281
token,
11911282
&format!("/v0/projects/{project_slug}/alerts"),
11921283
)
1193-
.await;
1194-
1195-
Captured { alerts }
1284+
.await
11961285
}
11971286

11981287
/// Capture the raw bytes of the metric endpoint, one response per seeded metric.

0 commit comments

Comments
 (0)