diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index d45d197f5..4a2d3fd84 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -85,8 +85,8 @@ jobs: run: cargo test --workspace --locked # The viewer's Python tests run its JavaScript in V8 through py_mini_racer; - # the ones that plan a query run target/debug/stage_pipeline, which the - # workspace test build above already linked for devtools' tests. + # they and the Node tests plan their documents with target/debug/asap_planner, + # which the workspace test build above already linked for devtools' tests. # A pinned Python and Node: the runner's system Python refuses a global # pip install (PEP 668). - name: Install Python diff --git a/crates/cost/src/lib.rs b/crates/cost/src/lib.rs index 98e989762..d381c59bb 100644 --- a/crates/cost/src/lib.rs +++ b/crates/cost/src/lib.rs @@ -252,7 +252,7 @@ pub enum ClusterObjective { /// The deployment's choices for plan selection (Stage 3): the cost model, /// how fast a core works, and how long one-off queries are spread over. -/// Loadable from JSON (`stage_pipeline --cost-inputs `); +/// Loadable from JSON (`asap_planner --cost-inputs `); /// an omitted field takes its default, an unknown one is refused. #[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)] #[serde(default, deny_unknown_fields)] diff --git a/crates/devtools/Cargo.toml b/crates/devtools/Cargo.toml index 8c71ffb2f..763e654fe 100644 --- a/crates/devtools/Cargo.toml +++ b/crates/devtools/Cargo.toml @@ -17,13 +17,13 @@ asap-physical-asap-optimization = { path = "../physical-asap-optimization", feat asap-cost = { path = "../cost" } asap-plan-selection = { path = "../plan-selection", features = ["reference-pricer"] } # The installable plan of the selected candidate (`installable_plan_of`), -# whose stored outputs `stage_pipeline` writes. +# whose stored outputs `asap_planner` writes. asap-planner = { path = "../planner" } -# The executor's supported summary families, which `stage_pipeline` plans +# The executor's supported summary families, which `asap_planner` plans # with when the deployment declares none, as `plan_installable` does. asap-executor = { path = "../executor" } -# Used by the show_ir / stage_pipeline / variant_coverage bins (catalog schemas, +# Used by the asap_planner / variant_coverage bins (catalog schemas, # async SQL path, JSON output) and by the topk_ir / canonical_examples # examples. Regular deps, not dev-deps: `[[bin]]` targets can't see dev-deps. asap-types = { path = "../types" } diff --git a/crates/devtools/src/bin/asap_planner.rs b/crates/devtools/src/bin/asap_planner.rs new file mode 100644 index 000000000..4e231d027 --- /dev/null +++ b/crates/devtools/src/bin/asap_planner.rs @@ -0,0 +1,1140 @@ +// cargo run -p asap-devtools --bin asap_planner -- \ +// --workload dashboard.json --run-until stage3 +// cargo run -p asap-devtools --bin asap_planner -- \ +// --workload promql.json --workload sql.json --sql-catalog-table '' \ +// --run-until stage1_pass2 --format json --out run.json +// +// Runs the planner stages over a workload up to and including +// `--run-until` (`stage0`, `stage1_pass1`, `stage1_pass2`, `stage2` or +// `stage3`) and writes each stage's output, as text (`--format text`, the default; +// `asap_devtools::planning_stages_text`) or as an `asap-stage-pipeline/v1` +// document for the Stage Viewer (`--format json`), to `--out` or standard +// output. +// +// The stages run as `run_planning` runs them +// (`asap_devtools::planning_stages::run_stages_until`), so the selected plan +// is the one `run_planning` selects. +// +// Inputs, one flag per field of the planner's `UserInput`: +// - `--workload ` (repeatable): a `PlanningWorkload` JSON file. Each +// file keeps its one language and is lowered by its own frontend; the +// queries are numbered `q1..qN` across the files in the order given, +// and the data workload is the union of the files' sources. A source +// several files declare must have the same facts in each, and the files +// must declare the same kind of arrival: ingesting (or mixed) arrivals +// add their `rows_per_second`. Plan selection shares a workload's +// `rows_per_second` out over all its sources, so when the data arrives, +// every source of several files must declare its own rate +// (`series_count` and `sample_interval`). A disagreement or a source +// without its own rate is refused. +// - `--sql-catalog-table ''` (repeatable): a table the SQL queries +// read: `{"name": ..., "columns": [{"name": ..., "type": +// "timestamp|utf8|string|float64|double|int64|bigint", "nullable": +// true}], "time_index": 0}` (`nullable` defaults to true; without +// `time_index` the table has no time column). +// - `--cost-inputs `: Stage 3's `CostInputs` (`cost_model`, +// `resource_speed`, `non_repeating_query_horizon_seconds`, +// `measured_operation_costs`, `frozen_calibration_factor`; an omitted +// field takes its default). +// - `--deployment-capabilities `: the `DeploymentCapabilities` +// (`supported_summaries`, `raw_data_retained`, +// `query_time_results_retained`; an omitted field takes its default). +// Without `supported_summaries`, the executor's summary families are +// supported, as `plan_installable` defaults it. +// - `--ignore-accuracy-guarantees`: Stage 3 accepts every query output +// regardless of its accuracy guarantee +// (`PlanningModels::ignore_accuracy_guarantees`), comparing candidates +// on cost alone. By default it enforces them, as `plan_installable` +// does. +// +// The document (`--format json`): +// - workload: the queries (`id`, `language`, `text`, `requirements`, +// `recurrence`, `predictability`) and the `data_workload` planned with; +// - stage0_query_language_frontends: the frontends' DAG, one root per query; +// - stage1_logical_asap_optimization (from `stage1_pass1`): `pruned_candidates`, +// the ways to compute a measure Pass 1 pruned (each on its aggregate's +// memo `group`, with `candidate` and `reason`); with `stage3`, +// `candidates` holds `L1`, the selected plan's roots (as Stage 2 built +// them, with Stage 3's weight proofs); `error` when Pass 1 failed; +// - stage2_physical_asap_optimization (from `stage2`): `pruned_candidates`, +// the physical candidates Stage 2 pruned (each with its memo `group`, +// `operator_node`, `operator`, `implementation_root`, `timing`, +// `materialization` and `reason`); with `stage3`, `candidates` holds +// `P1`, the selected plan as Stage 2 built it; +// - stage3_plan_selection (from `stage3`): Stage 3's `Selection`: +// `selected_dag` (`candidate_of_node`, `edges`, +// `node_of_output_reference`), `nodes_with_proven_non_negative_weights`, +// the plan's `cost` (its `usage`, `cost_per_second` and per-node +// charges), `accepted_accuracy_per_query`, `pruned_candidates` and +// `search_report`; `error` when no plan was selected. +// `tumbling_window_retention_ms` maps each tumbling-window chain the plan +// keeps beyond one evaluation (by node id in `P1`) to how far back its +// tumbling windows are kept. +// `materialized_views` are the plan's materialized views as +// `plan_installable` derives them, each with the node of `P1` it is +// installed from (`stage2_node_id`), its position among that node's +// installed nodes (`installed_position`) and their number +// (`installed_count`); `materialized_views_error` when the plan is not +// installable. `warnings`: a PromQL query applying `rate`, `increase` +// or `irate` to a declared gauge or combining a declared summary's +// quantile series across series (`metric_type_warnings`), and a peak memory over 10^12 bytes +// when the cost model declares no memory capacity; +// - memo: `nodes`, one node table for the section (each distinct node +// once, children as indices before their parents), and `steps`, the +// memo after each step run (`stage0`: the frontends' DAG hash-consed, +// groups without candidates; `pass1`; `pass2_local_window_composition`: +// local window composition's tumbling-window candidates; `pass2`: Pass +// 2's output, the memo Stage 2 implements). Each +// step lists its groups (`id`, `label`, `children`, `parents`, +// `queries`, `roots_of`, `node`) and their candidates (`index`, +// `implements`, `children`, `origin`, `accuracy_budget_parts`, `added` +// (not in the group at the step before), `label`, `node`, +// `group_of_node`). With `stage3`, `selected_choice` maps each group the +// plan uses to its candidate's index, or is null with +// `selected_choice_unavailable`; +// - `search`, `accuracy_guarantees` (`"enforced"` or `"ignored"`), +// `deployment_capabilities` (as planned with) and `run_until`. +// A section of a stage after `--run-until` is null. + +use std::collections::btree_map::Entry; +use std::collections::{BTreeSet, HashMap, HashSet}; +use std::rc::Rc; + +use asap_cost::{CostInputs, ResourceUsage}; +use asap_devtools::candidate_labels::{computation_label, node_label}; +use asap_devtools::planning_stages::{ + lower_promql_roots, lower_sql_roots, run_stages_until, sql_catalog, PlanningStagesError, + PlanningStagesRun, RunUntil, +}; +use asap_devtools::planning_stages_text::planning_stages_text; +use asap_devtools::SqlCatalog; +use asap_frontend_promql::{metric_type_warnings, promql_metric_names}; +use asap_logical_asap_optimization::{ChosenCandidatePerMemoGroup, DeploymentCapabilities}; +use asap_plan_selection::{large_peak_memory_warning, AcceptedAccuracy, Selection}; +use asap_planner::{ + installable_plan_of, selected_chosen_candidates, PlanningModels, PlanningRun, SelectionError, +}; +use asap_types::ir::flat::{flatten, FlatDag, FlatNode, NodeId}; +use asap_types::ir::operator::node::OperatorNode; +use asap_types::ir::properties::{default_metric_for_root, AccuracyTarget}; +use asap_types::ir::QueryRoot; +use asap_types::stages::{queries_rooted_at, MemoGroupId}; +use asap_types::stages::{Memo, OriginOfCandidateForMemoGroup}; +use asap_types::stages::{PerComputationLogicalMemoCandidates, PhysicalASAPMemoCandidates}; +use asap_types::workload::{ + DataArrival, DataWorkload, PlanningWorkload, QueryLanguage, QueryRecurrence, + QueryWorkloadEntry, RepeatedDemand, +}; +use serde_json::{json, Value}; + +const USAGE: &str = "usage: asap_planner (--workload )... \ +--run-until stage0|stage1_pass1|stage1_pass2|stage2|stage3 [--format text|json] [--out ] \ +[--sql-catalog-table ]... [--cost-inputs ] [--deployment-capabilities ] \ +[--ignore-accuracy-guarantees]"; + +fn main() { + match run(std::env::args().skip(1).collect()) { + Ok(()) => {} + Err(Failure::Usage(message)) => { + eprintln!("asap_planner: {message}\n{USAGE}"); + std::process::exit(2); + } + Err(Failure::Other(message)) => { + eprintln!("asap_planner: {message}"); + std::process::exit(1); + } + } +} + +/// Why a run failed: bad arguments (status 2, with the usage line) or +/// anything else (status 1). +enum Failure { + Usage(String), + Other(String), +} + +impl From for Failure { + fn from(message: String) -> Self { + Self::Other(message) + } +} + +/// `path`'s contents as `T`, errors naming `flag` and `path`. +fn read_json(flag: &str, path: &str) -> Result { + let text = std::fs::read_to_string(path).map_err(|e| format!("{flag} {path}: {e}"))?; + serde_json::from_str(&text).map_err(|e| format!("{flag} {path}: {e}")) +} + +fn run(args: Vec) -> Result<(), Failure> { + let mut workload_paths: Vec = Vec::new(); + let mut tables = Vec::new(); + let mut run_until = None; + let mut json_format = false; + let mut out = None; + let mut cost_inputs = CostInputs::default(); + let mut deployment_capabilities = DeploymentCapabilities::default(); + let mut ignore_accuracy_guarantees = false; + let mut args = args.into_iter(); + while let Some(flag) = args.next() { + let mut value = || { + args.next() + .ok_or_else(|| Failure::Usage(format!("{flag} needs a value"))) + }; + match flag.as_str() { + "--workload" => workload_paths.push(value()?), + "--sql-catalog-table" => tables.push(value()?), + "--run-until" => { + run_until = Some(value()?.parse::().map_err(Failure::Usage)?) + } + "--format" => { + json_format = match value()?.as_str() { + "text" => false, + "json" => true, + other => return Err(Failure::Usage(format!("unknown format {other}"))), + } + } + "--out" => out = Some(value()?), + "--cost-inputs" => cost_inputs = read_json(&flag, &value()?)?, + "--deployment-capabilities" => deployment_capabilities = read_json(&flag, &value()?)?, + "--ignore-accuracy-guarantees" => ignore_accuracy_guarantees = true, + other => return Err(Failure::Usage(format!("unknown argument {other}"))), + } + } + let run_until = run_until.ok_or(Failure::Usage("--run-until is required".into()))?; + if workload_paths.is_empty() { + return Err(Failure::Usage("--workload is required".into())); + } + let mut workloads = Vec::new(); + for path in &workload_paths { + let workload: PlanningWorkload = read_json("--workload", path)?; + workload + .validate() + .map_err(|e| format!("--workload {path}: {e}"))?; + workloads.push((path.as_str(), workload)); + } + let (catalog, table_names) = sql_catalog(&tables)?; + let (described, roots, entries, data) = lowered_together(&workloads, &catalog, &table_names)?; + // Plan with the executor's summary families when the deployment declares none. + if deployment_capabilities.supported_summaries.is_none() { + deployment_capabilities.supported_summaries = + Some(asap_executor::supported_summary_families()); + } + let models = PlanningModels::builtin() + .with_cost_inputs(&cost_inputs) + .with_deployment_capabilities(&deployment_capabilities) + .with_ignore_accuracy_guarantees(ignore_accuracy_guarantees); + let result = run_stages_until(roots.clone(), &entries, &data, models, run_until); + // A stage that failed: the output is still written, then the run exits 1. + let failure = match &result { + Err(PlanningStagesError::Pass1(error)) => Some(format!("Pass 1: {error}")), + Ok(run) => match &run.run.selection { + Err(SelectionError::NotRun) | Ok(_) => None, + Err(error) => Some(error.to_string()), + }, + Err(_) => None, + }; + // A rate, increase or irate over a declared gauge (the query still plans). + let mut warnings = metric_type_warnings_of(&described, &data)?; + // A selected plan's very large peak memory, when the deployment + // declares no memory capacity to check it against. + warnings.extend( + result + .as_ref() + .ok() + .and_then(|run| run.run.selection.as_ref().ok()) + .and_then(|selection| { + large_peak_memory_warning( + selection.cost.usage.peak_memory_bytes, + &cost_inputs.cost_model, + ) + }), + ); + let text = if json_format { + let mut document = document_of(described, &roots, &entries, result, run_until)?; + // Stage 3 is null when Pass 1 rejected a query. + if let Some(stage3) = document["stage3_plan_selection"].as_object_mut() { + if !warnings.is_empty() { + stage3.insert("warnings".into(), json!(warnings)); + } + } + document["workload"]["data_workload"] = + serde_json::to_value(&data).map_err(|e| e.to_string())?; + document["search"] = json!("candidate-connections"); + document["accuracy_guarantees"] = json!(if ignore_accuracy_guarantees { + "ignored" + } else { + "enforced" + }); + // Read-only: the deployment inputs the run planned with. + document["deployment_capabilities"] = + serde_json::to_value(&deployment_capabilities).map_err(|e| e.to_string())?; + serde_json::to_string_pretty(&document).map_err(|e| e.to_string())? + "\n" + } else { + let mut text = match result { + Ok(run) => planning_stages_text(&run), + Err(PlanningStagesError::Pass1(error)) => format!("error: Pass 1: {error}\n"), + Err(error) => return Err(error.to_string().into()), + }; + for warning in &warnings { + text.push_str(&format!("warning: {warning}\n")); + } + text + }; + match out { + Some(path) => std::fs::write(&path, text).map_err(|e| format!("{path}: {e}"))?, + None => print!("{text}"), + } + match failure { + Some(error) => Err(Failure::Other(error)), + None => Ok(()), + } +} + +/// The document for a run that ended with `result` over the frontends' +/// `roots`. A Pass 1 failure still gives a document: the Stage 0 DAG and +/// the error under Stage 1. A failed validation gives none. +fn document_of( + queries: Vec, + roots: &[QueryRoot], + entries: &[QueryWorkloadEntry], + result: Result, + run_until: RunUntil, +) -> Result { + let queries = with_default_metrics(queries, roots); + match result { + Ok(run) => Ok(document(queries, entries, &run)), + Err(PlanningStagesError::Pass1(error)) => Ok(json!({ + "format": "asap-stage-pipeline/v1", + "run_until": run_until.name(), + "workload": { "queries": queries }, + "stage0_query_language_frontends": { "dag": export(roots) }, + "stage1_logical_asap_optimization": { + "combinations": 0, + "capped": false, + "pruned_candidates": [], + "candidates": [], + "error": format!("Pass 1: {error}"), + }, + "stage2_physical_asap_optimization": null, + "stage3_plan_selection": null, + })), + Err(error @ PlanningStagesError::Validation(_)) => Err(error.to_string()), + } +} + +/// The document for `run`, whose workload queries are `queries`: each +/// section of a stage `run` ran, the others null. +fn document(queries: Vec, entries: &[QueryWorkloadEntry], run: &PlanningStagesRun) -> Value { + let until = run.run_until; + let labels = asap_devtools::candidate_labels::candidate_labels(run); + let logical: Vec<_> = run + .logical_candidates() + .iter() + .map(|c| match &c.roots { + Ok(roots) => { + json!({ "id": c.id, "display_name": labels[&c.id], "dag": export(roots) }) + } + Err(error) => { + json!({ "id": c.id, "display_name": labels[&c.id], "error": error }) + } + }) + .collect(); + let physical: Vec<_> = run + .physical_candidates() + .iter() + .map(|p| json!({ "id": p.id, "from_logical": p.from_logical, "display_name": labels[&p.id], "dag": p.dag })) + .collect(); + let stage1 = (until >= RunUntil::Pass1).then(|| { + let mut stage1 = json!({ + "combinations": logical.len(), + "capped": false, + "pruned_candidates": pass1_pruned_candidates_json(run.run.per_computation_candidates.as_ref()), + "candidates": logical, + }); + // Pass 1 or Pass 2 failed, or the deployment's capabilities + // contradict the data workload. + if run.run.logical_asap_candidates.is_none() { + if let Err(error) = &run.run.selection { + if !matches!(error, SelectionError::NotRun) { + stage1["error"] = json!(error.to_string()); + } + } + } + stage1 + }); + let stage2 = (until >= RunUntil::Stage2).then(|| { + let mut stage2 = json!({ + "candidates": physical, + "pruned": [], + "pruned_candidates": stage2_pruned_candidates_json(run.run.physical_asap_candidates.as_ref()), + }); + // Stage 2 itself failed. + if run.run.logical_asap_candidates.is_some() && run.run.physical_asap_candidates.is_none() { + if let Err(error) = &run.run.selection { + stage2["error"] = json!(error.to_string()); + } + } + stage2 + }); + let stage3 = (until == RunUntil::Stage3).then(|| stage3_json(run, entries)); + json!({ + "format": "asap-stage-pipeline/v1", + "run_until": until.name(), + "workload": { "queries": queries }, + "stage0_query_language_frontends": { "dag": export(&run.stage0_query_language_frontends) }, + "stage1_logical_asap_optimization": stage1, + "stage2_physical_asap_optimization": stage2, + "stage3_plan_selection": stage3, + "memo": memo_json(run), + }) +} + +/// Stage 3's section: the selection, the selected plan's tumbling-window retention and +/// its materialized views. +fn stage3_json(run: &PlanningStagesRun, entries: &[QueryWorkloadEntry]) -> Value { + let mut stage3 = selection_json(&run.run.selection); + match (&run.run.selection, &run.run.selected_plan) { + (Ok(selection), Some(roots)) => { + let tumbling_window_retention_ms = run + .run + .selected_physical_plan + .as_ref() + .map(|realized| realized.tumbling_window_retention_ms.clone()) + .unwrap_or_default(); + stage3["tumbling_window_retention_ms"] = tumbling_window_retention_ms + .iter() + .map(|(node, ms)| (node.to_string(), json!(ms))) + .collect::>() + .into(); + match materialized_views_json(roots, selection, &tumbling_window_retention_ms, entries) + { + Ok(views) => stage3["materialized_views"] = Value::Array(views), + Err(error) => { + stage3["materialized_views"] = json!([]); + stage3["materialized_views_error"] = json!(error); + } + } + } + _ => stage3["materialized_views"] = json!([]), + } + stage3 +} + +/// The document's `memo` section: the memo after each step `run` ran, and +/// with Stage 3 the selected plan's choice over the last step's memo. +fn memo_json(run: &PlanningStagesRun) -> Value { + let stage0 = run + .stage0_memo_groups + .as_ref() + .expect("run_stages_until builds Stage 0's memo"); + let mut named = vec![("stage0", "Stage 0, the frontends' DAG hash-consed", stage0)]; + if let Some(pass1) = &run.run.per_computation_candidates { + named.push(("pass1", "Stage 1, Pass 1", &pass1.memo)); + } + if let Some(pass2) = &run.pass2_local_window_composition { + named.push(( + "pass2_local_window_composition", + "Stage 1, Pass 2: local window composition", + pass2, + )); + } + if let Some(logical) = &run.run.logical_asap_candidates { + named.push(("pass2", "Stage 1, Pass 2", &logical.memo)); + } + let mut table = MemoNodes::default(); + let mut previous: Option<&Memo> = None; + let mut json_steps = Vec::new(); + for &(step, title, memo) in &named { + json_steps.push(json!({ + "step": step, + "title": title, + "groups": memo_groups_json(&mut table, memo, previous), + })); + previous = Some(memo); + } + let mut memo = json!({ "nodes": table.nodes, "steps": json_steps }); + if run.run_until == RunUntil::Stage3 { + match selected_choice(&run.run) { + Ok(choice) => memo["selected_choice"] = choice_json(&choice), + Err(reason) => { + memo["selected_choice"] = Value::Null; + memo["selected_choice_unavailable"] = json!(reason); + } + } + } + memo +} + +/// The selected plan's candidate for each group of the Pass 2 memo +/// (`run.logical_asap_candidates`), or why there is none. +fn selected_choice(run: &PlanningRun) -> Result { + match ( + &run.selection, + &run.logical_asap_candidates, + &run.physical_asap_candidates, + ) { + (Ok(selection), Some(logical), Some(physical)) => { + selected_chosen_candidates(&logical.memo, physical, &selection.selected_dag) + } + (Err(error), _, _) => Err(format!("no plan was selected: {error}")), + _ => Err("no plan was selected".into()), + } +} + +/// The [`metric_type_warnings`] of each PromQL query of `described` (the +/// document's queries) over `data`, prefixed with the query's id; the query +/// still plans. +fn metric_type_warnings_of( + described: &[Value], + data: &DataWorkload, +) -> Result, String> { + let mut warnings = Vec::new(); + for query in described.iter().filter(|q| q["language"] == "promql") { + let (id, text) = (&query["id"], query["text"].as_str().unwrap_or_default()); + let id = id.as_str().unwrap_or_default(); + warnings.extend( + metric_type_warnings(text, data) + .map_err(|e| format!("{id}: {e}"))? + .into_iter() + .map(|warning| format!("{id}: {warning}")), + ); + } + Ok(warnings) +} + +/// A workload's queries as the document describes them, the frontends' +/// roots, the workload entries and the data workload. +type Lowered = ( + Vec, + Vec, + Vec, + DataWorkload, +); + +/// `workload` lowered by its language's frontend (SQL over `catalog`); its +/// data workload is empty when it declares none. +fn lowered(workload: &PlanningWorkload, catalog: &SqlCatalog) -> Result { + let roots = match workload.query_workload.language { + QueryLanguage::PromQL => lower_promql_roots(workload)?, + _ => lower_sql_roots(workload, catalog)?, + }; + let entries = workload.query_workload.entries().collect(); + let data = workload.data_workload.clone().unwrap_or_default(); + Ok((workload_queries(workload), roots, entries, data)) +} + +/// `workloads` (each `--workload` file's path and workload, in order), +/// each lowered by its own language's frontend, as one workload: their +/// queries in order, numbered `q1..qN` across the files, and one data +/// workload with every file's sources. A source several files declare must +/// have the same facts in each, and their arrivals must agree +/// ([`arrival_together`]). When several files' data arrives, every source +/// must declare its own rate (`series_count` and `sample_interval`): each +/// declared source, each metric a PromQL query reads and, when a file is +/// SQL, each of `sql_tables` (the `--sql-catalog-table` names). Plan selection shares a +/// workload's `rows_per_second` out over all its sources, so one file's +/// rate would otherwise be priced on another file's sources. +fn lowered_together( + workloads: &[(&str, PlanningWorkload)], + catalog: &SqlCatalog, + sql_tables: &[String], +) -> Result { + let (mut described, mut roots, mut entries) = (Vec::new(), Vec::new(), Vec::new()); + let mut data: Option = None; + // The sources the queries read, beside the declared ones. + let mut read_sources: BTreeSet = BTreeSet::new(); + for (path, workload) in workloads { + match workload.query_workload.language { + QueryLanguage::PromQL => { + for entry in workload.query_workload.entries() { + read_sources.extend( + promql_metric_names(&entry.query.0) + .map_err(|e| format!("--workload {path}: {e}"))?, + ); + } + } + _ => read_sources.extend(sql_tables.iter().cloned()), + } + let (queries, more_roots, more_entries, more_data) = + lowered(workload, catalog).map_err(|e| format!("--workload {path}: {e}"))?; + described.extend(queries); + roots.extend(more_roots); + entries.extend(more_entries); + let Some(data) = data.as_mut() else { + data = Some(more_data); + continue; + }; + data.arrival = arrival_together(data.arrival, more_data.arrival).ok_or_else(|| { + format!( + "--workload {path}: its data arrival {:?} disagrees with the earlier files' {:?} \ + (a file without a data_workload declares an unknown arrival); files planned \ + together must declare the same kind of arrival", + more_data.arrival, data.arrival + ) + })?; + for (source, facts) in more_data.sources { + match data.sources.entry(source) { + Entry::Vacant(slot) => { + slot.insert(facts); + } + Entry::Occupied(earlier) if *earlier.get() != facts => { + return Err(format!( + "--workload {path}: source {} is declared with other facts in an earlier file", + earlier.key() + )) + } + Entry::Occupied(_) => {} + } + } + } + let data = data.unwrap_or_default(); + if workloads.len() > 1 && data.arrival.rows_per_second().is_some() { + let has_own_rate = |source: &String| { + data.sources.get(source).is_some_and(|facts| { + facts.series_count.is_some() && facts.sample_interval.is_some() + }) + }; + if let Some(source) = data + .sources + .keys() + .chain(&read_sources) + .find(|source| !has_own_rate(source)) + { + return Err(format!( + "--workload: source {source} declares no rate of its own (series_count and \ + sample_interval); with several files, each source needs one, since a file's \ + rows_per_second is shared out over every file's sources" + )); + } + } + data.validate().map_err(|e| e.to_string())?; + for (index, query) in described.iter_mut().enumerate() { + query["id"] = json!(format!("q{}", index + 1)); + } + Ok((described, roots, entries, data)) +} + +/// The arrival of two files' data planned together: ingesting (or mixed) +/// at the sum of their rates; the same arrival when both are at rest or +/// unknown; `None` when they differ. +fn arrival_together(earlier: DataArrival, later: DataArrival) -> Option { + match (earlier, later) { + ( + DataArrival::Ingesting { rows_per_second: a }, + DataArrival::Ingesting { rows_per_second: b }, + ) => Some(DataArrival::Ingesting { + rows_per_second: a + b, + }), + (DataArrival::Mixed { rows_per_second: a }, DataArrival::Mixed { rows_per_second: b }) => { + Some(DataArrival::Mixed { + rows_per_second: a + b, + }) + } + (DataArrival::AtRest, DataArrival::AtRest) => Some(DataArrival::AtRest), + (DataArrival::Unknown, DataArrival::Unknown) => Some(DataArrival::Unknown), + _ => None, + } +} + +/// The selected plan's materialized views, as the plan +/// [`installable_plan_of`] (what `plan_installable` returns) derives them: +/// each view's node id in the installable DAG, `refresh`, `coverage`, +/// `inputs` and `retention_ms`, +/// plus the node of the selected Stage 2 candidate it is installed from +/// (`stage2_node_id`, null when Stage 3 recorded no cost to map it by), its +/// position among that node's installed nodes (`installed_position`: a +/// tumbling-window chain's tumbling windows, newest first) and their number (`installed_count`). +fn materialized_views_json( + roots: &[QueryRoot], + selection: &Selection, + tumbling_window_retention_ms: &std::collections::BTreeMap< + asap_types::ir::physical_export::PhysicalASAPNodeId, + u64, + >, + entries: &[QueryWorkloadEntry], +) -> Result, String> { + let (plan, report) = installable_plan_of( + roots, + Some(selection), + tumbling_window_retention_ms, + entries, + ) + .map_err(|e| e.to_string())?; + let views = plan.materialized_views().map_err(|e| e.to_string())?; + let mut stage2_of = std::collections::HashMap::new(); + for node in report + .iter() + .flat_map(|report| &report.predicted_usage.per_node) + { + for (position, installed) in node.installed_node_ids.iter().enumerate() { + stage2_of.insert( + *installed, + (node.stage3_node_id, position, node.installed_node_ids.len()), + ); + } + } + views + .iter() + .map(|view| { + let mut value = serde_json::to_value(view).map_err(|e| e.to_string())?; + let (stage2_node_id, position, count) = match stage2_of.get(&view.node_id) { + Some((id, position, count)) => (json!(id), json!(position), json!(count)), + None => (Value::Null, Value::Null, Value::Null), + }; + value["stage2_node_id"] = stage2_node_id; + value["installed_position"] = position; + value["installed_count"] = count; + Ok(value) + }) + .collect() +} + +/// The ways to compute a measure Pass 1 pruned +/// (`PerComputationLogicalMemoCandidates::pruned_candidates`), in pruning +/// order: each with its aggregate's memo `group`, the `candidate` and the +/// `reason` (`PruneReason`, e.g. `UnsupportedByDeployment`). Empty when +/// Pass 1 failed. +fn pass1_pruned_candidates_json(pass1: Option<&PerComputationLogicalMemoCandidates>) -> Vec { + pass1 + .map_or(&[][..], |pass1| &pass1.pruned_candidates) + .iter() + .map(|pruned| { + json!({ + "group": pruned.group.0, + "candidate": computation_label(&pruned.candidate), + "reason": format!("{:?}", pruned.reason), + }) + }) + .collect() +} + +/// The physical candidates Stage 2 pruned +/// (`PhysicalASAPMemoCandidates::pruned_candidates`), in pruning order: +/// each with its memo `group`, its `operator_node` (index into Stage 2's +/// operator nodes), the node's `operator` label, whether it is an +/// `implementation_root`, its `timing` and `materialization`, and the +/// `reason` (`PruneReason`, e.g. +/// `RawDataNotRetained`). Empty without memo steps. +fn stage2_pruned_candidates_json(stage2: Option<&PhysicalASAPMemoCandidates>) -> Vec { + let Some(stage2) = stage2 else { + return Vec::new(); + }; + stage2 + .pruned_candidates + .iter() + .map(|pruned| { + let key = &pruned.candidate; + let node = &stage2.operator_nodes_of_candidates_for_memo_groups[key.operator_node.0]; + let flat = export(&[QueryRoot::Operator(node.physical_node.clone())]); + json!({ + "group": pruned.group.0, + "operator_node": key.operator_node.0, + "operator": flat.nodes.last().map(node_label).unwrap_or_default(), + "implementation_root": node.is_implementation_root, + "timing": key.timing, + "materialization": key.materialization, + "reason": format!("{:?}", pruned.reason), + }) + }) + .collect() +} + +/// One plan's cost: per second, its predicted CPU, unit and source, its +/// usage and each node's charge. +fn cost_json(cost: &asap_cost::PlanCost) -> Value { + let per_node: serde_json::Map<_, _> = cost + .per_node + .iter() + .map(|(node, c)| { + ( + node.to_string(), + json!({ + "operator": c.operator, + "timing": c.timing, + "cpu_core_seconds_per_second": c.cpu_core_seconds_per_second, + "cost_per_second": c.cost_per_second, + "multiplier": c.multiplier, + "detail": c.detail, + "price_source": c.price_source, + }), + ) + }) + .collect(); + json!({ + "cost_per_second": cost.cost_per_second, + // Report-only: the nodes' predicted CPU, summed as + // `plan_installable`'s predicted usage sums it. + "cpu_core_seconds_per_second": cost + .per_node + .values() + .map(|node| node.cpu_core_seconds_per_second) + .sum::(), + "cost_unit": cost.cost_unit, + "source": cost.source, + "usage": usage_json(&cost.usage), + "per_node": per_node, + }) +} + +/// Per query, the accuracy Stage 3 accepted: the target's metric, the +/// accepted metric, whether it was accepted regardless of its guarantee, +/// why the guarantee check would have rejected it, the sketches read and a +/// one-line summary. +fn accepted_json(queries: &[AcceptedAccuracy]) -> Value { + queries + .iter() + .map(|accepted| { + json!({ + "target_metric": accepted.target_metric, + "accepted_metric": accepted.accepted_metric, + "accepted_regardless_of_guarantee": accepted.accepted_regardless_of_guarantee, + "guarantee_check_rejection": accepted.guarantee_check_rejection, + "sketches": accepted.sketches, + "summary": accepted.to_string(), + }) + }) + .collect() +} + +/// Stage 3's selection ([`Selection`]): the selected DAG over the selected +/// round's Stage 2 candidates (one node per run of a candidate), the nodes +/// whose weights it proved non-negative, the plan's cost, the accuracy accepted per query, the +/// candidates it pruned and its search report; `error` when the facade +/// selected no plan. +fn selection_json(selection: &Result) -> Value { + let selection = match selection { + Ok(selection) => selection, + Err(error) => return json!({ "selected_dag": null, "error": error.to_string() }), + }; + let dag = &selection.selected_dag; + let nodes = |nodes: &[asap_types::stages::SelectedNodeId]| -> Vec { + nodes.iter().map(|node| node.0).collect() + }; + json!({ + "selected_dag": { + "candidate_of_node": dag.candidate_of_node.iter().map(|id| id.0).collect::>(), + "edges": dag + .edges + .iter() + .map(|e| json!({ "producer": e.producer.0, "consumer": e.consumer.0, "input": e.input })) + .collect::>(), + "node_of_output_reference": nodes(&dag.node_of_output_reference), + }, + "nodes_with_proven_non_negative_weights": + nodes(&selection.nodes_with_proven_non_negative_weights), + "cost": cost_json(&selection.cost), + "accepted_accuracy_per_query": accepted_json(&selection.accepted_accuracy_per_query), + "pruned_candidates": selection + .pruned_candidates + .iter() + .map(|p| json!({ "group": p.group.0, "candidate": p.candidate.0, "reason": format!("{:?}", p.reason) })) + .collect::>(), + "search_report": selection.search_report, + }) +} + +/// A candidate's resource usage, in the units its field names give. +fn usage_json(usage: &ResourceUsage) -> Value { + json!({ + "cpu_ops_per_second": usage.cpu_ops_per_second, + "scanned_bytes_per_second": usage.scanned_bytes_per_second, + "average_memory_bytes": usage.average_memory_bytes, + "peak_memory_bytes": usage.peak_memory_bytes, + "disk_bytes": usage.disk_bytes, + "peak_cpu_cores": usage.peak_cpu_cores, + "latency_per_query": usage + .latency_per_query + .iter() + .map(|l| json!({ "compaction_ms": l.compaction_ms, "query_ms": l.query_ms })) + .collect::>(), + }) +} + +/// A logical DAG as a flat node list: node `i` is `nodes[i]`, and children are +/// node indices inside each operator. +fn export(roots: &[QueryRoot]) -> FlatDag { + flatten(roots).0 +} + +fn workload_queries(workload: &PlanningWorkload) -> Vec { + let language = match workload.query_workload.language { + QueryLanguage::PromQL => "promql", + _ => "sql", + }; + workload + .query_workload + .entries() + .enumerate() + .map(|(index, entry)| { + let accuracy = json!(entry.requirements.accuracy.clone()); + let mut requirements = json!({ "accuracy": accuracy }); + if let Some(ms) = entry.requirements.max_latency_ms { + requirements["max_latency_ms"] = json!(ms); + } + if let QueryRecurrence::Repeated(RepeatedDemand::FixedInterval(interval)) = + &entry.recurrence + { + requirements["repeat_interval_ms"] = json!(interval.0); + } + if let Some(range) = entry.range_evaluation { + requirements["range_evaluation"] = json!(range); + } + json!({ + "id": format!("q{}", index + 1), + "language": language, + "text": entry.query.0, + "requirements": requirements, + // How often and when the query runs, as the workload entry + // declares it. + "recurrence": entry.recurrence, + "predictability": entry.predictability, + }) + }) + .collect() +} + +/// `queries` with, for each approximate target that sets no metric, the +/// metric its root's result is naturally stated in +/// ([`default_metric_for_root`]; null for a quantile, which has none) as +/// `requirements.default_metric`. Display only: Stage 3 accepts any metric. +fn with_default_metrics(mut queries: Vec, roots: &[QueryRoot]) -> Vec { + for (query, root) in queries.iter_mut().zip(roots) { + let accuracy = &query["requirements"]["accuracy"]; + let Ok(target) = serde_json::from_value::(accuracy.clone()) else { + continue; + }; + if target.accuracy_metric.is_none() && !target.is_exact() { + query["requirements"]["default_metric"] = json!(default_metric_for_root(root)); + } + } + queries +} + +/// Group id (as a string) to candidate index. +fn choice_json(choice: &ChosenCandidatePerMemoGroup) -> Value { + Value::Object( + choice + .iter() + .map(|(group, &candidate)| (group.0.to_string(), json!(candidate))) + .collect(), + ) +} + +/// The `memo` section's node table (`memo.nodes`): each distinct node once, +/// children before parents, its children as indices into the table. Nodes +/// are matched by their serialized JSON (children included), since the +/// steps rebuild equal nodes as different `Rc`s. +#[derive(Default)] +struct MemoNodes { + nodes: Vec, + index_of: HashMap, +} + +impl MemoNodes { + /// Adds `roots` and every node below them. Returns each root's index + /// and, for each node of that DAG, the original node and its index. + fn add( + &mut self, + roots: &[Rc], + ) -> (Vec, Vec<(Rc, NodeId)>) { + let roots: Vec<_> = roots.iter().cloned().map(QueryRoot::Operator).collect(); + let (dag, originals) = flatten(&roots); + let mut index_in_table: Vec = Vec::with_capacity(dag.nodes.len()); + for mut node in dag.nodes { + node.operator = node.operator.map_children(|&child| index_in_table[child]); + let key = serde_json::to_string(&node).expect("a flat node serializes"); + let index = *self.index_of.entry(key).or_insert_with(|| { + self.nodes.push(node); + self.nodes.len() - 1 + }); + index_in_table.push(index); + } + let roots = dag + .roots + .iter() + .map(|root| match root { + QueryRoot::Operator(id) => index_in_table[*id], + QueryRoot::Scalar(_) => unreachable!("memo DAGs have operator roots"), + }) + .collect(); + (roots, originals.into_iter().zip(index_in_table).collect()) + } +} + +/// `memo`'s groups, each with its candidates, their +/// nodes added to `table`; a candidate is `added` when `previous` (the memo +/// at the step before) has no candidate of that group with the same node and +/// children. +fn memo_groups_json(table: &mut MemoNodes, memo: &Memo, previous: Option<&Memo>) -> Vec { + let ids = |groups: &[MemoGroupId]| groups.iter().map(|g| g.0).collect::>(); + memo.groups + .iter() + .zip(&memo.candidates) + .enumerate() + .map(|(id, (group, candidates))| { + let node = table.add(std::slice::from_ref(&group.logical_node)).0[0]; + let mut by_origin = serde_json::Map::new(); + for candidate in candidates { + let count = by_origin + .entry(origin_name(&candidate.origin)) + .or_insert(json!(0)); + *count = json!(count.as_u64().unwrap_or(0) + 1); + } + let earlier = previous + .and_then(|memo| memo.candidates.get(id)) + .map_or(&[][..], Vec::as_slice); + let listed: Vec<_> = candidates + .iter() + .enumerate() + .map(|(index, candidate)| { + let (roots, originals) = table.add(std::slice::from_ref(&candidate.root_node)); + let group_of_node: serde_json::Map<_, _> = originals + .iter() + .filter_map(|(original, node)| { + candidate + .input_groups + .iter() + .find(|child| { + Rc::ptr_eq( + &memo.groups[child.0 as usize].logical_node, + original, + ) + }) + .map(|child| (node.to_string(), json!(child.0))) + }) + .collect(); + json!({ + "index": index, + "implements": std::iter::once(id as u32) + .chain(candidate.fused_groups.iter().map(|g| g.0)) + .collect::>(), + "children": ids(&candidate.input_groups), + "origin": origin_name(&candidate.origin), + "accuracy_budget_parts": candidate.accuracy_budget_parts, + "added": !earlier.iter().any(|before| { + Rc::ptr_eq(&before.root_node, &candidate.root_node) + && before.input_groups == candidate.input_groups + }), + "label": candidate_node_label(&table.nodes, roots[0], &group_of_node), + "node": roots[0], + "group_of_node": group_of_node, + }) + }) + .collect(); + json!({ + "id": id, + "label": node_label(&table.nodes[node]), + "children": ids(&group.child_groups), + "parents": ids(group.parent_groups()), + "queries": group.reached_by_queries(), + "roots_of": queries_rooted_at(&memo.query_roots, MemoGroupId(id as u32)), + "node": node, + "candidate_count": candidates.len(), + "candidate_count_by_origin": by_origin, + "candidates_capped": false, + "candidates": listed, + }) + }) + .collect() +} + +/// `OriginOfCandidateForMemoGroup` in snake case. +fn origin_name(origin: &OriginOfCandidateForMemoGroup) -> String { + match origin { + OriginOfCandidateForMemoGroup::Pass1 => "pass1", + OriginOfCandidateForMemoGroup::SharedBuild => "shared_build", + OriginOfCandidateForMemoGroup::SummaryResult => "summary_result", + OriginOfCandidateForMemoGroup::SummaryInput => "summary_input", + OriginOfCandidateForMemoGroup::TumblingWindows => "tumbling_windows", + OriginOfCandidateForMemoGroup::Rollup => "rollup", + OriginOfCandidateForMemoGroup::TopKReuse => "top_k_reuse", + OriginOfCandidateForMemoGroup::WidenedScans => "widened_scans", + } + .into() +} + +/// A candidate's own nodes (those above its child groups' nodes), root +/// first, as `node_label`s joined by " ← ". +fn candidate_node_label( + nodes: &[FlatNode], + root: NodeId, + group_of_node: &serde_json::Map, +) -> String { + let mut labels = Vec::new(); + let mut pending = vec![root]; + let mut seen = HashSet::new(); + while let Some(id) = pending.pop() { + if group_of_node.contains_key(&id.to_string()) || !seen.insert(id) { + continue; + } + labels.push(node_label(&nodes[id])); + pending.extend(nodes[id].operator.children().into_iter().rev().copied()); + } + labels.join(" ← ") +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A node two DAGs share is stored once in the memo's node table, even + /// when each DAG built it as its own `Rc`, and each DAG rebuilt from the + /// table equals its own `FlatDag` export. + #[test] + fn memo_node_table_stores_a_shared_node_once() { + let root = |query: &str| { + let workload: PlanningWorkload = serde_json::from_value(json!({ + "query_workload": {"language": "promql", "query_batch": [{ + "query": query, "invocations": 1, + "requirements": {"accuracy": {"max_error": 0.0, "failure_probability": 0.0}}, + "predictability": {"predictable": {"known_at": null}}, + }]}, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 1.0}}, + "sources": {"m": {"series_count": 10, "sample_interval": 15000, "metric_type": "gauge"}}, + }, + })) + .unwrap(); + match lower_promql_roots(&workload).unwrap().remove(0) { + QueryRoot::Operator(node) => node, + QueryRoot::Scalar(_) => unreachable!(), + } + }; + let (sum, max) = (root("sum(m)"), root("max(m)")); + let mut table = MemoNodes::default(); + let (sum_root, _) = table.add(std::slice::from_ref(&sum)); + let after_sum = table.nodes.len(); + let (max_root, _) = table.add(std::slice::from_ref(&max)); + // `max(m)` adds only its own aggregate: the scan is already there. + assert_eq!(table.nodes.len(), after_sum + 1); + // The DAG below `root`, renumbered in table order. + let rebuilt = |root: NodeId| { + let mut reached = BTreeSet::new(); + let mut pending = vec![root]; + while let Some(id) = pending.pop() { + if reached.insert(id) { + pending.extend(table.nodes[id].operator.children().into_iter().copied()); + } + } + let local: HashMap<_, _> = reached.iter().enumerate().map(|(i, &id)| (id, i)).collect(); + let nodes: Vec<_> = reached + .iter() + .map(|&id| { + let mut node = table.nodes[id].clone(); + node.operator = node.operator.map_children(|child| local[child]); + node + }) + .collect(); + FlatDag { + nodes, + roots: vec![QueryRoot::Operator(local[&root])], + } + }; + for (node, root) in [(sum, sum_root[0]), (max, max_root[0])] { + assert_eq!( + serde_json::to_value(rebuilt(root)).unwrap(), + serde_json::to_value(export(&[QueryRoot::Operator(node)])).unwrap() + ); + } + } +} diff --git a/crates/devtools/src/bin/convert_rqe_atomic_costs.rs b/crates/devtools/src/bin/convert_rqe_atomic_costs.rs index 85e440774..a08cdcca8 100644 --- a/crates/devtools/src/bin/convert_rqe_atomic_costs.rs +++ b/crates/devtools/src/bin/convert_rqe_atomic_costs.rs @@ -5,9 +5,8 @@ // and writes plan selection's measured table: a JSON array of // `{"sketch": , "operation": "update|merge|estimate", // "cpu_ns_per_operation": }`, three entries per converted row -// (`asap_devtools::rqe_atomic_costs`). Pass it to `stage_pipeline -// --measured-operation-costs`, or set it as -// `CostInputs::measured_operation_costs`. Skipped rows (exact +// (`asap_devtools::rqe_atomic_costs`). Set it as +// `CostInputs::measured_operation_costs` (`asap_planner --cost-inputs`). Skipped rows (exact // accumulators, Hydra structures, unknown names) are listed on stderr. use asap_devtools::rqe_atomic_costs::convert_rqe_atomic_costs; diff --git a/crates/devtools/src/bin/planner_fuzz.rs b/crates/devtools/src/bin/planner_fuzz.rs index 72eecde62..ce59b86ba 100644 --- a/crates/devtools/src/bin/planner_fuzz.rs +++ b/crates/devtools/src/bin/planner_fuzz.rs @@ -16,9 +16,9 @@ // // `--relations` takes relation names or the letters a–o (default: all); // `--search` is `candidate-connections` (the default search) or -// `exhaustive`; `--ignore-accuracy-guarantees` plans as `stage_pipeline` -// does by default (guarantees are enforced here by default, as -// `plan_installable` does). A relation is violated when it misses by more +// `exhaustive`; `--ignore-accuracy-guarantees` accepts every output +// regardless of its accuracy guarantee (guarantees are enforced by +// default, as `plan_installable` does). A relation is violated when it misses by more // than max(`--relative-tolerance` (default 0.01) × cost, // `--absolute-tolerance` (default 1e-6)). `--max-shrink-attempts` (default // 60) bounds the checks one shrink runs. Each case and each shrink runs in diff --git a/crates/devtools/src/bin/show_logical_dag.rs b/crates/devtools/src/bin/show_logical_dag.rs deleted file mode 100644 index 4b3812a5f..000000000 --- a/crates/devtools/src/bin/show_logical_dag.rs +++ /dev/null @@ -1,95 +0,0 @@ -// cargo run -p asap-devtools --bin show_logical_dag -- queries.txt -// (or pipe via stdin: cargo run -p asap-devtools --bin show_logical_dag < queries.txt) -// -// Lowers a batch of ad-hoc SQL/PromQL queries to **pre-ASAP IR** (the -// sketch-agnostic intent algebra: an `OperatorNode` DAG of `NonASAPOp` -// operators with `AggIntent` measures) and prints them. -// See `stage_pipeline` for the planner stages one layer -// downstream — this tool never picks a sketch, it only shows what a query -// means. -// -// File format: one query per line, prefixed with "sql>" or "promql>". -// Blank lines and lines starting with '#' are ignored. -// -// sql> SELECT service, COUNT(*) FROM metrics GROUP BY service -// promql> topk(5, rate(http_requests_total[5m])) -// -// SQL queries run against a fixed `metrics(ts, service, region, latency, -// bytes)` catalog — the same table used in cross_language.rs and topk_ir.rs. - -use asap_devtools::{lower_promql_with_sample_interval, lower_sql, SqlCatalog}; -use asap_types::ir::properties::AccuracyTarget; -use asap_types::ir::schema::{DataType, Field, Schema}; -use std::io::Read; - -fn col(name: &str, dtype: DataType) -> Field { - Field::plain(name, dtype, false) -} - -fn catalog() -> SqlCatalog { - SqlCatalog::new().with_table( - "metrics", - Schema::with_time_index( - vec![ - col("ts", DataType::Timestamp), - col("service", DataType::Utf8), - col("region", DataType::Utf8), - col("latency", DataType::Float64), - col("bytes", DataType::Int64), - ], - 0, - vec![], - ), - ) -} - -#[tokio::main] -async fn main() { - let mut args = std::env::args().skip(1); - assert_eq!( - args.next().as_deref(), - Some("--sample-interval-ms"), - "usage: show_logical_dag --sample-interval-ms [queries.txt]" - ); - let interval_ms = args - .next() - .expect("missing interval") - .parse() - .expect("interval must be an unsigned integer"); - let input = match args.next() { - Some(path) => { - std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("failed to read {path}: {e}")) - } - None => { - let mut buf = String::new(); - std::io::stdin() - .read_to_string(&mut buf) - .expect("failed to read stdin"); - buf - } - }; - - let catalog = catalog(); - for line in input.lines() { - let line = line.trim(); - if line.is_empty() || line.starts_with('#') { - continue; - } - println!("━━━ {line} ━━━"); - if let Some(q) = line.strip_prefix("sql>") { - match lower_sql(q.trim(), &catalog, AccuracyTarget::exact()).await { - Ok(qe) => println!("{qe:#?}"), - Err(e) => println!("ERR: {e}"), - } - } else if let Some(q) = line.strip_prefix("promql>") { - match lower_promql_with_sample_interval(q.trim(), AccuracyTarget::exact(), interval_ms) - { - Ok(qe) => println!("{qe:#?}"), - Err(e) => println!("ERR: {e}"), - } - } else { - println!("ERR: line must start with 'sql>' or 'promql>'"); - } - println!(); - } -} diff --git a/crates/devtools/src/bin/stage_pipeline.rs b/crates/devtools/src/bin/stage_pipeline.rs deleted file mode 100644 index 03c2809ae..000000000 --- a/crates/devtools/src/bin/stage_pipeline.rs +++ /dev/null @@ -1,2154 +0,0 @@ -// cargo run -p asap-devtools --bin stage_pipeline -- \ -// --example planner-layering-1 --exhaustive --enforce-accuracy-guarantees \ -// --max-displayed 100 --out planner-layering-example1.json -// (also planner-layering-2: #509 Example 2, three SQL flow statistics; -// planner-layering-3a and planner-layering-3b: #509 Example 3, Patterns A -// and B; 3a has 2,187 physical candidates, all priced, 64 shown by default) -// cargo run -p asap-devtools --bin stage_pipeline -- \ -// --workload dashboard.json --out run.json -// (a `PlanningWorkload` JSON file, e.g. with a repeating query's -// `"range_evaluation": {"step_ms": 60000, "span_ms": 3600000}`; PromQL, or -// SQL over `--table` declarations) -// cargo run -p asap-devtools --bin stage_pipeline -- \ -// --workload promql.json --workload sql.json --table '' --out run.json -// (`--workload` repeats: each file keeps its one language and is lowered by -// its own frontend, and their queries are planned as one workload; see -// below) -// cargo run -p asap-devtools --bin stage_pipeline -- \ -// --promql "topk by (job) (10, rate(x[1m]))" --max-error 0.01 --failure-probability 0.001 --out run.json -// cargo run -p asap-devtools --bin stage_pipeline -- \ -// --table '{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp"}, -// {"name": "service", "type": "utf8"}], "time_index": 0}' \ -// --sql "SELECT service, COUNT(*) FROM metrics GROUP BY service" --out run.json -// -// Writes an `asap-stage-pipeline/v1` document (tools/dag-viewer) with the -// four planner stages as the facade runs them (`asap_planner::run_planning`). -// The default document keeps the selected plan; `--exhaustive` adds the -// reference enumeration: Pass 2's memo choices, each implemented -// by Stage 2 with every materialization assignment, all priced by the -// reference pricer (`asap_plan_selection::reference_pricer`), under -// `reference_pricing`. -// Unlike `plan_installable`'s default settings, Stage 3 ignores accuracy -// guarantees unless `--enforce-accuracy-guarantees` is given (below); -// `accuracy_guarantees` is `"ignored"` or `"enforced"`. -// - stage0_query_language_frontends: the frontends' workload DAG, one root per query; -// - stage1_logical_asap_optimization: `L1`, the selected plan's roots (as -// Stage 2 built them, with Stage 3's weight proofs), then under -// `--exhaustive` the enumerated choices `E` (the memo itself, after -// each step, is under `memo`). The document shows the logical -// candidates of the physical candidates it shows (below), and those among -// the first `--max-displayed` that have none, with `combinations` (how -// many there are), `capped` (whether some are not shown) and -// `pruned_candidates` (the ways to compute a measure Pass 1 pruned, -// each on its aggregate's memo `group`: -// `PerComputationLogicalMemoCandidates::pruned_candidates`, with -// `candidate` and `reason`). A candidate that cannot be composed carries -// `error` instead of `dag`; -// - stage2_physical_asap_optimization: `P1`, the selected plan as Stage 2 -// built it, then under `--exhaustive` each enumerated choice's physical -// plans `E-`; an enumerated choice Stage 2 cannot implement is -// listed under `pruned` with the reason. `pruned_candidates` lists the -// physical candidates Stage 2 pruned -// (`PhysicalASAPMemoCandidates::pruned_candidates`): each with its memo -// `group`, `operator_node`, `operator`, `implementation_root`, `timing`, -// `materialization` and `reason` (`PruneReason`: -// `PerEntityAggregateNotExecutable`, `RawDataNotRetained`, -// `InputHasNoBuildableProducer`, `ReachesNoQueryOutput`, -// `TooFewQueryReaders`). The document shows at -// most `--max-displayed` (default 64) enumerated physical candidates: the -// reference pricer's pick, the other valid ones cheapest first, then the -// rejected ones taking each cause in turn; -// - stage3_plan_selection: Stage 3's `Selection`: `selected_dag` -// (`DagOfSelectedCandidates`: `candidate_of_node`, the candidate id of -// Stage 2's output each node runs, one node per run; -// `edges`, each `producer` node feeding input `input` of a `consumer` -// node; and `node_of_output_reference`), -// `nodes_with_proven_non_negative_weights`, the plan's `cost` (its -// `usage`, `cost_per_second` and per-node charges: each node's `cost` is -// one computation's price times `multiplier`, the times it is computed), -// `accepted_accuracy_per_query` (per query: the target's metric or null -// when not set, the accepted guarantee's metric or null when exact or -// accepted regardless of guarantee, `accepted_regardless_of_guarantee`, -// `guarantee_check_rejection`, the sketches read and a one-line summary), -// `pruned_candidates` and `search_report` (`SearchReport`: `status`, -// `cost_lower_bound_per_second`, MILP size and timings, and -// `inexact_objective` when the selection MILP's objective and `plan_cost` -// differ over terms the MILP encodes inexactly). Stage 3 runs #1095's -// selection MILP, which supports cost model 1 only (cost models 2 and 3 -// are an `UnsupportedCostModel` error, #888); `error` when no -// plan was selected. `tumbling_window_retention_ms` maps each pane chain the -// selected plan keeps beyond one evaluation (by node id in `P1`) to how -// far back the runtime must keep its tumbling windows (`RealizedPlan::tumbling_window_retention_ms`). -// `materialized_views` are the selected plan's materialized views as -// `plan_installable`'s plan derives them (`installable_plan_of`, then -// `InstallablePlan::materialized_views`: each view's installed node id, -// `refresh`, `coverage` with its `definition_id` and `selection`, -// `inputs` and `retention_ms`), each with the node of `P1` it is -// installed from (`stage2_node_id`), its position among that node's -// installed nodes (`installed_position`, a tumbling-window chain's tumbling windows newest -// first) and their number (`installed_count`); empty without a -// selection, with `materialized_views_error` when the plan is not -// installable. `warnings` are one-line warnings about the workload and -// the selected plan: a PromQL query applying `rate`, `increase` or -// `irate` to a metric the data workload declares a gauge, or combining -// a declared summary's quantile series across series, prefixed with -// its id (the query still plans; `metric_type_warnings`), -// and the plan's peak memory when over 10^12 bytes and the cost model -// declares no memory capacity (`large_peak_memory_warning`); absent when -// there are none; -// - planning_report: the facade's `PlanningReport`: each stage's time, -// Pass 2's and Stage 2's counts, and Stage 3's search report; -// - reference_pricing (with `--exhaustive`): the reference pricer's -// selection among the enumerated plans, named as Stage 3's `Selection` -// names its fields: the shown candidates' `cost_per_candidate` and -// `accepted_accuracy_per_candidate`, its pick `selected_candidate`, -// every other shown candidate under `pruned_candidates` with its -// `cause` (a failed check: `accuracy_target_missed`, -// `count_min_weights_not_proven_non_negative`, `no_accuracy_guarantee`, -// `latency_bound_exceeded`, `latency_over_repeat_interval`, -// `over_cluster_capacity`, `cost_not_representable`; or a loss under -// the cost model: `costlier`, -// `slower`, `more_memory`, `tied_lost_on_enumeration_order`), -// `pruned_candidates_not_enumerated` (rejections naming no enumerated -// candidate) and `pruned_candidate_count_by_cause`. -// `memo` (unless `--no-memo`) is the memo the search built, from the -// planning run: `nodes`, one node -// table for the whole section (`MemoNodes`: each distinct node once, matched -// by its serialized JSON, children as indices into the table and before -// their parents); `steps`, one per -// memo step in pipeline order (`stage0`: the frontends' DAG hash-consed, -// groups without candidates; `pass1`: Pass 1's candidates, with the -// accuracy-scoped sizes (`PlanningRun::per_computation_candidates`); -// `pass2`: Pass 2's output (`PlanningRun::logical_asap_candidates`): local -// window composition, the Pass 2 memo rules (scan labels, summary inputs, -// top-k reuse, finer aggregates, rollups, summary results, state -// configurations) and the concrete summary-state groups, the memo the -// search selects from; each candidate's `origin` names the rule that made -// it). Each step lists its groups (`id`, `label`, -// `children`, `parents`, `queries`, `roots_of`, the group's `node` in -// `nodes`) and their -// candidates (`candidate_count`, `candidate_count_by_origin`, and the first -// `--max-memo-candidates` (default 64) under `candidates`, with -// `candidates_capped`): each with its `index` in the group, `implements`, -// `children`, `origin` (`OriginOfCandidateForMemoGroup` in snake case), `accuracy_budget_parts`, -// `added` (not in the group at the step before), a one-line `label`, its -// `node` in `nodes` and `group_of_node` (the `nodes` indices below it that -// are a child group's node, to that group's id). -// `selected_choice` maps each group of the last step the selected plan uses -// (by id, as a string) to its candidate's index; it is null, with -// `selected_choice_unavailable` saying why, when the plan cannot be read as -// one candidate per group. `enumeration` counts each step's candidate DAGs -// (`memo::enumeration::enumerate`) up to `--memo-enumeration-limit` -// (default 1000) under `per_step` (`count`, `complete`; Stage 0 counts its -// one DAG), and lists the first `--max-enumerated` (default 200) `choices` -// of `step`: the last step enumerated completely within that many, else -// the last step. Each choice has its `id` (`E1`, ...), `choice`, `roots` -// (`memo::enumeration::build_logical_asap_dags_from_chosen_candidates`'s roots, one per root group, as indices -// into `nodes`; null with `error` when it does not -// compose), `accuracy_budget_parts_1` (every chosen candidate sized for the whole -// budget) and `selected` (it is `selected_choice`). With -// `--price-enumerated ` (off by default), each of the first `n` listed -// choices is also run through Stage 2 and the reference pricer, and its -// selected plan installed (`memo_pricing::price_choices`, under the -// run's models): `valid`, `rejection` (a Stage 3 rejection cause, or -// `compose_failed`, `invalid_structure`, `stage2_failed`, `stage3_failed`, -// `rejected`, `not_executable` (the executor cannot compile the selected -// plan), `not_installable` (installation refuses it otherwise)) with -// `rejection_reason`, -// `cost_per_second` (its cheapest valid physical plan's) and -// `cheaper_than_selected` (valid and cheaper than the selected plan); and -// `priced` is `{count, cheaper_than_selected, selected_cost_per_second}`. -// Without it `priced` is null. -// `workload.data_workload` is the `DataWorkload` the run planned with, as -// declared (`--workload`, `--example`) or built from `--sample-interval-ms` -// (`--promql`); each query also carries its `recurrence` and -// `predictability` as its workload entry declares them. -// When Pass 1 fails, the document still has the Stage 0 DAG; Stage 1 has -// `error` and no candidates, and nothing is selected. -// -// Under `--exhaustive`, `--max-combinations` (default 10^4) bounds the memo -// choices enumerated. Display truncation (`--max-displayed`) is separate. -// -// `--workload` may repeat, so one run plans queries in several languages -// (e.g. a PromQL file and a SQL file) together. Each file is lowered by -// its own language's frontend (SQL over the `--table` declarations), the -// queries are numbered `q1..qN` across the files in the order given, and -// the data workload is the union of the files' sources. A source several -// files declare must have the same facts in each, and the files must -// declare the same kind of arrival: ingesting (or mixed) arrivals add -// their `rows_per_second`. Plan selection shares a workload's -// `rows_per_second` out over all its sources, so when the data arrives, -// every source of several files must declare its own rate -// (`series_count` and `sample_interval`). A disagreement or a source -// without its own rate is refused. -// With `--workload`, each query's accuracy (ε, δ, metric) comes from the -// file: `--max-error`, `--failure-probability` and `--accuracy-metric` apply -// to `--promql`/`--sql` queries only. -// `--promql` and `--sql` may repeat and mix, in workload order; `--name` -// names the query before it (default `q`). `--max-error` (ε) and -// `--failure-probability` (δ) apply to every query; without them the -// queries are exact. `--accuracy-metric` (`absolute`, `absolute_stream_l1`, `absolute_stream_l2`, -// `relative` or `rank`) sets the error metric of that target; without it, -// a candidate is accepted in whatever metric its guarantee meets ε (and δ) -// in, and `stage3_plan_selection.accepted_accuracy_per_query` reports, per -// query, the metric accepted (e.g. "metric not set; accepted rank (Kll)"). -// `--cost-model` chooses how plan selection compares valid candidates: -// `usage-over-time` (cost model 1, the default: CPU, scanned bytes and -// memory byte-seconds per second, priced), `provisioned-capacity` (cost -// model 2: peak memory, peak CPU cores and disk, priced) or -// `cluster-capacity` (cost model 3: no prices; `--cluster-memory-bytes`, -// `--cluster-disk-bytes` and the optional `--cluster-cpu-cores` are hard -// limits, and `--cluster-objective` is `lowest-latency` or `least-memory`). -// Cost models 1 and 2 use the illustrative unit costs; -// `aws-fargate-us-east-1` is cost model 1 at AWS Fargate's us-east-1 -// Linux/x86 prices (`ResourceUnitCost::aws_fargate_us_east_1`, in US dollars -// per second). -// Each cost's `usage` lists what the plan uses; `cost_per_second` is null under -// cost model 3. -// `--deployment-capabilities ''` declares what the deployment's -// executor can run and keep (`DeploymentCapabilities`): -// `{"supported_summaries": [{"family": "Kll", "readouts": ["Quantile"]}, ...], -// "raw_data_retained": true, "query_time_results_retained": false}`. Every -// field is optional: by default the sketch families and readouts the -// Planner's executor runs are supported (`supported_summaries` null, as -// `plan_installable` defaults it), raw data is retained, -// and no query-time result is kept across evaluations. Pass 1 offers no -// summary the deployment does not support (exact computations are always -// supported); Stage 1's `pruned_candidates` lists every candidate Pass 1 -// pruned and why (e.g. `UnsupportedByDeployment`). Without raw data retained, Stage 2 -// does not generate a query-time scan of raw data (its `pruned_candidates` -// lists each with `RawDataNotRetained`), and stored data (at rest or -// mixed) is refused as a contradiction; with `query_time_results_retained`, -// a tumbling-window chain whose readers evaluate together at its tumbling-window width is -// also offered built at query time and kept across evaluations, beside being -// shared within one evaluation. The document's top-level -// `deployment_capabilities` shows the capabilities used, with the -// executor's families filled in. -// `--measured-operation-costs ` reads the deployment's measured CPU -// time per operator (`MeasuredOperationCosts`: `{"operators": [...], -// "ns_per_evaluation": ..., "provenance": ...}`, or the earlier JSON array -// `convert_rqe_atomic_costs` writes from sketch-bench's -// `rqe_atomic_costs.json`); a node with an entry is priced from it, the -// others from the formulas times the calibration factor, and each node's -// `price_source` says `measured` or `analytical`. -// `--cost-inputs ` reads all of Stage 3's settings from -// one JSON file (`CostInputs`: `cost_model`, e.g. -// `{"usage_over_time": {"cost_per_cpu_op": 1e-6, ...}}` with the -// deployment's own unit costs, `resource_speed`, -// `non_repeating_query_horizon_seconds` and `measured_operation_costs`; -// an omitted field takes its default). It replaces `--cost-model`, the -// `--cluster-*` flags and `--measured-operation-costs`. -// Accuracy guarantees are ignored by default: stage_pipeline plans with -// `PlanningModels::ignore_accuracy_guarantees`, so the facade passes Stage 3 -// an accuracy model that accepts every query output regardless of its -// accuracy guarantee, and candidates are compared on cost alone; -// executability checks still apply. `--enforce-accuracy-guarantees` clears -// it, and Stage 3 rejects an output whose guarantee does not meet its -// query's target, as `plan_installable` does with the default models; Pass 1 -// then also prunes each summary with which no query reading it can meet its -// target (`NoErrorBoundForQueryTarget`). The document's top-level `accuracy_guarantees` is -// `"ignored"` or `"enforced"`. -// `--sample-interval-ms` is the sample interval declared for every metric the -// PromQL queries read (default 15000). `--series-count `, -// `--rows-per-second ` (the data then ingests at that rate) and -// `--rows-held ` declare those facts for every source the queries read -// (metric or `--table`); nothing else is declared. Plan selection prices -// only from declared facts: a scan whose source has no arrival rate -// (`--rows-per-second`, or `--series-count` with the sample interval) and -// no `--rows-held` fails, naming the source and the missing fact. -// `--table ''` (repeatable) declares a table the SQL queries read: `{"name": ..., -// "columns": [{"name": ..., "type": "timestamp|utf8|string|float64|double| -// int64|bigint", "nullable": true}], "time_index": 0}` (`nullable` defaults -// to true; without `time_index` the table has no time column; a -// `time_index` that is not a column index is an error). - -use std::collections::btree_map::Entry; -use std::collections::{BTreeSet, HashMap, HashSet}; -use std::rc::Rc; - -use asap_cost::{ - ClusterLimits, ClusterObjective, CostInputs, CostModel, MeasuredOperationCosts, - ResourceUnitCost, ResourceUsage, -}; -use asap_devtools::candidate_labels::{candidate_labels, computation_label}; -use asap_devtools::memo_pricing::{price_choices, SelectedBaseline}; -use asap_devtools::planner_layering_examples::{ - example1, example2, pattern_a, pattern_b, PatternARecurrence, SHARED_ROWS_PER_SECOND, -}; -use asap_devtools::planning_stages::{ - lower_promql_roots, lower_sql_roots, run_search_over, sql_catalog, PlanningStagesError, - PlanningStagesRun, -}; -use asap_devtools::SqlCatalog; -use asap_frontend_promql::{metric_type_warnings, promql_metric_names}; -use asap_logical_asap_optimization::DeploymentCapabilities; -use asap_logical_asap_optimization::{ - build_logical_asap_dags_from_chosen_candidates, enumerate, ChosenCandidatePerMemoGroup, -}; -use asap_plan_selection::reference_pricer::{Rejection, WholePlanSelection}; -use asap_plan_selection::{ - large_peak_memory_warning, AcceptedAccuracy, ErrorSelectingPlan, Selection, -}; -use asap_planner::{ - installable_plan_of, selected_chosen_candidates, PlanSearchSettings, PlanningModels, - PlanningRun, SelectionError, -}; -use asap_types::ir::flat::{flatten, FlatDag, FlatNode, NodeId}; -use asap_types::ir::operator::node::OperatorNode; -use asap_types::ir::properties::{default_metric_for_root, AccuracyMetric, AccuracyTarget}; -use asap_types::ir::QueryRoot; -use asap_types::physical::PhysicalWorkloadPlan; -use asap_types::stages::{queries_rooted_at, MemoGroupId}; -use asap_types::stages::{Memo, OriginOfCandidateForMemoGroup}; -use asap_types::stages::{PerComputationLogicalMemoCandidates, PhysicalASAPMemoCandidates}; -use asap_types::workload::{ - BatchEntry, DataArrival, DataWorkload, DurationMs, PlanningWorkload, Query, QueryLanguage, - QueryRecurrence, QueryRequirements, QueryWorkload, QueryWorkloadEntry, RepeatedDemand, - SqlDialect, -}; -use serde_json::{json, Value}; - -const USAGE: &str = - "usage: stage_pipeline (--example planner-layering-{1,2,3a,3b} | (--workload )... [--table ...] | ((--promql | --sql) \ -[--name ]... [--table ...] [--max-error --failure-probability --accuracy-metric ] [--sample-interval-ms ] [--series-count ] [--rows-per-second ] [--rows-held ])) \ -[--cost-model usage-over-time|aws-fargate-us-east-1|provisioned-capacity|cluster-capacity [--cluster-memory-bytes \ ---cluster-disk-bytes [--cluster-cpu-cores ] --cluster-objective lowest-latency|least-memory]] \ -[--cost-inputs ] [--enforce-accuracy-guarantees] [--deployment-capabilities ] [--exhaustive [--max-combinations ]] [--max-displayed ] \ -[--no-memo] [--max-memo-candidates ] [--memo-enumeration-limit ] [--max-enumerated ] [--price-enumerated ] --out "; - -/// Default evaluator-assignment budget for an exhaustive archive run. -const MAX_COMBINATIONS: usize = 10_000; - -/// Why a run wrote no document. -enum Failure { - /// Bad arguments or a query that does not plan (exit status 2). - Usage(String), -} - -impl From for Failure { - fn from(message: String) -> Self { - Failure::Usage(message) - } -} - -impl From<&str> for Failure { - fn from(message: &str) -> Self { - Failure::Usage(message.into()) - } -} - -fn main() { - match run(std::env::args().skip(1).collect()) { - Ok(()) => {} - Err(Failure::Usage(message)) => { - eprintln!("stage_pipeline: {message}\n{USAGE}"); - std::process::exit(2); - } - } -} - -/// The document for a run that ended with `result` over the frontends' -/// `roots`. A Pass 1 failure still gives a document: the Stage 0 DAG, the -/// error under Stage 1 and no selection. Refusing too many combinations or -/// a failed validation gives none. -fn document_of( - queries: Vec, - roots: &[QueryRoot], - entries: &[QueryWorkloadEntry], - result: Result, - max_displayed: usize, -) -> Result { - let queries = with_default_metrics(queries, roots); - match result { - Ok(run) => Ok(stage_pipeline(queries, entries, &run, max_displayed)), - Err(PlanningStagesError::Pass1(error)) => Ok(json!({ - "format": "asap-stage-pipeline/v1", - "workload": { "queries": queries }, - "stage0_query_language_frontends": { "dag": export(roots) }, - "stage1_logical_asap_optimization": { - "combinations": 0, - "capped": false, - "pruned_candidates": [], - "candidates": [], - "error": format!("Pass 1: {error}"), - }, - "stage2_physical_asap_optimization": { "candidates": [], "pruned": [], "pruned_candidates": [] }, - "stage3_plan_selection": { "selected_dag": null, "materialized_views": [] }, - })), - Err(error @ PlanningStagesError::Validation(_)) => Err(error.to_string().into()), - } -} - -fn run(args: Vec) -> Result<(), Failure> { - let mut example = None; - // `--workload` files, planned together in this order. - let mut workload_paths: Vec = Vec::new(); - // (language, text, name) in workload order. - let mut queries: Vec<(QueryLanguage, String, Option)> = Vec::new(); - let mut tables = Vec::new(); - let (mut max_error, mut failure_probability, mut sample_interval_ms) = (None, None, 15_000u64); - // Facts declared for every source the --promql/--sql queries read. - let (mut series_count, mut rows_per_second, mut rows_held) = - (None::, None::, None::); - let mut accuracy_metric: Option = None; - let mut max_displayed = 64usize; - let mut max_combinations = MAX_COMBINATIONS; - let mut cost_model: Option = None; - let (mut cluster_memory, mut cluster_disk, mut cluster_cores) = (None, None, None); - let mut cluster_objective: Option = None; - let mut deployment_capabilities = DeploymentCapabilities::default(); - let mut out = None; - let mut measured_operation_costs_path: Option = None; - let mut cost_inputs_path: Option = None; - let mut enforce_accuracy_guarantees = false; - let mut exhaustive = false; - let mut max_combinations_given = false; - let mut write_memo = true; - let mut memo_caps = MemoCaps { - candidates_per_group: 64, - enumeration_limit: 1000, - enumerated: 200, - choices_to_price: 0, - }; - let mut args = args.into_iter(); - while let Some(flag) = args.next() { - let mut value = || args.next().ok_or(format!("{flag} needs a value")); - let number = |v: String| v.parse::().map_err(|e| format!("{v}: {e}")); - match flag.as_str() { - "--example" => example = Some(value()?), - "--workload" => workload_paths.push(value()?), - "--promql" => queries.push((QueryLanguage::PromQL, value()?, None)), - "--sql" => queries.push(( - QueryLanguage::SQL(SqlDialect::DataFusionSQL), - value()?, - None, - )), - "--name" => { - let name = value()?; - match queries.last_mut() { - Some((_, _, slot @ None)) => *slot = Some(name), - _ => return Err("--name follows a --promql or --sql query".into()), - } - } - "--table" => tables.push(value()?), - "--max-error" => max_error = Some(number(value()?)?), - "--failure-probability" => failure_probability = Some(number(value()?)?), - "--accuracy-metric" => { - let name = value()?; - accuracy_metric = Some( - serde_json::from_value(Value::String(name.clone())) - .map_err(|_| format!("unknown metric {name}"))?, - ) - } - "--sample-interval-ms" => { - sample_interval_ms = value()?.parse().map_err(|e| format!("{e}"))? - } - "--series-count" => series_count = Some(value()?.parse().map_err(|e| format!("{e}"))?), - "--rows-per-second" => rows_per_second = Some(number(value()?)?), - "--rows-held" => rows_held = Some(value()?.parse().map_err(|e| format!("{e}"))?), - "--max-displayed" => max_displayed = value()?.parse().map_err(|e| format!("{e}"))?, - "--max-combinations" => { - max_combinations = value()?.parse().map_err(|e| format!("{e}"))?; - max_combinations_given = true; - } - "--cost-model" => cost_model = Some(value()?), - "--cluster-memory-bytes" => { - cluster_memory = Some(value()?.parse::().map_err(|e| format!("{e}"))?) - } - "--cluster-disk-bytes" => { - cluster_disk = Some(value()?.parse::().map_err(|e| format!("{e}"))?) - } - "--cluster-cpu-cores" => { - cluster_cores = Some(value()?.parse::().map_err(|e| format!("{e}"))?) - } - "--cluster-objective" => cluster_objective = Some(value()?), - "--deployment-capabilities" => { - deployment_capabilities = serde_json::from_str(&value()?) - .map_err(|e| format!("--deployment-capabilities: {e}"))? - } - "--out" => out = Some(value()?), - "--measured-operation-costs" => measured_operation_costs_path = Some(value()?), - "--cost-inputs" => cost_inputs_path = Some(value()?), - "--enforce-accuracy-guarantees" => enforce_accuracy_guarantees = true, - "--exhaustive" => exhaustive = true, - "--no-memo" => write_memo = false, - "--max-memo-candidates" => { - memo_caps.candidates_per_group = value()?.parse().map_err(|e| format!("{e}"))? - } - "--memo-enumeration-limit" => { - memo_caps.enumeration_limit = value()?.parse().map_err(|e| format!("{e}"))? - } - "--max-enumerated" => { - memo_caps.enumerated = value()?.parse().map_err(|e| format!("{e}"))? - } - "--price-enumerated" => { - memo_caps.choices_to_price = value()?.parse().map_err(|e| format!("{e}"))? - } - other => return Err(format!("unknown argument {other}").into()), - } - } - let has_sql = queries - .iter() - .any(|(language, _, _)| *language != QueryLanguage::PromQL); - if !tables.is_empty() && !has_sql && workload_paths.is_empty() { - return Err("--table needs --sql".into()); - } - let out = out.ok_or("--out is required")?; - if (series_count.is_some() || rows_per_second.is_some() || rows_held.is_some()) - && (!workload_paths.is_empty() || example.is_some()) - { - return Err( - "--series-count, --rows-per-second and --rows-held declare facts for --promql/--sql \ - queries; a --workload or --example declares its own" - .into(), - ); - } - if max_combinations_given && !exhaustive { - return Err("--max-combinations limits the exhaustive search: it needs --exhaustive (bounds evaluated assignments)".into()); - } - if cost_inputs_path.is_some() - && (cost_model.is_some() - || cluster_memory.is_some() - || cluster_disk.is_some() - || cluster_cores.is_some() - || cluster_objective.is_some() - || measured_operation_costs_path.is_some()) - { - return Err( - "--cost-inputs replaces --cost-model, the --cluster-* flags \ - and --measured-operation-costs" - .into(), - ); - } - let cluster = (cluster_memory, cluster_disk, cluster_objective.as_deref()); - // The settings every flag below leaves at its default; the Fargate - // preset converts its prices at their `resource_speed`. - let default_settings = CostInputs::default(); - let cost_model = match (cost_model.as_deref(), cluster) { - (None | Some("usage-over-time"), (None, None, None)) if cluster_cores.is_none() => { - CostModel::UsageOverTime(ResourceUnitCost::illustrative()) - } - (Some("aws-fargate-us-east-1"), (None, None, None)) if cluster_cores.is_none() => { - CostModel::UsageOverTime(ResourceUnitCost::aws_fargate_us_east_1( - default_settings.resource_speed, - )) - } - (Some("provisioned-capacity"), (None, None, None)) if cluster_cores.is_none() => { - CostModel::ProvisionedCapacity(ResourceUnitCost::illustrative()) - } - (Some("cluster-capacity"), (Some(memory_bytes), Some(disk_bytes), Some(objective))) => { - CostModel::ClusterCapacity(ClusterLimits { - memory_bytes, - disk_bytes, - cpu_cores: cluster_cores, - objective: match objective { - "lowest-latency" => ClusterObjective::LowestLatency, - "least-memory" => ClusterObjective::LeastMemory, - other => return Err(format!("unknown cluster objective {other}").into()), - }, - }) - } - (Some("cluster-capacity"), _) => { - return Err( - "--cost-model cluster-capacity needs --cluster-memory-bytes, \ - --cluster-disk-bytes and --cluster-objective" - .into(), - ) - } - (None | Some("usage-over-time" | "aws-fargate-us-east-1" | "provisioned-capacity"), _) => { - return Err("--cluster-* flags need --cost-model cluster-capacity".into()) - } - (Some(other), _) => return Err(format!("unknown cost model {other}").into()), - }; - let measured_operation_costs: MeasuredOperationCosts = match &measured_operation_costs_path { - Some(path) => { - let text = std::fs::read_to_string(path) - .map_err(|e| format!("--measured-operation-costs {path}: {e}"))?; - serde_json::from_str(&text) - .map_err(|e| format!("--measured-operation-costs {path}: {e}"))? - } - None => MeasuredOperationCosts::default(), - }; - let cost_inputs: CostInputs = match &cost_inputs_path { - Some(path) => { - let text = - std::fs::read_to_string(path).map_err(|e| format!("--cost-inputs {path}: {e}"))?; - serde_json::from_str(&text).map_err(|e| format!("--cost-inputs {path}: {e}"))? - } - None => CostInputs { - cost_model, - measured_operation_costs, - ..default_settings - }, - }; - if memo_caps.choices_to_price > 0 { - if !write_memo { - return Err("--price-enumerated needs the memo (drop --no-memo)".into()); - } - // The exhaustive search reports no calibration factor to price each - // choice on the selection's scale. - if exhaustive - && cost_inputs.frozen_calibration_factor.is_none() - && !cost_inputs.measured_operation_costs.operators.is_empty() - { - return Err( - "--price-enumerated with --exhaustive needs a frozen_calibration_factor \ - when measured operation costs are given" - .into(), - ); - } - } - let from_files = if workload_paths.is_empty() { - None - } else if example.is_none() && queries.is_empty() { - let mut workloads = Vec::new(); - for path in &workload_paths { - let text = - std::fs::read_to_string(path).map_err(|e| format!("--workload {path}: {e}"))?; - let workload: PlanningWorkload = - serde_json::from_str(&text).map_err(|e| format!("--workload {path}: {e}"))?; - workload - .validate() - .map_err(|e| format!("--workload {path}: {e}"))?; - workloads.push((path.as_str(), workload)); - } - Some((workloads, sql_catalog(&tables)?)) - } else { - return Err("give exactly one of --example, --workload or --promql/--sql queries".into()); - }; - // The workload's queries as the document describes them, the - // frontends' roots, the workload entries and the data workload. - let (described, roots, entries, data) = match (example.as_deref(), queries.is_empty()) { - (None, true) if from_files.is_some() => { - let (workloads, catalog) = from_files.expect("checked"); - lowered_together(&workloads, &catalog, &catalog_table_names(&tables)?)? - } - (Some(example), true) => { - let (workload, catalog) = match example { - "planner-layering-1" => (example1(), SqlCatalog::new()), - "planner-layering-2" => example2(), - "planner-layering-3a" => ( - pattern_a( - PatternARecurrence::OnceAdHoc, - DataArrival::Mixed { - rows_per_second: SHARED_ROWS_PER_SECOND, - }, - ), - SqlCatalog::new(), - ), - "planner-layering-3b" => (pattern_b(), SqlCatalog::new()), - other => return Err(format!("unknown example {other}").into()), - }; - lowered(&workload, &catalog)? - } - (None, false) => { - let accuracy = match (max_error, failure_probability) { - (None, None) => AccuracyTarget::exact(), - (Some(max_error), None) => AccuracyTarget::max_error(max_error), - (Some(max_error), Some(failure_probability)) => { - AccuracyTarget::max_error(max_error).failure_probability(failure_probability) - } - (None, Some(_)) => return Err("--failure-probability needs --max-error".into()), - }; - let accuracy = match accuracy_metric { - Some(_) if max_error.is_none() => { - return Err("--accuracy-metric needs --max-error".into()) - } - Some(metric) => accuracy.accuracy_metric(metric), - None => accuracy, - }; - let catalog = sql_catalog(&tables)?; - let mut names = HashSet::new(); - let (mut described, mut roots, mut entries) = (Vec::new(), Vec::new(), Vec::new()); - // Every metric the PromQL queries read, each sampled every - // `--sample-interval-ms`. - let mut metrics = BTreeSet::new(); - for (index, (language, text, name)) in queries.into_iter().enumerate() { - let (name, defaulted) = match name { - Some(name) => (name, false), - None => (format!("q{}", index + 1), true), - }; - if !names.insert(name.clone()) { - return Err(if defaulted { - format!( - "duplicate query name {name} (the default name of query {})", - index + 1 - ) - } else { - format!("duplicate query name {name}") - } - .into()); - } - // Each query lowers in its own language; planning sees one workload. - let one = query_batch( - language.clone(), - &[text], - accuracy.clone(), - sample_interval_ms, - ); - roots.extend(match language { - QueryLanguage::PromQL => lower_promql_roots(&one)?, - _ => lower_sql_roots(&one, &catalog)?, - }); - entries.extend(one.query_workload.entries()); - metrics.extend( - one.data_workload - .iter() - .flat_map(|d| d.sources.keys().cloned()), - ); - let mut query = workload_queries(&one).remove(0); - query["id"] = json!(name); - described.push(query); - } - let mut data = DataWorkload::sampled_every(metrics, DurationMs(sample_interval_ms)); - for table in catalog_table_names(&tables)? { - data.sources.entry(table).or_default(); - } - for facts in data.sources.values_mut() { - facts.series_count = series_count; - facts.rows_held = rows_held; - } - if let Some(rows_per_second) = rows_per_second { - data.arrival = DataArrival::Ingesting { rows_per_second }; - } - data.validate().map_err(|e| e.to_string())?; - (described, roots, entries, data) - } - _ => { - return Err( - "give exactly one of --example, --workload or --promql/--sql queries".into(), - ) - } - }; - // Plan with the executor's summary families when the deployment declares none. - if deployment_capabilities.supported_summaries.is_none() { - deployment_capabilities.supported_summaries = - Some(asap_executor::supported_summary_families()); - } - let models = PlanningModels::builtin() - .with_cost_inputs(&cost_inputs) - .with_deployment_capabilities(&deployment_capabilities) - .with_ignore_accuracy_guarantees(!enforce_accuracy_guarantees); - let result = run_search_over( - roots.clone(), - &entries, - &data, - models, - // The document shows every stage's memo. - PlanSearchSettings { inspect: true }, - exhaustive.then_some(max_combinations), - ); - // A rate, increase or irate over a declared gauge, or an aggregation of - // a declared summary's quantiles (the query still plans). - let mut warnings = metric_type_warnings_of(&described, &data)?; - // A selected plan's very large peak memory, when the deployment - // declares no memory capacity to check it against. - warnings.extend( - result - .as_ref() - .ok() - .and_then(|run| run.run.selection.as_ref().ok()) - .and_then(|selection| { - large_peak_memory_warning( - selection.cost.usage.peak_memory_bytes, - &cost_inputs.cost_model, - ) - }), - ); - let memo = result - .as_ref() - .ok() - .and_then(|run| { - let pass1 = &run.run.per_computation_candidates.as_ref()?.memo; - let pass2 = &run.run.logical_asap_candidates.as_ref()?.memo; - Some((pass1, pass2, &run.run)) - }) - .filter(|_| write_memo) - .map(|(pass1, pass2, planning_run)| { - let pricing = Pricing { - stage0_roots: &roots, - entries: &entries, - data: &data, - models, - selected: result - .as_ref() - .ok() - .and_then(|run| run.run.selection.as_ref().ok()) - .map(SelectedBaseline::of), - }; - memo_json( - pass1, - pass2, - &selected_choice(planning_run), - &memo_caps, - &pricing, - ) - }); - let mut document = document_of(described, &roots, &entries, result, max_displayed)?; - if let Some(memo) = memo { - document["memo"] = memo; - } - if !warnings.is_empty() { - document["stage3_plan_selection"]["warnings"] = json!(warnings); - } - document["workload"]["data_workload"] = - serde_json::to_value(&data).map_err(|e| e.to_string())?; - document["search"] = json!(if exhaustive { - "exhaustive" - } else { - "candidate-connections" - }); - document["accuracy_guarantees"] = json!(if enforce_accuracy_guarantees { - "enforced" - } else { - "ignored" - }); - // Read-only: the deployment inputs the run planned with. - document["deployment_capabilities"] = - serde_json::to_value(&deployment_capabilities).map_err(|e| e.to_string())?; - let text = serde_json::to_string_pretty(&document).map_err(|e| e.to_string())? + "\n"; - std::fs::write(&out, text).map_err(|e| format!("{out}: {e}").into()) -} - -/// The [`metric_type_warnings`] of each PromQL query of `described` (the -/// document's queries) over `data`, prefixed with the query's id; the query -/// still plans. -fn metric_type_warnings_of( - described: &[Value], - data: &DataWorkload, -) -> Result, String> { - let mut warnings = Vec::new(); - for query in described.iter().filter(|q| q["language"] == "promql") { - let (id, text) = (&query["id"], query["text"].as_str().unwrap_or_default()); - let id = id.as_str().unwrap_or_default(); - warnings.extend( - metric_type_warnings(text, data) - .map_err(|e| format!("{id}: {e}"))? - .into_iter() - .map(|warning| format!("{id}: {warning}")), - ); - } - Ok(warnings) -} - -/// A workload's queries as the document describes them, the frontends' -/// roots, the workload entries and the data workload. -type Lowered = ( - Vec, - Vec, - Vec, - DataWorkload, -); - -/// `workload` lowered by its language's frontend (SQL over `catalog`); its -/// data workload is empty when it declares none. -fn lowered(workload: &PlanningWorkload, catalog: &SqlCatalog) -> Result { - let roots = match workload.query_workload.language { - QueryLanguage::PromQL => lower_promql_roots(workload)?, - _ => lower_sql_roots(workload, catalog)?, - }; - let entries = workload.query_workload.entries().collect(); - let data = workload.data_workload.clone().unwrap_or_default(); - Ok((workload_queries(workload), roots, entries, data)) -} - -/// `workloads` (each `--workload` file's path and workload, in order), -/// each lowered by its own language's frontend, as one workload: their -/// queries in order, numbered `q1..qN` across the files, and one data -/// workload with every file's sources. A source several files declare must -/// have the same facts in each, and their arrivals must agree -/// ([`arrival_together`]). When several files' data arrives, every source -/// must declare its own rate (`series_count` and `sample_interval`): each -/// declared source, each metric a PromQL query reads and, when a file is -/// SQL, each of `sql_tables` (the `--table` names). Plan selection shares a -/// workload's `rows_per_second` out over all its sources, so one file's -/// rate would otherwise be priced on another file's sources. -fn lowered_together( - workloads: &[(&str, PlanningWorkload)], - catalog: &SqlCatalog, - sql_tables: &[String], -) -> Result { - let (mut described, mut roots, mut entries) = (Vec::new(), Vec::new(), Vec::new()); - let mut data: Option = None; - // The sources the queries read, beside the declared ones. - let mut read_sources: BTreeSet = BTreeSet::new(); - for (path, workload) in workloads { - match workload.query_workload.language { - QueryLanguage::PromQL => { - for entry in workload.query_workload.entries() { - read_sources.extend( - promql_metric_names(&entry.query.0) - .map_err(|e| format!("--workload {path}: {e}"))?, - ); - } - } - _ => read_sources.extend(sql_tables.iter().cloned()), - } - let (queries, more_roots, more_entries, more_data) = - lowered(workload, catalog).map_err(|e| format!("--workload {path}: {e}"))?; - described.extend(queries); - roots.extend(more_roots); - entries.extend(more_entries); - let Some(data) = data.as_mut() else { - data = Some(more_data); - continue; - }; - data.arrival = arrival_together(data.arrival, more_data.arrival).ok_or_else(|| { - format!( - "--workload {path}: its data arrival {:?} disagrees with the earlier files' {:?} \ - (a file without a data_workload declares an unknown arrival); files planned \ - together must declare the same kind of arrival", - more_data.arrival, data.arrival - ) - })?; - for (source, facts) in more_data.sources { - match data.sources.entry(source) { - Entry::Vacant(slot) => { - slot.insert(facts); - } - Entry::Occupied(earlier) if *earlier.get() != facts => { - return Err(format!( - "--workload {path}: source {} is declared with other facts in an earlier file", - earlier.key() - )) - } - Entry::Occupied(_) => {} - } - } - } - let data = data.unwrap_or_default(); - if workloads.len() > 1 && data.arrival.rows_per_second().is_some() { - let has_own_rate = |source: &String| { - data.sources.get(source).is_some_and(|facts| { - facts.series_count.is_some() && facts.sample_interval.is_some() - }) - }; - if let Some(source) = data - .sources - .keys() - .chain(&read_sources) - .find(|source| !has_own_rate(source)) - { - return Err(format!( - "--workload: source {source} declares no rate of its own (series_count and \ - sample_interval); with several files, each source needs one, since a file's \ - rows_per_second is shared out over every file's sources" - )); - } - } - data.validate().map_err(|e| e.to_string())?; - for (index, query) in described.iter_mut().enumerate() { - query["id"] = json!(format!("q{}", index + 1)); - } - Ok((described, roots, entries, data)) -} - -/// The arrival of two files' data planned together: ingesting (or mixed) -/// at the sum of their rates; the same arrival when both are at rest or -/// unknown; `None` when they differ. -fn arrival_together(earlier: DataArrival, later: DataArrival) -> Option { - match (earlier, later) { - ( - DataArrival::Ingesting { rows_per_second: a }, - DataArrival::Ingesting { rows_per_second: b }, - ) => Some(DataArrival::Ingesting { - rows_per_second: a + b, - }), - (DataArrival::Mixed { rows_per_second: a }, DataArrival::Mixed { rows_per_second: b }) => { - Some(DataArrival::Mixed { - rows_per_second: a + b, - }) - } - (DataArrival::AtRest, DataArrival::AtRest) => Some(DataArrival::AtRest), - (DataArrival::Unknown, DataArrival::Unknown) => Some(DataArrival::Unknown), - _ => None, - } -} - -/// The document for `run`, whose workload queries are `queries`. -fn stage_pipeline( - queries: Vec, - entries: &[QueryWorkloadEntry], - run: &PlanningStagesRun, - max_displayed: usize, -) -> Value { - let logical = run.logical_candidates(); - let every_physical = run.physical_candidates(); - // The display keeps `max_displayed` physical candidates: under - // `--exhaustive` the reference pricer's selected one, the other valid - // ones cheapest first, then the rejected ones taking each cause in turn, - // so a cause with many candidates does not hide the others. - let shown_physical = displayed_physical(run, &every_physical, max_displayed); - let implemented: HashSet<&str> = every_physical - .iter() - .map(|p| p.from_logical.as_str()) - .collect(); - // The logical candidates of the shown physical ones, and those among - // the first `max_displayed` that have none (not composed or not - // implemented). - let shown: HashSet<&str> = logical - .iter() - .enumerate() - .filter(|(index, c)| { - every_physical - .iter() - .any(|p| p.from_logical == c.id && shown_physical.contains(p.id.as_str())) - || (*index < max_displayed && !implemented.contains(c.id.as_str())) - }) - .map(|(_, c)| c.id.as_str()) - .collect(); - // Labels are for this document only; planning never reads them. - let labels = candidate_labels(run); - // A choice that composition cannot substitute yet is listed with its - // error rather than ending the run. - let candidates: Vec<_> = logical - .iter() - .filter(|c| shown.contains(c.id.as_str())) - .map(|c| match &c.roots { - Ok(roots) => { - json!({ "id": c.id, "display_name": labels[&c.id], "dag": export(roots) }) - } - Err(error) => { - json!({ "id": c.id, "display_name": labels[&c.id], "error": error }) - } - }) - .collect(); - let physical: Vec<_> = every_physical - .iter() - .filter(|p| shown_physical.contains(p.id.as_str())) - .map(|p| json!({ "id": p.id, "from_logical": p.from_logical, "display_name": labels[&p.id], "dag": p.dag })) - .collect(); - let pruned: Vec<_> = run - .reference - .iter() - .flat_map(|reference| &reference.stage2_failed) - .filter(|(id, _)| shown.contains(id.as_str())) - .map(|(id, reason)| json!({ "from_logical": id, "display_name": labels[id], "reason": reason })) - .collect(); - let mut stage3 = selection_json(&run.run.selection); - match (&run.run.selection, &run.run.selected_plan) { - (Ok(selection), Some(roots)) => { - let tumbling_window_retention_ms = run - .run - .selected_physical_plan - .as_ref() - .map(|realized| realized.tumbling_window_retention_ms.clone()) - .unwrap_or_default(); - stage3["tumbling_window_retention_ms"] = tumbling_window_retention_ms - .iter() - .map(|(node, ms)| (node.to_string(), json!(ms))) - .collect::>() - .into(); - match materialized_views_json(roots, selection, &tumbling_window_retention_ms, entries) - { - Ok(views) => stage3["materialized_views"] = Value::Array(views), - Err(error) => { - stage3["materialized_views"] = json!([]); - stage3["materialized_views_error"] = json!(error); - } - } - } - _ => stage3["materialized_views"] = json!([]), - } - let mut document = json!({ - "format": "asap-stage-pipeline/v1", - "workload": { "queries": queries }, - "stage0_query_language_frontends": { "dag": export(&run.stage0_query_language_frontends) }, - "stage1_logical_asap_optimization": { - "combinations": logical.len(), - "capped": physical.len() < every_physical.len() || candidates.len() < logical.len() - || run.reference.as_ref().is_some_and(|reference| reference.capped), - "pruned_candidates": pass1_pruned_candidates_json(run.run.per_computation_candidates.as_ref()), - "candidates": candidates, - }, - "stage2_physical_asap_optimization": { - "candidates": physical, - "pruned": pruned, - "pruned_candidates": stage2_pruned_candidates_json(run.run.physical_asap_candidates.as_ref()), - }, - "stage3_plan_selection": stage3, - "planning_report": serde_json::to_value(&run.run.report).unwrap_or(Value::Null), - }); - if let Some(reference) = &run.reference { - let every: HashSet<&str> = every_physical.iter().map(|p| p.id.as_str()).collect(); - document["reference_pricing"] = reference_pricing_json( - &reference.selection, - |id| shown_physical.contains(id), - |id| every.contains(id), - ); - } - document -} - -/// The ids of the physical candidates the document shows: at most -/// `max_displayed`. Without a reference enumeration, the selected plan. -/// Under `--exhaustive`, the reference pricer's selected one first, then -/// the other valid ones cheapest first (in candidate order when unpriced), -/// then the rejected ones, one of each cause in turn in candidate order. -fn displayed_physical<'a>( - run: &PlanningStagesRun, - every: &'a [PhysicalWorkloadPlan], - max_displayed: usize, -) -> HashSet<&'a str> { - // The selected plan is always shown. - let selected_plan = every.iter().map(|p| p.id.as_str()).filter(|id| *id == "P1"); - let Some(reference) = &run.reference else { - return selected_plan.collect(); - }; - let shown: HashSet<&str> = displayed_reference(reference, every, max_displayed); - selected_plan.chain(shown).collect() -} - -/// [`displayed_physical`] among the reference enumeration's plans. -fn displayed_reference<'a>( - reference: &asap_devtools::memo_pricing::ReferenceEnumeration, - every: &'a [PhysicalWorkloadPlan], - max_displayed: usize, -) -> HashSet<&'a str> { - let every: Vec<&'a PhysicalWorkloadPlan> = every.iter().filter(|p| p.id != "P1").collect(); - let Ok(selection) = &reference.selection else { - // Without a selection (rejected, or not judged): each cause in turn. - let causes: Vec<_> = match &reference.selection { - Err(ErrorSelectingPlan::NoValidCandidate(rejected)) => rejected.iter().collect(), - _ => Vec::new(), - }; - let cause_of = |id: &str| causes.iter().find(|r| r.id == id).map(|r| r.cause); - return round_robin(every.iter().map(|p| (p.id.as_str(), cause_of(&p.id)))) - .take(max_displayed) - .collect(); - }; - let cause_of = |id: &str| { - selection - .rejected - .iter() - .find(|r| r.id == id) - .map(|r| r.cause) - }; - let mut valid: Vec<&str> = every - .iter() - .map(|p| p.id.as_str()) - .filter(|id| *id != selection.selected && selection.costs.contains_key(*id)) - .collect(); - let price = |id: &str| selection.costs[id].cost_per_second.unwrap_or(f64::INFINITY); - valid.sort_by(|a, b| price(a).total_cmp(&price(b))); - let rejected = every - .iter() - .filter(|p| p.id != selection.selected && !selection.costs.contains_key(&p.id)) - .map(|p| (p.id.as_str(), cause_of(&p.id))); - every - .iter() - .map(|p| p.id.as_str()) - .filter(|id| *id == selection.selected) - .chain(valid) - .chain(round_robin(rejected)) - // The selected candidate is always shown. - .take(max_displayed.max(1)) - .collect() -} - -/// `candidates` (id, cause) reordered to take one of each cause in turn, -/// each cause's in candidate order, causes in order of first appearance. -fn round_robin<'a, C: PartialEq>( - candidates: impl Iterator)>, -) -> impl Iterator { - let mut groups: Vec<(Option, std::collections::VecDeque<&'a str>)> = Vec::new(); - for (id, cause) in candidates { - match groups.iter_mut().find(|(c, _)| *c == cause) { - Some((_, ids)) => ids.push_back(id), - None => groups.push((cause, std::collections::VecDeque::from([id]))), - } - } - std::iter::from_fn(move || { - let ids: Vec<&'a str> = groups - .iter_mut() - .filter_map(|(_, ids)| ids.pop_front()) - .collect(); - (!ids.is_empty()).then_some(ids) - }) - .flatten() -} - -/// The selected plan's materialized views, as the plan -/// [`installable_plan_of`] (what `plan_installable` returns) derives them: -/// each view's node id in the installable DAG, `refresh`, `coverage`, -/// `inputs` and `retention_ms`, -/// plus the node of the selected Stage 2 candidate it is installed from -/// (`stage2_node_id`, null when Stage 3 recorded no cost to map it by), its -/// position among that node's installed nodes (`installed_position`: a -/// tumbling-window chain's tumbling windows, newest first) and their number (`installed_count`). -fn materialized_views_json( - roots: &[QueryRoot], - selection: &Selection, - tumbling_window_retention_ms: &std::collections::BTreeMap< - asap_types::ir::physical_export::PhysicalASAPNodeId, - u64, - >, - entries: &[QueryWorkloadEntry], -) -> Result, String> { - let (plan, report) = installable_plan_of( - roots, - Some(selection), - tumbling_window_retention_ms, - entries, - ) - .map_err(|e| e.to_string())?; - let views = plan.materialized_views().map_err(|e| e.to_string())?; - let mut stage2_of = std::collections::HashMap::new(); - for node in report - .iter() - .flat_map(|report| &report.predicted_usage.per_node) - { - for (position, installed) in node.installed_node_ids.iter().enumerate() { - stage2_of.insert( - *installed, - (node.stage3_node_id, position, node.installed_node_ids.len()), - ); - } - } - views - .iter() - .map(|view| { - let mut value = serde_json::to_value(view).map_err(|e| e.to_string())?; - let (stage2_node_id, position, count) = match stage2_of.get(&view.node_id) { - Some((id, position, count)) => (json!(id), json!(position), json!(count)), - None => (Value::Null, Value::Null, Value::Null), - }; - value["stage2_node_id"] = stage2_node_id; - value["installed_position"] = position; - value["installed_count"] = count; - Ok(value) - }) - .collect() -} - -/// The ways to compute a measure Pass 1 pruned -/// (`PerComputationLogicalMemoCandidates::pruned_candidates`), in pruning -/// order: each with its aggregate's memo `group`, the `candidate` and the -/// `reason` (`PruneReason`, e.g. `UnsupportedByDeployment`). Empty when -/// Pass 1 failed. -fn pass1_pruned_candidates_json(pass1: Option<&PerComputationLogicalMemoCandidates>) -> Vec { - pass1 - .map_or(&[][..], |pass1| &pass1.pruned_candidates) - .iter() - .map(|pruned| { - json!({ - "group": pruned.group.0, - "candidate": computation_label(&pruned.candidate), - "reason": format!("{:?}", pruned.reason), - }) - }) - .collect() -} - -/// The physical candidates Stage 2 pruned -/// (`PhysicalASAPMemoCandidates::pruned_candidates`), in pruning order: -/// each with its memo `group`, its `operator_node` (index into Stage 2's -/// operator nodes), the node's `operator` label, whether it is an -/// `implementation_root`, its `timing` and `materialization`, and the -/// `reason` (`PruneReason`, e.g. -/// `RawDataNotRetained`). Empty without a Stage 2 output. -fn stage2_pruned_candidates_json(stage2: Option<&PhysicalASAPMemoCandidates>) -> Vec { - let Some(stage2) = stage2 else { - return Vec::new(); - }; - stage2 - .pruned_candidates - .iter() - .map(|pruned| { - let key = &pruned.candidate; - let node = &stage2.operator_nodes_of_candidates_for_memo_groups[key.operator_node.0]; - let flat = export(&[QueryRoot::Operator(node.physical_node.clone())]); - json!({ - "group": pruned.group.0, - "operator_node": key.operator_node.0, - "operator": flat.nodes.last().map(node_label).unwrap_or_default(), - "implementation_root": node.is_implementation_root, - "timing": key.timing, - "materialization": key.materialization, - "reason": format!("{:?}", pruned.reason), - }) - }) - .collect() -} - -/// The reference pricer over every enumerated candidate (`--exhaustive`), -/// listing the costs and rejections of the `shown` ones. -fn reference_pricing_json( - selection: &Result, - shown: impl Fn(&str) -> bool, - is_candidate: impl Fn(&str) -> bool, -) -> Value { - let pruned_candidates_json = |rejected: &[Rejection]| -> Vec { - pruned_candidates_json(rejected.iter().filter(|r| shown(&r.id))) - }; - // The rejections of candidates the run did not keep (the default - // search keeps only the best; it records the first rejection of each - // cause) and the search's own, which name no candidate: a query whose - // every own candidate was discarded, or the candidates it skipped. - let not_enumerated = |rejected: &[Rejection]| -> Vec { - self::pruned_candidates_json(rejected.iter().filter(|r| !is_candidate(&r.id))) - }; - let selection = match selection { - Ok(selection) => selection, - Err(ErrorSelectingPlan::NoValidCandidate(rejected)) => { - return json!({ - "cost_per_candidate": {}, - "selected_candidate": null, - "pruned_candidates": pruned_candidates_json(rejected), - "pruned_candidates_not_enumerated": not_enumerated(rejected), - "pruned_candidate_count_by_cause": count_by_cause(rejected), - }); - } - Err(error) => { - return json!({ - "cost_per_candidate": {}, - "selected_candidate": null, - "pruned_candidates": [], - "error": error.to_string(), - }); - } - }; - let cost_per_candidate: serde_json::Map<_, _> = selection - .costs - .iter() - .filter(|(id, _)| shown(id)) - .map(|(id, cost)| (id.clone(), cost_json(cost))) - .collect(); - let accepted_accuracy_per_candidate: serde_json::Map<_, _> = selection - .accepted - .iter() - .filter(|(id, _)| shown(id)) - .map(|(id, queries)| (id.clone(), accepted_json(queries))) - .collect(); - json!({ - "cost_per_candidate": cost_per_candidate, - "accepted_accuracy_per_candidate": accepted_accuracy_per_candidate, - "selected_candidate": selection.selected, - "pruned_candidates": pruned_candidates_json(&selection.rejected), - "pruned_candidates_not_enumerated": not_enumerated(&selection.rejected), - "pruned_candidate_count_by_cause": count_by_cause(&selection.rejected), - "tumbling_window_retention_ms": selection - .tumbling_window_retention_ms - .iter() - .map(|(node, ms)| (node.to_string(), json!(ms))) - .collect::>(), - }) -} - -/// One plan's cost: per second, its predicted CPU, unit and source, its -/// usage and each node's charge. -fn cost_json(cost: &asap_cost::PlanCost) -> Value { - let per_node: serde_json::Map<_, _> = cost - .per_node - .iter() - .map(|(node, c)| { - ( - node.to_string(), - json!({ - "operator": c.operator, - "timing": c.timing, - "cpu_core_seconds_per_second": c.cpu_core_seconds_per_second, - "cost_per_second": c.cost_per_second, - "multiplier": c.multiplier, - "detail": c.detail, - "price_source": c.price_source, - }), - ) - }) - .collect(); - json!({ - "cost_per_second": cost.cost_per_second, - // Report-only: the nodes' predicted CPU, summed as - // `plan_installable`'s predicted usage sums it. - "cpu_core_seconds_per_second": cost - .per_node - .values() - .map(|node| node.cpu_core_seconds_per_second) - .sum::(), - "cost_unit": cost.cost_unit, - "source": cost.source, - "usage": usage_json(&cost.usage), - "per_node": per_node, - }) -} - -/// Per query, the accuracy Stage 3 accepted: the target's metric, the -/// accepted metric, whether it was accepted regardless of its guarantee, -/// why the guarantee check would have rejected it, the sketches read and a -/// one-line summary. -fn accepted_json(queries: &[AcceptedAccuracy]) -> Value { - queries - .iter() - .map(|accepted| { - json!({ - "target_metric": accepted.target_metric, - "accepted_metric": accepted.accepted_metric, - "accepted_regardless_of_guarantee": accepted.accepted_regardless_of_guarantee, - "guarantee_check_rejection": accepted.guarantee_check_rejection, - "sketches": accepted.sketches, - "summary": accepted.to_string(), - }) - }) - .collect() -} - -/// Stage 3's selection ([`Selection`]): the selected DAG over the selected -/// Stage 2 candidates (one node per run of a candidate), the nodes -/// whose weights it proved non-negative, the plan's cost, the accuracy accepted per query, the -/// candidates it pruned and its search report; `error` when the facade -/// selected no plan. -fn selection_json(selection: &Result) -> Value { - let selection = match selection { - Ok(selection) => selection, - Err(error) => return json!({ "selected_dag": null, "error": error.to_string() }), - }; - let dag = &selection.selected_dag; - let nodes = |nodes: &[asap_types::stages::SelectedNodeId]| -> Vec { - nodes.iter().map(|node| node.0).collect() - }; - json!({ - "selected_dag": { - "candidate_of_node": dag.candidate_of_node.iter().map(|id| id.0).collect::>(), - "edges": dag - .edges - .iter() - .map(|e| json!({ "producer": e.producer.0, "consumer": e.consumer.0, "input": e.input })) - .collect::>(), - "node_of_output_reference": nodes(&dag.node_of_output_reference), - }, - "nodes_with_proven_non_negative_weights": - nodes(&selection.nodes_with_proven_non_negative_weights), - "cost": cost_json(&selection.cost), - "accepted_accuracy_per_query": accepted_json(&selection.accepted_accuracy_per_query), - "pruned_candidates": selection - .pruned_candidates - .iter() - .map(|p| json!({ "group": p.group.0, "candidate": p.candidate.0, "reason": format!("{:?}", p.reason) })) - .collect::>(), - "search_report": selection.search_report, - }) -} - -/// A candidate's resource usage, in the units its field names give. -fn usage_json(usage: &ResourceUsage) -> Value { - json!({ - "cpu_ops_per_second": usage.cpu_ops_per_second, - "scanned_bytes_per_second": usage.scanned_bytes_per_second, - "average_memory_bytes": usage.average_memory_bytes, - "peak_memory_bytes": usage.peak_memory_bytes, - "disk_bytes": usage.disk_bytes, - "peak_cpu_cores": usage.peak_cpu_cores, - "latency_per_query": usage - .latency_per_query - .iter() - .map(|l| json!({ "compaction_ms": l.compaction_ms, "query_ms": l.query_ms })) - .collect::>(), - }) -} - -/// How many of `rejected` have each cause, by cause: over every candidate -/// under `--exhaustive`; under the default search, the rejections it -/// recorded (the first of each cause). -fn count_by_cause(rejected: &[Rejection]) -> Value { - let mut counts = serde_json::Map::new(); - for rejection in rejected { - let cause = serde_json::to_value(rejection.cause) - .ok() - .and_then(|v| v.as_str().map(str::to_owned)) - .unwrap_or_default(); - let count = counts.entry(cause).or_insert(json!(0)); - *count = json!(count.as_u64().unwrap_or(0) + 1); - } - Value::Object(counts) -} - -/// One entry per rejection, as `Selection.pruned_candidates` names it: -/// the `candidate` id, its `cause` and `reason`. -fn pruned_candidates_json<'a>(rejected: impl Iterator) -> Vec { - rejected - .map(|r| json!({ "candidate": r.id, "cause": r.cause, "reason": r.reason })) - .collect() -} - -/// A logical DAG as a flat node list: node `i` is `nodes[i]`, and children are -/// node indices inside each operator. -fn export(roots: &[QueryRoot]) -> FlatDag { - flatten(roots).0 -} - -fn workload_queries(workload: &PlanningWorkload) -> Vec { - let language = match workload.query_workload.language { - QueryLanguage::PromQL => "promql", - _ => "sql", - }; - workload - .query_workload - .entries() - .enumerate() - .map(|(index, entry)| { - let accuracy = json!(entry.requirements.accuracy.clone()); - let mut requirements = json!({ "accuracy": accuracy }); - if let Some(ms) = entry.requirements.max_latency_ms { - requirements["max_latency_ms"] = json!(ms); - } - if let QueryRecurrence::Repeated(RepeatedDemand::FixedInterval(interval)) = - &entry.recurrence - { - requirements["repeat_interval_ms"] = json!(interval.0); - } - if let Some(range) = entry.range_evaluation { - requirements["range_evaluation"] = json!(range); - } - json!({ - "id": format!("q{}", index + 1), - "language": language, - "text": entry.query.0, - "requirements": requirements, - // How often and when the query runs, as the workload entry - // declares it. - "recurrence": entry.recurrence, - "predictability": entry.predictability, - }) - }) - .collect() -} - -/// `queries` with, for each approximate target that sets no metric, the -/// metric its root's result is naturally stated in -/// ([`default_metric_for_root`]; null for a quantile, which has none) as -/// `requirements.default_metric`. Display only: Stage 3 accepts any metric. -fn with_default_metrics(mut queries: Vec, roots: &[QueryRoot]) -> Vec { - for (query, root) in queries.iter_mut().zip(roots) { - let accuracy = &query["requirements"]["accuracy"]; - let Ok(target) = serde_json::from_value::(accuracy.clone()) else { - continue; - }; - if target.accuracy_metric.is_none() && !target.is_exact() { - query["requirements"]["default_metric"] = json!(default_metric_for_root(root)); - } - } - queries -} - -/// The names of the tables `--table` declares. -fn catalog_table_names(tables: &[String]) -> Result, String> { - tables - .iter() - .map(|raw| { - let table: Value = serde_json::from_str(raw).map_err(|e| format!("--table: {e}"))?; - table["name"] - .as_str() - .map(str::to_owned) - .ok_or_else(|| "--table: name must be a string".to_string()) - }) - .collect() -} - -fn query_batch( - language: QueryLanguage, - queries: &[String], - accuracy: AccuracyTarget, - sample_interval_ms: u64, -) -> PlanningWorkload { - // Each metric a PromQL query reads is sampled every `sample_interval_ms`; a - // query that does not parse names none and fails when lowered. - let metrics: BTreeSet = match language { - QueryLanguage::PromQL => queries - .iter() - .flat_map(|query| promql_metric_names(query).unwrap_or_default()) - .collect(), - _ => BTreeSet::new(), - }; - PlanningWorkload { - query_workload: QueryWorkload { - language, - query_batch: Some( - queries - .iter() - .map(|query| BatchEntry { - query: Query(query.clone()), - requirements: QueryRequirements { - accuracy: accuracy.clone(), - ..Default::default() - }, - predictability: Default::default(), - invocations: 1, - execute_at: None, - range_evaluation: None, - }) - .collect(), - ), - repeating_queries: None, - }, - data_workload: Some(DataWorkload::sampled_every( - metrics, - DurationMs(sample_interval_ms), - )), - } -} - -/// How much of the memo the document writes (`--max-memo-candidates`, -/// `--memo-enumeration-limit`, `--max-enumerated`) and how many listed -/// choices it prices (`--price-enumerated`). -struct MemoCaps { - candidates_per_group: usize, - enumeration_limit: usize, - enumerated: usize, - choices_to_price: usize, -} - -/// What pricing an enumerated choice needs: the run's workload, models and -/// selection. -struct Pricing<'a> { - stage0_roots: &'a [QueryRoot], - entries: &'a [QueryWorkloadEntry], - data: &'a DataWorkload, - models: PlanningModels<'a>, - selected: Option, -} - -/// The selected plan's candidate for each group of the Pass 2 memo -/// (`run.logical_asap_candidates`), or why there is none: the selection's -/// error when no plan was selected or Stage 2 could not build it. -fn selected_choice(run: &PlanningRun) -> Result { - match ( - &run.selection, - &run.logical_asap_candidates, - &run.physical_asap_candidates, - ) { - (Ok(selection), Some(logical), Some(physical)) => { - selected_chosen_candidates(&logical.memo, physical, &selection.selected_dag) - } - (Err(error), _, _) => Err(format!("no plan was selected: {error}")), - _ => Err("no plan was selected".into()), - } -} - -/// The document's `memo` section: the memo after each stage or pass, the -/// selected plan's choice over the last step's memo (`selected`), and the -/// candidate DAGs each step's memo stands for -/// (`memo::enumeration::enumerate`; each listed choice composed by -/// `memo::enumeration::build_logical_asap_dags_from_chosen_candidates`). -/// `pass1` is Pass 1's memo, `pass2` Pass 2's (the memo Stage 2 -/// implements). -fn memo_json( - pass1: &Memo, - pass2: &Memo, - selected: &Result, - caps: &MemoCaps, - pricing: &Pricing<'_>, -) -> Value { - // Stage 0's groups are Pass 1's, without candidates. - let stage0 = Memo { - groups: pass1.groups.clone(), - query_roots: pass1.query_roots.clone(), - candidates: vec![Vec::new(); pass1.groups.len()], - }; - let named = [ - ("stage0", "Stage 0, the frontends' DAG hash-consed", &stage0), - ("pass1", "Stage 1, Pass 1", pass1), - ("pass2", "Stage 1, Pass 2", pass2), - ]; - let mut table = MemoNodes::default(); - let mut previous: Option<&Memo> = None; - let mut json_steps = Vec::new(); - let mut per_step = serde_json::Map::new(); - // The last step (with candidates) enumerated completely within `enumerated`. - let mut listed = None; - for &(step, title, memo) in &named { - json_steps.push(json!({ - "step": step, - "title": title, - "groups": memo_groups_json(&mut table, memo, previous, caps.candidates_per_group), - })); - previous = Some(memo); - if step == "stage0" { - // One DAG: the frontends' own. - per_step.insert(step.into(), json!({ "count": 1, "complete": true })); - continue; - } - let enumeration = enumerate(memo, caps.enumeration_limit); - if enumeration.complete && enumeration.all_chosen_candidates.len() <= caps.enumerated { - listed = Some((step, memo)); - } - per_step.insert( - step.into(), - json!({ "count": enumeration.all_chosen_candidates.len(), "complete": enumeration.complete }), - ); - } - let (step, memo) = listed.unwrap_or(("pass2", pass2)); - let chosen = selected.as_ref().ok(); - let enumerated = enumerate(memo, caps.enumerated).all_chosen_candidates; - let priced = (caps.choices_to_price > 0).then(|| { - price_choices( - memo, - &enumerated[..caps.choices_to_price.min(enumerated.len())], - pricing.stage0_roots, - pricing.entries, - pricing.data, - pricing.models, - pricing.selected, - ) - }); - let mut choices: Vec<_> = enumerated - .iter() - .enumerate() - .map(|(index, choice)| { - let mut entry = json!({ - "id": format!("E{}", index + 1), - "choice": choice_json(choice), - "accuracy_budget_parts_1": choice.iter().all(|(group, &candidate)| { - memo.candidates[group.0 as usize][candidate].accuracy_budget_parts == 1 - }), - "selected": step == "pass2" && chosen == Some(choice), - }); - match build_logical_asap_dags_from_chosen_candidates(memo, choice) { - Ok(roots) => { - let roots: Vec<_> = roots.into_iter().map(|(_, node)| node).collect(); - entry["roots"] = json!(table.add(&roots).0); - } - Err(error) => { - entry["roots"] = Value::Null; - entry["error"] = json!(error.to_string()); - } - } - entry - }) - .collect(); - for (entry, priced) in choices.iter_mut().zip(priced.iter().flatten()) { - entry["valid"] = json!(priced.valid); - entry["rejection"] = json!(priced.rejection); - entry["rejection_reason"] = json!(priced.rejection_reason); - entry["cost_per_second"] = json!(priced.cost_per_second); - entry["cheaper_than_selected"] = json!(priced.cheaper_than_selected); - } - let priced = priced.map_or(Value::Null, |priced| { - json!({ - "count": priced.len(), - "cheaper_than_selected": priced.iter().filter(|p| p.cheaper_than_selected).count(), - "selected_cost_per_second": pricing.selected.and_then(|s| s.cost_per_second), - }) - }); - let mut memo = json!({ - "nodes": table.nodes, - "steps": json_steps, - "selected_choice": chosen.map_or(Value::Null, choice_json), - "enumeration": { - "step": step, - "per_step": per_step, - "limit": caps.enumeration_limit, - "choices": choices, - "priced": priced, - }, - }); - if let Err(reason) = selected { - memo["selected_choice_unavailable"] = json!(reason); - } - memo -} - -/// Group id (as a string) to candidate index. -fn choice_json(choice: &ChosenCandidatePerMemoGroup) -> Value { - Value::Object( - choice - .iter() - .map(|(group, &candidate)| (group.0.to_string(), json!(candidate))) - .collect(), - ) -} - -/// The `memo` section's node table (`memo.nodes`): each distinct node once, -/// children before parents, its children as indices into the table. Nodes -/// are matched by their serialized JSON (children included), since the -/// steps rebuild equal nodes as different `Rc`s. -#[derive(Default)] -struct MemoNodes { - nodes: Vec, - index_of: HashMap, -} - -impl MemoNodes { - /// Adds `roots` and every node below them. Returns each root's index - /// and, for each node of that DAG, the original node and its index. - fn add( - &mut self, - roots: &[Rc], - ) -> (Vec, Vec<(Rc, NodeId)>) { - let roots: Vec<_> = roots.iter().cloned().map(QueryRoot::Operator).collect(); - let (dag, originals) = flatten(&roots); - let mut index_in_table: Vec = Vec::with_capacity(dag.nodes.len()); - for mut node in dag.nodes { - node.operator = node.operator.map_children(|&child| index_in_table[child]); - let key = serde_json::to_string(&node).expect("a flat node serializes"); - let index = *self.index_of.entry(key).or_insert_with(|| { - self.nodes.push(node); - self.nodes.len() - 1 - }); - index_in_table.push(index); - } - let roots = dag - .roots - .iter() - .map(|root| match root { - QueryRoot::Operator(id) => index_in_table[*id], - QueryRoot::Scalar(_) => unreachable!("memo DAGs have operator roots"), - }) - .collect(); - (roots, originals.into_iter().zip(index_in_table).collect()) - } -} - -/// `memo`'s groups, each with its first `max_candidates` candidates, their -/// nodes added to `table`; a candidate is `added` when `previous` (the memo -/// at the step before) has no candidate of that group with the same node and -/// children. -fn memo_groups_json( - table: &mut MemoNodes, - memo: &Memo, - previous: Option<&Memo>, - max_candidates: usize, -) -> Vec { - let ids = |groups: &[MemoGroupId]| groups.iter().map(|g| g.0).collect::>(); - memo.groups - .iter() - .zip(&memo.candidates) - .enumerate() - .map(|(id, (group, candidates))| { - let node = table.add(std::slice::from_ref(&group.logical_node)).0[0]; - let mut by_origin = serde_json::Map::new(); - for candidate in candidates { - let count = by_origin - .entry(origin_name(&candidate.origin)) - .or_insert(json!(0)); - *count = json!(count.as_u64().unwrap_or(0) + 1); - } - let earlier = previous - .and_then(|memo| memo.candidates.get(id)) - .map_or(&[][..], Vec::as_slice); - let listed: Vec<_> = candidates - .iter() - .take(max_candidates) - .enumerate() - .map(|(index, candidate)| { - let (roots, originals) = table.add(std::slice::from_ref(&candidate.root_node)); - let group_of_node: serde_json::Map<_, _> = originals - .iter() - .filter_map(|(original, node)| { - candidate - .input_groups - .iter() - .find(|child| { - Rc::ptr_eq( - &memo.groups[child.0 as usize].logical_node, - original, - ) - }) - .map(|child| (node.to_string(), json!(child.0))) - }) - .collect(); - json!({ - "index": index, - "implements": std::iter::once(id as u32) - .chain(candidate.fused_groups.iter().map(|g| g.0)) - .collect::>(), - "children": ids(&candidate.input_groups), - "origin": origin_name(&candidate.origin), - "accuracy_budget_parts": candidate.accuracy_budget_parts, - "added": !earlier.iter().any(|before| { - Rc::ptr_eq(&before.root_node, &candidate.root_node) - && before.input_groups == candidate.input_groups - }), - "label": candidate_node_label(&table.nodes, roots[0], &group_of_node), - "node": roots[0], - "group_of_node": group_of_node, - }) - }) - .collect(); - json!({ - "id": id, - "label": node_label(&table.nodes[node]), - "children": ids(&group.child_groups), - "parents": ids(group.parent_groups()), - "queries": group.reached_by_queries(), - "roots_of": queries_rooted_at(&memo.query_roots, MemoGroupId(id as u32)), - "node": node, - "candidate_count": candidates.len(), - "candidate_count_by_origin": by_origin, - "candidates_capped": listed.len() < candidates.len(), - "candidates": listed, - }) - }) - .collect() -} - -/// `OriginOfCandidateForMemoGroup` in snake case. -fn origin_name(origin: &OriginOfCandidateForMemoGroup) -> String { - match origin { - OriginOfCandidateForMemoGroup::Pass1 => "pass1", - OriginOfCandidateForMemoGroup::SharedBuild => "shared_build", - OriginOfCandidateForMemoGroup::SummaryResult => "summary_result", - OriginOfCandidateForMemoGroup::SummaryInput => "summary_input", - OriginOfCandidateForMemoGroup::TumblingWindows => "tumbling_windows", - OriginOfCandidateForMemoGroup::Rollup => "rollup", - OriginOfCandidateForMemoGroup::TopKReuse => "top_k_reuse", - OriginOfCandidateForMemoGroup::WidenedScans => "widened_scans", - } - .into() -} - -/// A candidate's own nodes (those above its child groups' nodes), root -/// first, as `node_label`s joined by " ← ". -fn candidate_node_label( - nodes: &[FlatNode], - root: NodeId, - group_of_node: &serde_json::Map, -) -> String { - let mut labels = Vec::new(); - let mut pending = vec![root]; - let mut seen = HashSet::new(); - while let Some(id) = pending.pop() { - if group_of_node.contains_key(&id.to_string()) || !seen.insert(id) { - continue; - } - labels.push(node_label(&nodes[id])); - pending.extend(nodes[id].operator.children().into_iter().rev().copied()); - } - labels.join(" ← ") -} - -/// One node's operator kind with its sketch family and parameters, its -/// summary query and its time range, when it has them: e.g. -/// `SummaryAgg Kll(k=200)`, `SummaryEstimate Quantile(q=0.5)`. -fn node_label(node: &asap_types::ir::flat::FlatNode) -> String { - let operator = serde_json::to_value(&node.operator).unwrap_or_default(); - // `{"ASAP": {"SummaryAgg": {...}}}` (or a unit variant's string). - let (kind, fields) = match operator.as_object().and_then(|o| o.values().next()) { - Some(Value::Object(inner)) => match inner.iter().next() { - Some((kind, fields)) => (kind.clone(), fields.clone()), - None => (String::new(), Value::Null), - }, - Some(Value::String(kind)) => (kind.clone(), Value::Null), - _ => (operator.to_string(), Value::Null), - }; - let mut details = Vec::new(); - find_details(&fields, &mut details, 0); - let mut label = kind; - for detail in details { - label.push(' '); - label.push_str(&detail); - } - label -} - -/// Sketch algorithms with parameters, summary queries and time ranges found -/// in an operator's fields (not inside its schemas). -fn find_details(value: &Value, details: &mut Vec, depth: usize) { - if depth > 6 || details.len() >= 3 { - return; - } - let compact = |value: &Value| -> String { - match value { - Value::Object(fields) => fields - .iter() - .map(|(key, value)| match value { - Value::Object(inner) if inner.len() == 1 => { - let (k, v) = inner.iter().next().unwrap(); - format!("{key}({k}={v})") - } - Value::Object(inner) => format!( - "{key}({})", - inner - .iter() - .filter(|(_, v)| !v.is_object() && !v.is_array()) - .map(|(k, v)| format!("{k}={v}")) - .collect::>() - .join(", ") - ), - other => format!("{key}={other}"), - }) - .collect::>() - .join(" "), - other => other.to_string(), - } - }; - match value { - Value::Object(fields) => { - if let (Some(Value::String(algorithm)), Some(params)) = - (fields.get("algorithm"), fields.get("params")) - { - let params = params - .as_object() - .and_then(|p| p.values().next()) - .map(compact) - .unwrap_or_default(); - details.push(format!("{algorithm}({params})")); - return; - } - for (key, field) in fields { - match key.as_str() { - "schema" | "input_schema" | "output_schema" | "predicates" | "cols" => {} - "query" => details.push(compact(field)), - // An aggregate's measures, e.g. `quantile(q=0.5)`. - "measures" => { - for measure in field.as_array().into_iter().flatten() { - let Some(kind) = measure.get("kind").and_then(Value::as_str) else { - continue; - }; - let params: Vec<_> = measure - .as_object() - .into_iter() - .flatten() - .filter(|(key, value)| { - *key != "kind" && (value.is_number() || value.is_string()) - }) - .map(|(key, value)| format!("{key}={value}")) - .collect(); - details.push(format!("{kind}({})", params.join(", "))); - } - } - "range" | "window" => details.push(format!("{key}={field}")), - _ => find_details(field, details, depth + 1), - } - } - } - Value::Array(items) => { - for item in items { - find_details(item, details, depth + 1); - } - } - _ => {} - } -} - -#[cfg(test)] -mod tests { - use super::*; - use asap_devtools::planning_stages::run_planning_stages; - - /// A node two DAGs share is stored once in the memo's node table, even - /// when each DAG built it as its own `Rc`, and each DAG rebuilt from the - /// table equals its own `FlatDag` export. - #[test] - fn memo_node_table_stores_a_shared_node_once() { - let root = |query: &str| { - let workload = query_batch( - QueryLanguage::PromQL, - &[query.into()], - AccuracyTarget::exact(), - 15_000, - ); - match lower_promql_roots(&workload).unwrap().remove(0) { - QueryRoot::Operator(node) => node, - QueryRoot::Scalar(_) => unreachable!(), - } - }; - let (sum, max) = (root("sum(m)"), root("max(m)")); - let mut table = MemoNodes::default(); - let (sum_root, _) = table.add(std::slice::from_ref(&sum)); - let after_sum = table.nodes.len(); - let (max_root, _) = table.add(std::slice::from_ref(&max)); - // `max(m)` adds only its own aggregate: the scan is already there. - assert_eq!(table.nodes.len(), after_sum + 1); - // The DAG below `root`, renumbered in table order. - let rebuilt = |root: NodeId| { - let mut reached = BTreeSet::new(); - let mut pending = vec![root]; - while let Some(id) = pending.pop() { - if reached.insert(id) { - pending.extend(table.nodes[id].operator.children().into_iter().copied()); - } - } - let local: HashMap<_, _> = reached.iter().enumerate().map(|(i, &id)| (id, i)).collect(); - let nodes: Vec<_> = reached - .iter() - .map(|&id| { - let mut node = table.nodes[id].clone(); - node.operator = node.operator.map_children(|child| local[child]); - node - }) - .collect(); - FlatDag { - nodes, - roots: vec![QueryRoot::Operator(local[&root])], - } - }; - for (node, root) in [(sum, sum_root[0]), (max, max_root[0])] { - assert_eq!( - serde_json::to_value(rebuilt(root)).unwrap(), - serde_json::to_value(export(&[QueryRoot::Operator(node)])).unwrap() - ); - } - } - - /// The exhaustive search's default `--max-combinations` is 10^4. - #[test] - fn exhaustive_default_candidate_limit_is_ten_thousand() { - assert_eq!(MAX_COMBINATIONS, 10_000); - } - - /// With no valid candidate, the document is still written: - /// `selected_candidate` is null and every rejection is listed. - #[test] - fn failed_selection_lists_rejections_without_a_selection() { - let rejected = vec![Rejection { - id: "P1".into(), - cause: asap_plan_selection::RejectionCause::AccuracyTargetMissed, - reason: "q1: misses target".into(), - }]; - let document = reference_pricing_json( - &Err(ErrorSelectingPlan::NoValidCandidate(rejected)), - |_| true, - |_| true, - ); - assert_eq!( - document, - json!({ - "cost_per_candidate": {}, - "selected_candidate": null, - "pruned_candidates": [{ - "candidate": "P1", - "cause": "accuracy_target_missed", - "reason": "q1: misses target", - }], - "pruned_candidates_not_enumerated": [], - "pruned_candidate_count_by_cause": { "accuracy_target_missed": 1 }, - }) - ); - } - - /// With nothing else displayed, the document still shows the selected - /// candidate and its cost, chosen among all candidates. - #[test] - fn display_cap_keeps_the_selected_candidate() { - let mut workload = query_batch( - QueryLanguage::PromQL, - &["sum(m)".into()], - AccuracyTarget::exact(), - 15_000, - ); - // `m` holds 1,000 series: pricing uses only declared facts. - workload.data_workload = workload - .data_workload - .map(|data| data.with_series_count(1_000)); - let run = - run_planning_stages(&workload, lower_promql_roots(&workload).unwrap(), None).unwrap(); - let entries: Vec<_> = workload.query_workload.entries().collect(); - let document = stage_pipeline(workload_queries(&workload), &entries, &run, 0); - let selected = document["reference_pricing"]["selected_candidate"] - .as_str() - .unwrap(); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - // The facade's selected plan (`P1`) and the reference pricer's. - assert_eq!(physical.len(), 2); - assert!(physical.iter().any(|p| p["id"] == selected)); - assert!(document["reference_pricing"]["cost_per_candidate"] - .get(selected) - .is_some()); - assert_eq!( - document["reference_pricing"]["pruned_candidates"], - json!([]) - ); - } -} diff --git a/crates/devtools/src/candidate_labels.rs b/crates/devtools/src/candidate_labels.rs index 762b19659..87ef5d797 100644 --- a/crates/devtools/src/candidate_labels.rs +++ b/crates/devtools/src/candidate_labels.rs @@ -1,11 +1,14 @@ //! Display labels for the planning stages' candidates, for the -//! `stage_pipeline` viewer document only. Planning never reads them: they +//! `asap_planner` viewer document and text +//! only. Planning never reads them: they //! are pure functions of the candidates' planning data (the logical roots //! and the materialization assignment). use std::collections::BTreeMap; use std::rc::Rc; +use serde_json::Value; + use asap_types::ir::physical_export::{PhysicalASAPDAG, PhysicalASAPNodeId}; use asap_types::ir::schema::{FieldDataType, SketchAlgorithm}; use asap_types::ir::NonASAPOp; @@ -262,6 +265,128 @@ fn sketch_name(algorithm: &SketchAlgorithm) -> String { } } +/// One node's operator kind with its sketch family and parameters, its +/// summary query, its time range and its tumbling-window layout, when it has them: e.g. +/// `SummaryAgg Kll(k=200)`, `SummaryEstimate Quantile(q=0.5)`. +pub fn node_label(node: &asap_types::ir::flat::FlatNode) -> String { + let operator = serde_json::to_value(&node.operator).unwrap_or_default(); + // `{"ASAP": {"SummaryAgg": {...}}}` (or a unit variant's string). + let (kind, fields) = match operator.as_object().and_then(|o| o.values().next()) { + Some(Value::Object(inner)) => match inner.iter().next() { + Some((kind, fields)) => (kind.clone(), fields.clone()), + None => (String::new(), Value::Null), + }, + Some(Value::String(kind)) => (kind.clone(), Value::Null), + _ => (operator.to_string(), Value::Null), + }; + let mut details = Vec::new(); + find_details(&fields, &mut details, 0); + let mut label = kind; + for detail in details { + label.push(' '); + label.push_str(&detail); + } + label +} + +/// Sketch algorithms with parameters, summary queries and time ranges found +/// in an operator's fields (not inside its schemas). +fn find_details(value: &Value, details: &mut Vec, depth: usize) { + if depth > 6 || details.len() >= 4 { + return; + } + let compact = |value: &Value| -> String { + match value { + Value::Object(fields) => fields + .iter() + .map(|(key, value)| match value { + Value::Object(inner) if inner.len() == 1 => { + let (k, v) = inner.iter().next().unwrap(); + format!("{key}({k}={v})") + } + Value::Object(inner) => format!( + "{key}({})", + inner + .iter() + .filter(|(_, v)| !v.is_object() && !v.is_array()) + .map(|(k, v)| format!("{k}={v}")) + .collect::>() + .join(", ") + ), + other => format!("{key}={other}"), + }) + .collect::>() + .join(" "), + other => other.to_string(), + } + }; + match value { + Value::Object(fields) => { + if let (Some(Value::String(algorithm)), Some(params)) = + (fields.get("algorithm"), fields.get("params")) + { + let params = params + .as_object() + .and_then(|p| p.values().next()) + .map(compact) + .unwrap_or_default(); + details.push(format!("{algorithm}({params})")); + return; + } + for (key, field) in fields { + match key.as_str() { + "schema" | "input_schema" | "output_schema" | "predicates" | "cols" => {} + "query" => details.push(compact(field)), + // An aggregate's measures, e.g. `quantile(q=0.5)`. + "measures" => { + for measure in field.as_array().into_iter().flatten() { + let Some(kind) = measure.get("kind").and_then(Value::as_str) else { + continue; + }; + let params: Vec<_> = measure + .as_object() + .into_iter() + .flatten() + .filter(|(key, value)| { + *key != "kind" && (value.is_number() || value.is_string()) + }) + .map(|(key, value)| format!("{key}={value}")) + .collect(); + details.push(format!("{kind}({})", params.join(", "))); + } + } + // A time range's length, e.g. `60s`. + "range" => details.push(duration_text(field)), + // A window, and a tumbling-window chain's layout. + "window" | "tumbling_window_width_ms" | "tumbling_window_count" => { + details.push(format!("{key}={field}")) + } + _ => find_details(field, details, depth + 1), + } + } + } + Value::Array(items) => { + for item in items { + find_details(item, details, depth + 1); + } + } + _ => {} + } +} + +/// A serialized `Duration` (`{"secs": 60, "nanos": 0}`) as `60s`, or +/// `1500ms` when it is not whole seconds; any other value as its JSON. +fn duration_text(value: &Value) -> String { + match ( + value.get("secs").and_then(Value::as_u64), + value.get("nanos").and_then(Value::as_u64), + ) { + (Some(secs), Some(0)) => format!("{secs}s"), + (Some(secs), Some(nanos)) => format!("{}ms", secs * 1000 + nanos / 1_000_000), + _ => value.to_string(), + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/devtools/src/lib.rs b/crates/devtools/src/lib.rs index 48ad1545c..f3c546d6e 100644 --- a/crates/devtools/src/lib.rs +++ b/crates/devtools/src/lib.rs @@ -13,7 +13,7 @@ //! other's parser. //! //! [`planning_stages`] and [`planner_layering_examples`] hold the one -//! pipeline and the #509 example workloads that `stage_pipeline` and the +//! pipeline and the #509 example workloads that `asap_planner` and the //! planner-layering acceptance tests share; [`candidate_labels`] names the //! candidates for the viewer document. [`corpora`] splits the checked-in //! query corpora and holds their SQL catalogs. [`rqe_atomic_costs`] converts @@ -27,6 +27,7 @@ pub mod memo_pricing; pub mod planner_fuzz; pub mod planner_layering_examples; pub mod planning_stages; +pub mod planning_stages_text; pub mod rqe_atomic_costs; use asap_frontend_promql::lower_promql_query_workload; diff --git a/crates/devtools/src/memo_pricing.rs b/crates/devtools/src/memo_pricing.rs index 469e2513e..128cc4911 100644 --- a/crates/devtools/src/memo_pricing.rs +++ b/crates/devtools/src/memo_pricing.rs @@ -253,7 +253,7 @@ pub struct EnumeratedChoice { pub roots: Result, String>, } -/// `stage_pipeline --exhaustive`: the first `max_choices` choices of the +/// The reference enumeration: the first `max_choices` choices of the /// Pass 2 memo (`run.logical_asap_candidates`), each composed, /// implemented by Stage 2 with every materialization assignment, and all /// their physical plans priced together by the reference pricer. diff --git a/crates/devtools/src/planner_fuzz/README.md b/crates/devtools/src/planner_fuzz/README.md index d6adb0c47..3370fa96f 100644 --- a/crates/devtools/src/planner_fuzz/README.md +++ b/crates/devtools/src/planner_fuzz/README.md @@ -3,8 +3,8 @@ A metamorphic fuzz tester for **missed reuse and sharing** in the planner. A missed-reuse bug is a plan that no candidate rule generates. The answers are -still correct, and `stage_pipeline --exhaustive` cannot see the bug, because it -only searches the generated candidates. Example (#903): Q1 is +still correct, and no search over the generated candidates can see the bug, +because it only searches the generated candidates. Example (#903): Q1 is `quantile_over_time(0.9, data[5m])` with an exact target, and Q2 is the same query at ε = 0.01. The planner builds a KLL for Q2 (0.166 per second) instead of reading Q1's exact result (0.144 per second). @@ -33,7 +33,7 @@ A run writes these files: design-doc coverage table below with this run's status. - `findings/-.md`: one issue draft per distinct shrunk violation, in the `asap-issue-authoring` layout. It starts with the example (both - workloads, both `stage_pipeline` commands, the costs, the selected and + workloads, both `asap_planner` commands, the costs, the selected and witness plans, and the violated relation). Next to it are `-base.workload.json` and `-related.workload.json`. **Nothing is filed**: a person reads the drafts, checks them against the open diff --git a/crates/devtools/src/planner_fuzz/generate.rs b/crates/devtools/src/planner_fuzz/generate.rs index b372031a5..d831c9894 100644 --- a/crates/devtools/src/planner_fuzz/generate.rs +++ b/crates/devtools/src/planner_fuzz/generate.rs @@ -351,7 +351,7 @@ fn sql_interval(ms: u64) -> String { } } -/// The SQL table declarations (`stage_pipeline --table` form) for `table`. +/// The SQL table declarations (`asap_planner --sql-catalog-table` form) for `table`. pub fn sql_table_declaration(table: &str, columns: &[&str]) -> serde_json::Value { let mut fields = vec![json!({"name": "ts", "type": "timestamp", "nullable": false})]; for column in columns { diff --git a/crates/devtools/src/planner_fuzz/mod.rs b/crates/devtools/src/planner_fuzz/mod.rs index 03a3bee36..e37b85e75 100644 --- a/crates/devtools/src/planner_fuzz/mod.rs +++ b/crates/devtools/src/planner_fuzz/mod.rs @@ -59,7 +59,7 @@ use crate::planning_stages::{ use generate::QuerySpec; /// The language of a fuzz workload: one per workload, as one -/// `stage_pipeline --workload` file has. +/// `asap_planner --workload` file has. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum Language { @@ -121,7 +121,7 @@ impl FuzzQuery { } /// A workload the tester plans: queries in one language, the data facts, -/// the SQL tables they read (`stage_pipeline --table` objects) and whether +/// the SQL tables they read (`asap_planner --sql-catalog-table` objects) and whether /// the deployment keeps raw data. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct FuzzWorkload { @@ -142,7 +142,7 @@ pub struct FuzzWorkload { } impl FuzzWorkload { - /// The `PlanningWorkload` (`stage_pipeline --workload` file) of these + /// The `PlanningWorkload` (`asap_planner --workload` file) of these /// queries. pub fn planning_workload(&self) -> PlanningWorkload { PlanningWorkload { @@ -217,29 +217,35 @@ impl FuzzWorkload { } } - /// The `stage_pipeline` invocation that plans this workload, saved as - /// `workload_file`, with `settings`. - /// Its output document is `.out.json`. - pub fn stage_pipeline_command(&self, workload_file: &str, settings: &FuzzSettings) -> String { - let mut command = format!( - "cargo run -p asap-devtools --bin stage_pipeline -- --workload {workload_file}" - ); + /// The `asap_planner` invocation (a bash command) that plans this + /// workload, saved as `workload_file`, with `settings`' accuracy + /// guarantees. `asap_planner` has no exhaustive search, so under + /// `--search exhaustive` the command starts with a comment saying it + /// may not reproduce. Its output document is `.out.json`. + pub fn asap_planner_command(&self, workload_file: &str, settings: &FuzzSettings) -> String { + let mut command = String::new(); + if matches!(settings.search, SearchKind::Exhaustive) { + command.push_str( + "# asap_planner runs the candidate-connection search, not the \ + exhaustive one; it may not reproduce this.\n", + ); + } + command.push_str(&format!( + "cargo run -p asap-devtools --bin asap_planner -- --workload {workload_file} \ + --run-until stage3 --format json" + )); for table in &self.tables { - command.push_str(&format!(" \\\n --table '{table}'")); + command.push_str(&format!(" \\\n --sql-catalog-table '{table}'")); } - if settings.enforce_accuracy_guarantees { - command.push_str(" --enforce-accuracy-guarantees"); + if !settings.enforce_accuracy_guarantees { + command.push_str(" --ignore-accuracy-guarantees"); } if !self.raw_data_retained || self.query_time_results_retained { command.push_str(&format!( - " --deployment-capabilities '{{\"raw_data_retained\": {}, \"query_time_results_retained\": {}}}'", + " --deployment-capabilities <(echo '{{\"raw_data_retained\": {}, \"query_time_results_retained\": {}}}')", self.raw_data_retained, self.query_time_results_retained )); } - match settings.search { - SearchKind::CandidateConnections => {} - SearchKind::Exhaustive => command.push_str(" --exhaustive"), - } let stem = workload_file.strip_suffix(".json").unwrap_or(workload_file); command.push_str(&format!(" --out {stem}.out.json")); command @@ -248,7 +254,7 @@ impl FuzzWorkload { /// Workloads read from a seed file: a `PlanningWorkload` JSON (its /// repeating queries), with the SQL tables of a `tables.json` (a JSON array -/// of `--table` objects) in the same directory, or an object +/// of `--sql-catalog-table` objects) in the same directory, or an object /// `{"workload": , "tables": [...]}`. pub fn read_seed_file(path: &std::path::Path) -> Result { let text = std::fs::read_to_string(path).map_err(|e| format!("{}: {e}", path.display()))?; @@ -302,11 +308,11 @@ pub fn read_seed_file(path: &std::path::Path) -> Result { }) } -/// Which search plans each workload (`stage_pipeline`'s choices). +/// Which search plans each workload. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "kebab-case")] pub enum SearchKind { - /// Joint candidate-connection planning (`stage_pipeline`'s default). + /// Joint candidate-connection planning, the facade's search. CandidateConnections, /// Every generated candidate (`--exhaustive`), up to /// [`EXHAUSTIVE_MAX_COMBINATIONS`]. @@ -409,13 +415,12 @@ impl Planner { Language::Promql => lower_promql_roots(&planning), Language::Sql => { let tables: Vec = workload.tables.iter().map(Value::to_string).collect(); - lower_sql_roots(&planning, &sql_catalog(&tables)?) + lower_sql_roots(&planning, &sql_catalog(&tables)?.0) } } } - /// `workload` planned by the configured search, as `stage_pipeline` - /// plans it. + /// `workload` planned by the configured search. pub fn plan(&mut self, workload: &FuzzWorkload) -> Result, String> { let key = serde_json::to_string(workload).expect("a workload serializes"); if let Some(planned) = self.cache.get(&key) { diff --git a/crates/devtools/src/planner_fuzz/report.rs b/crates/devtools/src/planner_fuzz/report.rs index 88abee049..330c8de94 100644 --- a/crates/devtools/src/planner_fuzz/report.rs +++ b/crates/devtools/src/planner_fuzz/report.rs @@ -889,8 +889,8 @@ fn write_draft( text.push_str(&format!("```json\n{}\n```\n\n", workload_json(&related))); text.push_str(&format!( "```sh\n{}\n{}\n```\n\n", - base.stage_pipeline_command("base.json", settings), - related.stage_pipeline_command("related.json", settings), + base.asap_planner_command("base.json", settings), + related.asap_planner_command("related.json", settings), )); text.push_str("| Workload | Selected plan | Cost per second |\n|---|---|---|\n"); text.push_str(&format!( diff --git a/crates/devtools/src/planner_layering_examples.rs b/crates/devtools/src/planner_layering_examples.rs index 8b4cab1e3..f4ac75793 100644 --- a/crates/devtools/src/planner_layering_examples.rs +++ b/crates/devtools/src/planner_layering_examples.rs @@ -1,6 +1,6 @@ //! The workloads of #509 Examples 1, 2, 3 and 4 //! (`docs/design_docs/proposals/planner-layering.md`), shared by -//! `stage_pipeline` and the planner-layering acceptance tests. +//! `tools/dag-viewer/workloads/planner-layering/` and the planner-layering acceptance tests. use asap_types::ir::properties::{AccuracyMetric, AccuracyTarget}; use asap_types::workload::{ diff --git a/crates/devtools/src/planning_stages.rs b/crates/devtools/src/planning_stages.rs index a0f6a7f6a..b40f36dea 100644 --- a/crates/devtools/src/planning_stages.rs +++ b/crates/devtools/src/planning_stages.rs @@ -1,29 +1,93 @@ -//! The four planner stages of #509 run end to end over one workload: the -//! one pipeline behind `stage_pipeline` and the planner-layering acceptance -//! tests, so both see the same candidates, ids and selection. +//! The four planner stages of #509 run over one workload: the facade's +//! search for the planner-layering acceptance tests, and +//! [`run_stages_until`] for `asap_planner`. use asap_frontend_sql::SqlCatalog; -use asap_planner::{run_planning, PlanSearchSettings, PlanningModels, PlanningRun}; +use asap_logical_asap_optimization::{ + generate_dag_of_logical_memo_groups, generate_per_computation_logical_memo_candidates, + pass2_after_local_window_composition, pass2_local_window_composition, + PerComputationLogicalMemoCandidatesError, +}; +use asap_physical_asap_optimization::{ + generate_physical_asap_memo_candidates, Stage2AssignmentRealizer, +}; +use asap_plan_selection::select_plan; +use asap_planner::{ + run_planning, PlanSearchSettings, PlanningModels, PlanningReport, PlanningRun, SelectionError, +}; use asap_types::ir::schema::schema_support::with_promql_series_identity; use asap_types::ir::schema::{DataType, Field, Schema}; use asap_types::ir::QueryRoot; use asap_types::physical::PhysicalWorkloadPlan; +use asap_types::stages::{AssignmentRealizer, DagOfLogicalMemoGroups, Memo}; +use asap_types::workload::bounds::with_confirmed_weights_at; use asap_types::workload::{DataWorkload, PlanningWorkload, QueryWorkloadEntry}; use serde_json::Value; use std::collections::HashSet; +use std::rc::Rc; use crate::memo_pricing::{reference_enumeration, ReferenceEnumeration}; -/// The memo choices `--exhaustive` prices when no limit is given. +/// The memo choices the reference enumeration prices when no limit is given. pub const DEFAULT_REFERENCE_CHOICES: usize = 10_000; +/// The last stage a run runs (`asap_planner --run-until`); the run's +/// fields for later stages are left empty. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub enum RunUntil { + Stage0, + Pass1, + Pass2, + Stage2, + Stage3, +} + +impl RunUntil { + const ALL: [Self; 5] = [ + Self::Stage0, + Self::Pass1, + Self::Pass2, + Self::Stage2, + Self::Stage3, + ]; + + /// The `--run-until` value, also written as the JSON `run_until`. + pub fn name(self) -> &'static str { + match self { + Self::Stage0 => "stage0", + Self::Pass1 => "stage1_pass1", + Self::Pass2 => "stage1_pass2", + Self::Stage2 => "stage2", + Self::Stage3 => "stage3", + } + } +} + +impl std::str::FromStr for RunUntil { + type Err = String; + fn from_str(stage: &str) -> Result { + Self::ALL + .into_iter() + .find(|until| until.name() == stage) + .ok_or_else(|| format!("unknown stage {stage}")) + } +} + #[derive(Debug)] pub struct PlanningStagesRun { + /// The last stage run; `run` holds nothing of the stages after it. + pub run_until: RunUntil, /// The query-language frontends' roots, one per workload entry. pub stage0_query_language_frontends: Vec, - /// The facade's run: every stage and the selection. + /// Stage 0's memo groups (the roots hash-consed), without candidates; + /// built by `run_stages_until` only (`asap_planner` prints them). + pub stage0_memo_groups: Option, + /// Stage 1 Pass 2's memo after local window composition (rule 1), before + /// the other rules; `None` before Pass 2. + pub pass2_local_window_composition: Option, + /// The stages' outputs: Pass 1, Pass 2, Stage 2 and the selection. pub run: PlanningRun, - /// `--exhaustive`: Pass 2's memo choices, each implemented + /// With a reference enumeration: Pass 2's memo choices, each implemented /// by Stage 2 and all priced by the reference pricer. pub reference: Option, } @@ -39,7 +103,7 @@ pub struct ShownLogicalCandidate { impl PlanningStagesRun { /// The logical candidates the document shows: the selected plan (`L1`; /// its roots as Stage 2 built them, with Stage 3's weight proofs), then - /// under `--exhaustive` the reference enumeration's choices (`E`). + /// with a reference enumeration, its's choices (`E`). pub fn logical_candidates(&self) -> Vec { let selected = self .run @@ -62,7 +126,7 @@ impl PlanningStagesRun { } /// The physical candidates the document shows: the selected plan as - /// Stage 2 built it (`P1`, from `L1`), then under `--exhaustive` every + /// Stage 2 built it (`P1`, from `L1`), then with a reference enumeration every /// plan the reference pricer priced (`E-`). pub fn physical_candidates(&self) -> Vec { let selected = @@ -232,12 +296,167 @@ pub fn run_search_over( let reference = reference_choices .and_then(|max| reference_enumeration(&run, &roots, entries, data, models, max)); Ok(PlanningStagesRun { + run_until: RunUntil::Stage3, + stage0_memo_groups: None, + pass2_local_window_composition: None, stage0_query_language_frontends: roots, run, reference, }) } +/// Stage 0's memo groups, `dag`, without candidates. +fn stage0_memo_groups(dag: DagOfLogicalMemoGroups) -> Memo { + Memo { + candidates: vec![Vec::new(); dag.groups.len()], + groups: dag.groups, + query_roots: dag.query_roots, + } +} + +/// The stages over `roots` up to and including `until`, as the facade runs +/// them. A stage's failure is reported in the run, as the facade does. +// ponytail: copies run_planning's stage order, which has no stop point; +// #1169 moves the stop point into run_planning. +pub fn run_stages_until( + roots: Vec, + entries: &[QueryWorkloadEntry], + data: &DataWorkload, + models: PlanningModels<'_>, + until: RunUntil, +) -> Result { + validate(&roots)?; + let stage0_dag = generate_dag_of_logical_memo_groups(&roots); + let mut stages = PlanningStagesRun { + run_until: until, + stage0_memo_groups: Some(stage0_memo_groups(stage0_dag.clone())), + pass2_local_window_composition: None, + stage0_query_language_frontends: roots.clone(), + run: PlanningRun { + per_computation_candidates: None, + logical_asap_candidates: None, + physical_asap_candidates: None, + selection: Err(SelectionError::NotRun), + selected_physical_plan: None, + selected_plan: None, + report: PlanningReport::default(), + }, + reference: None, + }; + if until == RunUntil::Stage0 { + return Ok(stages); + } + let run = &mut stages.run; + let accuracy_targets: Vec<_> = entries + .iter() + .map(|entry| entry.requirements.accuracy.clone()) + .collect(); + let accuracy = models.accuracy_check_inputs(); + let pass1 = generate_per_computation_logical_memo_candidates( + stage0_dag, + &accuracy_targets, + accuracy.model, + accuracy.evidence, + data, + models + .deployment_capabilities + .supported_summaries + .as_deref(), + ); + // As `run_planning`: a query Pass 1 cannot plan is reported before + // contradictory deployment capabilities. + if let Err(PerComputationLogicalMemoCandidatesError::PerComputation(error)) = &pass1 { + return Err(PlanningStagesError::Pass1(error.to_string())); + } + if let Err(error) = models.deployment_capabilities.validate(data) { + run.selection = Err(SelectionError::InvalidDeploymentCapabilities(error)); + return Ok(stages); + } + let pass1 = match pass1 { + Ok(pass1) => pass1, + Err(error) => { + run.selection = Err(SelectionError::LogicalASAPCandidates(error.to_string())); + return Ok(stages); + } + }; + run.per_computation_candidates = Some(pass1.clone()); + if until == RunUntil::Pass1 { + return Ok(stages); + } + let composed = pass2_local_window_composition(pass1, entries); + stages.pass2_local_window_composition = Some(composed.clone()); + let logical = match pass2_after_local_window_composition(composed) { + Ok(logical) => logical, + Err(error) => { + run.selection = Err(SelectionError::LogicalASAPCandidates(error.to_string())); + return Ok(stages); + } + }; + let run = &mut stages.run; + run.logical_asap_candidates = Some(logical); + if until == RunUntil::Pass2 { + return Ok(stages); + } + let logical = run.logical_asap_candidates.as_ref().expect("set above"); + let physical = match generate_physical_asap_memo_candidates( + logical, + entries, + data, + models.deployment_capabilities.query_time_results_retained, + models.deployment_capabilities.raw_data_retained, + ) { + Ok(physical) => physical, + Err(error) => { + run.selection = Err(SelectionError::PhysicalASAPCandidates(error.to_string())); + return Ok(stages); + } + }; + if until == RunUntil::Stage2 { + run.physical_asap_candidates = Some(physical); + return Ok(stages); + } + let realizer = Stage2AssignmentRealizer { + original_roots: &roots, + entries, + data, + query_time_results_retained: models.deployment_capabilities.query_time_results_retained, + }; + match select_plan( + &physical, + entries, + data, + &accuracy, + models.cost_inputs, + &realizer, + ) { + Ok(selection) => { + match realizer.realize(&physical, &selection.selected_dag) { + Ok(realized) => { + // As the facade does: the plan to run carries the weight + // proofs Stage 3 made. + let proven: HashSet<*const asap_types::ir::OperatorNode> = selection + .nodes_with_proven_non_negative_weights + .iter() + .filter_map(|node| realized.operator_node_of_selected_node.get(node.0)) + .map(Rc::as_ptr) + .collect(); + let plan = with_confirmed_weights_at(&realized.plan.roots, data, &|node| { + proven.contains(&Rc::as_ptr(node)) + }); + validate(&plan)?; + run.selected_plan = Some(plan); + run.selected_physical_plan = Some(realized); + run.selection = Ok(selection); + } + Err(error) => run.selection = Err(SelectionError::Realization(error)), + } + } + Err(error) => run.selection = Err(SelectionError::Selection(error)), + } + run.physical_asap_candidates = Some(physical); + Ok(stages) +} + fn validate(roots: &[QueryRoot]) -> Result<(), PlanningStagesError> { roots .iter() @@ -245,29 +464,32 @@ fn validate(roots: &[QueryRoot]) -> Result<(), PlanningStagesError> { .map_err(|e| PlanningStagesError::Validation(e.to_string())) } -/// The SQL catalog of the tables `tables` declare, each a `stage_pipeline -/// --table` JSON object: `{"name": ..., "columns": [{"name": ..., "type": +/// The SQL catalog of the tables `tables` declare, each a `asap_planner +/// --sql-catalog-table` JSON object: `{"name": ..., "columns": [{"name": ..., "type": /// "timestamp|utf8|string|float64|double|int64|bigint", "nullable": true}], -/// "time_index": 0}`. -pub fn sql_catalog(tables: &[String]) -> Result { +/// "time_index": 0}`, and the tables' names in declaration order. +pub fn sql_catalog(tables: &[String]) -> Result<(SqlCatalog, Vec), String> { let mut catalog = SqlCatalog::new(); - let mut names = HashSet::new(); + let mut names = Vec::new(); for raw in tables { - let table: Value = serde_json::from_str(raw).map_err(|e| format!("--table: {e}"))?; + let table: Value = + serde_json::from_str(raw).map_err(|e| format!("--sql-catalog-table: {e}"))?; let name = table["name"] .as_str() - .ok_or("--table: name must be a string")?; - if !names.insert(name.to_owned()) { - return Err(format!("--table: table {name} is declared twice")); + .ok_or("--sql-catalog-table: name must be a string")?; + if names.iter().any(|declared| declared == name) { + return Err(format!( + "--sql-catalog-table: table {name} is declared twice" + )); } let columns = table["columns"] .as_array() - .ok_or("--table: columns must be an array")? + .ok_or("--sql-catalog-table: columns must be an array")? .iter() .map(|column| { let column_name = column["name"] .as_str() - .ok_or("--table: column name must be a string")?; + .ok_or("--sql-catalog-table: column name must be a string")?; let dtype = match column["type"] .as_str() .map(str::to_ascii_lowercase) @@ -277,7 +499,11 @@ pub fn sql_catalog(tables: &[String]) -> Result { Some("utf8" | "string") => DataType::Utf8, Some("float64" | "double") => DataType::Float64, Some("int64" | "bigint") => DataType::Int64, - other => return Err(format!("--table: unsupported column type {other:?}")), + other => { + return Err(format!( + "--sql-catalog-table: unsupported column type {other:?}" + )) + } }; let nullable = column["nullable"].as_bool().unwrap_or(true); Ok(Field::plain(column_name, dtype, nullable)) @@ -291,13 +517,14 @@ pub fn sql_catalog(tables: &[String]) -> Result { } _ => { return Err(format!( - "--table {name}: time_index must be the index of one of its {} columns", + "--sql-catalog-table {name}: time_index must be the index of one of its {} columns", columns.len() )) } }, }; + names.push(name.to_owned()); catalog = catalog.with_table(name, schema); } - Ok(catalog) + Ok((catalog, names)) } diff --git a/crates/devtools/src/planning_stages_text.rs b/crates/devtools/src/planning_stages_text.rs new file mode 100644 index 000000000..f68332018 --- /dev/null +++ b/crates/devtools/src/planning_stages_text.rs @@ -0,0 +1,620 @@ +//! The planner stages' output as plain text: `asap_planner --format text`. +//! It prints, up to the run's last stage, the frontends' roots and their +//! memo groups (Stage 0), Pass 1's `PerComputationLogicalMemoCandidates`, +//! the memo after Pass 2's local window composition and Pass 2's +//! `LogicalASAPMemoCandidates` (Stage 2's input), Stage 2's +//! `PhysicalASAPMemoCandidates` (Stage 3's input), and Stage 3's +//! `Selection` with the selected memo choice, each node's cost and each +//! query's accepted accuracy. Planning never reads it. +//! +//! Notation: `G` is memo group `n` and `G[]` its candidate `i`; +//! `O` is Stage 2's operator node `n` and `C` physical candidate `n`; +//! `<- ...` lists what a line reads; `new` marks a candidate the step +//! added. A candidate's chain lists its own nodes root first, down to its +//! input groups: +//! +//! ```text +//! G1: TimeRange 60s <- G0 +//! G2: Aggregate rate() <- G1 +//! [0] Pass1 FinalizeExactAccumulator ← SummaryAgg inputs=[G1] +//! [1] TumblingWindows new FinalizeExactAccumulator ← SummaryMerge ← SummaryTumblingWindowChain tumbling_window_count=6 tumbling_window_width_ms=10000 inputs=[G0] +//! ``` + +use std::collections::HashSet; +use std::fmt::Write; +use std::rc::Rc; + +use asap_planner::selected_chosen_candidates; +use asap_types::ir::flat::flatten; +use asap_types::ir::{OperatorNode, QueryRoot}; +use asap_types::stages::{Memo, MemoGroupId, PhysicalASAPMemoCandidates}; + +use crate::candidate_labels::node_label; +use crate::planning_stages::{PlanningStagesRun, RunUntil}; + +/// The output of every stage `stages` ran, as text, one section per stage +/// boundary. A failure ends the text with its error. +pub fn planning_stages_text(stages: &PlanningStagesRun) -> String { + let mut out = String::new(); + let run = &stages.run; + let until = stages.run_until; + // Why the next stage has no output (a failure, not the stop point). + let failure = |out: &mut String| { + if let Err(error) = &run.selection { + let _ = writeln!(out, "\nerror: {error}"); + } + }; + + section( + &mut out, + "Stage 0 output: the frontends' roots (Vec)", + ); + write_dag(&mut out, &stages.stage0_query_language_frontends); + section( + &mut out, + "Stage 0 → Stage 1: DagOfLogicalMemoGroups (the roots hash-consed)", + ); + if let Some(memo) = &stages.stage0_memo_groups { + write_memo(&mut out, memo, None, false); + } + if until == RunUntil::Stage0 { + return out; + } + + let Some(pass1) = &run.per_computation_candidates else { + failure(&mut out); + return out; + }; + section( + &mut out, + "Stage 1 Pass 1 output: PerComputationLogicalMemoCandidates", + ); + write_memo(&mut out, &pass1.memo, None, true); + let _ = writeln!( + out, + "pruned_candidates ({}):", + pass1.pruned_candidates.len() + ); + let rows = pass1 + .pruned_candidates + .iter() + .map(|pruned| { + vec![ + format!("G{}:", pruned.group.0), + format!("{:?}", pruned.candidate), + format!("{:?}", pruned.reason), + ] + }) + .collect(); + write_rows(&mut out, " ", rows); + if until == RunUntil::Pass1 { + return out; + } + + let (Some(pass2), Some(logical)) = ( + &stages.pass2_local_window_composition, + &run.logical_asap_candidates, + ) else { + failure(&mut out); + return out; + }; + section( + &mut out, + "Stage 1 Pass 2, after local window composition (tumbling-window candidates)", + ); + write_memo(&mut out, pass2, Some(&pass1.memo), true); + section( + &mut out, + "Stage 1 Pass 2 output = Stage 2 input: LogicalASAPMemoCandidates", + ); + write_memo(&mut out, &logical.memo, Some(pass2), true); + if until == RunUntil::Pass2 { + return out; + } + + let Some(physical) = &run.physical_asap_candidates else { + failure(&mut out); + return out; + }; + section( + &mut out, + "Stage 2 output = Stage 3 input: PhysicalASAPMemoCandidates", + ); + write_physical(&mut out, physical); + if until == RunUntil::Stage2 { + return out; + } + + section(&mut out, "Stage 3 output: Selection"); + let chosen = match &run.selection { + Ok(selection) => { + selected_chosen_candidates(&logical.memo, physical, &selection.selected_dag) + } + Err(_) => Err("no plan was selected".into()), + }; + match &chosen { + Ok(choice) => { + let _ = writeln!(out, "selected choice: {}", choice_text(choice)); + } + Err(reason) => { + let _ = writeln!(out, "selected choice unavailable: {reason}"); + } + } + write_selection(&mut out, stages); + out +} + +fn section(out: &mut String, title: &str) { + let _ = writeln!(out, "\n==================== {title} ===================="); +} + +/// One line per node of the DAG of `roots` (children before parents), then +/// the query roots. +fn write_dag(out: &mut String, roots: &[QueryRoot]) { + let (flat, _) = flatten(roots); + let rows = flat + .nodes + .iter() + .enumerate() + .map(|(id, node)| { + let children: Vec = node + .operator + .children() + .into_iter() + .map(|child| format!("n{child}")) + .collect(); + vec![format!("n{id}:"), node_label(node), reads(&children)] + }) + .collect(); + write_rows(out, "", rows); + let roots: Vec = flat + .roots + .iter() + .map(|root| match root { + QueryRoot::Operator(id) => format!("n{id}"), + scalar => format!("{scalar:?}"), + }) + .collect(); + let _ = writeln!(out, "query_roots: [{}]", roots.join(", ")); +} + +/// `rows` with each column but the last padded to its widest cell, two +/// spaces apart, each line prefixed by `indent`; trailing empty cells are +/// dropped. +fn write_rows(out: &mut String, indent: &str, rows: Vec>) { + let columns = rows.iter().map(Vec::len).max().unwrap_or(0); + let widths: Vec = (0..columns) + .map(|column| { + rows.iter() + .filter_map(|row| row.get(column)) + .map(|cell| cell.chars().count()) + .max() + .unwrap_or(0) + }) + .collect(); + for row in rows { + let last = row.iter().rposition(|cell| !cell.is_empty()).unwrap_or(0); + let mut line = indent.to_owned(); + for (column, cell) in row.iter().enumerate().take(last + 1) { + line.push_str(cell); + if column < last { + let padding = widths[column] - cell.chars().count() + 2; + line.push_str(&" ".repeat(padding)); + } + } + let _ = writeln!(out, "{line}"); + } +} + +/// `<- a, b`, or nothing when there is nothing read. +fn reads(inputs: &[String]) -> String { + if inputs.is_empty() { + String::new() + } else { + format!("<- {}", inputs.join(", ")) + } +} + +/// `G0, G1`. +fn group_list(groups: &[MemoGroupId]) -> String { + groups + .iter() + .map(|group| format!("G{}", group.0)) + .collect::>() + .join(", ") +} + +/// `G2:c0 G3:c0`: each chosen group's candidate. +fn choice_text(choice: &asap_logical_asap_optimization::ChosenCandidatePerMemoGroup) -> String { + choice + .iter() + .map(|(group, candidate)| format!("G{}:c{candidate}", group.0)) + .collect::>() + .join(" ") +} + +/// `memo`'s groups, one line each with the child groups it reads, and with +/// `with_candidates` each group's candidates below it: index, origin, +/// `new` when `previous` (the memo at the step before) has no candidate of +/// that group with the same node and inputs, chain and input groups (fused +/// groups, and accuracy budget parts when more than one). +fn write_memo(out: &mut String, memo: &Memo, previous: Option<&Memo>, with_candidates: bool) { + // The group lines share one set of column widths, with each group's + // candidates printed under its line. + let group_rows: Vec> = memo + .groups + .iter() + .enumerate() + .map(|(id, group)| { + let children: Vec = group + .child_groups + .iter() + .map(|child| format!("G{}", child.0)) + .collect(); + vec![ + format!("G{id}:"), + root_label(&group.logical_node), + reads(&children), + ] + }) + .collect(); + let mut group_lines = String::new(); + write_rows(&mut group_lines, "", group_rows); + for (id, line) in group_lines.lines().enumerate() { + let _ = writeln!(out, "{line}"); + if !with_candidates { + continue; + } + let earlier = previous + .and_then(|memo| memo.candidates.get(id)) + .map_or(&[][..], Vec::as_slice); + let rows = memo.candidates[id] + .iter() + .enumerate() + .map(|(index, candidate)| { + let added = previous.is_some() + && !earlier.iter().any(|before| { + Rc::ptr_eq(&before.root_node, &candidate.root_node) + && before.input_groups == candidate.input_groups + }); + let mut inputs = format!("inputs=[{}]", group_list(&candidate.input_groups)); + if !candidate.fused_groups.is_empty() { + let _ = write!(inputs, " fuses=[{}]", group_list(&candidate.fused_groups)); + } + if candidate.accuracy_budget_parts != 1 { + let _ = write!( + inputs, + " accuracy_budget_parts={}", + candidate.accuracy_budget_parts + ); + } + vec![ + format!("[{index}]"), + format!("{:?}", candidate.origin), + if added { + "new".to_owned() + } else { + String::new() + }, + chain_label(memo, &candidate.root_node, &candidate.input_groups), + inputs, + ] + }) + .collect(); + write_rows(out, " ", rows); + } + let roots: Vec = memo + .query_roots + .iter() + .map(|root| match root { + QueryRoot::Operator(group) => format!("G{}", group.0), + scalar => format!("{scalar:?}"), + }) + .collect(); + let _ = writeln!(out, "query_roots: [{}]", roots.join(", ")); +} + +fn write_physical(out: &mut String, physical: &PhysicalASAPMemoCandidates) { + let nodes = &physical.operator_nodes_of_candidates_for_memo_groups; + let _ = writeln!( + out, + "operator_nodes_of_candidates_for_memo_groups ({}):", + nodes.len() + ); + let rows = nodes + .iter() + .enumerate() + .map(|(id, node)| { + let implements: Vec = node + .candidates_for_memo_group + .iter() + .map(|candidate| format!("G{}[{}]", candidate.group.0, candidate.index)) + .collect(); + let mut role = if node.is_implementation_root { + "implementation root".to_owned() + } else { + "private".to_owned() + }; + if node.accuracy_budget_parts != 1 { + let _ = write!( + role, + " accuracy_budget_parts={}", + node.accuracy_budget_parts + ); + } + vec![ + format!("O{id}:"), + root_label(&node.physical_node), + format!("implements=[{}]", implements.join(", ")), + role, + ] + }) + .collect(); + write_rows(out, " ", rows); + + let _ = writeln!(out, "candidates ({}):", physical.candidates.len()); + let rows = physical + .candidates + .iter() + .enumerate() + .map(|(id, candidate)| { + // One `{...}` per input: the candidates that may produce it. + let producers: Vec = candidate + .producer_candidates_per_input + .iter() + .map(|producers| { + let ids: Vec = producers.iter().map(|p| format!("C{}", p.0)).collect(); + format!("{{{}}}", ids.join(", ")) + }) + .collect(); + vec![ + format!("C{id}:"), + format!("O{}", candidate.operator_node.0), + format!("{:?}", candidate.timing), + format!("{:?}", candidate.materialization), + // It may produce a query's output. + if candidate.may_produce_query_output { + "output".to_owned() + } else { + String::new() + }, + reads(&producers), + ] + }) + .collect(); + write_rows(out, " ", rows); + + let output_groups: Vec = physical + .query_roots + .iter() + .map(|root| match root { + QueryRoot::Operator(group) => format!("G{}", group.0), + scalar => format!("{scalar:?}"), + }) + .collect(); + let _ = writeln!(out, "query_roots: [{}]", output_groups.join(", ")); + let _ = writeln!( + out, + "reader_constraints ({}):", + physical.reader_constraints.len() + ); + for constraint in &physical.reader_constraints { + let _ = writeln!(out, " {constraint:?}"); + } + let _ = writeln!( + out, + "pruned_candidates ({}):", + physical.pruned_candidates.len() + ); + let rows = physical + .pruned_candidates + .iter() + .map(|pruned| { + let key = &pruned.candidate; + vec![ + format!("G{}", pruned.group.0), + format!("O{}", key.operator_node.0), + format!("{:?}", key.timing), + format!("{:?}", key.materialization), + format!("{:?}", pruned.reason), + ] + }) + .collect(); + write_rows(out, " ", rows); +} + +/// Stage 3's selection: the search's outcome, the selected DAG (each node's +/// candidate, operator and producer nodes), the plan's cost with each plan +/// node's charge, and each query's accepted accuracy; or why none was +/// selected. +fn write_selection(out: &mut String, stages: &PlanningStagesRun) { + let run = &stages.run; + let selection = match &run.selection { + Ok(selection) => selection, + Err(error) => { + let _ = writeln!(out, "error: {error}"); + return; + } + }; + let report = &selection.search_report; + let _ = writeln!( + out, + "search: {:?}, {} MILP variables, {} constraints, {} ms solving", + report.status, report.milp_variables, report.milp_constraints, report.solver_ms + ); + let dag = &selection.selected_dag; + let rows = dag + .candidate_of_node + .iter() + .enumerate() + .map(|(node, candidate)| { + let mut inputs: Vec<_> = dag + .edges + .iter() + .filter(|edge| edge.consumer.0 == node) + .collect(); + inputs.sort_by_key(|edge| edge.input); + let producers: Vec = inputs + .iter() + .map(|edge| format!("node{}", edge.producer.0)) + .collect(); + // The operator the candidate runs. + let operator = + run.physical_asap_candidates + .as_ref() + .map_or(String::new(), |physical| { + let node = physical.candidates[candidate.0].operator_node; + root_label( + &physical.operator_nodes_of_candidates_for_memo_groups[node.0] + .physical_node, + ) + }); + vec![ + format!("node{node}:"), + format!("C{}", candidate.0), + operator, + reads(&producers), + ] + }) + .collect(); + write_rows(out, "", rows); + let outputs: Vec = dag + .node_of_output_reference + .iter() + .map(|node| format!("node{}", node.0)) + .collect(); + let _ = writeln!(out, "node_of_output_reference: [{}]", outputs.join(", ")); + let cost = &selection.cost; + let _ = writeln!( + out, + "cost_per_second: {}", + cost.cost_per_second + .map_or("none".to_owned(), |cost| format!("{cost:.6}")) + ); + // Keyed by the selected plan's node ids, as Stage 2 built it. + let rows = cost + .per_node + .iter() + .map(|(node, charge)| { + vec![ + format!("#{node}"), + format!("{:?}", charge.operator), + format!("{:?}", charge.timing), + charge + .cost_per_second + .map_or("none".to_owned(), |cost| format!("{cost:.6}")), + charge.detail.clone(), + ] + }) + .collect(); + write_rows(out, " ", rows); + let _ = writeln!(out, "accepted accuracy:"); + for (query, accepted) in selection.accepted_accuracy_per_query.iter().enumerate() { + let _ = writeln!(out, " q{}: {accepted}", query + 1); + } +} + +/// The label of `node`'s own operator, without its children. +fn root_label(node: &Rc) -> String { + let (flat, _) = flatten(&[QueryRoot::Operator(node.clone())]); + flat.nodes.last().map(node_label).unwrap_or_default() +} + +/// A candidate's own nodes, from `root` down to its `input_groups`' nodes +/// (excluded), root first, joined by " ← ". +fn chain_label(memo: &Memo, root: &Rc, input_groups: &[MemoGroupId]) -> String { + let (flat, originals) = flatten(&[QueryRoot::Operator(root.clone())]); + let is_input = |id: usize| { + input_groups + .iter() + .any(|group| Rc::ptr_eq(&memo.groups[group.0 as usize].logical_node, &originals[id])) + }; + let mut labels = Vec::new(); + let mut pending = vec![flat.nodes.len() - 1]; + let mut seen = HashSet::new(); + while let Some(id) = pending.pop() { + if is_input(id) || !seen.insert(id) { + continue; + } + labels.push(node_label(&flat.nodes[id])); + pending.extend( + flat.nodes[id] + .operator + .children() + .into_iter() + .rev() + .copied(), + ); + } + labels.join(" ← ") +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::planner_layering_examples::example1; + use crate::planning_stages::{lower_promql_roots, run_stages_until}; + use asap_planner::PlanningModels; + use asap_types::workload::Query; + + /// One repeating `sum(rate(...[1m]))` query: every section is printed, + /// Pass 2 adds tumbling-window candidates with their layout, Stage 2 offers + /// the tumbling-window chain at ingestion time, and Stage 3 names the selected choice. + #[test] + fn prints_every_stage_of_a_repeating_rate_query() { + let mut workload = example1(); + let queries = workload.query_workload.repeating_queries.as_mut().unwrap(); + queries.truncate(1); + queries[0].query = Query("sum by (job) (rate(http_requests_total[1m]))".into()); + let roots = lower_promql_roots(&workload).unwrap(); + let entries: Vec<_> = workload.query_workload.entries().collect(); + let stages = run_stages_until( + roots, + &entries, + &workload.data_workload.clone().unwrap(), + PlanningModels::builtin(), + RunUntil::Stage3, + ) + .unwrap(); + + let text = planning_stages_text(&stages); + + for heading in [ + "Stage 0 output", + "Stage 0 → Stage 1: DagOfLogicalMemoGroups", + "Stage 1 Pass 1 output: PerComputationLogicalMemoCandidates", + "Stage 1 Pass 2, after local window composition", + "Stage 2 input: LogicalASAPMemoCandidates", + "Stage 3 input: PhysicalASAPMemoCandidates", + "Stage 3 output: Selection", + ] { + assert!(text.contains(heading), "missing {heading}:\n{text}"); + } + // A group line names its operator and the groups it reads. + assert!(text.contains("G1: TimeRange 60s <- G0\n"), "{text}"); + // The line containing every one of `parts`, columns aside. + let has_line = |parts: &[&str]| { + text.lines() + .any(|line| parts.iter().all(|part| line.contains(part))) + }; + // Pass 2 adds a tumbling-window candidate reading the scan, its layout in its chain. + assert!( + has_line(&[ + "TumblingWindows new FinalizeExactAccumulator ← SummaryMerge ← SummaryTumblingWindowChain tumbling_window_count=6 tumbling_window_width_ms=10000", + "inputs=[G0]", + ]), + "{text}" + ); + // Stage 2 offers the tumbling-window chain at ingestion time, read from the + // ingestion-time scan. + assert!( + has_line(&["IngestionTime MatAtIngestion(Memory)", "<- {C1}"]), + "{text}" + ); + assert!( + text.contains("selected choice: G0:c0 G1:c0 G2:c0 G3:c0\n"), + "{text}" + ); + assert!(text.contains("accepted accuracy:\n q1: "), "{text}"); + } +} diff --git a/crates/devtools/tests/asap_planner.rs b/crates/devtools/tests/asap_planner.rs new file mode 100644 index 000000000..5f655d3b2 --- /dev/null +++ b/crates/devtools/tests/asap_planner.rs @@ -0,0 +1,2290 @@ +//! `asap_planner` writes valid stage documents: each stage's output over +//! #509's examples and ad-hoc workloads, the deployment and cost inputs it +//! plans with, and the inputs it refuses. +use std::collections::HashSet; +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; + +use asap_devtools::planning_stages::{lower_promql_roots, run_stages_until, RunUntil}; +use asap_types::ir::flat::FlatDag; +use asap_types::ir::physical_export::PhysicalASAPDAG; +use asap_types::ir::QueryRoot; +use asap_types::workload::PlanningWorkload; + +use serde_json::{json, Value}; + +/// The id of the selected plan among the physical candidates: the +/// selection is always shown as `P1`. +const SELECTED: &str = "P1"; + +/// A file under the system temp directory, removed when dropped. +struct TempFile(PathBuf); + +impl TempFile { + /// A new file holding `contents`. Tests run as threads of one process, + /// so the name needs a per-call counter. + fn new(contents: &str) -> Self { + static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + let path = std::env::temp_dir().join(format!( + "asap_planner_test_{}_{call}.json", + std::process::id() + )); + std::fs::write(&path, contents).unwrap(); + Self(path) + } + + fn json(value: &Value) -> Self { + Self::new(&value.to_string()) + } + + fn path(&self) -> &str { + self.0.to_str().unwrap() + } +} + +impl Drop for TempFile { + fn drop(&mut self) { + let _ = std::fs::remove_file(&self.0); + } +} + +/// `asap_planner` with `args`. +fn planner(args: &[&str]) -> Output { + Command::new(env!("CARGO_BIN_EXE_asap_planner")) + .args(args) + .output() + .unwrap() +} + +/// The JSON document `asap_planner` writes for `args`; the run exits 1 +/// exactly when a stage section records an error, else 0. +fn document_of(args: &[&str]) -> Value { + let mut args = args.to_vec(); + args.extend(["--format", "json"]); + let output = planner(&args); + let stderr = String::from_utf8_lossy(&output.stderr); + let document: Value = serde_json::from_str(&String::from_utf8(output.stdout).unwrap()) + .unwrap_or_else(|e| panic!("{e}: {stderr}")); + let failed = [ + "stage1_logical_asap_optimization", + "stage2_physical_asap_optimization", + "stage3_plan_selection", + ] + .iter() + .any(|stage| !document[stage]["error"].is_null()); + assert_eq!(output.status.code(), Some(i32::from(failed)), "{stderr}"); + document +} + +/// The document for the workload `workload` with `args`. +fn document(workload: &Value, args: &[&str]) -> Value { + let file = TempFile::json(workload); + let mut all = vec!["--workload", file.path()]; + all.extend_from_slice(args); + document_of(&all) +} + +/// `asap_planner`'s standard error for `args`, whose input it must refuse +/// (status 1, without the usage line, which is for bad arguments) without +/// writing output. +fn refused(args: &[&str]) -> String { + let output = planner(args); + assert_eq!(output.status.code(), Some(1), "{args:?}"); + assert!(output.stdout.is_empty(), "a refused run writes nothing"); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(!stderr.contains("usage:"), "{stderr}"); + stderr +} + +/// The committed workload file of #509's `example` (`example1`, `example2`, +/// `example3a`, `example3b`). +fn example(example: &str) -> String { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../../tools/dag-viewer/workloads/planner-layering") + .join(format!("{example}.json")) + .to_str() + .unwrap() + .to_owned() +} + +/// The document for `example` with `args`. +fn example_document(name: &str, args: &[&str]) -> Value { + let path = example(name); + let mut all = vec!["--workload", path.as_str()]; + all.extend_from_slice(args); + document_of(&all) +} + +/// `tables.json`'s tables, each as a `--sql-catalog-table` argument. +fn catalog_tables(tables_json: &Path) -> Vec { + serde_json::from_str::>(&std::fs::read_to_string(tables_json).unwrap()) + .unwrap() + .iter() + .map(Value::to_string) + .collect() +} + +/// An exact accuracy target. +fn exact() -> Value { + json!({"max_error": 0.0, "failure_probability": 0.0}) +} + +/// Within `max_error` with probability `1 - failure_probability`. +fn within(max_error: f64, failure_probability: f64) -> Value { + json!({"max_error": max_error, "failure_probability": failure_probability}) +} + +/// One-off PromQL `queries`, each run once with `accuracy` and unknown +/// predictability, reading `metrics`, each 1,000 series sampled every 15 s +/// and holding 60,000 rows, with an unknown arrival. +fn adhoc(queries: &[&str], accuracy: Value, metrics: &[&str]) -> Value { + let facts = json!({"series_count": 1000, "sample_interval": 15000, "rows_held": 60000}); + json!({ + "query_workload": {"language": "promql", "query_batch": queries.iter().map(|query| json!({ + "query": query, + "requirements": {"accuracy": accuracy}, + "predictability": "unknown", + "invocations": 1, + })).collect::>()}, + "data_workload": { + "arrival": "unknown", + "sources": metrics.iter().map(|m| (m.to_string(), facts.clone())).collect::>(), + }, + }) +} + +/// A repeating PromQL entry every 10 s within 1% (failure probability 0.001). +fn every_10s_within_1_percent(query: &str) -> Value { + json!({ + "query": query, + "demand": {"fixed_interval": 10000}, + "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, + "predictability": {"predictable": {"known_at": null}}, + }) +} + +/// A `--cost-inputs` file of `inputs`. +fn cost_inputs(inputs: &asap_cost::CostInputs) -> TempFile { + TempFile::new(&serde_json::to_string(inputs).unwrap()) +} + +/// Cost model 1 at AWS Fargate's us-east-1 prices. +fn aws_fargate() -> asap_cost::CostInputs { + let defaults = asap_cost::CostInputs::default(); + asap_cost::CostInputs { + cost_model: asap_cost::CostModel::UsageOverTime( + asap_cost::ResourceUnitCost::aws_fargate_us_east_1(defaults.resource_speed), + ), + ..defaults + } +} + +/// Stage 3 selected a plan: `stage3_plan_selection` is its `Selection`, +/// whose plan is the physical candidate [`SELECTED`] with its `cost` and +/// `accepted_accuracy_per_query`. +fn selects_a_plan(selection: &Value) -> bool { + selection["selected_dag"].is_object() && selection["cost"].is_object() +} + +fn validated_dag(value: &Value, queries: usize) { + let dag: FlatDag = serde_json::from_value(value.clone()).unwrap(); + assert_eq!(dag.roots.len(), queries); + // Children come before their parents. + for (id, node) in dag.nodes.iter().enumerate() { + assert!(node + .operator + .children() + .into_iter() + .all(|child| *child < id)); + } + for root in &dag.roots { + if let QueryRoot::Operator(id) = root { + assert!(*id < dag.nodes.len()); + } + } +} + +/// The selected plan `P1`, validated, with one root per query. +fn selected_physical(document: &Value, queries: usize) -> PhysicalASAPDAG { + let candidate = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap() + .iter() + .find(|c| c["id"] == SELECTED) + .expect("the selected plan is shown"); + let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); + dag.validate().unwrap(); + assert_eq!(dag.roots.len(), queries); + dag +} + +/// Stage 1's `pruned_candidates`: each candidate Pass 1 pruned, with its +/// reason, in pruning order. +fn pass1_pruned(document: &Value) -> Vec<(String, String)> { + document["stage1_logical_asap_optimization"]["pruned_candidates"] + .as_array() + .unwrap() + .iter() + .map(|pruned| { + ( + pruned["candidate"].as_str().unwrap().to_string(), + pruned["reason"].as_str().unwrap().to_string(), + ) + }) + .collect() +} + +/// Every candidate label of the memo step `step`. +fn memo_labels(document: &Value, step: &str) -> Vec { + document["memo"]["steps"] + .as_array() + .unwrap() + .iter() + .find(|s| s["step"] == step) + .unwrap()["groups"] + .as_array() + .unwrap() + .iter() + .flat_map(|g| g["candidates"].as_array().unwrap()) + .map(|c| c["label"].as_str().unwrap().to_owned()) + .collect() +} + +/// Example 1's document: every logical DAG is a well-formed flat DAG and +/// every physical DAG validates; each has one root per query; ids are +/// unique; Stage 2's candidates come from the shown logical ones and carry +/// no cost; Stage 3 selects `P1`. +#[test] +fn example1_document_validates_its_stage_dags() { + let document = example_document("example1", &["--run-until", "stage3"]); + assert_eq!(document["format"], "asap-stage-pipeline/v1"); + let queries = document["workload"]["queries"].as_array().unwrap().len(); + assert_eq!(queries, 2); + validated_dag(&document["stage0_query_language_frontends"]["dag"], queries); + + let logical = &document["stage1_logical_asap_optimization"]; + let candidates = logical["candidates"].as_array().unwrap(); + assert_eq!(logical["capped"], false); + assert_eq!(logical["combinations"], candidates.len()); + let mut ids = HashSet::new(); + let mut composed = HashSet::new(); + for candidate in candidates { + let id = candidate["id"].as_str().unwrap(); + assert!(ids.insert(id)); + if candidate.get("dag").is_some() { + composed.insert(id); + validated_dag(&candidate["dag"], queries); + } + } + + let physical = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + let sources: HashSet<_> = physical + .iter() + .map(|p| p["from_logical"].as_str().unwrap()) + .collect(); + assert_eq!(sources, composed); + for candidate in physical { + assert!(candidate.get("cost").is_none(), "Stage 2 has no cost"); + let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); + assert_eq!(dag.roots.len(), queries); + dag.validate().unwrap(); + } + assert!(selects_a_plan(&document["stage3_plan_selection"])); + selected_physical(&document, queries); +} + +/// A top-k partitioned by every label of its input has no item, so a heap +/// over it could not be composed; such a heap reads one row per item and +/// is pruned in Pass 1 (`TopKHeapOverOneRowPerItem`), listed under +/// `pruned_candidates`, and the selected plan's logical DAG composes. +#[test] +fn a_topk_heap_over_one_row_per_item_is_pruned_with_its_reason() { + let document = document( + &adhoc( + &["topk by (job) (3, sum by (job) (rate(m[5m])))"], + within(0.01, 0.001), + &["m"], + ), + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + assert_eq!( + pass1_pruned(&document), + [ + ("CMS+heap".into(), "TopKHeapOverOneRowPerItem".into()), + ( + "CountSketch+heap".into(), + "TopKHeapOverOneRowPerItem".into() + ), + ] + ); + let candidates = document["stage1_logical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + assert!(!candidates.is_empty()); + for candidate in candidates { + validated_dag(&candidate["dag"], 1); + } +} + +/// Example 3a (243 Pass 1 candidates) selects a plan whose physical DAG +/// validates with its five queries. +#[test] +fn example3a_selects_a_valid_plan() { + let document = example_document( + "example3a", + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + assert!(selects_a_plan(&document["stage3_plan_selection"])); + selected_physical(&document, 5); +} + +/// Example 2's three SQL queries over `flows` lower to one well-formed DAG +/// with a root per query. +#[test] +fn example2_lowers_its_three_sql_queries() { + let tables = catalog_tables( + &PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../../tools/dag-viewer/workloads/planner-layering/tables.json"), + ); + let document = example_document( + "example2", + &["--run-until", "stage0", "--sql-catalog-table", &tables[0]], + ); + let queries = document["workload"]["queries"].as_array().unwrap(); + assert_eq!(queries.len(), 3); + assert!(queries.iter().all(|q| q["language"] == "sql")); + validated_dag(&document["stage0_query_language_frontends"]["dag"], 3); +} + +/// Example 2 plans its three SQL queries and selects a plan. +#[test] +fn example2_plans_the_sql_workload() { + let tables = catalog_tables( + &PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../../tools/dag-viewer/workloads/planner-layering/tables.json"), + ); + let document = example_document( + "example2", + &[ + "--run-until", + "stage3", + "--ignore-accuracy-guarantees", + "--sql-catalog-table", + &tables[0], + ], + ); + assert!(selects_a_plan(&document["stage3_plan_selection"])); + selected_physical(&document, 3); +} + +/// `metrics(ts, service, latency)`. +const METRICS_TABLE: &str = r#"{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp", "nullable": false}, + {"name": "service", "type": "utf8", "nullable": false}, + {"name": "latency", "type": "float64", "nullable": false}], "time_index": 0}"#; + +/// One-off exact SQL `queries`, each run once, over `metrics`, which holds +/// 1,000 series and 60,000 rows. +fn adhoc_sql(queries: &[&str]) -> Value { + json!({ + "query_workload": {"language": {"sql": "datafusion_sql"}, "query_batch": queries.iter().map(|query| json!({ + "query": query, + "requirements": {"accuracy": exact()}, + "predictability": "unknown", + "invocations": 1, + })).collect::>()}, + "data_workload": {"arrival": "unknown", "sources": {"metrics": {"series_count": 1000, "rows_held": 60000}}}, + }) +} + +/// A count and an average by service over `metrics`. +fn two_sql_queries() -> Value { + adhoc_sql(&[ + "SELECT service, COUNT(*) FROM metrics GROUP BY service", + "SELECT service, AVG(latency) FROM metrics GROUP BY service", + ]) +} + +/// SQL queries over tables declared with `--sql-catalog-table` are lowered +/// by the SQL frontend, and the document records each query's language. +#[test] +fn sql_queries_over_declared_tables_are_lowered() { + let document = document( + &two_sql_queries(), + &[ + "--run-until", + "stage0", + "--sql-catalog-table", + METRICS_TABLE, + ], + ); + let queries = document["workload"]["queries"].as_array().unwrap(); + assert_eq!(queries.len(), 2); + assert!(queries.iter().all(|q| q["language"] == "sql")); + validated_dag(&document["stage0_query_language_frontends"]["dag"], 2); +} + +/// SQL queries over declared tables are planned through every stage to a +/// selected plan. +#[test] +fn sql_queries_over_declared_tables_are_planned() { + let document = document( + &two_sql_queries(), + &[ + "--run-until", + "stage3", + "--ignore-accuracy-guarantees", + "--sql-catalog-table", + METRICS_TABLE, + ], + ); + assert!(selects_a_plan(&document["stage3_plan_selection"])); + selected_physical(&document, 2); +} + +/// Nine approximate quantile queries over nine metrics, three candidates +/// each: 3^9 = 19683 Pass 1 combinations. +fn nine_quantile_queries() -> Value { + let queries: Vec = (1..=9) + .map(|i| format!("quantile_over_time(0.99, m{i}[5m])")) + .collect(); + let metrics: Vec = (1..=9).map(|i| format!("m{i}")).collect(); + adhoc( + &queries.iter().map(String::as_str).collect::>(), + within(0.01, 0.001), + &metrics.iter().map(String::as_str).collect::>(), + ) +} + +/// Nine quantile queries plan with the candidate-connection search, which +/// searches Stage 2's catalog without enumerating the combinations. The +/// document holds the selected plan and the search's report. +#[test] +fn nine_queries_plan_without_cartesian_enumeration() { + let document = document( + &nine_quantile_queries(), + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + assert_eq!(document["search"], "candidate-connections"); + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{selection}"); + let physical = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + assert_eq!(physical.len(), 1); + assert_eq!(physical[0]["id"], SELECTED); + assert!(selection["search_report"].is_object(), "{selection}"); +} + +/// A deployment that keeps neither raw data nor query-time results. +const NO_RAW: &str = r#"{"raw_data_retained": false, "query_time_results_retained": false}"#; + +/// ASAPQuery's random workload (seed 862, workload-01): nine PromQL queries. +const NINE_QUERY_WORKLOAD: &str = + "../../tools/dag-viewer/workloads/random-mixed-seed-862/workload-01.json"; + +/// The nine-query workload's document up to `run_until`, planned as a +/// deployment that keeps no raw data at AWS Fargate prices. +fn nine_query_document(run_until: &str) -> Value { + let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(NINE_QUERY_WORKLOAD); + let capabilities = TempFile::new(NO_RAW); + let fargate = cost_inputs(&aws_fargate()); + document_of(&[ + "--workload", + path.to_str().unwrap(), + "--run-until", + run_until, + "--deployment-capabilities", + capabilities.path(), + "--cost-inputs", + fargate.path(), + ]) +} + +/// The nine-query workload, planned as a deployment that keeps no raw data +/// at AWS Fargate prices with guarantees enforced: `asap_planner` lowers it +/// to one root per query, and installation returns a validated plan that +/// accepts no query without its guarantee. +#[test] +fn the_nine_query_workload_lowers_and_installs() { + let document = nine_query_document("stage2"); + assert_eq!(document["search"], "candidate-connections"); + assert_eq!(document["accuracy_guarantees"], "enforced"); + let queries = document["workload"]["queries"].as_array().unwrap().len(); + assert_eq!(queries, 9); + validated_dag(&document["stage0_query_language_frontends"]["dag"], queries); + + let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(NINE_QUERY_WORKLOAD); + let workload: PlanningWorkload = + serde_json::from_str(&std::fs::read_to_string(&path).unwrap()).unwrap(); + let settings = aws_fargate(); + let capabilities: asap_planner::DeploymentCapabilities = serde_json::from_str(NO_RAW).unwrap(); + let models = asap_planner::PlanningModels::builtin() + .with_cost_inputs(&settings) + .with_deployment_capabilities(&capabilities); + let (plan, report) = tokio::runtime::Builder::new_current_thread() + .build() + .unwrap() + .block_on(asap_planner::plan_installable( + asap_planner::UserInput::new(&workload, asap_planner::FrontendInput::Promql, models), + )) + .unwrap(); + plan.validate().unwrap(); + assert_eq!(plan.query_texts.len(), queries); + let report = report.unwrap(); + assert!(report.queries_accepted_without_guarantee.is_empty()); + assert!(report + .cost_per_second + .is_some_and(|cost| cost.is_finite() && cost > 0.0)); + assert_eq!(report.predicted_usage.per_query.len(), queries); +} + +/// `asap_planner` selects a validated plan for the nine-query workload +/// (no raw data, AWS Fargate prices, guarantees enforced), with an +/// accepted accuracy per query and a MILP search report. +#[test] +fn the_nine_query_workload_selects_a_valid_plan() { + let document = nine_query_document("stage3"); + let queries = 9; + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{selection}"); + let physical = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + assert_eq!(physical.len(), 1); + selected_physical(&document, queries); + assert!( + selection["search_report"]["milp_variables"] + .as_u64() + .unwrap() + > 0 + ); + assert_eq!( + selection["accepted_accuracy_per_query"] + .as_array() + .unwrap() + .len(), + queries + ); +} + +/// A bare instant selector looks back its metric's sample interval, as +/// the data workload declares it: `sum(up)` over `up` sampled every 30 s +/// reads a 30 s range. +#[test] +fn the_sample_interval_sets_the_instant_selector_lookback() { + let mut workload = adhoc(&["sum(up)"], exact(), &["up"]); + workload["data_workload"]["sources"]["up"]["sample_interval"] = json!(30000); + let document = document(&workload, &["--run-until", "stage0"]); + let ranges: Vec<&Value> = document["stage0_query_language_frontends"]["dag"]["nodes"] + .as_array() + .unwrap() + .iter() + .filter_map(|node| node["operator"]["NonASAP"]["TimeRange"]["range"].as_object()) + .map(|range| &range["secs"]) + .collect(); + assert_eq!(ranges, [&Value::from(30)]); +} + +/// A table declared twice is refused. +#[test] +fn table_declared_twice_is_rejected() { + let table = + r#"{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp"}], "time_index": 0}"#; + let workload = TempFile::json(&adhoc_sql(&["SELECT COUNT(*) FROM metrics"])); + let stderr = refused(&[ + "--workload", + workload.path(), + "--run-until", + "stage0", + "--sql-catalog-table", + table, + "--sql-catalog-table", + table, + ]); + assert!( + stderr.contains("--sql-catalog-table: table metrics is declared twice"), + "{stderr}" + ); +} + +/// When Pass 1 fails, the document still carries the Stage 0 DAG, with the +/// error under Stage 1 and no Stage 2 or Stage 3 section, so the viewer can +/// show both, and the run exits 1. +#[test] +fn pass1_failure_still_writes_stage0_and_the_error() { + let document = document( + &adhoc(&["topk(5, rate(m[5m]))"], within(1.5, 0.1), &["m"]), + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + validated_dag(&document["stage0_query_language_frontends"]["dag"], 1); + let logical = &document["stage1_logical_asap_optimization"]; + assert!( + logical["error"] + .as_str() + .unwrap() + .starts_with("Pass 1: approximate accuracy"), + "{logical}" + ); + assert_eq!(logical["combinations"], 0); + assert_eq!(logical["candidates"], json!([])); + assert_eq!(document["stage2_physical_asap_optimization"], Value::Null); + assert!(!selects_a_plan(&document["stage3_plan_selection"])); + // Text mode also prints the error and exits 1. + let file = TempFile::json(&adhoc(&["topk(5, rate(m[5m]))"], within(1.5, 0.1), &["m"])); + let output = planner(&["--workload", file.path(), "--run-until", "stage3"]); + assert_eq!(output.status.code(), Some(1)); + assert!(String::from_utf8(output.stdout) + .unwrap() + .contains("error: Pass 1: ")); +} + +/// A query Pass 1 rejects is reported before deployment capabilities that +/// contradict the data workload (as `run_planning` orders them), and a +/// metric-type warning does not turn the null Stage 3 section into one. +#[test] +fn pass1_failure_comes_before_capabilities_and_keeps_stage3_null() { + let mut workload = adhoc(&["topk(5, rate(m[5m]))"], within(1.5, 0.1), &["m"]); + workload["data_workload"]["arrival"] = json!("at_rest"); + workload["data_workload"]["sources"]["m"]["metric_type"] = json!("gauge"); + let no_raw = TempFile::new(NO_RAW); + let document = document( + &workload, + &[ + "--run-until", + "stage3", + "--ignore-accuracy-guarantees", + "--deployment-capabilities", + no_raw.path(), + ], + ); + let error = document["stage1_logical_asap_optimization"]["error"] + .as_str() + .unwrap(); + assert!(error.starts_with("Pass 1: approximate accuracy"), "{error}"); + assert_eq!(document["stage3_plan_selection"], Value::Null); +} + +/// Standalone scalar queries first, in the middle and last keep their +/// query positions in the selected physical DAG's `scalar_roots` (the +/// viewer's test_viewer.py draws this workload). +#[test] +fn scalar_queries_keep_their_positions_in_scalar_roots() { + let document = document( + &adhoc( + &[ + "scalar(sum(m))", + "sum(n)", + "time()", + "sum by (job) (k)", + "1 + 2", + ], + exact(), + &["k", "m", "n"], + ), + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + assert!(selects_a_plan(&document["stage3_plan_selection"])); + let physical = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap() + .iter() + .find(|c| c["id"] == SELECTED) + .unwrap(); + let positions: Vec<_> = physical["dag"]["scalar_roots"] + .as_array() + .unwrap() + .iter() + .map(|entry| entry[0].as_u64().unwrap()) + .collect(); + assert_eq!(positions, [0, 2, 4]); +} + +/// `time_index` must be a column index; anything else is an error rather +/// than a table without a time column. +#[test] +fn table_time_index_must_be_a_column_index() { + let workload = TempFile::json(&adhoc_sql(&["SELECT COUNT(*) FROM metrics"])); + for index in [r#""0""#, "-1", "1.5", "3"] { + let table = format!( + r#"{{"name": "metrics", "columns": [{{"name": "ts", "type": "timestamp"}}], "time_index": {index}}}"# + ); + let stderr = refused(&[ + "--workload", + workload.path(), + "--run-until", + "stage0", + "--sql-catalog-table", + &table, + ]); + assert!( + stderr.contains( + "--sql-catalog-table metrics: time_index must be the index of one of its 1 columns" + ), + "{index}: {stderr}" + ); + } +} + +/// A query whose target sets no metric shows the metric its result is +/// naturally stated in (the default-metric table, display only): a top-k's +/// is the L1-scaled absolute error; an exact query shows none. +#[test] +fn a_target_without_a_metric_shows_the_query_default() { + let document = example_document("example1", &["--run-until", "stage0"]); + let queries = &document["workload"]["queries"]; + assert!(queries[0]["requirements"].get("default_metric").is_none()); + assert_eq!( + queries[1]["requirements"]["default_metric"], + "absolute_stream_l1" + ); +} + +/// A quantile query's target may leave its metric unset: then Pass 1 keeps +/// both KLL (rank) and DDSketch (relative) candidates, which compete on +/// cost, and the selection reports the metric it accepted and that the +/// target's metric was not set. With `accuracy_metric: rank`, Pass 1 prunes +/// DDSketch and the selection accepts rank (or exact). The workload's +/// accuracy is the target's own JSON form. Guarantees are enforced (the +/// default), since the test is about which guarantees meet the target. +#[test] +fn a_quantile_target_without_a_metric_accepts_rank_or_relative() { + let query = |metric: Option<&str>| { + let mut accuracy = json!({"max_error": 0.01}); + if let Some(metric) = metric { + accuracy["accuracy_metric"] = metric.into(); + } + let workload = json!({ + "query_workload": {"language": "promql", "query_batch": [{ + "query": "quantile_over_time(0.99, m[1m])", + "requirements": {"accuracy": accuracy}, + "predictability": "unknown", "invocations": 1 + }]}, + // Relative DDS interpolation requires this explicit domain proof. + "data_workload": {"arrival": "unknown", "sources": {"m": { + "sample_interval": 1000, "series_count": 1, "values_non_negative": true + }}} + }); + document(&workload, &["--run-until", "stage3"]) + }; + let summary = |document: &Value| -> String { + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{selection}"); + selection["accepted_accuracy_per_query"][0]["summary"] + .as_str() + .unwrap() + .to_owned() + }; + let unset = query(None); + assert_eq!(unset["accuracy_guarantees"], "enforced"); + assert_eq!( + unset["workload"]["queries"][0]["requirements"]["accuracy"], + json!({"max_error": 0.01, "accuracy_scope": "whole_result"}) + ); + // A quantile has no single default metric to show. + assert!(unset["workload"]["queries"][0]["requirements"] + .get("default_metric") + .is_some_and(Value::is_null)); + assert!( + pass1_pruned(&unset).is_empty(), + "{:?}", + pass1_pruned(&unset) + ); + let labels = memo_labels(&unset, "pass1"); + for family in ["Kll", "DDSketch"] { + assert!( + labels.iter().any(|l| l.contains(family)), + "{family}: {labels:?}" + ); + } + let accepted = summary(&unset); + assert!( + accepted.starts_with("metric not set; accepted "), + "{accepted}" + ); + assert!( + unset["stage3_plan_selection"]["accepted_accuracy_per_query"][0]["accepted_metric"] + .is_string() + ); + + let rank = query(Some("rank")); + assert_eq!( + pass1_pruned(&rank), + [("DDSketch".into(), "NoErrorBoundForQueryTarget".into())] + ); + let accepted = summary(&rank); + assert!( + accepted == "metric rank; accepted rank (Kll)" || accepted == "metric rank; accepted exact", + "{accepted}" + ); +} + +/// The `--cost-inputs` cost model decides how Example 4's candidates are +/// compared: cost model 1 (the default) selects a priced plan, and Stage 3's +/// selection MILP refuses cost models 2 and 3 with an error naming the +/// model (#888). +#[test] +fn the_cost_model_chooses_how_candidates_are_compared() { + use asap_cost::{ClusterLimits, ClusterObjective, CostInputs, CostModel, ResourceUnitCost}; + let selected = |cost_model: CostModel| { + let file = cost_inputs(&CostInputs { + cost_model, + ..Default::default() + }); + let args = ["--run-until", "stage3", "--cost-inputs", file.path()]; + example_document("example3b", &args)["stage3_plan_selection"].clone() + }; + let usage = selected(CostModel::UsageOverTime(ResourceUnitCost::illustrative())); + assert!(selects_a_plan(&usage), "{usage}"); + assert!(usage["cost"]["cost_per_second"].is_number()); + for (cost_model, name) in [ + ( + CostModel::ProvisionedCapacity(ResourceUnitCost::illustrative()), + "ProvisionedCapacity (cost model 2)", + ), + ( + CostModel::ClusterCapacity(ClusterLimits { + memory_bytes: 256_000_000_000, + disk_bytes: 1_000_000_000_000, + cpu_cores: None, + objective: ClusterObjective::LowestLatency, + }), + "ClusterCapacity (cost model 3)", + ), + ] { + let refused = selected(cost_model); + assert!(!selects_a_plan(&refused), "{refused}"); + assert!( + refused["error"].as_str().unwrap().contains(name), + "{refused}" + ); + } +} + +/// AWS Fargate's us-east-1 prices Example 4 under cost model 1 in US +/// dollars per second: the selected plan costs no more than the +/// whole-window KLL, $2.60278e-8/s (its 2.1 CPU-ms/s at $1.1244e-8 per +/// CPU-ms, and its states held while the query runs). +#[test] +fn aws_fargate_prices_example_4_in_dollars() { + let fargate = cost_inputs(&aws_fargate()); + let document = example_document( + "example3b", + &[ + "--run-until", + "stage3", + "--ignore-accuracy-guarantees", + "--cost-inputs", + fargate.path(), + ], + ); + let cost = document["stage3_plan_selection"]["cost"]["cost_per_second"] + .as_f64() + .unwrap(); + assert!(cost.is_finite() && cost > 0.0); + assert!(cost <= 2.602778532914e-8 * (1.0 + 1e-6), "{cost}"); +} + +/// `--deployment-capabilities` declares the executor's capabilities, which +/// the document's `deployment_capabilities` shows (defaults filled in; +/// without the flag, every summary supported, raw data retained, no +/// query-time result retained). For Pattern B with KLL only and query-time +/// results retained: no DDSketch candidate is offered, and Stage 1's +/// `pruned_candidates` says why. Without raw data, a plan is still selected +/// with guarantees enforced, from materialized views. A deployment with an +/// unknown field is refused. +#[test] +fn deployment_capabilities_flag_sets_the_capabilities_and_shows_them() { + let with_capabilities = |capabilities: &str, extra: &[&str]| { + let file = TempFile::new(capabilities); + let mut args = vec![ + "--run-until", + "stage3", + "--deployment-capabilities", + file.path(), + ]; + args.extend_from_slice(extra); + example_document("example3b", &args) + }; + let default = example_document( + "example3b", + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + assert_eq!( + default["deployment_capabilities"], + json!({ + "supported_summaries": asap_executor::supported_summary_families(), + "raw_data_retained": true, + "query_time_results_retained": false, + }) + ); + let kll_kept = with_capabilities( + r#"{"supported_summaries": [{"family": "Kll", "readouts": ["Quantile"]}], + "query_time_results_retained": true}"#, + &["--ignore-accuracy-guarantees"], + ); + assert_eq!( + kll_kept["deployment_capabilities"], + json!({ + "supported_summaries": [{ "family": "Kll", "readouts": ["Quantile"] }], + "raw_data_retained": true, + "query_time_results_retained": true, + }) + ); + // Pass 1 lists why DDSketch is gone. + assert_eq!( + pass1_pruned(&kll_kept), + [("DDSketch".into(), "UnsupportedByDeployment".into())] + ); + assert!(pass1_pruned(&default).is_empty()); + assert!(selects_a_plan(&kll_kept["stage3_plan_selection"])); + let physical = kll_kept["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + assert!(!physical.is_empty()); + assert!(physical + .iter() + .all(|c| !c["dag"].to_string().contains("DDSketch"))); + + let no_raw = with_capabilities(r#"{"raw_data_retained": false}"#, &[]); + let selection = &no_raw["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{selection}"); + assert!(!selection["materialized_views"] + .as_array() + .unwrap() + .is_empty()); + + let unknown = TempFile::new(r#"{"raw_data": false}"#); + let stderr = refused(&[ + "--workload", + &example("example3b"), + "--run-until", + "stage0", + "--deployment-capabilities", + unknown.path(), + ]); + assert!( + stderr.contains("--deployment-capabilities") && stderr.contains("unknown field `raw_data`"), + "{stderr}" + ); +} + +/// Two sums and two maxima over two metrics share each metric's scan and +/// range between its own pair of queries: the selected logical candidate's +/// label names which queries share each node. +#[test] +fn logical_candidate_labels_name_each_query() { + let document = document( + &adhoc( + &["sum(a)", "max(a)", "sum(b)", "max(b)"], + exact(), + &["a", "b"], + ), + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + let labels: Vec<&str> = document["stage1_logical_asap_optimization"]["candidates"] + .as_array() + .unwrap() + .iter() + .map(|c| c["display_name"].as_str().unwrap()) + .collect(); + assert!(!labels.is_empty()); + assert!(labels + .iter() + .all(|label| (1..=4).all(|q| label.contains(&format!("Q{q} "))))); +} + +/// The MILP search's reported lower bound never exceeds the selected +/// plan's cost, and an optimal search reports a relative gap of at most +/// 1e-4, on `topk(32, sum by (label_0) (sum_over_time(data[1h])))`, whose +/// largest charge (the query-time top-k tumbling windows) is about 308 times the +/// selected plan's. +#[test] +fn milp_lower_bound_is_consistent_with_the_selected_cost() { + let workload = json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [{ + "query": "topk(32, sum by (label_0) (sum_over_time(data[1h])))", + "demand": {"fixed_interval": 60000}, + "requirements": { + "accuracy": { + "max_error": 0.05, + "failure_probability": 0.01, + "accuracy_scope": "whole_result", + }, + "max_latency_ms": null, + }, + "predictability": {"predictable": {"known_at": null}}, + }], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 2000000.0}}, + "sources": {"data": { + "series_count": 10000, + "metric_type": "gauge", + "values_non_negative": true, + "sample_interval": 5, + }}, + }, + }); + let capabilities = + TempFile::new(r#"{"raw_data_retained": true, "query_time_results_retained": false}"#); + let fargate = cost_inputs(&aws_fargate()); + let document = document( + &workload, + &[ + "--run-until", + "stage3", + "--deployment-capabilities", + capabilities.path(), + "--cost-inputs", + fargate.path(), + ], + ); + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{selection}"); + let report = &selection["search_report"]; + let incumbent = selection["cost"]["cost_per_second"].as_f64().unwrap(); + if let Some(lower_bound) = report["cost_lower_bound_per_second"].as_f64() { + assert!(lower_bound <= incumbent * (1.0 + 1e-9), "{report}"); + if report["status"] == "optimal" { + let relative_gap = (incumbent - lower_bound).max(0.0) / incumbent; + assert!(relative_gap <= 1e-4, "{report}"); + } + } +} + +/// A dashboard panel (`max_over_time(m[5m])` every minute over 1M series +/// sampled every 1.5 s) answered as the last hour at 1-minute steps +/// (`range_evaluation`) is priced as 61 evaluations per request: the +/// selected plan costs no more than the tumbling windows kept across evaluations, and +/// the document reports their retention, 1 h + 5 min. The workload section +/// shows the range. An invalid range is refused. +#[test] +fn workload_file_with_a_range_evaluation_is_planned() { + let workload = json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [{ + "query": "max_over_time(m[5m])", + "demand": {"fixed_interval": 60000}, + "requirements": {}, + "predictability": {"predictable": {"known_at": null}}, + "range_evaluation": {"step_ms": 60000, "span_ms": 3600000}, + }], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 666666.67}}, + "sources": {"m": {"series_count": 1000000, "sample_interval": 1500}}, + }, + }); + let capabilities = TempFile::new(r#"{"query_time_results_retained": true}"#); + let document = document( + &workload, + &[ + "--run-until", + "stage3", + "--ignore-accuracy-guarantees", + "--deployment-capabilities", + capabilities.path(), + ], + ); + assert_eq!( + document["workload"]["queries"][0]["requirements"]["range_evaluation"], + json!({"step_ms": 60000, "span_ms": 3600000}) + ); + let selection = &document["stage3_plan_selection"]; + let cost = selection["cost"]["cost_per_second"].as_f64().unwrap(); + // The reference: the window merged from tumbling windows kept across evaluations, + // priced under the same models. Its absolute cost is pinned in + // integration-tests/tests/range_evaluation.rs. + let kept_tumbling_windows = { + use asap_accuracy::propagation::recompose_guarantees; + use asap_logical_asap_optimization::test_support::{ + compose_logical_candidate, generate_per_computation_candidates, + }; + use asap_logical_asap_optimization::test_support::{ + tumbling_window_candidates, window_composition_pass2_candidates, + }; + use asap_types::ir::physical_export::PhysicalASAPOperatorPayload; + use asap_types::ir::ASAPOp; + use asap_types::physical::{Materialization, Medium}; + let parsed: PlanningWorkload = serde_json::from_value(workload.clone()).unwrap(); + let entries: Vec<_> = parsed.query_workload.entries().collect(); + let data = parsed.data_workload.clone().unwrap(); + let inventory = generate_per_computation_candidates( + &asap_logical_asap_optimization::generate_dag_of_logical_memo_groups( + &lower_promql_roots(&parsed).unwrap(), + ), + ) + .unwrap(); + let exact: Vec<_> = compose_logical_candidate(&inventory, &[0]).unwrap(); + let grids: Vec<_> = entries.iter().map(|e| e.evaluation_grid_ms()).collect(); + let tumbling_windows = tumbling_window_candidates(&exact, &grids); + let windowed = window_composition_pass2_candidates(&exact, &tumbling_windows) + .unwrap() + .remove(0) + .roots + .unwrap(); + let deployment = asap_planner::DeploymentCapabilities { + query_time_results_retained: true, + ..Default::default() + }; + let kept = asap_physical_asap_optimization::optimize_physical_asap( + "kept-tumbling-windows", + &recompose_guarantees(&windowed).unwrap(), + &entries, + &data, + deployment.query_time_results_retained, + ) + .unwrap() + .into_iter() + .find(|plan| { + plan.dag.nodes.iter().any(|n| { + matches!( + n.payload, + PhysicalASAPOperatorPayload::ASAP(ASAPOp::SummaryTumblingWindowChain { .. }) + ) && n.materialization + == Materialization::MatAtQueryKeptAcrossEvaluations(Medium::Memory) + }) + }) + .expect("a plan keeps its tumbling windows across evaluations"); + let settings = asap_cost::CostInputs::default(); + let priced = asap_devtools::memo_pricing::reference_select( + std::slice::from_ref(&kept), + &entries, + &data, + asap_planner::PlanningModels::builtin() + .with_cost_inputs(&settings) + .with_deployment_capabilities(&deployment), + ) + .unwrap(); + priced.costs[&priced.selected].cost_per_second.unwrap() + }; + assert!( + cost <= kept_tumbling_windows * (1.0 + 1e-9), + "selected {SELECTED} costs {cost}, more than the kept tumbling windows' {kept_tumbling_windows}" + ); + assert!(selection["tumbling_window_retention_ms"] + .as_object() + .unwrap() + .values() + .any(|retention| retention.as_u64() == Some(3_900_000))); + + let mut invalid = workload.clone(); + invalid["query_workload"]["repeating_queries"][0]["range_evaluation"]["step_ms"] = json!(0); + let invalid = TempFile::json(&invalid); + let stderr = refused(&["--workload", invalid.path(), "--run-until", "stage0"]); + assert!( + stderr.contains("step_ms must be greater than zero"), + "{stderr}" + ); +} + +/// The default is to enforce accuracy guarantees: Pass 1 prunes every +/// sketch candidate of an average of p99s, which has no guarantee +/// (`NoErrorBoundForQueryTarget`). With `--ignore-accuracy-guarantees`, the +/// selected sketch is accepted regardless of its guarantee and records why +/// the guarantee check would have rejected it. The document says which +/// mode it planned under, and a `--cost-inputs` file cannot carry the mode. +#[test] +fn accuracy_guarantees_are_enforced_by_default_and_ignored_on_request() { + let workload = adhoc( + &["avg by (g) (quantile_over_time(0.99, x[100s]))"], + json!({"max_error": 0.01, "failure_probability": 0.01, "accuracy_metric": "rank"}), + &["x"], + ); + let enforced = document(&workload, &["--run-until", "stage1_pass1"]); + assert_eq!(enforced["accuracy_guarantees"], "enforced"); + let no_bound: Vec<&str> = enforced["stage1_logical_asap_optimization"]["pruned_candidates"] + .as_array() + .unwrap() + .iter() + .filter(|pruned| pruned["reason"] == "NoErrorBoundForQueryTarget") + .map(|pruned| pruned["candidate"].as_str().unwrap()) + .collect(); + assert!(!no_bound.is_empty(), "{enforced}"); + + let ignored = document( + &workload, + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + assert_eq!(ignored["accuracy_guarantees"], "ignored"); + assert!(pass1_pruned(&ignored).is_empty()); + let accepted = &ignored["stage3_plan_selection"]["accepted_accuracy_per_query"][0]; + assert_eq!( + accepted["accepted_regardless_of_guarantee"], true, + "{accepted}" + ); + assert_eq!( + accepted["guarantee_check_rejection"], + "no_accuracy_guarantee" + ); + + let settings = TempFile::new(r#"{"ignore_accuracy_guarantees": true}"#); + let file = TempFile::json(&workload); + let stderr = refused(&[ + "--workload", + file.path(), + "--run-until", + "stage0", + "--cost-inputs", + settings.path(), + ]); + assert!(stderr.contains("ignore_accuracy_guarantees"), "{stderr}"); +} + +/// The workload sets the viewer lists (`tools/dag-viewer/workloads`): every +/// file the index names parses as a `PlanningWorkload` and validates, every +/// set has a name, description and source, and a workload's precomputed +/// document, when it names one, exists. +#[test] +fn workload_set_index_and_files_validate() { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tools/dag-viewer"); + let index: Value = + serde_json::from_str(&std::fs::read_to_string(root.join("workloads/index.json")).unwrap()) + .unwrap(); + assert_eq!(index["format"], "asap-workload-sets/v1"); + let sets = index["sets"].as_array().unwrap(); + assert!(!sets.is_empty()); + for set in sets { + for field in ["id", "name", "description", "source"] { + assert!( + set[field].as_str().is_some_and(|s| !s.is_empty()), + "{field}: {set}" + ); + } + if let Some(capabilities) = set.get("deployment_capabilities") { + serde_json::from_value::( + capabilities.clone(), + ) + .unwrap(); + } + if let Some(mode) = set.get("accuracy_guarantees") { + assert!(mode == "ignored" || mode == "enforced", "{set}"); + } + for entry in set["workloads"].as_array().unwrap() { + let file = root.join("workloads").join(entry["file"].as_str().unwrap()); + let text = std::fs::read_to_string(&file).unwrap(); + let workload: PlanningWorkload = + serde_json::from_str(&text).unwrap_or_else(|e| panic!("{}: {e}", file.display())); + workload + .validate() + .unwrap_or_else(|e| panic!("{}: {e}", file.display())); + if let Some(labels) = entry.get("query_labels") { + assert_eq!( + labels.as_array().unwrap().len(), + workload.query_workload.entries().count(), + "{}", + file.display() + ); + } + if let Some(document) = entry.get("document") { + assert!(root.join(document.as_str().unwrap()).exists(), "{document}"); + } + } + } +} + +/// Each workload example (`examples.json` entries with a `workload`), run +/// from its workload file with the SQL tables in the `tables.json` next to +/// it, its deployment capabilities and cost inputs (`cost-inputs/.json`) +/// when it names them, and its accuracy guarantee mode, +/// carries its data workload as the file declares it and selects a valid +/// plan, whose materialized views, as `plan_installable`'s plan derives +/// them, each map to a node of the selected Stage 2 candidate. +#[test] +fn workload_examples_select_valid_plans_with_linked_materialized_views() { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tools/dag-viewer"); + let examples: Value = + serde_json::from_str(&std::fs::read_to_string(root.join("examples.json")).unwrap()) + .unwrap(); + let workload_examples: Vec<&Value> = examples + .as_array() + .unwrap() + .iter() + .filter(|e| e.get("workload").is_some()) + .collect(); + assert!(!workload_examples.is_empty()); + for example in workload_examples { + check_workload_example(&root, example); + } +} + +fn check_workload_example(root: &Path, example: &Value) { + let path = root.join(example["workload"].as_str().unwrap()); + let mut args = vec![ + "--workload", + path.to_str().unwrap(), + "--run-until", + "stage3", + ]; + let capabilities = example.get("deployment_capabilities").map(TempFile::json); + if let Some(capabilities) = &capabilities { + args.extend(["--deployment-capabilities", capabilities.path()]); + } + let inputs = example.get("cost_model").map(|name| { + root.join("cost-inputs") + .join(format!("{}.json", name.as_str().unwrap())) + }); + if let Some(inputs) = &inputs { + args.extend(["--cost-inputs", inputs.to_str().unwrap()]); + } + match example.get("accuracy_guarantees") { + None => args.push("--ignore-accuracy-guarantees"), + Some(mode) if mode == "ignored" => args.push("--ignore-accuracy-guarantees"), + Some(mode) if mode == "enforced" => {} + Some(mode) => panic!("accuracy_guarantees {mode}"), + } + let tables_path = path.parent().unwrap().join("tables.json"); + let tables = match tables_path.exists() { + true => catalog_tables(&tables_path), + false => Vec::new(), + }; + for table in &tables { + args.extend(["--sql-catalog-table", table.as_str()]); + } + let document = document_of(&args); + + let workload: PlanningWorkload = + serde_json::from_str(&std::fs::read_to_string(&path).unwrap()).unwrap(); + assert_eq!( + document["workload"]["data_workload"], + serde_json::to_value(workload.data_workload.as_ref().unwrap()).unwrap() + ); + let queries = document["workload"]["queries"].as_array().unwrap(); + // Each query carries its recurrence and predictability as declared. + assert!(queries + .iter() + .all(|q| !q["recurrence"].is_null() && !q["predictability"].is_null())); + + let selection = &document["stage3_plan_selection"]; + assert!( + selects_a_plan(selection), + "{}: no validated selected plan: {selection}", + example["id"] + ); + let dag = selected_physical(&document, queries.len()); + let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); + assert_eq!(accepted.len(), queries.len()); + if document["accuracy_guarantees"] == "enforced" { + for query in accepted { + assert_eq!(query["accepted_regardless_of_guarantee"], false); + assert!(query["guarantee_check_rejection"].is_null()); + } + } + assert!( + selection.get("materialized_views_error").is_none(), + "{selection}" + ); + let nodes: HashSet<_> = dag.nodes.iter().map(|node| node.id as u64).collect(); + for view in selection["materialized_views"].as_array().unwrap() { + assert!(nodes.contains(&view["stage2_node_id"].as_u64().unwrap())); + assert!( + view["installed_position"].as_u64().unwrap() + < view["installed_count"].as_u64().unwrap() + ); + } + assert!( + selection["search_report"].is_object(), + "{}: {selection}", + example["id"] + ); +} + +/// With no supported summaries declared, `asap_planner` uses the executor's +/// families, as `plan_installable` does. CountSketch cannot answer a total, so +/// Pass 1 neither generates it for count nor reports it deployment-pruned. +#[test] +fn undeclared_supported_summaries_default_to_the_executors_families() { + let document = document( + &adhoc(&["count by (job) (m)"], within(0.01, 0.01), &["m"]), + &[ + "--run-until", + "stage1_pass1", + "--ignore-accuracy-guarantees", + ], + ); + let labels = memo_labels(&document, "pass1"); + for family in ["FinalizeExactAccumulator", "Cms", "UnivMon"] { + assert!( + labels.iter().any(|label| label.contains(family)), + "missing {family}: {labels:?}" + ); + } + assert!(labels.iter().all(|label| !label.contains("CountSketch"))); + assert!(pass1_pruned(&document).is_empty()); + assert_eq!( + document["deployment_capabilities"]["supported_summaries"], + serde_json::to_value(asap_executor::supported_summary_families()).unwrap() + ); +} + +/// Two windowed quantiles over one metric (Q1 over 1 m, Q2 over 5 m, both +/// every 10 s): with a shared scan, Stage 2 also offers the scan built once +/// per evaluation, which reads the 5 m window once (≈0.1364/s) instead of +/// once per reader (0.2203/s); the selected plan costs at most 0.143/s. +#[test] +fn a_scan_two_queries_share_is_offered_built_once_per_evaluation() { + let workload = json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [ + every_10s_within_1_percent("quantile_over_time(0.5, data[1m])"), + every_10s_within_1_percent("quantile_over_time(0.9, data[5m])"), + ], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 666.67}}, + "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}, + }, + }); + let document = document( + &workload, + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + let physical = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + assert!(physical.iter().any(|p| p["id"] == SELECTED)); + let cost = document["stage3_plan_selection"]["cost"]["cost_per_second"] + .as_f64() + .unwrap(); + assert!(cost <= 0.143, "{cost}"); +} + +/// A scan and a time range that already have every label of their metric +/// are not widened, so no memo group's candidate reads its own group, and +/// Stage 2's candidates, whose producers are its input groups' candidates, +/// read no candidate that reads them back. +#[test] +fn widened_scans_keep_memo_groups_and_stage2_candidates_acyclic() { + use asap_logical_asap_optimization::DeploymentCapabilities; + let workload: PlanningWorkload = serde_json::from_value(json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [ + every_10s_within_1_percent("sum by (job, region) (rate(data[5m]))"), + every_10s_within_1_percent("sum by (job) (rate(data[5m]))"), + every_10s_within_1_percent("sum by (job, region) (data)"), + every_10s_within_1_percent("sum by (job) (data)"), + ], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 666.67}}, + "sources": {"data": {"series_count": 10000, "metric_type": "counter", "sample_interval": 15000}}, + }, + })) + .unwrap(); + let entries: Vec<_> = workload.query_workload.entries().collect(); + let deployment = DeploymentCapabilities { + raw_data_retained: true, + ..DeploymentCapabilities::default() + }; + let run = run_stages_until( + lower_promql_roots(&workload).unwrap(), + &entries, + workload.data_workload.as_ref().unwrap(), + asap_planner::PlanningModels::builtin().with_deployment_capabilities(&deployment), + RunUntil::Stage2, + ) + .unwrap() + .run; + let memo = &run.logical_asap_candidates.as_ref().unwrap().memo; + assert!(memo + .candidates + .iter() + .flatten() + .any(|candidate| candidate.origin + == asap_types::stages::OriginOfCandidateForMemoGroup::WidenedScans)); + // Whether `to` is reachable from `from` along `edges`. + fn reaches(edges: &dyn Fn(usize) -> Vec, from: usize, to: usize) -> bool { + let mut seen = HashSet::new(); + let mut pending = edges(from); + while let Some(next) = pending.pop() { + if next == to { + return true; + } + if seen.insert(next) { + pending.extend(edges(next)); + } + } + false + } + let group_reads = |group: usize| -> Vec { + memo.candidates[group] + .iter() + .flat_map(|candidate| candidate.input_groups.iter().map(|g| g.0 as usize)) + .collect() + }; + for group in 0..memo.groups.len() { + assert!( + !reaches(&group_reads, group, group), + "memo group {group} reads itself" + ); + } + let physical = run.physical_asap_candidates.as_ref().unwrap(); + let candidate_reads = |candidate: usize| -> Vec { + physical.candidates[candidate] + .producer_candidates_per_input + .iter() + .flat_map(|producers| producers.iter().map(|id| id.0)) + .collect() + }; + for candidate in 0..physical.candidates.len() { + assert!( + !reaches(&candidate_reads, candidate, candidate), + "Stage 2 candidate {candidate} reads itself" + ); + } +} + +/// #840's workload: Q1 and Q2 window `data` (1 m and 5 m), Q3 and Q4 read +/// its latest samples by (host, job) and by job, so the four scans project +/// different labels. The widening rule's one scan containing every label +/// installs, compiles and is charged once per evaluation, and the selected +/// plan costs no more than it or than the partial-sharing plan (#994). +#[test] +fn scans_projecting_different_labels_are_shared() { + let workload = json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [ + every_10s_within_1_percent("quantile_over_time(0.5, data[1m])"), + every_10s_within_1_percent("quantile_over_time(0.9, data[5m])"), + every_10s_within_1_percent("quantile by (host, job) (0.5, data)"), + every_10s_within_1_percent("quantile by (job) (0.5, data)"), + ], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 666.67}}, + "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}, + }, + }); + let document = document( + &workload, + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + let workload: PlanningWorkload = serde_json::from_value(workload).unwrap(); + let entries: Vec<_> = workload.query_workload.entries().collect(); + let data = workload.data_workload.as_ref().unwrap(); + let roots = lower_promql_roots(&workload).unwrap(); + use asap_logical_asap_optimization::test_support::with_union_scan_labels; + use asap_types::physical::{Materialization, Medium}; + // Plans `roots` with identical sub-DAGs made one node, as the memo's + // groups are, and a node read by several roots kept for the evaluation. + let plan = |name: &str, roots: &[QueryRoot]| { + let shared: Vec<_> = asap_logical_asap_optimization::test_support::share_common_sub_dags( + roots + .iter() + .enumerate() + .map(|(id, root)| match root { + QueryRoot::Operator(node) => (id, std::rc::Rc::clone(node)), + QueryRoot::Scalar(_) => panic!("operator root"), + }) + .collect(), + ) + .into_iter() + .map(|(_, node)| QueryRoot::Operator(node)) + .collect(); + let physical = asap_physical_asap_optimization::optimize_physical_asap_assigned( + name, + &shared, + &entries, + data, + false, + &mut |readers| { + Some(if readers.len() > 1 { + Materialization::MatAtQuery(Medium::Memory) + } else { + Materialization::NotMaterialized + }) + }, + ) + .unwrap() + .expect("a node read by several roots can be kept for the evaluation"); + let offered = asap_devtools::memo_pricing::reference_select( + std::slice::from_ref(&physical), + &entries, + data, + asap_planner::PlanningModels::builtin(), + ) + .unwrap(); + (physical, offered) + }; + // The widened scan: one scan with every label, shared by all four. + let widened = with_union_scan_labels(&roots).unwrap().unwrap(); + let (physical, offered) = plan("widened-scan-rule", &widened); + // The partial-sharing reference: the two windows share a narrow scan, and + // the two instant selectors a scan widened to their labels. + let mut partial = roots[..2].to_vec(); + partial.extend(with_union_scan_labels(&roots[2..]).unwrap().unwrap()); + let (_, reference) = plan("partial-sharing", &partial); + let (installed, _) = asap_planner::installable_plan_of( + &physical.roots, + None, + &offered.tumbling_window_retention_ms, + &entries, + ) + .unwrap(); + installed.validate().unwrap(); + installed.check_compiles().unwrap(); + let dag = &physical.dag; + let scans: Vec<_> = dag + .nodes + .iter() + .filter(|n| n.payload.kind_name() == "Scan") + .collect(); + let [scan] = &scans[..] else { + panic!("one scan: {scans:?}") + }; + let field_names: Vec<&str> = scan + .output_schema + .fields + .iter() + .map(|f| f.name.as_str()) + .collect(); + assert_eq!( + field_names, + ["ts", "value", "host", "job", "$promql_series_identity"] + ); + let cost = &offered.costs[&offered.selected]; + let scan_cost = &cost.per_node[&scan.id]; + assert_eq!(scan_cost.multiplier, 1, "{scan_cost:?}"); + // 10,000 series × 20 samples in 5 m, once per 10 s evaluation. + let detail = &scan_cost.detail; + assert!( + detail.starts_with("scan 200000 samples; x 0.1000/s"), + "{detail}" + ); + let shared_cost = cost.cost_per_second.unwrap(); + assert!(shared_cost.is_finite() && shared_cost > 0.0); + let selected_cost = document["stage3_plan_selection"]["cost"]["cost_per_second"] + .as_f64() + .unwrap(); + let reference_cost = reference.costs[&reference.selected] + .cost_per_second + .unwrap(); + assert!( + selected_cost <= shared_cost, + "{selected_cost} > {shared_cost}" + ); + assert!( + selected_cost <= reference_cost * (1.0 + 1e-9), + "{selected_cost} > partial sharing {reference_cost}" + ); +} + +/// The search is the candidate-connection search; its report has no +/// legacy route. +#[test] +fn the_search_reports_candidate_connections() { + let document = document( + &adhoc(&["sum(m)"], exact(), &["m"]), + &["--run-until", "stage3"], + ); + assert_eq!(document["search"], "candidate-connections"); + let report = &document["stage3_plan_selection"]["search_report"]; + assert!(report.is_object(), "{report}"); + assert!(report.get("routed_to_legacy").is_none()); + assert!(report.get("algorithm").is_none(), "{report}"); +} + +/// The document names its selected plan's logical candidate under the id +/// its physical candidates read (`from_logical`), as the viewer's +/// validator requires. +#[test] +fn the_document_names_the_selected_logical_candidate() { + let document = document( + &adhoc( + &[ + "quantile_over_time(0.5, data[5m])", + "quantile_over_time(0.9, data[5m])", + ], + exact(), + &["data"], + ), + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + let logical: HashSet<&str> = document["stage1_logical_asap_optimization"]["candidates"] + .as_array() + .unwrap() + .iter() + .map(|c| c["id"].as_str().unwrap()) + .collect(); + let physical = document["stage2_physical_asap_optimization"]["candidates"] + .as_array() + .unwrap(); + assert!(!physical.is_empty()); + for candidate in physical { + let from = candidate["from_logical"].as_str().unwrap(); + assert!(logical.contains(from), "{from} is not in {logical:?}"); + } + assert!(selects_a_plan(&document["stage3_plan_selection"])); + assert!(physical.iter().any(|c| c["id"] == SELECTED)); +} + +/// The selected plan's `per_node` entries carry the node's operator, +/// timing and predicted CPU, as `plan_installable`'s predicted usage names +/// them, and its `cpu_core_seconds_per_second` is their sum: enough to +/// compare a plan's predicted CPU with a runtime's measurement without +/// `plan_installable`. +#[test] +fn per_node_costs_show_operator_timing_and_predicted_cpu() { + let document = example_document( + "example1", + &["--run-until", "stage3", "--ignore-accuracy-guarantees"], + ); + let selection = &document["stage3_plan_selection"]; + assert!( + selects_a_plan(selection), + "missing evaluated costs: {selection}" + ); + let cost = &selection["cost"]; + let mut timings = HashSet::new(); + let mut sum = 0.0; + for node in cost["per_node"].as_object().unwrap().values() { + assert!(node["operator"].is_string(), "{node}"); + timings.insert(node["timing"].as_str().unwrap().to_owned()); + let cpu = node["cpu_core_seconds_per_second"].as_f64().unwrap(); + assert!(cpu >= 0.0, "{node}"); + sum += cpu; + } + let total = cost["cpu_core_seconds_per_second"].as_f64().unwrap(); + assert!(total > 0.0); + assert!((total - sum).abs() <= 1e-9 * total, "{total} vs {sum}"); + assert!( + timings.is_subset(&HashSet::from([ + "ingestion_time".to_owned(), + "query_time".to_owned() + ])), + "{timings:?}" + ); +} + +/// A rate by job and a 5 m maximum, every minute, over `m`, declared a +/// `metric_type` with 1,000 series sampled every 15 s. +fn rate_and_max_over(metric_type: &str) -> Value { + let entry = |query: &str| { + json!({ + "query": query, + "demand": {"fixed_interval": 60000}, + "requirements": {}, + "predictability": {"predictable": {"known_at": null}}, + }) + }; + json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [ + entry("sum by (job) (rate(m[5m]))"), + entry("max_over_time(m[5m])"), + ], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 1000.0}}, + "sources": {"m": {"series_count": 1000, "metric_type": metric_type, + "sample_interval": 15000}}, + }, + }) +} + +/// The warning for `rate_and_max_over("gauge")`'s first query. +const RATE_OVER_GAUGE_WARNING: &str = "q1: rate over m, which the data workload declares a \ + gauge: rate assumes a counter (a drop is a reset); planned as written"; + +/// `rate`, `increase` and `irate` over a source the workload declares a +/// gauge get a warning, whichever stage the run stops at, naming the query, +/// the function and the metric; the text output prints it. A counter gets +/// none. +#[test] +fn a_counter_function_over_a_declared_gauge_is_warned_about_in_text() { + let text = |metric_type: &str| { + let file = TempFile::json(&rate_and_max_over(metric_type)); + let output = planner(&["--workload", file.path(), "--run-until", "stage0"]); + assert!(output.status.success()); + String::from_utf8(output.stdout).unwrap() + }; + let warnings = |text: &str| -> Vec { + text.lines() + .filter_map(|line| line.strip_prefix("warning: ")) + .map(str::to_owned) + .collect() + }; + assert_eq!(warnings(&text("gauge")), [RATE_OVER_GAUGE_WARNING]); + assert!(warnings(&text("counter")).is_empty()); +} + +/// `rate`, `increase` and `irate` over a source the workload declares a +/// gauge still plan, with a warning under `stage3_plan_selection.warnings` +/// (which the viewer shows) naming the query, the function and the metric. +/// A counter gets none. +#[test] +fn a_counter_function_over_a_declared_gauge_plans_with_a_warning() { + let args = ["--run-until", "stage3", "--ignore-accuracy-guarantees"]; + let gauge = document(&rate_and_max_over("gauge"), &args); + let counter = document(&rate_and_max_over("counter"), &args); + assert!(selects_a_plan(&gauge["stage3_plan_selection"])); + assert_eq!( + gauge["stage3_plan_selection"]["warnings"], + json!([RATE_OVER_GAUGE_WARNING]) + ); + assert!(selects_a_plan(&counter["stage3_plan_selection"])); + assert!(counter["stage3_plan_selection"].get("warnings").is_none()); +} + +/// A PromQL file and a SQL file, each a `PlanningWorkload` in its own +/// language. +fn mixed_language_workload_files(sql_arrival: Value, sql_facts: Value) -> (TempFile, TempFile) { + let entry = |query: &str| { + json!({ + "query": query, + "demand": {"fixed_interval": 60000}, + "requirements": {}, + "predictability": {"predictable": {"known_at": null}}, + }) + }; + let promql = json!({ + "query_workload": { + "language": "promql", + "query_batch": null, + "repeating_queries": [entry("sum by (job) (rate(http_requests_total[1m]))")], + }, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 1000.0}}, + "sources": {"http_requests_total": {"series_count": 15000, "metric_type": "counter", "sample_interval": 15000}}, + }, + }); + let sql = json!({ + "query_workload": { + "language": {"sql": "datafusion_sql"}, + "query_batch": null, + "repeating_queries": [entry( + "SELECT service, COUNT(*) FROM flows \ + WHERE ts >= NOW() - INTERVAL '1 minute' GROUP BY service" + )], + }, + "data_workload": {"arrival": sql_arrival, "sources": {"flows": sql_facts}}, + }); + (TempFile::json(&promql), TempFile::json(&sql)) +} + +/// `flows(ts, service)`. The time index is declared non-null: the executor +/// builds no count over a nullable time column, so a plan of the SQL query +/// over it would not compile. +const FLOWS_TABLE: &str = r#"{"name": "flows", "columns": [{"name": "ts", "type": "timestamp", "nullable": false}, {"name": "service", "type": "utf8", "nullable": false}], "time_index": 0}"#; + +/// The document for the mixed-language files (SQL at 500 rows/s over +/// `flows`, 10 series sampled every second), run until `run_until`. +fn mixed_language_document(run_until: &str) -> Value { + let (promql, sql) = mixed_language_workload_files( + json!({"ingesting": {"rows_per_second": 500.0}}), + json!({"series_count": 10, "sample_interval": 1000}), + ); + document_of(&[ + "--workload", + promql.path(), + "--workload", + sql.path(), + "--sql-catalog-table", + FLOWS_TABLE, + "--run-until", + run_until, + "--ignore-accuracy-guarantees", + ]) +} + +/// One run can lower PromQL and SQL queries together: `--workload` +/// repeats, each file is lowered by its own language's frontend (SQL over +/// the `--sql-catalog-table` declarations), and their queries are one +/// workload. The document numbers the queries across files (q1 from the +/// PromQL file, q2 from the SQL file) and its data workload is the union +/// of the files' sources, with their arrival rates added (1000 + 500 +/// rows/s). +#[test] +fn several_workload_files_in_different_languages_are_lowered_together() { + let document = mixed_language_document("stage0"); + let queries = document["workload"]["queries"].as_array().unwrap(); + let ids_and_languages: Vec<(&str, &str)> = queries + .iter() + .map(|q| (q["id"].as_str().unwrap(), q["language"].as_str().unwrap())) + .collect(); + assert_eq!(ids_and_languages, [("q1", "promql"), ("q2", "sql")]); + let data = &document["workload"]["data_workload"]; + assert_eq!( + data["arrival"], + json!({"ingesting": {"rows_per_second": 1500.0}}) + ); + let sources: Vec<&String> = data["sources"].as_object().unwrap().keys().collect(); + assert_eq!(sources, ["flows", "http_requests_total"]); + assert_eq!( + document["stage0_query_language_frontends"]["dag"]["roots"] + .as_array() + .unwrap() + .len(), + 2 + ); +} + +/// PromQL and SQL queries from several workload files are planned together +/// to one selected plan. +#[test] +fn several_workload_files_in_different_languages_are_planned_together() { + let document = mixed_language_document("stage3"); + assert!( + selects_a_plan(&document["stage3_plan_selection"]), + "{}", + document["stage3_plan_selection"] + ); + selected_physical(&document, 2); +} + +/// Files planned together must agree on how the data arrives and on any +/// source they both declare: a run is refused rather than picking one. +/// While data arrives, each source must also declare its own rate: plan +/// selection shares a workload's `rows_per_second` out over all its +/// sources, so `flows` (10 groups, no sample interval) would be priced at +/// 1500 × 10 / 15010 ≈ 1 row/s instead of its file's 500. +#[test] +fn workload_files_that_disagree_on_the_data_are_refused() { + let refused_together = |first: &TempFile, second: &TempFile| { + refused(&[ + "--workload", + first.path(), + "--workload", + second.path(), + "--sql-catalog-table", + FLOWS_TABLE, + "--run-until", + "stage0", + ]) + }; + let (promql, sql) = mixed_language_workload_files( + json!({"ingesting": {"rows_per_second": 500.0}}), + json!({"series_count": 10}), + ); + let stderr = refused_together(&promql, &sql); + assert!( + stderr.contains("source flows declares no rate of its own"), + "{stderr}" + ); + + // A table the SQL file reads but does not declare has no rate either. + let (_, sql) = mixed_language_workload_files( + json!({"ingesting": {"rows_per_second": 500.0}}), + json!({"series_count": 10, "sample_interval": 1000}), + ); + let mut text: Value = serde_json::from_str(&std::fs::read_to_string(&sql.0).unwrap()).unwrap(); + text["data_workload"]["sources"] = json!({}); + let sql = TempFile::json(&text); + let stderr = refused_together(&promql, &sql); + assert!( + stderr.contains("source flows declares no rate of its own"), + "{stderr}" + ); + + let (promql, sql) = mixed_language_workload_files(json!("at_rest"), json!({"rows_held": 1000})); + let stderr = refused_together(&promql, &sql); + assert!(stderr.contains("arrival"), "{stderr}"); + + // The same source declared twice with different facts. + let mut text: Value = + serde_json::from_str(&std::fs::read_to_string(&promql.0).unwrap()).unwrap(); + text["data_workload"]["sources"]["http_requests_total"]["series_count"] = json!(7); + let other = TempFile::json(&text); + let stderr = refused_together(&promql, &other); + assert!(stderr.contains("http_requests_total"), "{stderr}"); +} + +/// Issue #890: a classic-bucket `histogram_quantile` with exact accuracy +/// enforced gets a selected plan the executor compiles, as its bucket rates +/// alone (the control) already did. +#[test] +fn a_classic_histogram_quantile_gets_a_selected_plan() { + let buckets = r#"sum by (le) (rate(http_request_duration_seconds_bucket{service="customer-orders-service"}[5m]))"#; + for query in [ + format!("histogram_quantile(0.5, {buckets})"), + buckets.to_string(), + ] { + let document = document( + &adhoc( + &[&query], + exact(), + &["http_request_duration_seconds_bucket"], + ), + &["--run-until", "stage3"], + ); + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{query}: {selection}"); + } +} + +/// The issue's example workload (#1003): one ratio of per-series quantiles, +/// every minute, ε = 0.01, δ = 0.001. +const MEMO_RATIO_OF_QUANTILES: &str = + "../../tools/dag-viewer/memo-examples/promql/ratio-of-quantiles-5m.json"; + +/// The `memo` section: the memo after each step and its node table, on the +/// issue's example with guarantees ignored (so Pass 1 keeps every sketch), +/// and with guarantees enforced the selected choice. +#[test] +fn memo_section_lists_each_step_and_the_selected_choice() { + let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(MEMO_RATIO_OF_QUANTILES); + // Up to Stage 2: Stage 3 over every sketch candidate takes minutes. + let document = document_of(&[ + "--workload", + path.to_str().unwrap(), + "--run-until", + "stage2", + "--ignore-accuracy-guarantees", + ]); + let memo = &document["memo"]; + let steps = memo["steps"].as_array().unwrap(); + let names: Vec<_> = steps.iter().map(|s| s["step"].as_str().unwrap()).collect(); + assert_eq!( + names, + ["stage0", "pass1", "pass2_local_window_composition", "pass2"] + ); + let candidates = |step: &Value| -> u64 { + step["groups"] + .as_array() + .unwrap() + .iter() + .map(|g| g["candidate_count"].as_u64().unwrap()) + .sum() + }; + // The PromQL frontend gives each series its identity as a column, so the + // two `quantile_over_time(0.5, data[5m])` have a unique key and share one + // group: 7 groups (scan, range, two quantile groups, ratio, sum, avg), + // 5 + 5 + 5 Pass 1 candidates (each quantile group: exact, DDSketch and + // a KLL sized for the whole result for the whole budget, DDSketch and a + // KLL for half of it). + assert_eq!(steps[0]["groups"].as_array().unwrap().len(), 7); + assert_eq!(candidates(&steps[0]), 0); + assert_eq!(steps[1]["groups"].as_array().unwrap().len(), 7); + assert_eq!(candidates(&steps[1]), 15); + // The node table is a valid FlatDag (children before parents) holding + // each node once; every group's and candidate's `node` names a node of + // it, and each boundary names a node and one of the candidate's child + // groups. + let nodes = memo["nodes"].as_array().unwrap(); + validated_dag(&json!({ "nodes": nodes, "roots": [] }), 0); + let distinct: HashSet = nodes.iter().map(Value::to_string).collect(); + assert_eq!(distinct.len(), nodes.len()); + let in_table = |id: &Value| (id.as_u64().unwrap() as usize) < nodes.len(); + for step in steps { + for group in step["groups"].as_array().unwrap() { + assert!(in_table(&group["node"]) && group.get("dag").is_none()); + for candidate in group["candidates"].as_array().unwrap() { + assert!(in_table(&candidate["node"]) && candidate.get("dag").is_none()); + let children = candidate["children"].as_array().unwrap(); + for (node, child) in candidate["group_of_node"].as_object().unwrap() { + assert!(node.parse::().unwrap() < nodes.len()); + assert!(children.contains(child)); + } + } + } + } + assert!(memo.get("selected_choice").is_none()); + // Every candidate of the selected choice exists in the last step. + let enforced = document_of(&[ + "--workload", + path.to_str().unwrap(), + "--run-until", + "stage3", + ]); + let memo = &enforced["memo"]; + let last = memo["steps"].as_array().unwrap().last().unwrap()["groups"] + .as_array() + .unwrap(); + let choice = memo["selected_choice"].as_object().unwrap(); + assert!(!choice.is_empty()); + for (group, index) in choice { + let group = &last[group.parse::().unwrap()]; + assert!(index.as_u64().unwrap() < group["candidate_count"].as_u64().unwrap()); + } +} + +/// The text output ends with Stage 3's selection and the selected plan's +/// cost per second. +#[test] +fn text_output_shows_the_selected_plans_cost() { + let output = planner(&["--workload", &example("example1"), "--run-until", "stage3"]); + assert!(output.status.success()); + let text = String::from_utf8(output.stdout).unwrap(); + assert!(text.contains("Stage 3 output: Selection"), "{text}"); + assert!(text.contains("\ncost_per_second: "), "{text}"); +} + +/// A memo example under `tools/dag-viewer/memo-examples`. +fn memo_example(path: &str) -> String { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../../tools/dag-viewer/memo-examples") + .join(path) + .to_str() + .unwrap() + .to_owned() +} + +/// The arguments for raw data retained, query-time results not retained, +/// AWS Fargate prices and enforced guarantees, with `capabilities` the +/// temporary deployment capabilities file they name. +fn milp_args(capabilities: &TempFile) -> Vec { + let aws = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("../../tools/dag-viewer/cost-inputs/aws-fargate-us-east-1.json"); + vec![ + "--run-until".into(), + "stage3".into(), + "--deployment-capabilities".into(), + capabilities.path().into(), + "--cost-inputs".into(), + aws.to_str().unwrap().into(), + ] +} + +fn raw_data_retained() -> TempFile { + TempFile::json(&json!({"raw_data_retained": true, "query_time_results_retained": false})) +} + +/// #1095 C5: example 1 and the RangeHeavy T5 q0.5 + q0.9 workload select a +/// plan whose `plan_cost` equals the MILP's objective (no +/// `inexact_objective`), solved to optimality, and which passes every +/// query's accuracy check. +#[test] +fn milp_search_objective_equals_plan_cost_on_example1_and_rangeheavy() { + let capabilities = raw_data_retained(); + let rangeheavy = memo_example("promql/rangeheavy-t5-q05-q09-1h.json"); + let example1 = example("example1"); + let milp = milp_args(&capabilities); + let runs: [Vec<&str>; 2] = [ + vec!["--workload", example1.as_str(), "--run-until", "stage3"], + std::iter::once("--workload") + .chain([rangeheavy.as_str()]) + .chain(milp.iter().map(String::as_str)) + .collect(), + ]; + for args in runs { + let document = document_of(&args); + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{args:?}: {selection}"); + let report = &selection["search_report"]; + assert_eq!(report["status"], "optimal", "{args:?}: {report}"); + assert!( + report.get("inexact_objective").is_none(), + "{args:?}: {report}" + ); + assert!(report["milp_variables"].as_u64().unwrap() > 0); + let queries = document["workload"]["queries"].as_array().unwrap().len(); + let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); + assert_eq!(accepted.len(), queries); + for query in accepted { + assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); + } + } +} + +/// #1127 regressions: with guarantees enforced, ratio-of-quantiles (KLL +/// rank errors compose no guarantee through the quotient) and +/// rollupheavy-t11 (HLL's failure probability over 10^6 distinct users is +/// above its contract's) select a plan that passes every query's accuracy +/// check and uses no summary of the family Pass 1 pruned. +#[test] +fn milp_search_selects_checked_plans_on_ratio_and_rollupheavy() { + let capabilities = raw_data_retained(); + let milp = milp_args(&capabilities); + let tables = catalog_tables(Path::new(&memo_example("sql/tables.json"))); + for (workload, pruned_family) in [ + ("promql/ratio-of-quantiles-5m.json", "Kll"), + ("sql/rollupheavy-t11-service-endpoint-5m.json", "Hll"), + ] { + let path = memo_example(workload); + let mut args = vec!["--workload", path.as_str()]; + args.extend(milp.iter().map(String::as_str)); + if workload.starts_with("sql/") { + for table in &tables { + args.extend(["--sql-catalog-table", table.as_str()]); + } + } + let document = document_of(&args); + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{workload}: {selection}"); + let queries = document["workload"]["queries"].as_array().unwrap().len(); + let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); + assert_eq!(accepted.len(), queries); + for query in accepted { + assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); + assert!( + !query["sketches"] + .as_array() + .unwrap() + .iter() + .any(|sketch| sketch == pruned_family), + "{workload}: {query}" + ); + } + let pruned = pass1_pruned(&document); + assert!( + pruned.iter().any(|(family, reason)| family == pruned_family + && reason == "NoErrorBoundForQueryTarget"), + "{workload}: {pruned:?}" + ); + } +} + +/// #1095 C5: on RangeHeavy T8+T5 (`q0.9 / q0.5` with both operands also +/// queried alone), Stage 3 selects a DDSketch plan that passes every +/// query's accuracy check (KLL fails the check there). +#[test] +#[ignore = "Stage 3 over RangeHeavy T8+T5 takes about 4.5 minutes in debug; run with --ignored"] +fn milp_search_selects_a_checked_ddsketch_plan_for_a_ratio_and_its_operands() { + let capabilities = raw_data_retained(); + let path = memo_example("promql/rangeheavy-t8-t5-1h.json"); + let mut args = vec!["--workload", path.as_str()]; + let milp = milp_args(&capabilities); + args.extend(milp.iter().map(String::as_str)); + let document = document_of(&args); + let selection = &document["stage3_plan_selection"]; + assert!(selects_a_plan(selection), "{selection}"); + for query in selection["accepted_accuracy_per_query"].as_array().unwrap() { + assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); + assert_eq!(query["sketches"], json!(["DDSketch"]), "{query}"); + } +} + +/// The viewer's cost-input files carry the prices of the presets they are +/// named after, so a change to `crates/cost`'s constants fails here instead +/// of leaving the viewer and the snapshots on the old numbers. +#[test] +fn cost_input_files_match_their_presets() { + use asap_cost::{CostInputs, CostModel, ResourceUnitCost}; + let dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tools/dag-viewer/cost-inputs"); + let presets = [ + ("usage-over-time.json", ResourceUnitCost::illustrative()), + ( + "aws-fargate-us-east-1.json", + ResourceUnitCost::aws_fargate_us_east_1(CostInputs::default().resource_speed), + ), + ]; + for (file, unit_cost) in presets { + let loaded: CostInputs = + serde_json::from_str(&std::fs::read_to_string(dir.join(file)).unwrap()).unwrap(); + let expected = CostInputs { + cost_model: CostModel::UsageOverTime(unit_cost), + ..CostInputs::default() + }; + // Through JSON, as the file was read: serde_json's float parsing + // may differ from the computed value in the last bit. + let expected: CostInputs = + serde_json::from_str(&serde_json::to_string(&expected).unwrap()).unwrap(); + assert_eq!(loaded, expected, "{file}"); + } +} diff --git a/crates/devtools/tests/asap_planner_acceptance.rs b/crates/devtools/tests/asap_planner_acceptance.rs new file mode 100644 index 000000000..7043d0767 --- /dev/null +++ b/crates/devtools/tests/asap_planner_acceptance.rs @@ -0,0 +1,239 @@ +//! #1161's acceptance tests for `asap_planner`: `--run-until` stops after +//! the stage it names, Pass 2 holds every legal tumbling-window width, Stage 3 selects +//! among the candidates Stage 2 lists, and the CLI refuses what it no +//! longer takes. + +use std::collections::BTreeSet; +use std::process::{Command, Output}; + +use serde_json::{json, Value}; + +/// One repeating PromQL `query` every `interval_ms`, within `max_error` +/// (failure probability 0.001 when approximate), over `metric`, a counter +/// with 1,000 series sampled every 15 s. +fn workload(query: &str, interval_ms: u64, max_error: f64, metric: &str) -> Value { + let failure_probability = if max_error == 0.0 { 0.0 } else { 0.001 }; + json!({ + "query_workload": {"language": "promql", "query_batch": null, "repeating_queries": [{ + "query": query, + "demand": {"fixed_interval": interval_ms}, + "requirements": {"accuracy": {"max_error": max_error, "failure_probability": failure_probability}}, + "predictability": {"predictable": {"known_at": null}}, + }]}, + "data_workload": { + "arrival": {"ingesting": {"rows_per_second": 1000.0 / 15.0}}, + "sources": {metric: {"series_count": 1000, "sample_interval": 15000, "metric_type": "counter"}}, + }, + }) +} + +/// A rate every 10 s: Stage 3 solves it in well under a second. +fn rate_workload() -> Value { + workload( + "sum by (job) (rate(http_requests_total[1m]))", + 10_000, + 0.0, + "http_requests_total", + ) +} + +/// `asap_planner` on `workload` with `args`. +fn planner(workload: &Value, args: &[&str]) -> Output { + static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); + let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); + let path = std::env::temp_dir().join(format!( + "asap_planner_acceptance_{}_{call}.json", + std::process::id() + )); + std::fs::write(&path, workload.to_string()).unwrap(); + let output = Command::new(env!("CARGO_BIN_EXE_asap_planner")) + .arg("--workload") + .arg(&path) + .args(args) + .output() + .unwrap(); + std::fs::remove_file(path).unwrap(); + output +} + +/// `asap_planner`'s standard output on success. +fn stdout(output: Output) -> String { + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8(output.stdout).unwrap() +} + +/// The text sections, in order, that each `--run-until` prints after the +/// ones before it. +const SECTIONS: [(&str, &[&str]); 5] = [ + ( + "stage0", + &[ + "Stage 0 output: the frontends' roots", + "Stage 0 → Stage 1: DagOfLogicalMemoGroups", + ], + ), + ( + "stage1_pass1", + &["Stage 1 Pass 1 output: PerComputationLogicalMemoCandidates"], + ), + ( + "stage1_pass2", + &[ + "Stage 1 Pass 2, after local window composition", + "Stage 1 Pass 2 output = Stage 2 input: LogicalASAPMemoCandidates", + ], + ), + ( + "stage2", + &["Stage 2 output = Stage 3 input: PhysicalASAPMemoCandidates"], + ), + ("stage3", &["Stage 3 output: Selection"]), +]; + +/// `--run-until ` prints exactly the sections of every stage up to +/// and including that one, in pipeline order. +#[test] +fn run_until_prints_exactly_the_stages_up_to_it() { + for (last, (stage, _)) in SECTIONS.iter().enumerate() { + let text = stdout(planner(&rate_workload(), &["--run-until", stage])); + let printed: Vec<&str> = text + .lines() + .filter_map(|line| line.strip_prefix("==================== ")) + .collect(); + let expected: Vec<&str> = SECTIONS[..=last] + .iter() + .flat_map(|(_, headings)| headings.iter().copied()) + .collect(); + assert_eq!( + printed.len(), + expected.len(), + "--run-until {stage}:\n{text}" + ); + for (line, heading) in printed.iter().zip(&expected) { + assert!(line.starts_with(heading), "--run-until {stage}: {line}"); + } + } +} + +/// Pass 2 composes tumbling-window candidates over every legal width, not only +/// the seed grid: a 5 m quantile within 1% every 60 s gets a mergeable +/// summary's tumbling-window chain for every width that divides 60 s (in ms), from +/// 1 ms to 60 s. +#[test] +fn pass2_holds_a_tumbling_window_candidate_for_every_width_dividing_the_grid() { + let quantile = workload("quantile_over_time(0.5, data[5m])", 60_000, 0.01, "data"); + let text = stdout(planner(&quantile, &["--run-until", "stage1_pass2"])); + let widths: BTreeSet = text + .split("tumbling_window_width_ms=") + .skip(1) + .map(|rest| { + let digits: String = rest.chars().take_while(char::is_ascii_digit).collect(); + digits.parse().unwrap() + }) + .collect(); + let divisors: BTreeSet = (1..=60_000).filter(|w| 60_000 % w == 0).collect(); + assert_eq!(widths, divisors); +} + +/// Stage 3 selects among the candidates Stage 2 lists: every candidate a +/// selected node runs (`nodeN: C`) is a line of Stage 2's output. +#[test] +fn stage3_selects_candidates_stage2_lists() { + let text = stdout(planner(&rate_workload(), &["--run-until", "stage3"])); + let (stage2, stage3) = text.split_once("Stage 3 output: Selection").unwrap(); + let selected: Vec<&str> = stage3 + .lines() + .filter(|line| line.starts_with("node") && line.contains(": C")) + .filter_map(|line| line.split_whitespace().nth(1)) + .collect(); + assert!(!selected.is_empty(), "{stage3}"); + for candidate in selected { + assert!( + stage2.contains(&format!("\n {candidate}:")), + "{candidate} is not in Stage 2's output" + ); + } +} + +/// `--format json` leaves the sections of stages it did not run null: after +/// `stage1_pass2`, Stage 1 and the memo up to Pass 2 are there and Stages 2 and 3 +/// are null; after `stage3`, every section is there. +#[test] +fn json_sections_of_stages_not_run_are_null() { + let document = |stage: &str| -> Value { + serde_json::from_str(&stdout(planner( + &rate_workload(), + &["--run-until", stage, "--format", "json"], + ))) + .unwrap() + }; + let pass2 = document("stage1_pass2"); + assert_eq!(pass2["run_until"], "stage1_pass2"); + assert!(pass2["stage0_query_language_frontends"].is_object()); + assert!(pass2["stage1_logical_asap_optimization"].is_object()); + assert_eq!(pass2["stage2_physical_asap_optimization"], Value::Null); + assert_eq!(pass2["stage3_plan_selection"], Value::Null); + let steps: Vec<&str> = pass2["memo"]["steps"] + .as_array() + .unwrap() + .iter() + .map(|step| step["step"].as_str().unwrap()) + .collect(); + assert_eq!( + steps, + ["stage0", "pass1", "pass2_local_window_composition", "pass2"] + ); + + let stage3 = document("stage3"); + assert!(stage3["stage2_physical_asap_optimization"].is_object()); + assert!(stage3["stage3_plan_selection"]["selected_dag"].is_object()); + assert_eq!( + stage3["stage1_logical_asap_optimization"]["candidates"][0]["id"], + "L1" + ); + assert_eq!( + stage3["stage2_physical_asap_optimization"]["candidates"][0]["id"], + "P1" + ); +} + +/// A missing `--run-until`, the old `pass1`/`pass2` stage names, and each +/// flag `stage_pipeline` took that `asap_planner` does not, is a usage error +/// (status 2, with the usage). +#[test] +fn missing_run_until_and_removed_flags_are_usage_errors() { + let refused = |args: &[&str], message: &str| { + let output = planner(&rate_workload(), args); + assert_eq!(output.status.code(), Some(2), "{args:?}"); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains(message), "{args:?}: {stderr}"); + assert!(stderr.contains("usage: asap_planner"), "{args:?}: {stderr}"); + }; + refused(&[], "--run-until is required"); + for stage in ["pass1", "pass2"] { + refused(&["--run-until", stage], &format!("unknown stage {stage}")); + } + for flag in [ + "--example", + "--promql", + "--sql", + "--table", + "--exhaustive", + "--milp-time-limit-s", + "--max-displayed", + "--no-memo", + "--max-enumerated", + "--enforce-accuracy-guarantees", + "--cost-model", + "--text", + ] { + refused( + &["--run-until", "stage0", flag], + &format!("unknown argument {flag}"), + ); + } +} diff --git a/crates/devtools/tests/corpus_snapshots.rs b/crates/devtools/tests/corpus_snapshots.rs index 3f347a364..40bf88751 100644 --- a/crates/devtools/tests/corpus_snapshots.rs +++ b/crates/devtools/tests/corpus_snapshots.rs @@ -10,13 +10,16 @@ //! that query's snapshot. Regenerate with //! `UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored`. //! +//! Each query is planned by `asap_planner --run-until stage3`: one exact +//! search over every legal tumbling-window width, with no time limit. +//! //! A snapshot holds only the plan, never the search that found it: the //! selected plan's DAG (Stage 2 candidate `P1`) and display name, its cost //! and accepted accuracy per query, the materialized views, tumbling-window retention, //! and whether the search proved the plan optimal; for a query that does not -//! plan, the error. Candidate ids, round and evaluation counters, timings, the -//! Stage 1 and Stage 2 candidate lists and the memo are left out. Floats are -//! rounded to 12 significant digits. A query with a `skip_reason` is skipped. +//! plan, the error. Candidate ids, timings, the Stage 1 and Stage 2 +//! candidate lists and the memo are left out. Floats are rounded to 12 +//! significant digits. A query with a `skip_reason` is skipped. //! //! The same binary holds the sketch plan snapshots (`sketch_plan_snapshots.rs`), //! which reuse these helpers on multi-query workloads with accuracy targets. @@ -51,23 +54,24 @@ fn workload(corpus: &Value, query: &str) -> Value { }) } -/// Runs `stage_pipeline` (without the memo section) on `workload`; panics -/// (a planner crash) on failure. +/// Runs `asap_planner` on `workload`; panics (a planner crash) on failure. fn plan(workload: &Value, args: &[String], scratch: &Path) -> Value { let input = scratch.join("workload.json"); std::fs::write(&input, workload.to_string()).unwrap(); let mut input_args = vec!["--workload".to_owned(), input.display().to_string()]; input_args.extend_from_slice(args); - stage_pipeline(&input_args, scratch) + asap_planner(&input_args, scratch) } -/// Runs `stage_pipeline` (without the memo section) with `args`, which name -/// its input; panics (a planner crash) on failure. -fn stage_pipeline(args: &[String], scratch: &Path) -> Value { +/// Runs `asap_planner --run-until stage3 --format json` with `args`, which +/// name its input (paths relative to the repository root); panics (a +/// planner crash) on failure. +fn asap_planner(args: &[String], scratch: &Path) -> Value { let output = scratch.join("document.json"); - let mut command = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")); + let mut command = Command::new(env!("CARGO_BIN_EXE_asap_planner")); command - .arg("--no-memo") + .current_dir(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..")) + .args(["--run-until", "stage3", "--format", "json"]) .args(args) .arg("--out") .arg(&output); @@ -105,7 +109,7 @@ fn rounded(value: &Value) -> Value { } } -/// The stable part of a one-query `stage_pipeline` document: the query, the +/// The stable part of a one-query `asap_planner` document: the query, the /// selected plan and its cost, or why there is none. See the module docs. fn snapshot_of(document: &Value) -> Value { let mut snapshot = plan_snapshot(document); @@ -218,7 +222,7 @@ fn corpus_plans_match_snapshots() { .unwrap_or_else(|poisoned| poisoned.into_inner()); let update = std::env::var_os("UPDATE_SNAPSHOTS").is_some(); // Regenerated snapshots of failing queries, with their full - // `stage_pipeline` documents for the DAG viewer, for CI to upload. + // `asap_planner` documents for the DAG viewer, for CI to upload. // Each corpus clears only its own directory: the sketch plan test writes // beside them. let regenerated = PathBuf::from(env!("CARGO_TARGET_TMPDIR")).join("corpus_snapshots"); @@ -238,7 +242,7 @@ fn corpus_plans_match_snapshots() { serde_json::from_str(&std::fs::read_to_string(dir.join("corpus.json")).unwrap()) .unwrap(); let args: Vec = - serde_json::from_value(corpus["stage_pipeline_args"].clone()).unwrap(); + serde_json::from_value(corpus["asap_planner_args"].clone()).unwrap(); // A query with a `skip_reason` is not planned. let queries: Vec<&Value> = corpus["queries"] .as_array() diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/corpus.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/corpus.json index 1b55ea4d7..753ba32a1 100644 --- a/crates/devtools/tests/corpus_snapshots/o11y_bench/corpus.json +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/corpus.json @@ -1,8 +1,6 @@ { "source": "crates/frontend-promql/tests/observability/data/o11y_bench_promql.txt", - "stage_pipeline_args": [ - "--enforce-accuracy-guarantees" - ], + "asap_planner_args": [], "query_workload": { "language": "promql", "repeating_query": { @@ -93,8 +91,7 @@ }, { "id": "q18", - "query": "sum(increase(http_requests_total{job=\"payment-service\",status=~\"5..\"}[1h] offset 6h)) / sum(increase(http_requests_total{job=\"payment-service\"}[1h] offset 6h))", - "skip_reason": "planning runs past the time limit on main (#1023)" + "query": "sum(increase(http_requests_total{job=\"payment-service\",status=~\"5..\"}[1h] offset 6h)) / sum(increase(http_requests_total{job=\"payment-service\"}[1h] offset 6h))" }, { "id": "q19", @@ -106,8 +103,7 @@ }, { "id": "q21", - "query": "sum(rate(http_requests_total{job=\"order-service\"}[5m] offset 1h))", - "skip_reason": "a pane round can end on its wall-clock budget (status time_limit), so the result depends on machine speed; about 70 s and 400 MB on one core" + "query": "sum(rate(http_requests_total{job=\"order-service\"}[5m] offset 1h))" }, { "id": "q22", diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json new file mode 100644 index 000000000..cc49899a7 --- /dev/null +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q18.json @@ -0,0 +1,8365 @@ +{ + "accuracy_guarantees": "enforced", + "materialized_views": [ + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21660000 + }, + { + "included": -21600000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21600000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 0, + "node_id": 3, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21720000 + }, + { + "included": -21660000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21660000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 1, + "node_id": 6, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21780000 + }, + { + "included": -21720000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21720000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 2, + "node_id": 9, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21840000 + }, + { + "included": -21780000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21780000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 3, + "node_id": 12, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21900000 + }, + { + "included": -21840000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21840000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 4, + "node_id": 15, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21960000 + }, + { + "included": -21900000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21900000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 5, + "node_id": 18, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22020000 + }, + { + "included": -21960000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21960000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 6, + "node_id": 21, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22080000 + }, + { + "included": -22020000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22020000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 7, + "node_id": 24, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22140000 + }, + { + "included": -22080000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22080000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 8, + "node_id": 27, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22200000 + }, + { + "included": -22140000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22140000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 9, + "node_id": 30, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22260000 + }, + { + "included": -22200000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22200000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 10, + "node_id": 33, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22320000 + }, + { + "included": -22260000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22260000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 11, + "node_id": 36, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22380000 + }, + { + "included": -22320000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22320000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 12, + "node_id": 39, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22440000 + }, + { + "included": -22380000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22380000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 13, + "node_id": 42, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22500000 + }, + { + "included": -22440000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22440000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 14, + "node_id": 45, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22560000 + }, + { + "included": -22500000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22500000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 15, + "node_id": 48, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22620000 + }, + { + "included": -22560000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22560000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 16, + "node_id": 51, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22680000 + }, + { + "included": -22620000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22620000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 17, + "node_id": 54, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22740000 + }, + { + "included": -22680000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22680000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 18, + "node_id": 57, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22800000 + }, + { + "included": -22740000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22740000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 19, + "node_id": 60, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22860000 + }, + { + "included": -22800000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22800000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 20, + "node_id": 63, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22920000 + }, + { + "included": -22860000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22860000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 21, + "node_id": 66, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22980000 + }, + { + "included": -22920000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22920000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 22, + "node_id": 69, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23040000 + }, + { + "included": -22980000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22980000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 23, + "node_id": 72, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23100000 + }, + { + "included": -23040000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23040000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 24, + "node_id": 75, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23160000 + }, + { + "included": -23100000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23100000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 25, + "node_id": 78, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23220000 + }, + { + "included": -23160000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23160000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 26, + "node_id": 81, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23280000 + }, + { + "included": -23220000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23220000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 27, + "node_id": 84, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23340000 + }, + { + "included": -23280000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23280000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 28, + "node_id": 87, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23400000 + }, + { + "included": -23340000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23340000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 29, + "node_id": 90, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23460000 + }, + { + "included": -23400000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23400000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 30, + "node_id": 93, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23520000 + }, + { + "included": -23460000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23460000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 31, + "node_id": 96, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23580000 + }, + { + "included": -23520000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23520000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 32, + "node_id": 99, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23640000 + }, + { + "included": -23580000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23580000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 33, + "node_id": 102, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23700000 + }, + { + "included": -23640000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23640000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 34, + "node_id": 105, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23760000 + }, + { + "included": -23700000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23700000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 35, + "node_id": 108, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23820000 + }, + { + "included": -23760000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23760000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 36, + "node_id": 111, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23880000 + }, + { + "included": -23820000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23820000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 37, + "node_id": 114, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23940000 + }, + { + "included": -23880000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23880000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 38, + "node_id": 117, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24000000 + }, + { + "included": -23940000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23940000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 39, + "node_id": 120, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24060000 + }, + { + "included": -24000000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24000000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 40, + "node_id": 123, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24120000 + }, + { + "included": -24060000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24060000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 41, + "node_id": 126, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24180000 + }, + { + "included": -24120000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24120000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 42, + "node_id": 129, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24240000 + }, + { + "included": -24180000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24180000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 43, + "node_id": 132, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24300000 + }, + { + "included": -24240000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24240000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 44, + "node_id": 135, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24360000 + }, + { + "included": -24300000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24300000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 45, + "node_id": 138, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24420000 + }, + { + "included": -24360000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24360000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 46, + "node_id": 141, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24480000 + }, + { + "included": -24420000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24420000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 47, + "node_id": 144, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24540000 + }, + { + "included": -24480000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24480000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 48, + "node_id": 147, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24600000 + }, + { + "included": -24540000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24540000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 49, + "node_id": 150, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24660000 + }, + { + "included": -24600000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24600000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 50, + "node_id": 153, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24720000 + }, + { + "included": -24660000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24660000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 51, + "node_id": 156, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24780000 + }, + { + "included": -24720000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24720000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 52, + "node_id": 159, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24840000 + }, + { + "included": -24780000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24780000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 53, + "node_id": 162, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24900000 + }, + { + "included": -24840000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24840000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 54, + "node_id": 165, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24960000 + }, + { + "included": -24900000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24900000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 55, + "node_id": 168, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25020000 + }, + { + "included": -24960000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24960000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 56, + "node_id": 171, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25080000 + }, + { + "included": -25020000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 25020000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 57, + "node_id": 174, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25140000 + }, + { + "included": -25080000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 25080000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 58, + "node_id": 177, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6fea1c63a2fdd67d8f43d0106be9d627", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25200000 + }, + { + "included": -25140000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 25140000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 59, + "node_id": 180, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21660000 + }, + { + "included": -21600000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21600000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 0, + "node_id": 188, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21720000 + }, + { + "included": -21660000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21660000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 1, + "node_id": 191, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21780000 + }, + { + "included": -21720000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21720000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 2, + "node_id": 194, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21840000 + }, + { + "included": -21780000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21780000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 3, + "node_id": 197, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21900000 + }, + { + "included": -21840000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21840000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 4, + "node_id": 200, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -21960000 + }, + { + "included": -21900000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21900000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 5, + "node_id": 203, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22020000 + }, + { + "included": -21960000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 21960000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 6, + "node_id": 206, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22080000 + }, + { + "included": -22020000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22020000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 7, + "node_id": 209, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22140000 + }, + { + "included": -22080000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22080000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 8, + "node_id": 212, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22200000 + }, + { + "included": -22140000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22140000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 9, + "node_id": 215, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22260000 + }, + { + "included": -22200000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22200000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 10, + "node_id": 218, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22320000 + }, + { + "included": -22260000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22260000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 11, + "node_id": 221, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22380000 + }, + { + "included": -22320000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22320000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 12, + "node_id": 224, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22440000 + }, + { + "included": -22380000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22380000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 13, + "node_id": 227, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22500000 + }, + { + "included": -22440000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22440000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 14, + "node_id": 230, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22560000 + }, + { + "included": -22500000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22500000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 15, + "node_id": 233, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22620000 + }, + { + "included": -22560000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22560000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 16, + "node_id": 236, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22680000 + }, + { + "included": -22620000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22620000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 17, + "node_id": 239, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22740000 + }, + { + "included": -22680000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22680000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 18, + "node_id": 242, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22800000 + }, + { + "included": -22740000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22740000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 19, + "node_id": 245, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22860000 + }, + { + "included": -22800000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22800000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 20, + "node_id": 248, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22920000 + }, + { + "included": -22860000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22860000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 21, + "node_id": 251, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -22980000 + }, + { + "included": -22920000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22920000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 22, + "node_id": 254, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23040000 + }, + { + "included": -22980000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 22980000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 23, + "node_id": 257, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23100000 + }, + { + "included": -23040000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23040000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 24, + "node_id": 260, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23160000 + }, + { + "included": -23100000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23100000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 25, + "node_id": 263, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23220000 + }, + { + "included": -23160000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23160000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 26, + "node_id": 266, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23280000 + }, + { + "included": -23220000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23220000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 27, + "node_id": 269, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23340000 + }, + { + "included": -23280000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23280000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 28, + "node_id": 272, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23400000 + }, + { + "included": -23340000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23340000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 29, + "node_id": 275, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23460000 + }, + { + "included": -23400000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23400000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 30, + "node_id": 278, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23520000 + }, + { + "included": -23460000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23460000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 31, + "node_id": 281, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23580000 + }, + { + "included": -23520000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23520000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 32, + "node_id": 284, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23640000 + }, + { + "included": -23580000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23580000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 33, + "node_id": 287, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23700000 + }, + { + "included": -23640000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23640000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 34, + "node_id": 290, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23760000 + }, + { + "included": -23700000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23700000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 35, + "node_id": 293, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23820000 + }, + { + "included": -23760000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23760000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 36, + "node_id": 296, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23880000 + }, + { + "included": -23820000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23820000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 37, + "node_id": 299, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -23940000 + }, + { + "included": -23880000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23880000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 38, + "node_id": 302, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24000000 + }, + { + "included": -23940000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 23940000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 39, + "node_id": 305, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24060000 + }, + { + "included": -24000000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24000000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 40, + "node_id": 308, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24120000 + }, + { + "included": -24060000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24060000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 41, + "node_id": 311, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24180000 + }, + { + "included": -24120000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24120000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 42, + "node_id": 314, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24240000 + }, + { + "included": -24180000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24180000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 43, + "node_id": 317, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24300000 + }, + { + "included": -24240000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24240000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 44, + "node_id": 320, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24360000 + }, + { + "included": -24300000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24300000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 45, + "node_id": 323, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24420000 + }, + { + "included": -24360000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24360000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 46, + "node_id": 326, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24480000 + }, + { + "included": -24420000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24420000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 47, + "node_id": 329, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24540000 + }, + { + "included": -24480000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24480000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 48, + "node_id": 332, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24600000 + }, + { + "included": -24540000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24540000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 49, + "node_id": 335, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24660000 + }, + { + "included": -24600000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24600000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 50, + "node_id": 338, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24720000 + }, + { + "included": -24660000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24660000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 51, + "node_id": 341, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24780000 + }, + { + "included": -24720000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24720000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 52, + "node_id": 344, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24840000 + }, + { + "included": -24780000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24780000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 53, + "node_id": 347, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24900000 + }, + { + "included": -24840000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24840000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 54, + "node_id": 350, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -24960000 + }, + { + "included": -24900000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24900000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 55, + "node_id": 353, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25020000 + }, + { + "included": -24960000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 24960000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 56, + "node_id": 356, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25080000 + }, + { + "included": -25020000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 25020000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 57, + "node_id": 359, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25140000 + }, + { + "included": -25080000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 25080000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 58, + "node_id": 362, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + }, + { + "coverage": { + "definition_id": "6e0a4719202a0469f55a3a8469d009cc", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "payment-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -25200000 + }, + { + "included": -25140000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 25140000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 60, + "installed_position": 59, + "node_id": 365, + "refresh": "at_ingestion", + "retention_ms": 28800000, + "stage2_node_id": 9 + } + ], + "proven_optimal": true, + "query": "sum(increase(http_requests_total{job=\"payment-service\",status=~\"5..\"}[1h] offset 6h)) / sum(increase(http_requests_total{job=\"payment-service\"}[1h] offset 6h))", + "selected": { + "accepted": [ + { + "accepted_metric": null, + "accepted_regardless_of_guarantee": false, + "guarantee_check_rejection": null, + "sketches": [], + "summary": "metric not set; accepted exact", + "target_metric": null + } + ], + "cost": { + "cost_per_second": 6.87727148353, + "cost_unit": "cost_per_second", + "cpu_core_seconds_per_second": 0.000143269716667, + "per_node": { + "0": { + "cost_per_second": 0.0074, + "cpu_core_seconds_per_second": 7.4e-6, + "detail": "scan 1000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "1": { + "cost_per_second": 0.0, + "cpu_core_seconds_per_second": 0.0, + "detail": "time shift: pass 1000 rows, free; x 1.0000/s", + "multiplier": 1, + "operator": "TimeShift", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "10": { + "cost_per_second": 0.061000882488, + "cpu_core_seconds_per_second": 0.000061, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "price_source": "analytical", + "timing": "query_time" + }, + "11": { + "cost_per_second": 0.00101666666667, + "cpu_core_seconds_per_second": 1.01666666667e-6, + "detail": "finalize 1000 accumulators; x 1.0167/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "price_source": "analytical", + "timing": "query_time" + }, + "12": { + "cost_per_second": 0.00101666754915, + "cpu_core_seconds_per_second": 1.01666666667e-6, + "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "multiplier": 1, + "operator": "SummaryAgg", + "price_source": "analytical", + "timing": "query_time" + }, + "13": { + "cost_per_second": 1.01666666667e-6, + "cpu_core_seconds_per_second": 1.01666666667e-9, + "detail": "finalize 1 accumulators; x 1.0167/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "price_source": "analytical", + "timing": "query_time" + }, + "14": { + "cost_per_second": 1.01679273638e-6, + "cpu_core_seconds_per_second": 1.01666666667e-9, + "detail": "operator: pass 1 rows; x 1.0167/s; holds 8 bytes for 0.1 ms", + "multiplier": 1, + "operator": "BinaryOp", + "price_source": "analytical", + "timing": "query_time" + }, + "2": { + "cost_per_second": 3.369, + "cpu_core_seconds_per_second": 2e-6, + "detail": "build exact Increase accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 26936000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "3": { + "cost_per_second": 0.061000882488, + "cpu_core_seconds_per_second": 0.000061, + "detail": "merge 60000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.1 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "price_source": "analytical", + "timing": "query_time" + }, + "4": { + "cost_per_second": 0.00101666666667, + "cpu_core_seconds_per_second": 1.01666666667e-6, + "detail": "finalize 1000 accumulators; x 1.0167/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "price_source": "analytical", + "timing": "query_time" + }, + "5": { + "cost_per_second": 0.00101666754915, + "cpu_core_seconds_per_second": 1.01666666667e-6, + "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.1 ms", + "multiplier": 1, + "operator": "SummaryAgg", + "price_source": "analytical", + "timing": "query_time" + }, + "6": { + "cost_per_second": 1.01666666667e-6, + "cpu_core_seconds_per_second": 1.01666666667e-9, + "detail": "finalize 1 accumulators; x 1.0167/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "price_source": "analytical", + "timing": "query_time" + }, + "7": { + "cost_per_second": 0.0058, + "cpu_core_seconds_per_second": 5.8e-6, + "detail": "scan 1000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "8": { + "cost_per_second": 0.0, + "cpu_core_seconds_per_second": 0.0, + "detail": "time shift: pass 1000 rows, free; x 1.0000/s", + "multiplier": 1, + "operator": "TimeShift", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "9": { + "cost_per_second": 3.369, + "cpu_core_seconds_per_second": 2e-6, + "detail": "build exact Increase accumulator in 60 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 26936000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "price_source": "analytical", + "timing": "ingestion_time" + } + }, + "source": "analytical-cost-v2 (illustrative statistics)", + "usage": { + "average_memory_bytes": 53872014.1349, + "cpu_ops_per_second": 132069.716667, + "disk_bytes": 0, + "latency_per_query": [ + { + "compaction_ms": 0.0, + "query_ms": 0.124003 + } + ], + "peak_cpu_cores": 2, + "peak_memory_bytes": 53984232, + "scanned_bytes_per_second": 112000.0 + } + }, + "dag": { + "edges": [ + { + "consumer": 1, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "producer": 0, + "role": "Input", + "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + }, + { + "consumer": 2, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "producer": 1, + "role": "Input", + "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + }, + { + "consumer": 3, + "data_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "producer": 2, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 4, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "producer": 3, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 5, + "data_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "producer": 4, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 6, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "name": "state", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "producer": 5, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 8, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 7, + "role": "Input", + "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + }, + { + "consumer": 9, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 8, + "role": "Input", + "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + }, + { + "consumer": 10, + "data_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 9, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 11, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 10, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 12, + "data_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 11, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 13, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "name": "state", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "producer": 12, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 14, + "data_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "float64" + }, + "name": "sum", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "producer": 6, + "role": "Left", + "window": "NotApplicable" + }, + { + "consumer": 14, + "data_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "float64" + }, + "name": "sum", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "producer": 13, + "role": "Right", + "window": "NotApplicable" + } + ], + "nodes": [ + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 0, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "payload": { + "NonASAP": { + "Scan": { + "predicates": [ + { + "Compare": { + "left": { + "Column": 2 + }, + "op": "Eq", + "right": { + "Literal": { + "Utf8": "payment-service" + } + }, + "semantics": "promql" + } + }, + { + "Compare": { + "left": { + "Column": 3 + }, + "op": "Regex", + "right": { + "Literal": { + "Utf8": "5.." + } + }, + "semantics": "promql" + } + } + ], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 1, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "payload": { + "NonASAP": { + "TimeShift": { + "child": 0, + "shift": { + "at": null, + "offset_ms": 21600000 + } + } + } + } + }, + { + "guarantee": null, + "id": 2, + "materialization": { + "MatAtIngestion": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "payload": { + "ASAP": { + "SummaryTumblingWindowChain": { + "child": 1, + "family": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "filter": null, + "grouping": "PerSubpopulationInstance", + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + }, + "weight_domain": { + "kind": "unknown_or_signed" + } + }, + "reduction": "PerEntity", + "tumbling_window_count": 60, + "tumbling_window_width_ms": 60000 + } + } + } + }, + { + "guarantee": null, + "id": 3, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "SummaryMerge": { + "children": [ + 2 + ] + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact accumulator over exact input" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 4, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "status", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 4, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "FinalizeExactAccumulator": { + "child": 3 + } + } + } + }, + { + "guarantee": null, + "id": 5, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "name": "state", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "SummaryAgg": { + "child": 4, + "family": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "filter": null, + "grouping": "PerSubpopulationInstance", + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + }, + "weight_domain": { + "kind": "unknown_or_signed" + } + }, + "reduction": { + "Reduce": [] + } + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact accumulator over exact input" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 6, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "float64" + }, + "name": "sum", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "FinalizeExactAccumulator": { + "child": 5 + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 7, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "payload": { + "NonASAP": { + "Scan": { + "predicates": [ + { + "Compare": { + "left": { + "Column": 2 + }, + "op": "Eq", + "right": { + "Literal": { + "Utf8": "payment-service" + } + }, + "semantics": "promql" + } + } + ], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 8, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "payload": { + "NonASAP": { + "TimeShift": { + "child": 7, + "shift": { + "at": null, + "offset_ms": 21600000 + } + } + } + } + }, + { + "guarantee": null, + "id": 9, + "materialization": { + "MatAtIngestion": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "payload": { + "ASAP": { + "SummaryTumblingWindowChain": { + "child": 8, + "family": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "filter": null, + "grouping": "PerSubpopulationInstance", + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + }, + "weight_domain": { + "kind": "unknown_or_signed" + } + }, + "reduction": "PerEntity", + "tumbling_window_count": 60, + "tumbling_window_width_ms": 60000 + } + } + } + }, + { + "guarantee": null, + "id": 10, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Increase", + "Increase" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "SummaryMerge": { + "children": [ + 9 + ] + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact accumulator over exact input" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 11, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "FinalizeExactAccumulator": { + "child": 10 + } + } + } + }, + { + "guarantee": null, + "id": 12, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "name": "state", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "SummaryAgg": { + "child": 11, + "family": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "filter": null, + "grouping": "PerSubpopulationInstance", + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + }, + "weight_domain": { + "kind": "unknown_or_signed" + } + }, + "reduction": { + "Reduce": [] + } + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact accumulator over exact input" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 13, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "float64" + }, + "name": "sum", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "FinalizeExactAccumulator": { + "child": 12 + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 14, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "float64" + }, + "name": "sum", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "NonASAP": { + "BinaryOp": { + "lhs": 6, + "operator": { + "checked_finite_division": false, + "checked_relative_division": false, + "kind": { + "Arithmetic": "Div" + }, + "vector_match": null + }, + "return_bool": false, + "rhs": 13 + } + } + } + } + ], + "roots": [ + 14 + ], + "scalar_roots": [] + }, + "display_name": "Q1 Sum acc + Increase acc ×60 tumbling windows · ingestion time: Increase acc for Q1, 60 × 1m tumbling windows, Increase acc for Q1, 60 × 1m tumbling windows" + }, + "tumbling_window_retention_ms": { + "2": 28800000, + "9": 28800000 + } +} diff --git a/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json b/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json new file mode 100644 index 000000000..8546dd2b2 --- /dev/null +++ b/crates/devtools/tests/corpus_snapshots/o11y_bench/q21.json @@ -0,0 +1,1239 @@ +{ + "accuracy_guarantees": "enforced", + "materialized_views": [ + { + "coverage": { + "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -3660000 + }, + { + "included": -3600000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 3600000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 5, + "installed_position": 0, + "node_id": 3, + "refresh": "at_ingestion", + "retention_ms": 7500000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -3720000 + }, + { + "included": -3660000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 3660000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 5, + "installed_position": 1, + "node_id": 6, + "refresh": "at_ingestion", + "retention_ms": 7500000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -3780000 + }, + { + "included": -3720000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 3720000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 5, + "installed_position": 2, + "node_id": 9, + "refresh": "at_ingestion", + "retention_ms": 7500000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -3840000 + }, + { + "included": -3780000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 3780000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 5, + "installed_position": 3, + "node_id": 12, + "refresh": "at_ingestion", + "retention_ms": 7500000, + "stage2_node_id": 2 + }, + { + "coverage": { + "definition_id": "b167c0953f55d24c1c15591dddb42cb4", + "selection": [ + { + "columns": [ + { + "column": { + "name": "job", + "table": null + }, + "constraint": { + "in": [ + { + "Utf8": "order-service" + } + ] + } + } + ], + "relative_time": [ + { + "excluded": -3900000 + }, + { + "included": -3840000 + } + ] + } + ] + }, + "inputs": { + "materialized_views": [], + "raw_data_windows": [ + { + "end_offset_ms": 3840000, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + }, + "width_ms": 60000 + } + ] + }, + "installed_count": 5, + "installed_position": 4, + "node_id": 15, + "refresh": "at_ingestion", + "retention_ms": 7500000, + "stage2_node_id": 2 + } + ], + "proven_optimal": true, + "query": "sum(rate(http_requests_total{job=\"order-service\"}[5m] offset 1h))", + "selected": { + "accepted": [ + { + "accepted_metric": null, + "accepted_regardless_of_guarantee": false, + "guarantee_check_rejection": null, + "sketches": [], + "summary": "metric not set; accepted exact", + "target_metric": null + } + ], + "cost": { + "cost_per_second": 0.896917733207, + "cost_unit": "cost_per_second", + "cpu_core_seconds_per_second": 0.0000149176833333, + "per_node": { + "0": { + "cost_per_second": 0.0058, + "cpu_core_seconds_per_second": 5.8e-6, + "detail": "scan 1000 samples; x 1.0000/s", + "multiplier": 1, + "operator": "Scan", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "1": { + "cost_per_second": 0.0, + "cpu_core_seconds_per_second": 0.0, + "detail": "time shift: pass 1000 rows, free; x 1.0000/s", + "multiplier": 1, + "operator": "TimeShift", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "2": { + "cost_per_second": 0.884, + "cpu_core_seconds_per_second": 2e-6, + "detail": "build exact Rate accumulator in 5 tumbling windows of 60000 ms for 1000 groups: 1000 rows x (1 + depth 1); x 1.0000/s; keeps 7056000 bytes", + "multiplier": 1, + "operator": "SummaryTumblingWindowChain", + "price_source": "analytical", + "timing": "ingestion_time" + }, + "3": { + "cost_per_second": 0.00508338315712, + "cpu_core_seconds_per_second": 5.08333333333e-6, + "detail": "merge 5000 states into 1000; x 1.0167/s; holds 56000 bytes for 0.0 ms", + "multiplier": 1, + "operator": "SummaryMerge", + "price_source": "analytical", + "timing": "query_time" + }, + "4": { + "cost_per_second": 0.00101666666667, + "cpu_core_seconds_per_second": 1.01666666667e-6, + "detail": "finalize 1000 accumulators; x 1.0167/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "price_source": "analytical", + "timing": "query_time" + }, + "5": { + "cost_per_second": 0.00101666671649, + "cpu_core_seconds_per_second": 1.01666666667e-6, + "detail": "build exact Sum accumulator into 1 states: 1000 rows x depth 1; x 1.0167/s; holds 56 bytes for 0.0 ms", + "multiplier": 1, + "operator": "SummaryAgg", + "price_source": "analytical", + "timing": "query_time" + }, + "6": { + "cost_per_second": 1.01666666667e-6, + "cpu_core_seconds_per_second": 1.01666666667e-9, + "detail": "finalize 1 accumulators; x 1.0167/s", + "multiplier": 1, + "operator": "FinalizeExactAccumulator", + "price_source": "analytical", + "timing": "query_time" + } + }, + "source": "analytical-cost-v2 (illustrative statistics)", + "usage": { + "average_memory_bytes": 7056000.39899, + "cpu_ops_per_second": 10117.6833333, + "disk_bytes": 0, + "latency_per_query": [ + { + "compaction_ms": 0.0, + "query_ms": 0.007001 + } + ], + "peak_cpu_cores": 2, + "peak_memory_bytes": 7112104, + "scanned_bytes_per_second": 48000.0 + } + }, + "dag": { + "edges": [ + { + "consumer": 1, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 0, + "role": "Input", + "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + }, + { + "consumer": 2, + "data_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 1, + "role": "Input", + "window": "RequiresAlignedPanePhaseOrExactBoundaryResidual" + }, + { + "consumer": 3, + "data_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 2, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 4, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 3, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 5, + "data_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "producer": 4, + "role": "Input", + "window": "NotApplicable" + }, + { + "consumer": 6, + "data_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "grouping": "NotApplicable", + "intermediate_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "name": "state", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "producer": 5, + "role": "Input", + "window": "NotApplicable" + } + ], + "nodes": [ + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 0, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "payload": { + "NonASAP": { + "Scan": { + "predicates": [ + { + "Compare": { + "left": { + "Column": 2 + }, + "op": "Eq", + "right": { + "Literal": { + "Utf8": "order-service" + } + }, + "semantics": "promql" + } + } + ], + "schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "source": { + "TimeSeries": { + "metric": "http_requests_total" + } + } + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact over exact inputs" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 1, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "ingestion_time" + }, + "payload": { + "NonASAP": { + "TimeShift": { + "child": 0, + "shift": { + "at": null, + "offset_ms": 3600000 + } + } + } + } + }, + { + "guarantee": null, + "id": 2, + "materialization": { + "MatAtIngestion": "Memory" + }, + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "ingestion_time" + }, + "payload": { + "ASAP": { + "SummaryTumblingWindowChain": { + "child": 1, + "family": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "filter": null, + "grouping": "PerSubpopulationInstance", + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + }, + "weight_domain": { + "kind": "unknown_or_signed" + } + }, + "reduction": "PerEntity", + "tumbling_window_count": 5, + "tumbling_window_width_ms": 60000 + } + } + } + }, + { + "guarantee": null, + "id": 3, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "ExactAggregate": [ + "Rate", + "Rate" + ] + }, + "name": "state", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "SummaryMerge": { + "children": [ + 2 + ] + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact accumulator over exact input" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 4, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "timestamp" + }, + "name": "ts", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "float64" + }, + "name": "value", + "nullable": false, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "job", + "nullable": true, + "table": null + }, + { + "dtype": { + "Plain": "utf8" + }, + "name": "$promql_series_identity", + "nullable": false, + "table": null + } + ], + "time_index": 0, + "unique_keys": [ + [ + 3, + 0 + ] + ] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "FinalizeExactAccumulator": { + "child": 3 + } + } + } + }, + { + "guarantee": null, + "id": 5, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "name": "state", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "SummaryState", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "SummaryAgg": { + "child": 4, + "family": { + "ExactAggregate": [ + "Sum", + "Sum" + ] + }, + "filter": null, + "grouping": "PerSubpopulationInstance", + "input": { + "item": null, + "weight": { + "Column": "SampleValue" + }, + "weight_domain": { + "kind": "unknown_or_signed" + } + }, + "reduction": { + "Reduce": [] + } + } + } + } + }, + { + "guarantee": { + "provenance": [ + { + "kind": "exact", + "reason": "exact accumulator over exact input" + } + ], + "spec": { + "accuracy_metric": "absolute", + "accuracy_scope": "per_value", + "error": { + "op": "zero" + }, + "failure_probability": { + "op": "zero" + } + } + }, + "id": 6, + "materialization": "NotMaterialized", + "output_schema": { + "closed": true, + "fields": [ + { + "dtype": { + "Plain": "float64" + }, + "name": "sum", + "nullable": false, + "table": null + } + ], + "time_index": null, + "unique_keys": [] + }, + "output_state": { + "primitive": "Raw", + "timing": "query_time" + }, + "payload": { + "ASAP": { + "FinalizeExactAccumulator": { + "child": 5 + } + } + } + } + ], + "roots": [ + 6 + ], + "scalar_roots": [] + }, + "display_name": "Q1 Sum acc + Rate acc ×5 tumbling windows · ingestion time: Rate acc for Q1, 5 × 1m tumbling windows" + }, + "tumbling_window_retention_ms": { + "2": 7500000 + } +} diff --git a/crates/devtools/tests/main.rs b/crates/devtools/tests/main.rs index 5fcc40e19..4952ae75e 100644 --- a/crates/devtools/tests/main.rs +++ b/crates/devtools/tests/main.rs @@ -2,8 +2,9 @@ //! its own copy of every dependency, which outgrew CI's disk. //! One file: `cargo test -p asap-devtools --test devtools ::`. +mod asap_planner; +mod asap_planner_acceptance; mod cross_language; mod memo_pricing; mod planner_fuzz; -mod stage_pipeline; mod viewer_contract; diff --git a/crates/devtools/tests/memo_pricing.rs b/crates/devtools/tests/memo_pricing.rs index 68b85028d..1bdc9e850 100644 --- a/crates/devtools/tests/memo_pricing.rs +++ b/crates/devtools/tests/memo_pricing.rs @@ -4,7 +4,7 @@ use asap_cost::CostInputs; use asap_devtools::memo_pricing::{price_choices, PricedChoice, SelectedBaseline}; use asap_devtools::planner_layering_examples::example1; -use asap_devtools::planning_stages::{lower_promql_roots, run_default_search_over}; +use asap_devtools::planning_stages::{lower_promql_roots, run_stages_until, RunUntil}; use asap_logical_asap_optimization::enumerate; use asap_logical_asap_optimization::generate_dag_of_logical_memo_groups; use asap_logical_asap_optimization::test_support::generate_per_computation_candidates; @@ -19,8 +19,6 @@ use asap_types::workload::{ QueryRequirements, QueryWorkload, QueryWorkloadEntry, RepeatedDemand, RepeatingEntry, RepetitionInterval, }; -use serde_json::{json, Value}; -use std::process::Command; /// The issue's workload file: the query every minute, ε = 0.01, δ = 0.001. const RATIO_OF_QUANTILES_WORKLOAD: &str = @@ -65,7 +63,7 @@ struct Planned { memo: Memo, } -/// `stage_pipeline` lowers each series' identity as a column +/// `asap_planner` lowers each series' identity as a column /// (`lower_promql_roots`); without it (`series_identity` false, as the /// memo-enumeration tests lower), the ratio's two `quantile 0.5` have no /// provable unique key, but they are still one group. @@ -91,7 +89,7 @@ fn planned(workload: &PlanningWorkload, series_identity: bool) -> Planned { } } -/// `stage_pipeline`'s models: the executor's summary families, +/// `asap_planner`'s models: the executor's summary families, /// `settings` (accuracy guarantees ignored unless enforced). fn models<'a>( settings: &'a Settings, @@ -110,7 +108,7 @@ fn deployment() -> DeploymentCapabilities { } } -/// `stage_pipeline`'s cost inputs, and whether it ignores accuracy +/// `asap_planner`'s cost inputs, and whether it ignores accuracy /// guarantees. struct Settings { cost_inputs: CostInputs, @@ -124,14 +122,14 @@ fn settings(enforce_accuracy_guarantees: bool) -> Settings { } } -/// The default search's selection of `planned`'s workload, as -/// `stage_pipeline` runs it. +/// `asap_planner`'s selection of `planned`'s workload. fn selected(planned: &Planned, models: PlanningModels<'_>) -> SelectedBaseline { - let run = run_default_search_over( + let run = run_stages_until( planned.roots.clone(), &planned.entries, &planned.data, models, + RunUntil::Stage3, ) .unwrap(); SelectedBaseline::of(run.run.selection.as_ref().expect("a selection")) @@ -157,94 +155,82 @@ fn price_every_choice( ) } -/// `stage_pipeline --price-enumerated 200` on `args`' workload: the listed -/// choices of the enumeration's step, each priced. -fn priced_document(args: &[&str]) -> Value { - static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); - let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_{call}_priced.json", - std::process::id() - )); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(args) - .args(["--price-enumerated", "200", "--out"]) - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - document -} - -/// Every listed choice is priced; the enumeration of its step is complete, -/// and no valid choice is cheaper than the selected plan, whose cost is -/// the one Stage 3 reports. Returns the choices. -fn assert_no_listed_choice_is_cheaper(document: &Value) -> Vec { - let enumeration = &document["memo"]["enumeration"]; - let step = enumeration["step"].as_str().unwrap(); - assert_eq!(enumeration["per_step"][step]["complete"], true); - let choices = enumeration["choices"].as_array().unwrap().clone(); - assert_eq!( - enumeration["per_step"][step]["count"].as_u64().unwrap(), - choices.len() as u64 - ); - let stage3 = &document["stage3_plan_selection"]; - let selected = &stage3["cost"]["cost_per_second"]; - assert_eq!( - enumeration["priced"], - json!({"count": choices.len(), "cheaper_than_selected": 0, "selected_cost_per_second": selected}) +/// Every choice of `workload`'s Pass 1 memo, as `asap_planner` builds it +/// (the enumeration is complete), priced against the plan `asap_planner` +/// selects: none is cheaper, every valid one costs at least as much, and +/// the others say why they were rejected. Returns the priced choices. +fn assert_no_pass1_choice_is_cheaper( + workload: &PlanningWorkload, + enforce_accuracy_guarantees: bool, +) -> Vec { + let planned = planned(workload, true); + let (settings, deployment) = (settings(enforce_accuracy_guarantees), deployment()); + let models = models(&settings, &deployment); + let run = run_stages_until( + planned.roots.clone(), + &planned.entries, + &planned.data, + models, + RunUntil::Stage3, + ) + .unwrap(); + let selected = SelectedBaseline::of(run.run.selection.as_ref().expect("a selection")); + let memo = &run.run.per_computation_candidates.as_ref().unwrap().memo; + let enumeration = enumerate(memo, 100_000); + assert!(enumeration.complete); + let priced = price_choices( + memo, + &enumeration.all_chosen_candidates, + &planned.roots, + &planned.entries, + &planned.data, + models, + Some(selected), ); - for choice in &choices { - assert_eq!(choice["cheaper_than_selected"], false, "{}", choice["id"]); - if choice["valid"] == true { - assert!(choice["cost_per_second"].as_f64().unwrap() >= selected.as_f64().unwrap()); - assert_eq!(choice["rejection"], Value::Null); + let selected = selected.cost_per_second.unwrap(); + for choice in &priced { + assert!(!choice.cheaper_than_selected, "{choice:?}"); + if choice.valid { + assert!(choice.cost_per_second.unwrap() >= selected, "{choice:?}"); + assert_eq!(choice.rejection, None); } else { - assert!(choice["rejection"].is_string(), "{}", choice["id"]); + assert!(choice.rejection.is_some(), "{choice:?}"); } } - assert!(choices.iter().any(|c| c["valid"] == true)); - choices + assert!(priced.iter().any(|choice| choice.valid)); + priced } -/// #509 Example 1: its Pass 1 memo (the listed step) stands for 3 DAGs +/// #509 Example 1: its Pass 1 memo stands for 3 DAGs /// (Q2's top-k group: exact, Count-Min with a heap and Count Sketch with a /// heap); /// Q2's summary reads miss its 100 ms latency bound, and no valid DAG is /// cheaper than the selection. #[test] -fn example1_no_listed_choice_is_cheaper_than_the_selected_plan() { - let document = priced_document(&["--example", "planner-layering-1"]); - let choices = assert_no_listed_choice_is_cheaper(&document); - assert_eq!(document["memo"]["enumeration"]["step"], "pass1"); +fn example1_no_pass1_choice_is_cheaper_than_the_selected_plan() { + let choices = assert_no_pass1_choice_is_cheaper(&example1(), false); assert_eq!(choices.len(), 3); assert!(choices .iter() - .any(|c| c["rejection"] == "latency_bound_exceeded")); + .any(|c| c.rejection.as_deref() == Some("latency_bound_exceeded"))); } -/// The issue's ratio of quantiles as `stage_pipeline` lowers it (series -/// identity: the two `quantile 0.5` share one group): no listed DAG is -/// cheaper than the selection, with accuracy guarantees ignored (the -/// default) or enforced. Enforced, Pass 1 prunes every summary: no query +/// The issue's ratio of quantiles as `asap_planner` lowers it (series +/// identity: the two `quantile 0.5` share one group): no Pass 1 DAG is +/// cheaper than the selection, with accuracy guarantees ignored or +/// enforced (the default). Enforced, Pass 1 prunes every summary: no query /// has a guarantee through `/` and `+` over a KLL rank error or a DDSketch /// relative one, so every listed DAG is exact and valid. #[test] -fn ratio_of_quantiles_no_listed_choice_is_cheaper_than_the_selected_plan() { - let ignored = assert_no_listed_choice_is_cheaper(&priced_document(&[ - "--workload", - RATIO_OF_QUANTILES_WORKLOAD, - ])); +fn ratio_of_quantiles_no_pass1_choice_is_cheaper_than_the_selected_plan() { + let workload: PlanningWorkload = + serde_json::from_str(&std::fs::read_to_string(RATIO_OF_QUANTILES_WORKLOAD).unwrap()) + .unwrap(); + let ignored = assert_no_pass1_choice_is_cheaper(&workload, false); assert!(ignored.len() >= 25); - let enforced = assert_no_listed_choice_is_cheaper(&priced_document(&[ - "--workload", - RATIO_OF_QUANTILES_WORKLOAD, - "--enforce-accuracy-guarantees", - ])); + let enforced = assert_no_pass1_choice_is_cheaper(&workload, true); assert!(!enforced.is_empty()); - assert!(enforced.iter().all(|choice| choice["valid"] == true)); + assert!(enforced.iter().all(|choice| choice.valid)); } /// Lowered without series identity, the two `quantile 0.5` have no unique diff --git a/crates/devtools/tests/sketch_plan_snapshots.rs b/crates/devtools/tests/sketch_plan_snapshots.rs index 1196f1e1f..e3c9d1be5 100644 --- a/crates/devtools/tests/sketch_plan_snapshots.rs +++ b/crates/devtools/tests/sketch_plan_snapshots.rs @@ -10,24 +10,26 @@ //! diff. //! //! `tests/sketch_plan_snapshots/manifest.json` lists each workload by id: -//! its `workload` file (or built-in `example`) relative to the repository -//! root, SQL `tables`, `deployment`, and whether accuracy guarantees are -//! `ignored` (enforced otherwise). `.json` beside it is its snapshot, +//! its `workload` file relative to the repository root, SQL `tables`, +//! `deployment` (the `asap_planner` arguments of its deployment +//! capabilities and cost inputs, files in `deployments/`), and whether +//! accuracy guarantees are `ignored` (enforced otherwise). `.json` beside it is its snapshot, //! the corpus snapshot of the selected plan plus every query's text and the //! search status. Workloads copied from elsewhere live in `workloads/`. //! Regenerate with //! `UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored sketch_plans_match_snapshots`. //! -//! Planning has no time limit, so every workload must end with status -//! `optimal`; any other ending fails instead of being recorded. +//! Each workload is planned by `asap_planner --run-until stage3`. Planning +//! has no time limit, so every workload must end with status `optimal`; any +//! other ending fails instead of being recorded. use std::path::PathBuf; use serde_json::{json, Value}; use super::{ - first_difference, first_differing_path, plan_snapshot, proven_optimal, rounded, search_status, - stage_pipeline, + asap_planner, first_difference, first_differing_path, plan_snapshot, proven_optimal, rounded, + search_status, }; fn repository_root() -> PathBuf { @@ -38,23 +40,19 @@ fn snapshots_dir() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/sketch_plan_snapshots") } -/// The `stage_pipeline` arguments for `entry` under `manifest`'s deployments. +/// The `asap_planner` arguments for `entry` under `manifest`'s deployments. fn arguments(manifest: &Value, entry: &Value) -> Vec { let path = |key: &str| repository_root().join(entry[key].as_str().unwrap()); let mut args = Vec::new(); - match (entry["workload"].as_str(), entry["example"].as_str()) { - (Some(_), None) => args.extend([ - "--workload".to_owned(), - path("workload").display().to_string(), - ]), - (None, Some(example)) => args.extend(["--example".to_owned(), example.to_owned()]), - _ => panic!("{entry}: give exactly one of `workload` and `example`"), - } + args.extend([ + "--workload".to_owned(), + path("workload").display().to_string(), + ]); if entry.get("tables").is_some() { let tables: Vec = serde_json::from_str(&std::fs::read_to_string(path("tables")).unwrap()).unwrap(); for table in tables { - args.extend(["--table".to_owned(), table.to_string()]); + args.extend(["--sql-catalog-table".to_owned(), table.to_string()]); } } let deployment = entry["deployment"].as_str().unwrap(); @@ -63,8 +61,8 @@ fn arguments(manifest: &Value, entry: &Value) -> Vec { .unwrap_or_else(|_| panic!("{entry}: unknown deployment {deployment}")); args.extend(deployment); match entry["accuracy_guarantees"].as_str() { - None | Some("enforced") => args.push("--enforce-accuracy-guarantees".to_owned()), - Some("ignored") => {} + None | Some("enforced") => {} + Some("ignored") => args.push("--ignore-accuracy-guarantees".to_owned()), Some(other) => { panic!("{entry}: accuracy_guarantees must be enforced or ignored, not {other}") } @@ -72,7 +70,7 @@ fn arguments(manifest: &Value, entry: &Value) -> Vec { args } -/// The stable part of a `stage_pipeline` document: every query's text, the +/// The stable part of an `asap_planner` document: every query's text, the /// selected plan (see [`plan_snapshot`]) and how its search ended. fn snapshot_of(document: &Value) -> Value { let mut snapshot = plan_snapshot(document); @@ -125,7 +123,7 @@ fn sketch_plans_match_snapshots() { for entry in entries { let id = entry["id"].as_str().unwrap(); - let document = stage_pipeline(&arguments(&manifest, entry), &scratch); + let document = asap_planner(&arguments(&manifest, entry), &scratch); if !proven_optimal(&document) { failures.push(format!( "{id}: the plan is not proven optimal (status {})", @@ -190,9 +188,12 @@ fn manifest_entries_build_their_arguments() { } } let args = arguments(&manifest, entry); + for path in args.iter().skip(2).filter(|arg| arg.ends_with(".json")) { + assert!(repository_root().join(path).is_file(), "{entry}: no {path}"); + } assert_eq!( - args.iter().any(|a| a == "--enforce-accuracy-guarantees"), - entry["accuracy_guarantees"] != json!("ignored") + args.iter().any(|a| a == "--ignore-accuracy-guarantees"), + entry["accuracy_guarantees"] == json!("ignored") ); } } diff --git a/crates/devtools/tests/sketch_plan_snapshots/deployments/mvp.capabilities.json b/crates/devtools/tests/sketch_plan_snapshots/deployments/mvp.capabilities.json new file mode 100644 index 000000000..f5c9e4fe8 --- /dev/null +++ b/crates/devtools/tests/sketch_plan_snapshots/deployments/mvp.capabilities.json @@ -0,0 +1 @@ +{ "raw_data_retained": false, "query_time_results_retained": false } diff --git a/crates/devtools/tests/sketch_plan_snapshots/deployments/raw-retained.capabilities.json b/crates/devtools/tests/sketch_plan_snapshots/deployments/raw-retained.capabilities.json new file mode 100644 index 000000000..967cdbdf9 --- /dev/null +++ b/crates/devtools/tests/sketch_plan_snapshots/deployments/raw-retained.capabilities.json @@ -0,0 +1 @@ +{ "raw_data_retained": true, "query_time_results_retained": false } diff --git a/crates/devtools/tests/sketch_plan_snapshots/manifest.json b/crates/devtools/tests/sketch_plan_snapshots/manifest.json index c7e77493d..a8f59d22a 100644 --- a/crates/devtools/tests/sketch_plan_snapshots/manifest.json +++ b/crates/devtools/tests/sketch_plan_snapshots/manifest.json @@ -2,15 +2,15 @@ "deployments": { "raw_retained": [ "--deployment-capabilities", - "{\"raw_data_retained\":true,\"query_time_results_retained\":false}", - "--cost-model", - "aws-fargate-us-east-1" + "crates/devtools/tests/sketch_plan_snapshots/deployments/raw-retained.capabilities.json", + "--cost-inputs", + "tools/dag-viewer/cost-inputs/aws-fargate-us-east-1.json" ], "mvp": [ "--deployment-capabilities", - "{\"raw_data_retained\":false,\"query_time_results_retained\":false}", - "--cost-model", - "aws-fargate-us-east-1" + "crates/devtools/tests/sketch_plan_snapshots/deployments/mvp.capabilities.json", + "--cost-inputs", + "tools/dag-viewer/cost-inputs/aws-fargate-us-east-1.json" ] }, "workloads": [ diff --git a/crates/devtools/tests/stage_pipeline.rs b/crates/devtools/tests/stage_pipeline.rs deleted file mode 100644 index df58ba3e1..000000000 --- a/crates/devtools/tests/stage_pipeline.rs +++ /dev/null @@ -1,2651 +0,0 @@ -//! `stage_pipeline` writes valid four-stage viewer documents for #509's examples. -use std::collections::HashSet; -use std::path::PathBuf; -use std::process::Command; - -use asap_devtools::planning_stages::lower_promql_roots; -use asap_plan_selection::RejectionCause; -use asap_types::ir::flat::FlatDag; -use asap_types::ir::physical_export::PhysicalASAPDAG; -use asap_types::ir::QueryRoot; -use asap_types::workload::PlanningWorkload; - -use serde_json::Value; - -const COMMITTED: &str = "../../tools/dag-viewer/examples/planner-layering-example1.json"; - -fn generate(extra: &[&str]) -> Value { - generate_example("planner-layering-1", extra) -} - -/// `example`'s document from the default search (no `--exhaustive`). -fn generate_default_search(example: &str) -> Value { - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_default_{example}.json", - std::process::id(), - )); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(["--example", example, "--out"]) - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - document -} - -fn generate_example(example: &str, extra: &[&str]) -> Value { - let mut options = vec!["--exhaustive"]; - options.extend_from_slice(extra); - generate_bounded_example(example, &options) -} - -fn generate_bounded_example(example: &str, extra: &[&str]) -> Value { - // Tests run as threads of one process, so the name needs a per-call counter. - static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); - let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_{call}_{example}.json", - std::process::id(), - )); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(["--example", example, "--out"]) - .arg(&out) - .args(extra) - .status() - .unwrap(); - assert!(status.success()); - let document = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - document -} - -/// Stage 3 selected a plan: `stage3_plan_selection` is its `Selection`, -/// whose plan is the physical candidate [`SELECTED`] with its `cost` and -/// `accepted_accuracy_per_query`. -fn selects_a_plan(selection: &Value) -> bool { - selection["selected_dag"].is_object() && selection["cost"].is_object() -} - -/// The id of the selected plan among the physical candidates: the facade's -/// selection is always shown as `P1`. -const SELECTED: &str = "P1"; - -fn validated_dag(value: &Value, queries: usize) { - let dag: FlatDag = serde_json::from_value(value.clone()).unwrap(); - assert_eq!(dag.roots.len(), queries); - // Children come before their parents. - for (id, node) in dag.nodes.iter().enumerate() { - assert!(node - .operator - .children() - .into_iter() - .all(|child| *child < id)); - } - for root in &dag.roots { - if let QueryRoot::Operator(id) = root { - assert!(*id < dag.nodes.len()); - } - } -} - -/// Every logical DAG is a well-formed flat DAG and every physical DAG validates; -/// each has one root per query; ids are unique; Stage 2 implements or prunes -/// (with a reason) each composed logical candidate, with no cost; Stage 3 -/// selects `P1`, and the reference pricer accounts for every enumerated -/// Stage 2 candidate once and costs exactly the valid ones; the committed -/// fixture's workload is current. -#[test] -fn example1_document_preserves_workload_and_validates_archived_plans() { - // Example 1's story is about which plans miss their accuracy target; - // archive enough evaluated plans to check the cross-stage linkage. - let document = generate(&["--enforce-accuracy-guarantees", "--max-displayed", "100000"]); - let committed: Value = serde_json::from_str( - &std::fs::read_to_string(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(COMMITTED)) - .unwrap(), - ) - .unwrap(); - assert_eq!(document["workload"], committed["workload"]); - - assert_eq!(document["format"], "asap-stage-pipeline/v1"); - let queries = document["workload"]["queries"].as_array().unwrap().len(); - assert_eq!(queries, 2); - validated_dag(&document["stage0_query_language_frontends"]["dag"], queries); - - let logical = &document["stage1_logical_asap_optimization"]; - let candidates = logical["candidates"].as_array().unwrap(); - assert_eq!(logical["capped"], false); - assert_eq!(logical["combinations"], candidates.len()); - let mut ids = HashSet::new(); - let mut composed = HashSet::new(); - for candidate in candidates { - let id = candidate["id"].as_str().unwrap(); - assert!(ids.insert(id)); - if candidate.get("dag").is_some() { - composed.insert(id); - validated_dag(&candidate["dag"], queries); - } - } - - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - let pruned = document["stage2_physical_asap_optimization"]["pruned"] - .as_array() - .unwrap(); - assert!(pruned - .iter() - .all(|p| !p["reason"].as_str().unwrap().is_empty())); - let sources: HashSet<_> = physical - .iter() - .chain(pruned) - .map(|p| p["from_logical"].as_str().unwrap()) - .collect(); - assert_eq!(sources, composed); - for candidate in physical { - assert!(candidate.get("cost").is_none(), "Stage 2 has no cost"); - let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); - assert_eq!(dag.roots.len(), queries); - dag.validate().unwrap(); - } - - // The facade's selection is `P1`; the reference pricer selects among - // the enumerated plans and rejects every other one. - assert!(selects_a_plan(&document["stage3_plan_selection"])); - let reference = &document["reference_pricing"]; - let costs = reference["cost_per_candidate"].as_object().unwrap(); - let selected = reference["selected_candidate"].as_str().unwrap(); - let mut accounted = HashSet::from([SELECTED, selected]); - for rejection in reference["pruned_candidates"].as_array().unwrap() { - let id = rejection["candidate"].as_str().unwrap(); - assert!(accounted.insert(id), "{id} accounted for once"); - assert!(!rejection["reason"].as_str().unwrap().is_empty()); - // A candidate that failed a check is not priced; one that lost - // under the cost model is. - let cause: RejectionCause = serde_json::from_value(rejection["cause"].clone()).unwrap(); - let failed_a_check = cause.failed_a_check(); - assert_eq!(failed_a_check, !costs.contains_key(id), "{rejection}"); - } - assert!(costs.contains_key(selected)); - let all: HashSet<_> = physical.iter().map(|p| p["id"].as_str().unwrap()).collect(); - assert_eq!(accounted, all); -} - -/// `--max-displayed` cuts only what the document shows, says so, and keeps -/// the selection made over all candidates. -#[test] -fn display_cap_is_recorded_and_keeps_the_selection() { - // Instant quantiles provide several implementations without expanding a - // tumbling-window-width domain. Exhaustive mode ensures the display cap is exercised. - let document = document_of(&[ - "--promql", - "quantile(0.5, data)", - "--promql", - "quantile(0.9, data)", - "--sample-interval-ms", - "15000", - "--exhaustive", - "--max-displayed", - "1", - ]); - let logical = &document["stage1_logical_asap_optimization"]; - assert_eq!(logical["capped"], true); - // Logical parents of displayed physical plans are always retained: the - // selected plan's (`L1`) and the reference pricer's pick's; the cap - // additionally allows one unimplemented logical candidate. - assert!(logical["candidates"].as_array().unwrap().len() <= 3); - assert!(logical["combinations"].as_u64().unwrap() > 1); - assert!( - selects_a_plan(&document["stage3_plan_selection"]), - "selection survives display cap: {}", - document["stage3_plan_selection"] - ); - let reference = &document["reference_pricing"]; - let reference_selected = reference["selected_candidate"].as_str().unwrap(); - assert!(reference["cost_per_candidate"] - .get(reference_selected) - .is_some()); - // One enumerated plan shown, the reference pricer's pick, beside the - // selected plan. - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - let ids: Vec<&str> = physical.iter().map(|p| p["id"].as_str().unwrap()).collect(); - assert_eq!(ids, [SELECTED, reference_selected]); - for candidate in physical { - assert!(logical["candidates"] - .as_array() - .unwrap() - .iter() - .any(|logical| logical["id"] == candidate["from_logical"])); - let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); - dag.validate().unwrap(); - assert_eq!(dag.roots.len(), 2); - } -} - -/// Stage 1's `pruned_candidates`: each candidate Pass 1 pruned, with its -/// reason, in pruning order. -fn pass1_pruned(document: &serde_json::Value) -> Vec<(String, String)> { - document["stage1_logical_asap_optimization"]["pruned_candidates"] - .as_array() - .unwrap() - .iter() - .map(|pruned| { - ( - pruned["candidate"].as_str().unwrap().to_string(), - pruned["reason"].as_str().unwrap().to_string(), - ) - }) - .collect() -} - -/// A top-k partitioned by every label of its input has no item, so a heap -/// over it could not be composed; such a heap reads one row per item and -/// is pruned in Pass 1 (`TopKHeapOverOneRowPerItem`), listed under -/// `pruned_candidates`, and every candidate left composes. -#[test] -fn a_topk_heap_over_one_row_per_item_is_pruned_with_its_reason() { - let document = document_of(&[ - "--promql", - "topk by (job) (3, sum by (job) (rate(m[5m])))", - "--max-error", - "0.01", - "--failure-probability", - "0.001", - ]); - let stage1 = &document["stage1_logical_asap_optimization"]; - assert_eq!( - pass1_pruned(&document), - [ - ("CMS+heap".into(), "TopKHeapOverOneRowPerItem".into()), - ( - "CountSketch+heap".into(), - "TopKHeapOverOneRowPerItem".into() - ), - ] - ); - for candidate in stage1["candidates"].as_array().unwrap() { - validated_dag(&candidate["dag"], 1); - } -} - -/// Example 3a (243 Pass 1 candidates, each with 3 Pass 2 candidates) -/// selects what the planning stages select over -/// every candidate, which the acceptance tests also run: the document's -/// display cap does not change the selection, and the selected plan is -/// displayed with its cost. -#[test] -fn example3a_archive_and_viewer_select_consistently() { - // Candidate IDs are local to one time-bounded search. Validate the viewer - // references within its own archive instead of comparing independent runs. - let document = generate_example("planner-layering-3a", &[]); - assert!(selects_a_plan(&document["stage3_plan_selection"])); - let reference = &document["reference_pricing"]; - let reference_selected = reference["selected_candidate"].as_str().unwrap(); - assert!(reference["cost_per_candidate"] - .get(reference_selected) - .is_some()); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - for selected in [SELECTED, reference_selected] { - let candidate = physical.iter().find(|p| p["id"] == selected).unwrap(); - let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); - dag.validate().unwrap(); - assert_eq!(dag.roots.len(), 5); - } -} - -/// Example 2 plans its three SQL queries with the default search and -/// selects a plan. -#[test] -fn example2_plans_the_sql_workload() { - // The default search: its SQL windows get tumbling windows, so an - // exhaustive run has over 10^4 Stage 2 candidates. The integration test - // `stage1_logical_asap_optimization_sql_adds_one_univmon_for_all_three` - // checks the shared UnivMon candidate. - let document = generate_default_search("planner-layering-2"); - let queries = document["workload"]["queries"].as_array().unwrap(); - assert_eq!(queries.len(), 3); - assert!(queries.iter().all(|q| q["language"] == "sql")); - validated_dag(&document["stage0_query_language_frontends"]["dag"], 3); - assert!(selects_a_plan(&document["stage3_plan_selection"])); -} - -/// SQL queries over tables declared with `--table` are planned through the -/// same stages, and the document records each query's language. -#[test] -fn sql_queries_over_declared_tables_are_planned() { - let out = std::env::temp_dir().join(format!("stage_pipeline_{}_sql.json", std::process::id())); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args([ - "--table", - r#"{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp", "nullable": false}, - {"name": "service", "type": "utf8", "nullable": false}, - {"name": "latency", "type": "float64", "nullable": false}], "time_index": 0}"#, - "--sql", - "SELECT service, COUNT(*) FROM metrics GROUP BY service", - "--sql", - "SELECT service, AVG(latency) FROM metrics GROUP BY service", - "--series-count", - "1000", - "--rows-held", - "60000", - "--out", - ]) - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document: Value = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - let queries = document["workload"]["queries"].as_array().unwrap(); - assert_eq!(queries.len(), 2); - assert!(queries.iter().all(|q| q["language"] == "sql")); - validated_dag(&document["stage0_query_language_frontends"]["dag"], 2); - assert!(selects_a_plan(&document["stage3_plan_selection"])); -} - -/// One run mixes SQL (over declared tables) and PromQL queries, in the -/// order given, each named with `--name`. -#[test] -fn mixed_sql_and_promql_queries_are_planned_with_their_names() { - let out = - std::env::temp_dir().join(format!("stage_pipeline_{}_mixed.json", std::process::id())); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args([ - "--table", - r#"{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp", "nullable": false}, - {"name": "service", "type": "utf8", "nullable": false}], "time_index": 0}"#, - "--sql", - "SELECT service, COUNT(*) FROM metrics GROUP BY service", - "--name", - "count_by_service", - "--promql", - "topk(5, rate(http_requests_total[5m]))", - "--max-error", - "0.01", - "--series-count", - "1000", - "--rows-held", - "60000", - "--out", - ]) - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document: Value = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - let queries = document["workload"]["queries"].as_array().unwrap(); - let described: Vec<_> = queries - .iter() - .map(|q| (q["id"].as_str().unwrap(), q["language"].as_str().unwrap())) - .collect(); - assert_eq!(described, [("count_by_service", "sql"), ("q2", "promql")]); - validated_dag(&document["stage0_query_language_frontends"]["dag"], 2); - assert!(selects_a_plan(&document["stage3_plan_selection"])); -} - -/// Runs `stage_pipeline` with `args` (and a throwaway `--out`), expecting it -/// to fail: its exit status and stderr. -fn failure(args: &[&str]) -> (Option, String) { - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_failure.json", - std::process::id() - )); - let output = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(args) - .arg("--out") - .arg(&out) - .output() - .unwrap(); - assert!(!output.status.success()); - assert!(!out.exists(), "a rejected run writes no document"); - ( - output.status.code(), - String::from_utf8(output.stderr).unwrap(), - ) -} - -/// Exhaustive mode bounds evaluated assignments and archives the examined prefix. -#[test] -fn workload_above_the_candidate_limit_reports_partial_search() { - let document = generate(&["--max-combinations", "3", "--max-displayed", "100"]); - // The memo has more than three choices: the reference enumeration - // stops at three and says it is partial. - let logical = &document["stage1_logical_asap_optimization"]; - assert_eq!(logical["capped"], true); - let enumerated = logical["candidates"] - .as_array() - .unwrap() - .iter() - .filter(|c| c["id"].as_str().unwrap().starts_with('E')) - .count(); - assert!(enumerated <= 3, "{enumerated}"); - // The bound is the reference enumeration's only: the facade still - // selects its plan. - assert!(selects_a_plan(&document["stage3_plan_selection"])); -} - -/// Nine approximate quantile queries over nine metrics, three candidates -/// each: 3^9 = 19683 Pass 1 combinations. -fn nine_quantile_queries() -> Vec { - let mut args = Vec::new(); - for i in 1..=9 { - args.extend([ - "--promql".to_string(), - format!("quantile_over_time(0.99, m{i}[5m])"), - ]); - } - args.extend(["--max-error", "0.01", "--failure-probability", "0.001"].map(String::from)); - args -} - -/// Without `--exhaustive`, the same nine queries plan with the default -/// search, `plan_installable`'s, which searches Stage 2's catalog without -/// enumerating the combinations. The document holds the selected plan and -/// the search's report. -#[test] -fn default_search_plans_nine_queries_without_cartesian_enumeration() { - let args = nine_quantile_queries(); - let args: Vec<&str> = args.iter().map(String::as_str).collect(); - let document = document_of(&args); - assert_eq!(document["search"], "candidate-connections"); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert_eq!(physical.len(), 1); - assert_eq!(physical[0]["id"], SELECTED); - assert!(selection["search_report"].is_object(), "{selection}"); -} - -/// `--max-combinations` limits only the exhaustive search. -#[test] -fn max_combinations_needs_exhaustive() { - let (status, stderr) = failure(&["--promql", "sum(m)", "--max-combinations", "10"]); - assert_eq!(status, Some(2)); - assert!(stderr.contains("needs --exhaustive"), "{stderr}"); -} - -/// A 9-query PromQL workload converted from ASAPQuery's random workloads -/// (seed 862, workload-01), planned as a deployment that keeps no raw data: -/// both CLI planning and installation return validated plans with guarantees -/// enforced. Timed searches may choose different incumbents and archive IDs; -/// the API contract is output coverage, accuracy, and executable installation. -#[test] -fn default_search_and_installation_validate_the_nine_query_workload() { - const WORKLOAD: &str = - "../../tools/dag-viewer/workloads/random-mixed-seed-862/workload-01.json"; - const NO_RAW: &str = r#"{"raw_data_retained": false, "query_time_results_retained": false}"#; - let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(WORKLOAD); - let document = document_of(&[ - "--workload", - path.to_str().unwrap(), - "--deployment-capabilities", - NO_RAW, - "--cost-model", - "aws-fargate-us-east-1", - "--enforce-accuracy-guarantees", - ]); - assert_eq!(document["search"], "candidate-connections"); - assert_eq!(document["accuracy_guarantees"], "enforced"); - let queries = document["workload"]["queries"].as_array().unwrap().len(); - assert_eq!(queries, 9); - validated_dag(&document["stage0_query_language_frontends"]["dag"], queries); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert_eq!(physical.len(), 1); - assert_eq!(physical[0]["id"], SELECTED); - let dag: PhysicalASAPDAG = serde_json::from_value(physical[0]["dag"].clone()).unwrap(); - assert_eq!(dag.roots.len(), queries); - assert!( - selection["search_report"]["milp_variables"] - .as_u64() - .unwrap() - > 0 - ); - - let text = std::fs::read_to_string(&path).unwrap(); - let workload: asap_types::workload::PlanningWorkload = serde_json::from_str(&text).unwrap(); - let defaults = asap_cost::CostInputs::default(); - let settings = asap_cost::CostInputs { - cost_model: asap_cost::CostModel::UsageOverTime( - asap_cost::ResourceUnitCost::aws_fargate_us_east_1(defaults.resource_speed), - ), - ..defaults - }; - let capabilities: asap_planner::DeploymentCapabilities = serde_json::from_str(NO_RAW).unwrap(); - let models = asap_planner::PlanningModels::builtin() - .with_cost_inputs(&settings) - .with_deployment_capabilities(&capabilities); - let (plan, report) = tokio::runtime::Builder::new_current_thread() - .build() - .unwrap() - .block_on(asap_planner::plan_installable( - asap_planner::UserInput::new(&workload, asap_planner::FrontendInput::Promql, models), - )) - .unwrap(); - plan.validate().unwrap(); - assert_eq!(plan.query_texts.len(), queries); - assert_eq!( - selection["accepted_accuracy_per_query"] - .as_array() - .unwrap() - .len(), - queries - ); - let report = report.unwrap(); - assert!(report.queries_accepted_without_guarantee.is_empty()); - assert!(report - .cost_per_second - .is_some_and(|cost| cost.is_finite() && cost > 0.0)); - assert_eq!(report.predicted_usage.per_query.len(), queries); -} - -/// `--sample-interval-ms` is the sample interval of every metric the -/// queries read, as in the other lowering tools: `sum(up)`'s bare selector -/// looks back 30 s. The former `--interval-ms` is unknown. -#[test] -fn sample_interval_ms_sets_the_instant_selector_lookback() { - let document = document_of(&["--promql", "sum(up)", "--sample-interval-ms", "30000"]); - let ranges: Vec<&Value> = document["stage0_query_language_frontends"]["dag"]["nodes"] - .as_array() - .unwrap() - .iter() - .filter_map(|node| node["operator"]["NonASAP"]["TimeRange"]["range"].as_object()) - .map(|range| &range["secs"]) - .collect(); - assert_eq!(ranges, [&Value::from(30)]); - let (status, stderr) = failure(&["--promql", "sum(up)", "--interval-ms", "30000"]); - assert_eq!(status, Some(2)); - assert!( - stderr.contains("unknown argument --interval-ms"), - "{stderr}" - ); -} - -#[test] -fn table_declared_twice_is_rejected() { - let table = - r#"{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp"}], "time_index": 0}"#; - let (status, stderr) = failure(&[ - "--table", - table, - "--table", - table, - "--sql", - "SELECT COUNT(*) FROM metrics", - ]); - assert_eq!(status, Some(2)); - assert!( - stderr.contains("--table: table metrics is declared twice"), - "{stderr}" - ); -} - -/// A defaulted name that collides says it is the default. -#[test] -fn duplicate_default_name_says_it_is_the_default() { - let (_, stderr) = failure(&["--promql", "sum(a)", "--name", "q2", "--promql", "sum(b)"]); - assert!( - stderr.contains("duplicate query name q2 (the default name of query 2)"), - "{stderr}" - ); - let (_, stderr) = failure(&["--promql", "sum(a)", "--promql", "sum(b)", "--name", "q1"]); - assert!(stderr.contains("duplicate query name q1\n"), "{stderr}"); -} - -/// Runs `stage_pipeline` with `args`, expecting a document; queries given -/// on the command line read sources declared to hold 1,000 series each, -/// and a SQL table 60,000 rows (pricing uses only declared facts). -fn document_of(args: &[&str]) -> Value { - static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); - let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_{call}_document.json", - std::process::id() - )); - let facts: &[&str] = if args.iter().any(|a| *a == "--promql" || *a == "--sql") { - &["--series-count", "1000", "--rows-held", "60000"] - } else { - &[] - }; - let output = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(args) - .args(facts) - .arg("--out") - .arg(&out) - .output() - .unwrap(); - assert!( - output.status.success(), - "{}", - String::from_utf8_lossy(&output.stderr) - ); - let document = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - document -} - -/// CLI planning reports Pass 1's time as the production planner does. -#[test] -fn cli_reports_pass1_generation_time() { - let document = document_of(&["--promql", "sum(data)"]); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - let generation = document["planning_report"]["pass1_ms"] - .as_f64() - .expect("local generation is timed by the production entry point"); - assert!(generation.is_finite() && generation >= 0.0); -} - -/// When Pass 1 fails, the document still carries the Stage 0 DAG, with the -/// error under Stage 1 and no selection, so the viewer can show both. -#[test] -fn pass1_failure_still_writes_stage0_and_the_error() { - let document = document_of(&[ - "--promql", - "topk(5, rate(m[5m]))", - "--max-error", - "1.5", - "--failure-probability", - "0.1", - ]); - validated_dag(&document["stage0_query_language_frontends"]["dag"], 1); - let logical = &document["stage1_logical_asap_optimization"]; - assert!( - logical["error"] - .as_str() - .unwrap() - .starts_with("Pass 1: approximate accuracy"), - "{logical}" - ); - assert_eq!(logical["combinations"], 0); - assert_eq!(logical["candidates"], serde_json::json!([])); - assert_eq!( - document["stage2_physical_asap_optimization"], - serde_json::json!({ "candidates": [], "pruned": [], "pruned_candidates": [] }) - ); - assert!(!selects_a_plan(&document["stage3_plan_selection"])); -} - -/// Standalone scalar queries first, in the middle and last, which the -/// physical DAG keeps in `scalar_roots` by query position (the viewer's -/// test_viewer.py draws this workload). -const SCALAR_QUERIES_ARGS: &[&str] = &[ - "--promql", - "scalar(sum(m))", - "--promql", - "sum(n)", - "--promql", - "time()", - "--promql", - "sum by (job) (k)", - "--promql", - "1 + 2", -]; - -#[test] -fn scalar_queries_keep_their_positions_in_scalar_roots() { - let document = document_of(SCALAR_QUERIES_ARGS); - assert!(selects_a_plan(&document["stage3_plan_selection"])); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .find(|c| c["id"] == SELECTED) - .unwrap(); - let positions: Vec<_> = physical["dag"]["scalar_roots"] - .as_array() - .unwrap() - .iter() - .map(|entry| entry[0].as_u64().unwrap()) - .collect(); - assert_eq!(positions, [0, 2, 4]); -} - -/// `time_index` must be a column index; anything else is an error rather -/// than a table without a time column. -#[test] -fn table_time_index_must_be_a_column_index() { - for index in [r#""0""#, "-1", "1.5", "3"] { - let table = format!( - r#"{{"name": "metrics", "columns": [{{"name": "ts", "type": "timestamp"}}], "time_index": {index}}}"# - ); - let (status, stderr) = - failure(&["--table", &table, "--sql", "SELECT COUNT(*) FROM metrics"]); - assert_eq!(status, Some(2), "{index}"); - assert!( - stderr - .contains("--table metrics: time_index must be the index of one of its 1 columns"), - "{index}: {stderr}" - ); - } -} - -/// A query whose target sets no metric shows the metric its result is -/// naturally stated in (the default-metric table, display only): a top-k's -/// is the L1-scaled absolute error; an exact query shows none. -#[test] -fn a_target_without_a_metric_shows_the_query_default() { - let document = generate(&[]); - let queries = &document["workload"]["queries"]; - assert!(queries[0]["requirements"].get("default_metric").is_none()); - assert_eq!( - queries[1]["requirements"]["default_metric"], - "absolute_stream_l1" - ); -} - -/// A quantile query's target may leave its metric unset: then KLL (rank) -/// and DDSketch (relative) candidates both meet it and compete on cost, and -/// the document reports, per valid candidate, the metric each was accepted -/// in and that the target's metric was not set. With `--accuracy-metric rank` only -/// rank guarantees (and exact plans) are accepted, and the report says so. The workload's -/// accuracy is the target's own JSON form. Guarantees are enforced, since -/// the test is about which guarantees meet the target. -#[test] -fn a_quantile_target_without_a_metric_accepts_rank_or_relative() { - let query = |metric: Option<&str>| { - let mut accuracy = serde_json::json!({"max_error": 0.01}); - if let Some(metric) = metric { - accuracy["accuracy_metric"] = metric.into(); - } - let workload = serde_json::json!({ - "query_workload": {"language": "promql", "query_batch": [{ - "query": "quantile_over_time(0.99, m[1m])", - "requirements": {"accuracy": accuracy}, - "predictability": "unknown", "invocations": 1 - }]}, - // Relative DDS interpolation requires this explicit domain proof. - "data_workload": {"arrival": "unknown", "sources": {"m": { - "sample_interval": 1000, "series_count": 1, "values_non_negative": true - }}} - }); - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_quantile_metric_{}.json", - std::process::id(), - metric.unwrap_or("unset") - )); - std::fs::write(&path, workload.to_string()).unwrap(); - let document = document_of(&[ - "--enforce-accuracy-guarantees", - "--exhaustive", - "--workload", - path.to_str().unwrap(), - ]); - std::fs::remove_file(path).unwrap(); - document - }; - let accepted = |document: &Value| -> Vec { - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - // Every valid enumerated plan, as the reference pricer accepted it. - document["reference_pricing"]["accepted_accuracy_per_candidate"] - .as_object() - .unwrap() - .values() - .map(|queries| queries[0]["summary"].as_str().unwrap().to_owned()) - .collect() - }; - let unset = query(None); - assert_eq!( - unset["workload"]["queries"][0]["requirements"]["accuracy"], - serde_json::json!({"max_error": 0.01, "accuracy_scope": "whole_result"}) - ); - // A quantile has no single default metric to show. - assert!(unset["workload"]["queries"][0]["requirements"] - .get("default_metric") - .is_some_and(Value::is_null)); - let summaries = accepted(&unset); - assert!( - summaries - .iter() - .all(|s| s.starts_with("metric not set; accepted ")), - "{summaries:?}" - ); - for wanted in ["accepted rank (Kll)", "accepted relative (DDSketch)"] { - assert!( - summaries.iter().any(|s| s.contains(wanted)), - "{wanted}: {summaries:?}" - ); - } - assert!( - unset["stage3_plan_selection"]["accepted_accuracy_per_query"][0]["accepted_metric"] - .is_string() - ); - - let rank = query(Some("rank")); - let summaries = accepted(&rank); - assert!( - summaries - .iter() - .all(|s| s == "metric rank; accepted rank (Kll)" || s == "metric rank; accepted exact"), - "{summaries:?}" - ); -} - -/// `--cost-model` chooses how plan selection compares Example 4's -/// candidates: each of cost models 1 (the default) and 2 selects a priced -/// plan, and cost model 3 with a 256 GB cluster (lowest latency) selects a -/// plan that fits the cluster, with no price. The selected plan reports its -/// usage. Cost model 3 is run with guarantees enforced: ignoring them, a -/// candidate without a guarantee fits the cluster better. -#[test] -#[ignore = "fails on main too: the selection MILP does not support cost models 2 and 3 (UnsupportedCostModel); not a wall-clock issue; #888"] -fn cost_model_flag_chooses_how_candidates_are_compared() { - let selected = |extra: &[&str]| { - let document = generate_example("planner-layering-3b", extra); - document["stage3_plan_selection"].clone() - }; - for options in [&[][..], &["--cost-model", "provisioned-capacity"][..]] { - let selection = selected(options); - assert!(selects_a_plan(&selection), "{selection}"); - } - let cluster = selected(&[ - "--enforce-accuracy-guarantees", - "--cost-model", - "cluster-capacity", - "--cluster-memory-bytes", - "256000000000", - "--cluster-disk-bytes", - "1000000000000", - "--cluster-objective", - "lowest-latency", - ]); - assert!(selects_a_plan(&cluster), "{cluster}"); - let cost = &cluster["cost"]; - assert!(cost["cost_per_second"].is_null()); - assert!(cost["usage"]["peak_memory_bytes"].as_f64().unwrap() <= 256000000000.0); -} - -/// `--cost-model aws-fargate-us-east-1` prices Example 4 under cost model 1 -/// at AWS Fargate's us-east-1 prices, in US dollars per second: the -/// whole-window KLL (P2) is still selected, at $2.60278e-8/s (its 2.1 -/// CPU-ms/s at $1.1244e-8 per CPU-ms, and its states held while the query -/// runs), and the tumbling windows -/// maintained at ingestion time (P3-m1) cost about $6.0238e-6/s, almost all -/// for their 5.232 GB kept in memory (six tumbling windows of 1,000,000 series, each a -/// KLL of 872 bytes: its fixed 744 and an item array grown for its 4 -/// samples). A costlier candidate's reason shows -/// such small costs in scientific notation. -#[test] -fn aws_fargate_preset_prices_example_4_in_dollars() { - // Pricing acceptance exercises the production bounded planner; exhaustive - // archive traversal is tested separately on small catalogs. - let document = generate_bounded_example( - "planner-layering-3b", - &["--cost-model", "aws-fargate-us-east-1"], - ); - let cost = document["stage3_plan_selection"]["cost"]["cost_per_second"] - .as_f64() - .unwrap(); - assert!(cost.is_finite() && cost > 0.0); - // Preserve the historical feasible whole-window KLL price as an upper bound. - assert!(cost <= 2.602778532914e-8 * (1.0 + 1e-6), "{cost}"); -} - -/// `--deployment-capabilities` declares the executor's capabilities, which -/// the document's `deployment_capabilities` shows (defaults filled in; -/// without the flag, every summary supported, raw data retained, no -/// query-time result retained). For Pattern B with KLL only and query-time -/// results retained: no DDSketch candidate is offered, and Stage 1's -/// `pruned_candidates` says why; the KLL tumbling windows gain B3, built at query time and -/// kept (`P3-m2`, 436.59 cost/s); the whole-window KLL (`P2`) is still -/// selected. Without raw data, every query-time scan is invalid and B1 -/// (`P3-m1`) is selected when guarantees are enforced. A deployment with an -/// unknown field is refused. -#[test] -fn deployment_capabilities_flag_sets_the_capabilities_and_shows_them() { - let default = generate_example("planner-layering-3b", &[]); - assert_eq!( - default["deployment_capabilities"], - serde_json::json!({ - "supported_summaries": asap_executor::supported_summary_families(), - "raw_data_retained": true, - "query_time_results_retained": false, - }) - ); - let kll_kept = generate_example( - "planner-layering-3b", - &[ - "--deployment-capabilities", - r#"{"supported_summaries": [{"family": "Kll", "readouts": ["Quantile"]}], - "query_time_results_retained": true}"#, - ], - ); - assert_eq!( - kll_kept["deployment_capabilities"], - serde_json::json!({ - "supported_summaries": [{ "family": "Kll", "readouts": ["Quantile"] }], - "raw_data_retained": true, - "query_time_results_retained": true, - }) - ); - let physical: Vec<&str> = kll_kept["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .map(|c| c["id"].as_str().unwrap()) - .collect(); - assert!(!physical.is_empty()); - // Pass 1 lists why DDSketch is gone. - assert_eq!( - pass1_pruned(&kll_kept), - [("DDSketch".into(), "UnsupportedByDeployment".into())] - ); - assert!(pass1_pruned(&default).is_empty()); - assert!(selects_a_plan(&kll_kept["stage3_plan_selection"])); - assert!(kll_kept["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .all(|c| !c["dag"].to_string().contains("DDSketch"))); - - let no_raw = generate_example( - "planner-layering-3b", - &[ - "--enforce-accuracy-guarantees", - "--deployment-capabilities", - r#"{"raw_data_retained": false}"#, - ], - ); - let selection = &no_raw["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - assert!(!selection["materialized_views"] - .as_array() - .unwrap() - .is_empty()); - - let (status, stderr) = failure(&[ - "--exhaustive", - "--example", - "planner-layering-3b", - "--deployment-capabilities", - r#"{"raw_data": false}"#, - ]); - assert_eq!(status, Some(2)); - assert!( - stderr.contains("--deployment-capabilities: unknown field `raw_data`"), - "{stderr}" - ); -} - -/// Cluster limits need cost model 3, and cost model 3 needs them. -#[test] -fn cluster_flags_and_cost_model_3_go_together() { - let (code, stderr) = failure(&[ - "--exhaustive", - "--example", - "planner-layering-3b", - "--cluster-memory-bytes", - "1", - ]); - assert_eq!(code, Some(2)); - assert!( - stderr.contains("need --cost-model cluster-capacity"), - "{stderr}" - ); - let (code, stderr) = failure(&[ - "--exhaustive", - "--example", - "planner-layering-3b", - "--cost-model", - "cluster-capacity", - ]); - assert_eq!(code, Some(2)); - assert!(stderr.contains("needs --cluster-memory-bytes"), "{stderr}"); -} - -/// Two sums and two maxima over two metrics share each metric's scan and -/// range between its own pair of queries: every logical candidate's label -/// names which queries share each node, so no two labels are the same. -#[test] -fn logical_candidate_labels_name_each_query() { - let document = document_of(&[ - "--exhaustive", - "--promql", - "sum(a)", - "--promql", - "max(a)", - "--promql", - "sum(b)", - "--promql", - "max(b)", - ]); - let labels: Vec<&str> = document["stage1_logical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .map(|c| c["display_name"].as_str().unwrap()) - .collect(); - assert!(!labels.is_empty()); - assert!(labels - .iter() - .all(|label| (1..=4).all(|q| label.contains(&format!("Q{q} "))))); -} - -/// The MILP search's final lower bound never drops below a bound an earlier -/// round proved. `topk(32, sum by (label_0) (sum_over_time(data[1h])))` -/// alone: round 1 proves the selected plan optimal within 1e-5, and the -/// last round finds nothing below the incumbent. That round's bound (the -/// incumbent less the solver's pruning tolerance, scaled by the largest -/// charge, here the query-time top-k tumbling windows at about 308 times the -/// selected plan) used to replace round 1's, reporting a gap of 3.09e-4. -#[test] -fn milp_final_bound_keeps_the_bound_an_earlier_round_proved() { - let workload = serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [{ - "query": "topk(32, sum by (label_0) (sum_over_time(data[1h])))", - "demand": {"fixed_interval": 60000}, - "requirements": { - "accuracy": { - "max_error": 0.05, - "failure_probability": 0.01, - "accuracy_scope": "whole_result", - }, - "max_latency_ms": null, - }, - "predictability": {"predictable": {"known_at": null}}, - }], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 2000000.0}}, - "sources": {"data": { - "series_count": 10000, - "metric_type": "gauge", - "values_non_negative": true, - "sample_interval": 5, - }}, - }, - }); - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_milp_final_bound.json", - std::process::id() - )); - std::fs::write(&path, workload.to_string()).unwrap(); - let document = document_of(&[ - "--workload", - path.to_str().unwrap(), - "--deployment-capabilities", - r#"{"raw_data_retained": true, "query_time_results_retained": false}"#, - "--cost-model", - "aws-fargate-us-east-1", - "--enforce-accuracy-guarantees", - ]); - std::fs::remove_file(&path).unwrap(); - let selection = &document["stage3_plan_selection"]; - let report = &selection["search_report"]; - let incumbent = selection["cost"]["cost_per_second"].as_f64().unwrap(); - if let Some(lower_bound) = report["cost_lower_bound_per_second"].as_f64() { - assert!(lower_bound <= incumbent * (1.0 + 1e-9), "{report}"); - if report["status"] == "optimal" { - let relative_gap = (incumbent - lower_bound).max(0.0) / incumbent; - assert!(relative_gap <= 1e-4, "{report}"); - } - } -} - -/// `--workload` plans a `PlanningWorkload` JSON file. A dashboard panel -/// (`max_over_time(m[5m])` every minute over 1M series sampled every -/// 1.5 s) answered as the last hour at 1-minute steps -/// (`range_evaluation`) is priced as 61 evaluations per request: the kept -/// tumbling windows (`P2-m2`, 460.04 cost/s) beat rebuilding the window at every step -/// (`P1`, 1,270.52), and the document reports their retention, 1 h + 5 min. -/// The workload section shows the range. An invalid range is refused. -#[test] -fn workload_file_with_a_range_evaluation_is_planned() { - let workload = serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [{ - "query": "max_over_time(m[5m])", - "demand": {"fixed_interval": 60000}, - "requirements": {}, - "predictability": {"predictable": {"known_at": null}}, - "range_evaluation": {"step_ms": 60000, "span_ms": 3600000}, - }], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 666666.67}}, - "sources": {"m": {"series_count": 1000000, "sample_interval": 1500}}, - }, - }); - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_range_workload.json", - std::process::id() - )); - std::fs::write(&path, workload.to_string()).unwrap(); - let document = document_of(&[ - "--workload", - path.to_str().unwrap(), - "--deployment-capabilities", - r#"{"query_time_results_retained": true}"#, - ]); - assert_eq!( - document["workload"]["queries"][0]["requirements"]["range_evaluation"], - serde_json::json!({"step_ms": 60000, "span_ms": 3600000}) - ); - let selection = &document["stage3_plan_selection"]; - let cost = selection["cost"]["cost_per_second"].as_f64().unwrap(); - // The reference: the window merged from tumbling windows kept across evaluations, - // priced under the same models. Its absolute cost is pinned in - // integration-tests/tests/range_evaluation.rs. - let kept_tumbling_windows = { - use asap_accuracy::propagation::recompose_guarantees; - use asap_logical_asap_optimization::test_support::{ - compose_logical_candidate, generate_per_computation_candidates, - }; - use asap_logical_asap_optimization::test_support::{ - tumbling_window_candidates, window_composition_pass2_candidates, - }; - use asap_types::ir::physical_export::PhysicalASAPOperatorPayload; - use asap_types::ir::ASAPOp; - use asap_types::physical::{Materialization, Medium}; - let parsed: PlanningWorkload = serde_json::from_value(workload.clone()).unwrap(); - let entries: Vec<_> = parsed.query_workload.entries().collect(); - let data = parsed.data_workload.clone().unwrap(); - let inventory = generate_per_computation_candidates( - &asap_logical_asap_optimization::generate_dag_of_logical_memo_groups( - &lower_promql_roots(&parsed).unwrap(), - ), - ) - .unwrap(); - let exact: Vec<_> = compose_logical_candidate(&inventory, &[0]).unwrap(); - let grids: Vec<_> = entries.iter().map(|e| e.evaluation_grid_ms()).collect(); - let tumbling_windows = tumbling_window_candidates(&exact, &grids); - let windowed = window_composition_pass2_candidates(&exact, &tumbling_windows) - .unwrap() - .remove(0) - .roots - .unwrap(); - let deployment = asap_planner::DeploymentCapabilities { - query_time_results_retained: true, - ..Default::default() - }; - let kept = asap_physical_asap_optimization::optimize_physical_asap( - "kept-tumbling-windows", - &recompose_guarantees(&windowed).unwrap(), - &entries, - &data, - deployment.query_time_results_retained, - ) - .unwrap() - .into_iter() - .find(|plan| { - plan.dag.nodes.iter().any(|n| { - matches!( - n.payload, - PhysicalASAPOperatorPayload::ASAP(ASAPOp::SummaryTumblingWindowChain { .. }) - ) && n.materialization - == Materialization::MatAtQueryKeptAcrossEvaluations(Medium::Memory) - }) - }) - .expect("a plan keeps its tumbling windows across evaluations"); - let settings = asap_cost::CostInputs::default(); - let priced = asap_devtools::memo_pricing::reference_select( - std::slice::from_ref(&kept), - &entries, - &data, - asap_planner::PlanningModels::builtin() - .with_cost_inputs(&settings) - .with_deployment_capabilities(&deployment), - ) - .unwrap(); - priced.costs[&priced.selected].cost_per_second.unwrap() - }; - assert!( - cost <= kept_tumbling_windows * (1.0 + 1e-9), - "selected {SELECTED} costs {cost}, more than the kept tumbling windows' {kept_tumbling_windows}" - ); - assert!(selection["tumbling_window_retention_ms"] - .as_object() - .unwrap() - .values() - .any(|retention| retention.as_u64() == Some(3_900_000))); - - let mut invalid = workload.clone(); - invalid["query_workload"]["repeating_queries"][0]["range_evaluation"]["step_ms"] = - serde_json::json!(0); - std::fs::write(&path, invalid.to_string()).unwrap(); - let (status, stderr) = failure(&["--workload", path.to_str().unwrap()]); - assert_ne!(status, Some(0)); - assert!( - stderr.contains("step_ms must be greater than zero"), - "{stderr}" - ); - std::fs::remove_file(path).unwrap(); -} - -/// `--cost-inputs ` loads Stage 3's settings from JSON: -/// cost model 2 with the illustrative prices selects what -/// `--cost-model provisioned-capacity` selects for Example 4 (P1). The file -/// replaces the cost-model flags, so combining them is refused. -#[test] -#[ignore = "fails on main too: the selection MILP does not support cost models 2 and 3 (UnsupportedCostModel); not a wall-clock issue; #888"] -fn plan_selection_settings_flag_loads_the_settings_from_json() { - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_plan_selection_settings.json", - std::process::id() - )); - let settings = asap_cost::CostInputs { - cost_model: asap_cost::CostModel::ProvisionedCapacity( - asap_cost::ResourceUnitCost::illustrative(), - ), - ..Default::default() - }; - std::fs::write(&path, serde_json::to_string(&settings).unwrap()).unwrap(); - let path_arg = path.to_str().unwrap(); - let document = generate_example("planner-layering-3b", &["--cost-inputs", path_arg]); - assert!(selects_a_plan(&document["stage3_plan_selection"])); - let (code, stderr) = failure(&[ - "--exhaustive", - "--example", - "planner-layering-3b", - "--cost-inputs", - path_arg, - "--cost-model", - "usage-over-time", - ]); - assert_eq!(code, Some(2)); - assert!( - stderr.contains("--cost-inputs replaces --cost-model"), - "{stderr}" - ); - std::fs::remove_file(path).unwrap(); -} - -/// stage_pipeline ignores accuracy guarantees by default: every sketch -/// candidate of an average of p99s, which has no guarantee and which Pass 1 -/// prunes under `--enforce-accuracy-guarantees` -/// (`NoErrorBoundForQueryTarget`), is valid with no flag, accepted -/// regardless of its guarantee, and records why the guarantee check would -/// have rejected it. The document says which mode it -/// planned under; there is no `--ignore-accuracy-guarantees`. -#[test] -fn accuracy_guarantees_are_ignored_by_default_and_enforced_on_request() { - let args = [ - "--exhaustive", - "--promql", - "avg by (g) (quantile_over_time(0.99, x[100s]))", - "--max-error", - "0.01", - "--failure-probability", - "0.01", - "--accuracy-metric", - "rank", - ]; - let mut enforcing = args.to_vec(); - enforcing.push("--enforce-accuracy-guarantees"); - let enforced = document_of(&enforcing); - assert_eq!(enforced["accuracy_guarantees"], "enforced"); - let no_bound: Vec<&str> = enforced["stage1_logical_asap_optimization"]["pruned_candidates"] - .as_array() - .unwrap() - .iter() - .filter(|pruned| pruned["reason"] == "NoErrorBoundForQueryTarget") - .map(|pruned| pruned["candidate"].as_str().unwrap()) - .collect(); - assert!(!no_bound.is_empty(), "{enforced}"); - assert!( - !enforced["reference_pricing"]["pruned_candidates"] - .as_array() - .unwrap() - .iter() - .any(|r| r["cause"] == "no_accuracy_guarantee"), - "{enforced}" - ); - let document = document_of(&args); - assert_eq!(document["accuracy_guarantees"], "ignored"); - let accepted = &document["reference_pricing"]["accepted_accuracy_per_candidate"]; - assert!(accepted.as_object().unwrap().values().any(|queries| { - queries.as_array().unwrap().iter().any(|query| { - query["accepted_regardless_of_guarantee"] == true - && query["guarantee_check_rejection"] == "no_accuracy_guarantee" - }) - })); - // A settings file no longer carries `ignore_accuracy_guarantees`: the - // flag alone decides, and the file refuses the field. - let settings = std::env::temp_dir().join(format!( - "stage_pipeline_{}_ignoring_settings.json", - std::process::id() - )); - std::fs::write(&settings, r#"{"ignore_accuracy_guarantees": true}"#).unwrap(); - let mut from_file = enforcing.clone(); - from_file.extend(["--cost-inputs", settings.to_str().unwrap()]); - let (code, stderr) = failure(&from_file); - assert_eq!(code, Some(2), "{stderr}"); - assert!(stderr.contains("ignore_accuracy_guarantees"), "{stderr}"); - std::fs::remove_file(settings).unwrap(); - let mut ignoring = args.to_vec(); - ignoring.push("--ignore-accuracy-guarantees"); - let (code, stderr) = failure(&ignoring); - assert_eq!(code, Some(2)); - assert!( - stderr.contains("unknown argument --ignore-accuracy-guarantees"), - "{stderr}" - ); -} - -/// The workload sets the viewer lists (`tools/dag-viewer/workloads`): every -/// file the index names parses as a `PlanningWorkload` and validates, every -/// set has a name, description and source, and a workload's precomputed -/// document, when it names one, exists. -#[test] -fn workload_set_index_and_files_validate() { - let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tools/dag-viewer"); - let index: Value = - serde_json::from_str(&std::fs::read_to_string(root.join("workloads/index.json")).unwrap()) - .unwrap(); - assert_eq!(index["format"], "asap-workload-sets/v1"); - let sets = index["sets"].as_array().unwrap(); - assert!(!sets.is_empty()); - for set in sets { - for field in ["id", "name", "description", "source"] { - assert!( - set[field].as_str().is_some_and(|s| !s.is_empty()), - "{field}: {set}" - ); - } - if let Some(capabilities) = set.get("deployment_capabilities") { - serde_json::from_value::( - capabilities.clone(), - ) - .unwrap(); - } - if let Some(mode) = set.get("accuracy_guarantees") { - assert!(mode == "ignored" || mode == "enforced", "{set}"); - } - for entry in set["workloads"].as_array().unwrap() { - let file = root.join("workloads").join(entry["file"].as_str().unwrap()); - let text = std::fs::read_to_string(&file).unwrap(); - let workload: PlanningWorkload = - serde_json::from_str(&text).unwrap_or_else(|e| panic!("{}: {e}", file.display())); - workload - .validate() - .unwrap_or_else(|e| panic!("{}: {e}", file.display())); - if let Some(labels) = entry.get("query_labels") { - assert_eq!( - labels.as_array().unwrap().len(), - workload.query_workload.entries().count(), - "{}", - file.display() - ); - } - if let Some(document) = entry.get("document") { - assert!(root.join(document.as_str().unwrap()).exists(), "{document}"); - } - } - } -} - -/// The committed workload examples (`examples.json` entries with a -/// `workload`) are current: written with the default search (or the -/// entry's `search`) from their workload file, the SQL tables in the -/// `tables.json` next to it, deployment capabilities, cost model and -/// accuracy guarantee mode (`accuracy_guarantees`, ignored when absent). -/// Candidate-search examples validate the selected DAG, query coverage, and -/// materialized-view linkage without depending on a particular timed-search ID. -/// Each document carries its -/// data workload as the file declares it and the selected plan's -/// materialized views as `plan_installable`'s plan derives them, each mapped -/// to a node of the selected Stage 2 candidate. -#[test] -#[ignore = "slow without a planning budget: over 30 min in a debug build; #1197"] -fn workload_examples_preserve_data_and_valid_materialized_views() { - let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tools/dag-viewer"); - let examples: Value = - serde_json::from_str(&std::fs::read_to_string(root.join("examples.json")).unwrap()) - .unwrap(); - let workload_examples: Vec<&Value> = examples - .as_array() - .unwrap() - .iter() - .filter(|e| e.get("workload").is_some()) - .collect(); - assert_eq!(workload_examples.len(), 10); - // Bound fixture concurrency so this acceptance test does not starve its - // own solvers. - for batch in workload_examples.chunks(2) { - std::thread::scope(|scope| { - for example in batch { - let root = &root; - scope.spawn(move || check_workload_example(root, example)); - } - }); - } -} - -fn check_workload_example(root: &std::path::Path, example: &Value) { - let path = root.join(example["workload"].as_str().unwrap()); - let capabilities = example["deployment_capabilities"].to_string(); - let cost_model = example["cost_model"].as_str().unwrap(); - let enforce = match example.get("accuracy_guarantees") { - None => false, - Some(mode) if mode == "ignored" => false, - Some(mode) if mode == "enforced" => true, - Some(mode) => panic!("accuracy_guarantees {mode}"), - }; - let mut args = vec![ - "--workload", - path.to_str().unwrap(), - "--deployment-capabilities", - &capabilities, - "--cost-model", - cost_model, - ]; - if enforce { - args.push("--enforce-accuracy-guarantees"); - } - let tables_path = path.parent().unwrap().join("tables.json"); - let tables: Vec = match tables_path.exists() { - true => serde_json::from_str::>(&std::fs::read_to_string(&tables_path).unwrap()) - .unwrap() - .iter() - .map(Value::to_string) - .collect(), - false => Vec::new(), - }; - for table in &tables { - args.extend(["--table", table.as_str()]); - } - let document = document_of(&args); - let committed: Value = serde_json::from_str( - &std::fs::read_to_string(root.join(example["file"].as_str().unwrap())).unwrap(), - ) - .unwrap(); - assert_eq!( - document["workload"], committed["workload"], - "{} input contract", - example["id"] - ); - - let workload: PlanningWorkload = - serde_json::from_str(&std::fs::read_to_string(&path).unwrap()).unwrap(); - assert_eq!( - document["workload"]["data_workload"], - serde_json::to_value(workload.data_workload.as_ref().unwrap()).unwrap() - ); - let queries = document["workload"]["queries"].as_array().unwrap(); - assert!(queries - .iter() - .all(|q| q["recurrence"]["repeated"].is_object() && q["predictability"].is_object())); - check_candidate_example(document, committed, example); -} - -/// Current candidates may differ from historical snapshots; validate the emitted DAG linkage. -fn check_candidate_example(document: Value, _committed: Value, example: &Value) { - let selection = &document["stage3_plan_selection"]; - assert!( - selects_a_plan(selection), - "{}: no validated selected plan: {selection}", - example["id"] - ); - let candidates = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - let candidate = candidates - .iter() - .find(|candidate| candidate["id"] == SELECTED) - .expect("selected plan is present in the physical archive"); - let dag: PhysicalASAPDAG = serde_json::from_value(candidate["dag"].clone()).unwrap(); - dag.validate().unwrap(); - let queries = document["workload"]["queries"].as_array().unwrap().len(); - assert_eq!(dag.roots.len(), queries); - let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); - assert_eq!(accepted.len(), queries); - if document["accuracy_guarantees"] == "enforced" { - for query in accepted { - assert_eq!(query["accepted_regardless_of_guarantee"], false); - assert!(query["guarantee_check_rejection"].is_null()); - } - } - assert!( - selection.get("materialized_views_error").is_none(), - "{selection}" - ); - let nodes: HashSet<_> = dag.nodes.iter().map(|node| node.id as u64).collect(); - for view in selection["materialized_views"].as_array().unwrap() { - assert!(nodes.contains(&view["stage2_node_id"].as_u64().unwrap())); - assert!( - view["installed_position"].as_u64().unwrap() - < view["installed_count"].as_u64().unwrap() - ); - } - assert!( - selection["search_report"].is_object(), - "{}: {selection}", - example["id"] - ); - let report = &document["planning_report"]; - assert!( - report["physical_candidates"].as_u64().unwrap() > 0, - "{report}" - ); -} - -/// The `--promql` path's data workload is what it plans with: every metric -/// the queries read, sampled every `--sample-interval-ms`. -#[test] -fn promql_queries_carry_their_sampled_data_workload() { - // `document_of` declares 1,000 series and 60,000 rows held per source. - let document = document_of(&[ - "--promql", - "sum(rate(a[1m])) / sum(b)", - "--sample-interval-ms", - "30000", - ]); - assert_eq!( - document["workload"]["data_workload"]["sources"], - serde_json::json!({ - "a": {"series_count": 1000, "metric_type": null, "sample_interval": 30000, "stream_norms": null, "max_distinct_per_evaluation": null, "rows_held": 60000}, - "b": {"series_count": 1000, "metric_type": null, "sample_interval": 30000, "stream_norms": null, "max_distinct_per_evaluation": null, "rows_held": 60000}, - }) - ); - // No materialized view: every query computes at query time. - assert_eq!( - document["stage3_plan_selection"]["materialized_views"], - serde_json::json!([]) - ); -} - -/// Under `--exhaustive` with a display cap, the document shows the selected -/// candidate first, then the other valid ones cheapest first, then the -/// rejected ones, each cause in turn; `pruned_candidate_count_by_cause` counts every -/// rejection. -#[test] -fn display_cap_shows_the_selected_then_valid_cheapest_then_each_cause() { - let document = generate_example("planner-layering-3a", &["--max-displayed", "6"]); - let selection = &document["reference_pricing"]; - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - // Six enumerated plans and the facade's selection. - assert_eq!(physical.len(), 7); - let ids: HashSet<&str> = physical.iter().map(|p| p["id"].as_str().unwrap()).collect(); - assert!(ids.contains(SELECTED)); - assert!(ids.contains(selection["selected_candidate"].as_str().unwrap())); - let counts = selection["pruned_candidate_count_by_cause"] - .as_object() - .unwrap(); - let total: u64 = counts.values().map(|n| n.as_u64().unwrap()).sum(); - // 2,187 candidates: one selected, the rest rejected for a failed check - // or a loss under the cost model. - assert!(total >= physical.len().saturating_sub(1) as u64); - // The shown valid ones (priced) are the cheapest of all valid ones. - let costs = selection["cost_per_candidate"].as_object().unwrap(); - assert!(costs.len() >= 2, "{costs:?}"); -} - -/// With no supported summaries declared, `stage_pipeline` uses the executor's -/// families, as `plan_installable` does. CountSketch cannot answer a total, so -/// it is neither generated for count nor reported as deployment-pruned. -#[test] -fn undeclared_supported_summaries_default_to_the_executors_families() { - let document = document_of(&[ - "--exhaustive", - "--promql", - "count by (job) (m)", - "--max-error", - "0.01", - "--failure-probability", - "0.01", - ]); - let labels: Vec<&str> = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .map(|c| c["display_name"].as_str().unwrap()) - .collect(); - for family in ["Count acc", "Cms", "UnivMon"] { - assert!( - labels.iter().any(|label| label.contains(family)), - "missing {family}: {labels:?}" - ); - } - assert!(labels.iter().all(|label| !label.contains("CountSketch"))); - assert!(pass1_pruned(&document).is_empty()); - assert_eq!( - document["deployment_capabilities"]["supported_summaries"], - serde_json::to_value(asap_executor::supported_summary_families()).unwrap() - ); -} - -/// #840's workload: two `quantile_over_time` queries and two instant -/// `quantile by (G)` queries over 10,000 gauge series, each at 1% rank error. -const QUANTILE_BY_WORKLOAD: &str = r#"{"query_workload": {"language": "promql", "query_batch": null, "repeating_queries": [ - {"query": "quantile_over_time(0.5, data[1m])", "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}}, - {"query": "quantile_over_time(0.9, data[5m])", "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}}, - {"query": "quantile by (host, job) (0.5, data)", "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}}, - {"query": "quantile by (job) (0.5, data)", "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}}]}, - "data_workload": {"arrival": {"ingesting": {"rows_per_second": 666.67}}, - "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}}}"#; - -/// The reference pricer prices the exact baseline of an instant -/// `quantile by (G)` next to the sketches: every enumerated plan but its -/// pick is rejected for its cost or, for a sketch whose guarantee misses -/// the target (DDSketch's relative bound needs non-negative values, which -/// the workload does not declare), for its accuracy; none fails to -/// execute. Accepted plans answer Q3 and Q4 exactly. -#[test] -fn exact_grouped_quantile_plans_compile_and_are_compared() { - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_quantile_by_workload.json", - std::process::id() - )); - std::fs::write(&path, QUANTILE_BY_WORKLOAD).unwrap(); - // Guarantees enforced, so accepted plans passed the guarantee check. - let document = document_of(&[ - "--workload", - path.to_str().unwrap(), - "--enforce-accuracy-guarantees", - "--exhaustive", - "--max-combinations", - "200", - "--no-memo", - ]); - std::fs::remove_file(path).unwrap(); - assert!(selects_a_plan(&document["stage3_plan_selection"])); - let selection = &document["reference_pricing"]; - assert_eq!(document["accuracy_guarantees"], "enforced"); - let by_cause = selection["pruned_candidate_count_by_cause"] - .as_object() - .unwrap(); - assert!( - by_cause.keys().all(|cause| [ - "costlier", - "tied_lost_on_enumeration_order", - "accuracy_target_missed" - ] - .contains(&cause.as_str())), - "{by_cause:?}" - ); - assert!(by_cause.contains_key("costlier"), "{by_cause:?}"); - // Accepted plans passed the guarantee check (the selected one among them). - let accepted = selection["accepted_accuracy_per_candidate"] - .as_object() - .unwrap(); - // Accepted plans include ones that answer Q3 and Q4 without a sketch. - let exact_q3_q4 = accepted - .values() - .filter(|queries| { - let queries = queries.as_array().unwrap(); - queries[2]["sketches"].as_array().unwrap().is_empty() - && queries[3]["sketches"].as_array().unwrap().is_empty() - }) - .count(); - assert!(exact_q3_q4 > 0); -} - -/// Two windowed quantiles over one metric (Q1 over 1 m, Q2 over 5 m, both -/// every 10 s): with a shared scan, Stage 2 also offers the scan built once -/// per evaluation (`query time, shared: Scan`). It reads the 5 m window -/// once (≈0.1364/s) instead of once per reader (0.2203/s), and beats the -/// separate scans (0.1530/s). -#[test] -fn a_scan_two_queries_share_is_offered_built_once_per_evaluation() { - let query = |q: &str| { - serde_json::json!({ - "query": q, - "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}, - }) - }; - let workload = serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [ - query("quantile_over_time(0.5, data[1m])"), - query("quantile_over_time(0.9, data[5m])"), - ], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 666.67}}, - "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}, - }, - }); - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_shared_scan_workload.json", - std::process::id() - )); - std::fs::write(&path, workload.to_string()).unwrap(); - // This checks the production planner's feasible cost, not exhaustive - // archival coverage of all full-width assignments. - let document = document_of(&["--workload", path.to_str().unwrap()]); - std::fs::remove_file(path).unwrap(); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - let selection = &document["stage3_plan_selection"]; - assert!(physical.iter().any(|p| p["id"] == SELECTED)); - let cost = selection["cost"]["cost_per_second"].as_f64().unwrap(); - // The shared-scan feasible price as an upper bound. The KLL for both - // queries (k = 567, sized in Pass 1 for the whole result, ≈0.1364/s) is - // valid, but the search, pricing the KLL at that final size, now - // proposes DDSketch for Q2 first (≈0.1399–0.1421/s within budget). - assert!(cost <= 0.143, "{cost}"); -} - -/// #840's workload: Q1 and Q2 window `data` (1 m and 5 m), Q3 and Q4 read -/// its latest samples by (host, job) and by job. The four scans project -/// different labels. The widening rule offers one scan containing every -/// required label, with column projections above the requested time ranges. -/// This explicit candidate must install and compile, and its shared scan is -/// charged once per evaluation. The optimizer may instead select cheaper -/// partial sharing: a wide five-minute scan can cost more than a narrow long -/// scan plus a wide fifteen-second scan. -/// A scan and a time range that already have every label of their metric -/// are not widened, so no memo group's candidate reads its own group, and -/// Stage 2's candidates, whose producers are its input groups' candidates, -/// read no candidate that reads them back. Before, each such group got an -/// identity `Project` over itself, and Stage 2 wired that candidate (and -/// its execution-scope copies) to itself. -#[test] -fn widened_scans_keep_memo_groups_and_stage2_candidates_acyclic() { - use asap_logical_asap_optimization::DeploymentCapabilities; - let query = |q: &str| { - serde_json::json!({ - "query": q, - "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}, - }) - }; - let workload: PlanningWorkload = serde_json::from_value(serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [ - query("sum by (job, region) (rate(data[5m]))"), - query("sum by (job) (rate(data[5m]))"), - query("sum by (job, region) (data)"), - query("sum by (job) (data)"), - ], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 666.67}}, - "sources": {"data": {"series_count": 10000, "metric_type": "counter", "sample_interval": 15000}}, - }, - })) - .unwrap(); - let entries: Vec<_> = workload.query_workload.entries().collect(); - let deployment = DeploymentCapabilities { - raw_data_retained: true, - ..DeploymentCapabilities::default() - }; - let run = asap_devtools::planning_stages::run_default_search_over( - lower_promql_roots(&workload).unwrap(), - &entries, - workload.data_workload.as_ref().unwrap(), - asap_planner::PlanningModels::builtin().with_deployment_capabilities(&deployment), - ) - .unwrap() - .run; - let memo = &run.logical_asap_candidates.as_ref().unwrap().memo; - assert!(memo - .candidates - .iter() - .flatten() - .any(|candidate| candidate.origin - == asap_types::stages::OriginOfCandidateForMemoGroup::WidenedScans)); - // Whether `to` is reachable from `from` along `edges`. - fn reaches(edges: &dyn Fn(usize) -> Vec, from: usize, to: usize) -> bool { - let mut seen = HashSet::new(); - let mut pending = edges(from); - while let Some(next) = pending.pop() { - if next == to { - return true; - } - if seen.insert(next) { - pending.extend(edges(next)); - } - } - false - } - let group_reads = |group: usize| -> Vec { - memo.candidates[group] - .iter() - .flat_map(|candidate| candidate.input_groups.iter().map(|g| g.0 as usize)) - .collect() - }; - for group in 0..memo.groups.len() { - assert!( - !reaches(&group_reads, group, group), - "memo group {group} reads itself" - ); - } - let physical = run.physical_asap_candidates.as_ref().unwrap(); - let candidate_reads = |candidate: usize| -> Vec { - physical.candidates[candidate] - .producer_candidates_per_input - .iter() - .flat_map(|producers| producers.iter().map(|id| id.0)) - .collect() - }; - for candidate in 0..physical.candidates.len() { - assert!( - !reaches(&candidate_reads, candidate, candidate), - "Stage 2 candidate {candidate} reads itself" - ); - } -} - -#[test] -fn scans_projecting_different_labels_are_shared() { - let query = |q: &str| { - serde_json::json!({ - "query": q, - "demand": {"fixed_interval": 10000}, - "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.001}}, - "predictability": {"predictable": {"known_at": null}}, - }) - }; - let workload = serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [ - query("quantile_over_time(0.5, data[1m])"), - query("quantile_over_time(0.9, data[5m])"), - query("quantile by (host, job) (0.5, data)"), - query("quantile by (job) (0.5, data)"), - ], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 666.67}}, - "sources": {"data": {"series_count": 10000, "metric_type": "gauge", "sample_interval": 15000}}, - }, - }); - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_union_labels_workload.json", - std::process::id() - )); - std::fs::write(&path, workload.to_string()).unwrap(); - let document = document_of(&[ - "--max-displayed", - "100000", - "--workload", - path.to_str().unwrap(), - ]); - std::fs::remove_file(path).unwrap(); - // Inspect an offered widened candidate directly. A bounded optimizer may - // correctly prefer a narrow long scan plus a short wide scan; its archive - // is not an enumeration of every supported sharing configuration. - let workload: PlanningWorkload = serde_json::from_value(workload).unwrap(); - let entries: Vec<_> = workload.query_workload.entries().collect(); - let data = workload.data_workload.as_ref().unwrap(); - let roots = lower_promql_roots(&workload).unwrap(); - use asap_logical_asap_optimization::test_support::with_union_scan_labels; - use asap_types::physical::{Materialization, Medium}; - // Plans `roots` with identical sub-DAGs made one node, as the memo's - // groups are, and a node read by several roots kept for the evaluation. - let plan = |name: &str, roots: &[asap_types::ir::QueryRoot]| { - let shared: Vec<_> = asap_logical_asap_optimization::test_support::share_common_sub_dags( - roots - .iter() - .enumerate() - .map(|(id, root)| match root { - asap_types::ir::QueryRoot::Operator(node) => (id, std::rc::Rc::clone(node)), - asap_types::ir::QueryRoot::Scalar(_) => panic!("operator root"), - }) - .collect(), - ) - .into_iter() - .map(|(_, node)| asap_types::ir::QueryRoot::Operator(node)) - .collect(); - let physical = asap_physical_asap_optimization::optimize_physical_asap_assigned( - name, - &shared, - &entries, - data, - false, - &mut |readers| { - Some(if readers.len() > 1 { - Materialization::MatAtQuery(Medium::Memory) - } else { - Materialization::NotMaterialized - }) - }, - ) - .unwrap() - .expect("a node read by several roots can be kept for the evaluation"); - let offered = asap_devtools::memo_pricing::reference_select( - std::slice::from_ref(&physical), - &entries, - data, - asap_planner::PlanningModels::builtin(), - ) - .unwrap(); - (physical, offered) - }; - // The widened scan: one scan with every label, shared by all four. - let widened = with_union_scan_labels(&roots).unwrap().unwrap(); - let (physical, offered) = plan("widened-scan-rule", &widened); - // The partial-sharing reference: the two windows share a narrow scan, and - // the two instant selectors a scan widened to their labels. - let mut partial = roots[..2].to_vec(); - partial.extend(with_union_scan_labels(&roots[2..]).unwrap().unwrap()); - let (_, reference) = plan("partial-sharing", &partial); - let (installed, _) = asap_planner::installable_plan_of( - &physical.roots, - None, - &offered.tumbling_window_retention_ms, - &entries, - ) - .unwrap(); - installed.validate().unwrap(); - installed.check_compiles().unwrap(); - let dag = &physical.dag; - let scans: Vec<_> = dag - .nodes - .iter() - .filter(|n| n.payload.kind_name() == "Scan") - .collect(); - let [scan] = &scans[..] else { - panic!("one scan: {scans:?}") - }; - let field_names: Vec<&str> = scan - .output_schema - .fields - .iter() - .map(|f| f.name.as_str()) - .collect(); - assert_eq!( - field_names, - ["ts", "value", "host", "job", "$promql_series_identity"] - ); - let cost = &offered.costs[&offered.selected]; - let scan_cost = &cost.per_node[&scan.id]; - assert_eq!(scan_cost.multiplier, 1, "{scan_cost:?}"); - // 10,000 series × 20 samples in 5 m, once per 10 s evaluation. - let detail = &scan_cost.detail; - assert!( - detail.starts_with("scan 200000 samples; x 0.1000/s"), - "{detail}" - ); - let selection = &document["stage3_plan_selection"]; - let shared_cost = cost.cost_per_second.unwrap(); - assert!(shared_cost.is_finite() && shared_cost > 0.0); - let selected_cost = selection["cost"]["cost_per_second"].as_f64().unwrap(); - // Offering maximal sharing must not force a costlier plan, and the - // search must find a plan no dearer than the partial-sharing one (#994). - let reference_cost = reference.costs[&reference.selected] - .cost_per_second - .unwrap(); - assert!( - selected_cost <= shared_cost, - "{selected_cost} > {shared_cost}" - ); - assert!( - selected_cost <= reference_cost * (1.0 + 1e-9), - "{selected_cost} > partial sharing {reference_cost}" - ); -} - -/// The default search is the candidate-connection search; its report has -/// no legacy route. -#[test] -fn default_search_reports_candidate_connections() { - let document = document_of(&["--promql", "sum(m)"]); - assert_eq!(document["search"], "candidate-connections"); - let report = &document["stage3_plan_selection"]["search_report"]; - assert!(report.is_object(), "{report}"); - assert!(report.get("routed_to_legacy").is_none()); - assert!(report.get("algorithm").is_none(), "{report}"); -} - -/// There is one search: `--search` is not an option, so no engine name, -/// current or retired, is accepted. -#[test] -fn search_flag_is_rejected() { - for engine in ["candidate-connections", "legacy", "memo-milp"] { - let (status, stderr) = failure(&["--promql", "sum(m)", "--search", engine]); - assert_eq!(status, Some(2)); - assert!(stderr.contains("unknown argument --search"), "{stderr}"); - } -} - -/// The memo search's document names its selected plan's logical candidate -/// under the id its physical candidates read (`from_logical`), as the -/// viewer's validator requires: the brute force numbers its logical plans, -/// so the selected one need not be the first. -#[test] -fn the_memo_search_document_names_the_selected_logical_candidate() { - let document = document_of(&[ - "--promql", - "quantile_over_time(0.5, data[5m])", - "--promql", - "quantile_over_time(0.9, data[5m])", - "--sample-interval-ms", - "15000", - ]); - assert_eq!(document["search"], "candidate-connections"); - let logical: HashSet<&str> = document["stage1_logical_asap_optimization"]["candidates"] - .as_array() - .unwrap() - .iter() - .map(|c| c["id"].as_str().unwrap()) - .collect(); - let physical = document["stage2_physical_asap_optimization"]["candidates"] - .as_array() - .unwrap(); - assert!(!physical.is_empty()); - for candidate in physical { - let from = candidate["from_logical"].as_str().unwrap(); - assert!(logical.contains(from), "{from} is not in {logical:?}"); - } - assert!(selects_a_plan(&document["stage3_plan_selection"])); - assert!(physical.iter().any(|c| c["id"] == SELECTED)); -} - -/// Each shown candidate's `per_node` entries carry the node's operator, -/// timing and predicted CPU, as `plan_installable`'s predicted usage names -/// them, and the candidate's `cpu_core_seconds_per_second` is their sum: -/// enough to compare a plan's predicted CPU with a runtime's measurement -/// without `plan_installable`. Report-only: prices are unchanged. -#[test] -fn per_node_costs_show_operator_timing_and_predicted_cpu() { - // The selected plan supplies the serialization evidence; exhaustive - // candidate coverage is tested separately by the archive tests. - let document = generate_bounded_example("planner-layering-1", &[]); - let selection = &document["stage3_plan_selection"]; - assert!( - selects_a_plan(selection), - "missing evaluated costs: {selection}" - ); - let mut timings = HashSet::new(); - for (id, cost) in [(SELECTED, &selection["cost"])] { - let per_node = cost["per_node"].as_object().unwrap(); - let mut sum = 0.0; - for node in per_node.values() { - assert!(node["operator"].is_string(), "{id}: {node}"); - timings.insert(node["timing"].as_str().unwrap().to_owned()); - let cpu = node["cpu_core_seconds_per_second"].as_f64().unwrap(); - assert!(cpu >= 0.0, "{id}: {node}"); - sum += cpu; - } - let total = cost["cpu_core_seconds_per_second"].as_f64().unwrap(); - assert!(total > 0.0, "{id}"); - assert!( - (total - sum).abs() <= 1e-9 * total, - "{id}: {total} vs {sum}" - ); - } - assert!( - timings.is_subset(&HashSet::from([ - "ingestion_time".to_owned(), - "query_time".to_owned() - ])), - "{timings:?}" - ); -} - -/// `rate`, `increase` and `irate` over a source the workload declares a -/// gauge still plan, with a warning under `stage3_plan_selection.warnings` -/// (which the viewer shows) naming the query, the function and the metric. -/// A counter gets none. -#[test] -fn a_counter_function_over_a_declared_gauge_plans_with_a_warning() { - let workload = |metric_type: &str| { - serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [ - { - "query": "sum by (job) (rate(m[5m]))", - "demand": {"fixed_interval": 60000}, - "requirements": {}, - "predictability": {"predictable": {"known_at": null}}, - }, - { - "query": "max_over_time(m[5m])", - "demand": {"fixed_interval": 60000}, - "requirements": {}, - "predictability": {"predictable": {"known_at": null}}, - }, - ], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 1000.0}}, - "sources": {"m": {"series_count": 1000, "metric_type": metric_type, - "sample_interval": 15000}}, - }, - }) - }; - let path = std::env::temp_dir().join(format!( - "stage_pipeline_{}_gauge_rate_workload.json", - std::process::id() - )); - std::fs::write(&path, workload("gauge").to_string()).unwrap(); - let gauge = document_of(&["--workload", path.to_str().unwrap()]); - std::fs::write(&path, workload("counter").to_string()).unwrap(); - let counter = document_of(&["--workload", path.to_str().unwrap()]); - std::fs::remove_file(path).unwrap(); - assert!(selects_a_plan(&gauge["stage3_plan_selection"])); - assert_eq!( - gauge["stage3_plan_selection"]["warnings"], - serde_json::json!([ - "q1: rate over m, which the data workload declares a gauge: rate assumes a \ - counter (a drop is a reset); planned as written" - ]) - ); - assert!(selects_a_plan(&counter["stage3_plan_selection"])); - assert!(counter["stage3_plan_selection"].get("warnings").is_none()); -} - -/// A PromQL file and a SQL file, each a `PlanningWorkload` in its own -/// language. -fn mixed_language_workload_files(sql_arrival: Value, sql_facts: Value) -> (PathBuf, PathBuf) { - let entry = |query: &str| { - serde_json::json!({ - "query": query, - "demand": {"fixed_interval": 60000}, - "requirements": {}, - "predictability": {"predictable": {"known_at": null}}, - }) - }; - let promql = serde_json::json!({ - "query_workload": { - "language": "promql", - "query_batch": null, - "repeating_queries": [entry("sum by (job) (rate(http_requests_total[1m]))")], - }, - "data_workload": { - "arrival": {"ingesting": {"rows_per_second": 1000.0}}, - "sources": {"http_requests_total": {"series_count": 15000, "metric_type": "counter", "sample_interval": 15000}}, - }, - }); - let sql = serde_json::json!({ - "query_workload": { - "language": {"sql": "datafusion_sql"}, - "query_batch": null, - "repeating_queries": [entry( - "SELECT service, COUNT(*) FROM flows \ - WHERE ts >= NOW() - INTERVAL '1 minute' GROUP BY service" - )], - }, - "data_workload": {"arrival": sql_arrival, "sources": {"flows": sql_facts}}, - }); - // Tests run as threads of one process, so the names need a per-call - // counter. - static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); - let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); - let path = |name: &str| { - std::env::temp_dir().join(format!( - "stage_pipeline_{}_{call}_mixed_{name}.json", - std::process::id() - )) - }; - let (promql_path, sql_path) = (path("promql"), path("sql")); - std::fs::write(&promql_path, promql.to_string()).unwrap(); - std::fs::write(&sql_path, sql.to_string()).unwrap(); - (promql_path, sql_path) -} - -/// `flows(ts, service)`. The time index is declared non-null: the executor -/// builds no count over a nullable time column, so a plan of the SQL query -/// over it would not compile. -const FLOWS_TABLE: &str = r#"{"name": "flows", "columns": [{"name": "ts", "type": "timestamp", "nullable": false}, {"name": "service", "type": "utf8", "nullable": false}], "time_index": 0}"#; - -/// One run can plan PromQL and SQL queries together: `--workload` repeats, -/// each file is lowered by its own language's frontend (SQL over the -/// `--table` declarations), and their queries are planned as one workload. -/// The document numbers the queries across files (q1 from the PromQL -/// file, q2 from the SQL file) and its data workload is the union of the -/// files' sources, with their arrival rates added (1000 + 500 rows/s). -#[test] -fn several_workload_files_in_different_languages_are_planned_together() { - let (promql, sql) = mixed_language_workload_files( - serde_json::json!({"ingesting": {"rows_per_second": 500.0}}), - serde_json::json!({"series_count": 10, "sample_interval": 1000}), - ); - let document = document_of(&[ - "--workload", - promql.to_str().unwrap(), - "--workload", - sql.to_str().unwrap(), - "--table", - FLOWS_TABLE, - ]); - let queries = document["workload"]["queries"].as_array().unwrap(); - let ids_and_languages: Vec<(&str, &str)> = queries - .iter() - .map(|q| (q["id"].as_str().unwrap(), q["language"].as_str().unwrap())) - .collect(); - assert_eq!(ids_and_languages, [("q1", "promql"), ("q2", "sql")]); - let data = &document["workload"]["data_workload"]; - assert_eq!( - data["arrival"], - serde_json::json!({"ingesting": {"rows_per_second": 1500.0}}) - ); - let sources: Vec<&String> = data["sources"].as_object().unwrap().keys().collect(); - assert_eq!(sources, ["flows", "http_requests_total"]); - assert_eq!( - document["stage0_query_language_frontends"]["dag"]["roots"] - .as_array() - .unwrap() - .len(), - 2 - ); - assert!( - selects_a_plan(&document["stage3_plan_selection"]), - "{}", - document["stage3_plan_selection"] - ); - std::fs::remove_file(promql).unwrap(); - std::fs::remove_file(sql).unwrap(); -} - -/// Files planned together must agree on how the data arrives and on any -/// source they both declare: a run is refused rather than picking one. -/// While data arrives, each source must also declare its own rate: plan -/// selection shares a workload's `rows_per_second` out over all its -/// sources, so `flows` (10 groups, no sample interval) would be priced at -/// 1500 × 10 / 15010 ≈ 1 row/s instead of its file's 500. -#[test] -fn workload_files_that_disagree_on_the_data_are_refused() { - let (promql, sql) = mixed_language_workload_files( - serde_json::json!({"ingesting": {"rows_per_second": 500.0}}), - serde_json::json!({"series_count": 10}), - ); - let (_, stderr) = failure(&[ - "--workload", - promql.to_str().unwrap(), - "--workload", - sql.to_str().unwrap(), - "--table", - FLOWS_TABLE, - ]); - assert!( - stderr.contains("source flows declares no rate of its own"), - "{stderr}" - ); - std::fs::remove_file(&sql).unwrap(); - - // A table the SQL file reads but does not declare has no rate either. - let (_, sql) = mixed_language_workload_files( - serde_json::json!({"ingesting": {"rows_per_second": 500.0}}), - serde_json::json!({"series_count": 10, "sample_interval": 1000}), - ); - let mut text: Value = serde_json::from_str(&std::fs::read_to_string(&sql).unwrap()).unwrap(); - text["data_workload"]["sources"] = serde_json::json!({}); - std::fs::write(&sql, text.to_string()).unwrap(); - let (_, stderr) = failure(&[ - "--workload", - promql.to_str().unwrap(), - "--workload", - sql.to_str().unwrap(), - "--table", - FLOWS_TABLE, - ]); - assert!( - stderr.contains("source flows declares no rate of its own"), - "{stderr}" - ); - - let (promql, sql) = mixed_language_workload_files( - serde_json::json!("at_rest"), - serde_json::json!({"rows_held": 1000}), - ); - let (_, stderr) = failure(&[ - "--workload", - promql.to_str().unwrap(), - "--workload", - sql.to_str().unwrap(), - "--table", - FLOWS_TABLE, - ]); - assert!(stderr.contains("arrival"), "{stderr}"); - std::fs::remove_file(&sql).unwrap(); - - // The same source declared twice with different facts. - let other = std::env::temp_dir().join(format!( - "stage_pipeline_{}_mixed_promql_other.json", - std::process::id() - )); - let mut text: Value = serde_json::from_str(&std::fs::read_to_string(&promql).unwrap()).unwrap(); - text["data_workload"]["sources"]["http_requests_total"]["series_count"] = serde_json::json!(7); - std::fs::write(&other, text.to_string()).unwrap(); - let (_, stderr) = failure(&[ - "--workload", - promql.to_str().unwrap(), - "--workload", - other.to_str().unwrap(), - ]); - assert!(stderr.contains("http_requests_total"), "{stderr}"); - std::fs::remove_file(promql).unwrap(); - std::fs::remove_file(other).unwrap(); -} - -/// Issue #890: a classic-bucket `histogram_quantile` with exact accuracy -/// enforced gets a selected plan the executor compiles, as its bucket rates -/// alone (the control) already did. -#[test] -fn a_classic_histogram_quantile_gets_a_selected_plan() { - let buckets = r#"sum by (le) (rate(http_request_duration_seconds_bucket{service="customer-orders-service"}[5m]))"#; - for query in [ - format!("histogram_quantile(0.5, {buckets})"), - buckets.to_string(), - ] { - let document = document_of(&[ - "--promql", - &query, - "--sample-interval-ms", - "15000", - "--enforce-accuracy-guarantees", - ]); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{query}: {selection}"); - } -} - -/// The issue's example workload (#1003): one ratio of per-series quantiles, -/// every minute, ε = 0.01, δ = 0.001. -const MEMO_RATIO_OF_QUANTILES: &str = - "../../tools/dag-viewer/memo-examples/promql/ratio-of-quantiles-5m.json"; - -/// The `memo` section: the memo after each step, the selected choice and -/// the enumeration, on the issue's example. -#[test] -fn memo_section_lists_each_step_the_selected_choice_and_the_enumeration() { - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_memo_ratio.json", - std::process::id() - )); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(["--workload", MEMO_RATIO_OF_QUANTILES, "--out"]) - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document: Value = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - let memo = &document["memo"]; - let steps = memo["steps"].as_array().unwrap(); - let names: Vec<_> = steps.iter().map(|s| s["step"].as_str().unwrap()).collect(); - assert_eq!(names, ["stage0", "pass1", "pass2"]); - let candidates = |step: &Value| -> u64 { - step["groups"] - .as_array() - .unwrap() - .iter() - .map(|g| g["candidate_count"].as_u64().unwrap()) - .sum() - }; - // The PromQL frontend gives each series its identity as a column, so the - // two `quantile_over_time(0.5, data[5m])` have a unique key and share one - // group: 7 groups (scan, range, two quantile groups, ratio, sum, avg), - // 5 + 5 + 5 Pass 1 candidates (each quantile group: exact, DDSketch and - // a KLL sized for the whole result for the whole budget, DDSketch and a - // KLL for half of it), 5 × 5 candidate DAGs. - assert_eq!(steps[0]["groups"].as_array().unwrap().len(), 7); - assert_eq!(candidates(&steps[0]), 0); - assert_eq!(steps[1]["groups"].as_array().unwrap().len(), 7); - assert_eq!(candidates(&steps[1]), 15); - let per_step = &memo["enumeration"]["per_step"]; - assert_eq!( - per_step["stage0"], - serde_json::json!({"count": 1, "complete": true}) - ); - assert_eq!( - per_step["pass1"], - serde_json::json!({"count": 25, "complete": true}) - ); - // The listed step is the last one enumerated completely within 200 - // choices (Stage 0 has no candidates to choose), else the last step. - let listed = ["pass1", "pass2"] - .into_iter() - .rfind(|step| { - per_step[step]["complete"] == true && per_step[step]["count"].as_u64().unwrap() <= 200 - }) - .unwrap_or("pass2"); - assert_eq!(memo["enumeration"]["step"], listed); - if listed != "pass2" { - assert_eq!( - memo["enumeration"]["choices"].as_array().unwrap().len() as u64, - per_step[listed]["count"].as_u64().unwrap() - ); - } - // The node table is a valid FlatDag (children before parents) holding - // each node once; every group's and listed candidate's `node` names a - // node of it, and each boundary names a node and one of the candidate's - // child groups. - let nodes = memo["nodes"].as_array().unwrap(); - validated_dag(&serde_json::json!({ "nodes": nodes, "roots": [] }), 0); - let distinct: HashSet = nodes.iter().map(Value::to_string).collect(); - assert_eq!(distinct.len(), nodes.len()); - let in_table = |id: &Value| (id.as_u64().unwrap() as usize) < nodes.len(); - for step in steps { - for group in step["groups"].as_array().unwrap() { - assert!(in_table(&group["node"]) && group.get("dag").is_none()); - for candidate in group["candidates"].as_array().unwrap() { - assert!(in_table(&candidate["node"]) && candidate.get("dag").is_none()); - let children = candidate["children"].as_array().unwrap(); - for (node, child) in candidate["group_of_node"].as_object().unwrap() { - assert!(node.parse::().unwrap() < nodes.len()); - assert!(children.contains(child)); - } - } - } - } - // Every candidate of the selected choice exists in the last step. - let last = steps.last().unwrap()["groups"].as_array().unwrap(); - let choice = memo["selected_choice"].as_object().unwrap(); - assert!(!choice.is_empty()); - for (group, index) in choice { - let group = &last[group.parse::().unwrap()]; - assert!(index.as_u64().unwrap() < group["candidate_count"].as_u64().unwrap()); - } - let enumeration = &memo["enumeration"]; - let choices = enumeration["choices"].as_array().unwrap(); - assert!(!choices.is_empty() && choices.len() <= 200); - for choice in choices { - let roots = choice["roots"].as_array().unwrap(); - assert!(!roots.is_empty() && roots.iter().all(in_table)); - // Pricing is off without `--price-enumerated`. - assert!(choice.get("valid").is_none()); - } - assert_eq!(enumeration["priced"], Value::Null); - // `--no-memo` leaves the section out. - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_no_memo_ratio.json", - std::process::id() - )); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(["--workload", MEMO_RATIO_OF_QUANTILES, "--no-memo", "--out"]) - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document: Value = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - assert!(document.get("memo").is_none()); -} - -/// `stage_pipeline`'s document for `args`, run with Stage 3's group -/// selection MILP (#1095). -fn document_with_milp_search(args: &[&str]) -> Value { - static CALLS: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0); - let call = CALLS.fetch_add(1, std::sync::atomic::Ordering::Relaxed); - let out = std::env::temp_dir().join(format!( - "stage_pipeline_{}_milp_{call}.json", - std::process::id(), - )); - let status = Command::new(env!("CARGO_BIN_EXE_stage_pipeline")) - .args(args) - .arg("--out") - .arg(&out) - .status() - .unwrap(); - assert!(status.success()); - let document = serde_json::from_str(&std::fs::read_to_string(&out).unwrap()).unwrap(); - std::fs::remove_file(out).unwrap(); - document -} - -/// #1095 C5: with the group selection MILP, example 1 and the RangeHeavy T5 -/// q0.5 + q0.9 workload select a plan through the real pipeline, solved to -/// optimality, whose `plan_cost` equals the MILP's objective exactly (the -/// search errors on a difference and reports `inexact_objective` when the -/// model encoded a term inexactly) and which passes every query's accuracy -/// check. -#[test] -fn milp_search_objective_equals_plan_cost_on_example1_and_rangeheavy() { - let rangeheavy = PathBuf::from(env!("CARGO_MANIFEST_DIR")) - .join("../../tools/dag-viewer/memo-examples/promql/rangeheavy-t5-q05-q09-1h.json"); - let runs: [Vec<&str>; 2] = [ - vec![ - "--example", - "planner-layering-1", - "--enforce-accuracy-guarantees", - ], - vec![ - "--workload", - rangeheavy.to_str().unwrap(), - "--deployment-capabilities", - r#"{"raw_data_retained": true, "query_time_results_retained": false}"#, - "--cost-model", - "aws-fargate-us-east-1", - "--enforce-accuracy-guarantees", - ], - ]; - for args in runs { - let document = document_with_milp_search(&args); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{args:?}: {selection}"); - let report = &selection["search_report"]; - // The MILP runs to proven optimality, and the selected plan's - // `plan_cost` equals its objective: the search errors on a - // difference, and no term is encoded inexactly here. - assert_eq!(report["status"], "optimal", "{args:?}: {report}"); - assert!( - report.get("inexact_objective").is_none(), - "{args:?}: {report}" - ); - assert!(report["milp_variables"].as_u64().unwrap() > 0); - let queries = document["workload"]["queries"].as_array().unwrap().len(); - let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); - assert_eq!(accepted.len(), queries); - for query in accepted { - assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); - } - } -} - -/// #1127 regressions: with the group selection MILP and guarantees enforced, the -/// memo examples on which the A1 rows admitted a plan that `satisfies` -/// rejects now select a plan that passes every query's accuracy check: -/// ratio-of-quantiles (KLL rank errors compose no guarantee through the -/// quotient, so no query can use KLL) and rollupheavy-t11 (HLL's -/// failure probability is 1 over the declared 10^6 distinct users, above -/// its contract's 2^17, so Pass 1 prunes it for both queries). -#[test] -fn milp_search_selects_checked_plans_on_ratio_and_rollupheavy() { - let root = - PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tools/dag-viewer/memo-examples"); - let tables: Vec = serde_json::from_str::>( - &std::fs::read_to_string(root.join("sql/tables.json")).unwrap(), - ) - .unwrap() - .iter() - .map(Value::to_string) - .collect(); - for (workload, pruned_family) in [ - ("promql/ratio-of-quantiles-5m.json", "Kll"), - ("sql/rollupheavy-t11-service-endpoint-5m.json", "Hll"), - ] { - let path = root.join(workload); - let mut args = vec![ - "--workload", - path.to_str().unwrap(), - "--deployment-capabilities", - r#"{"raw_data_retained": true, "query_time_results_retained": false}"#, - "--cost-model", - "aws-fargate-us-east-1", - "--enforce-accuracy-guarantees", - ]; - if workload.starts_with("sql/") { - for table in &tables { - args.extend(["--table", table.as_str()]); - } - } - let document = document_with_milp_search(&args); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{workload}: {selection}"); - let queries = document["workload"]["queries"].as_array().unwrap().len(); - let accepted = selection["accepted_accuracy_per_query"].as_array().unwrap(); - assert_eq!(accepted.len(), queries); - for query in accepted { - assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); - assert!( - !query["sketches"] - .as_array() - .unwrap() - .iter() - .any(|sketch| sketch == pruned_family), - "{workload}: {query}" - ); - } - let pruned = pass1_pruned(&document); - assert!( - pruned.iter().any(|(family, reason)| family == pruned_family - && reason == "NoErrorBoundForQueryTarget"), - "{workload}: {pruned:?}" - ); - } -} - -/// #1095 C5: on RangeHeavy T8+T5 (`q0.9 / q0.5` with both operands also -/// queried alone), the group selection MILP selects a DDSketch plan that passes every -/// query's accuracy check (the A1 rows admitted KLL there, which the check -/// rejects). The weights -/// that let one chain serve all three queries are checked in -/// `groups::tests::one_pane_chain_serves_a_ratio_and_its_operands`. -#[test] -#[ignore = "slow without a planning budget: over 30 min in a debug build; #1197"] -fn milp_search_selects_a_checked_ddsketch_plan_for_a_ratio_and_its_operands() { - let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")) - .join("../../tools/dag-viewer/memo-examples/promql/rangeheavy-t8-t5-1h.json"); - let document = document_with_milp_search(&[ - "--workload", - path.to_str().unwrap(), - "--deployment-capabilities", - r#"{"raw_data_retained": true, "query_time_results_retained": false}"#, - "--cost-model", - "aws-fargate-us-east-1", - "--enforce-accuracy-guarantees", - ]); - let selection = &document["stage3_plan_selection"]; - assert!(selects_a_plan(selection), "{selection}"); - for query in selection["accepted_accuracy_per_query"].as_array().unwrap() { - assert_eq!(query["accepted_regardless_of_guarantee"], false, "{query}"); - assert_eq!( - query["sketches"], - serde_json::json!(["DDSketch"]), - "{query}" - ); - } -} diff --git a/crates/integration-tests/tests/memo_enumeration.rs b/crates/integration-tests/tests/memo_enumeration.rs index 83df2cf2d..6566b08e1 100644 --- a/crates/integration-tests/tests/memo_enumeration.rs +++ b/crates/integration-tests/tests/memo_enumeration.rs @@ -77,7 +77,7 @@ fn with_queries(queries: &[&str], interval_ms: u32) -> PlanningWorkload { } /// The frontend's roots, each with the series-identity column, as -/// `Stage1ToStage3Pass` and `stage_pipeline` lower a PromQL workload. +/// `Stage1ToStage3Pass` and `asap_planner` lower a PromQL workload. fn production_roots(workload: &PlanningWorkload) -> Vec { lower_promql_roots(workload).unwrap() } diff --git a/crates/integration-tests/tests/planner_layering_common/mod.rs b/crates/integration-tests/tests/planner_layering_common/mod.rs index 6227dfea7..47357ac32 100644 --- a/crates/integration-tests/tests/planner_layering_common/mod.rs +++ b/crates/integration-tests/tests/planner_layering_common/mod.rs @@ -4,7 +4,7 @@ //! //! [`run_stages`] runs the real stages through //! `asap_devtools::planning_stages::run_planning_stages`, the pipeline -//! `stage_pipeline` also runs; the example workloads come from +//! `asap_devtools` provides; the example workloads come from //! `asap_devtools::planner_layering_examples`. The functions under "Pending adapters" read properties the IR //! cannot express yet (sliding and EH window summaries) or derive what it //! does not record (retention). They return today's diff --git a/crates/integration-tests/tests/sql_two_percentiles.rs b/crates/integration-tests/tests/sql_two_percentiles.rs index b868d1ed0..6ae9ff645 100644 --- a/crates/integration-tests/tests/sql_two_percentiles.rs +++ b/crates/integration-tests/tests/sql_two_percentiles.rs @@ -82,8 +82,8 @@ async fn run(queries: &[&str]) -> Run { .unwrap_or_else(|e| panic!("{}: {e}", entry.query.0)); roots.push(QueryRoot::Operator(root)); } - // Every output accepted whatever its guarantee, as `stage_pipeline` - // plans by default: this test checks the answers against the bound. + // Every output accepted whatever its guarantee: this test checks the + // answers against the bound. let settings = CostInputs { ..CostInputs::default() }; diff --git a/crates/logical-asap-optimization/src/deployment_capabilities.rs b/crates/logical-asap-optimization/src/deployment_capabilities.rs index 945d72cb3..f449513b6 100644 --- a/crates/logical-asap-optimization/src/deployment_capabilities.rs +++ b/crates/logical-asap-optimization/src/deployment_capabilities.rs @@ -41,7 +41,7 @@ use thiserror::Error; /// What the deployment's executor can run and keep. Planning reads it in /// Stage 1 (`supported_summaries`), Stage 2 (`query_time_results_retained`) -/// and Stage 3 (`raw_data_retained`). It is data only; `stage_pipeline` reads it from +/// and Stage 3 (`raw_data_retained`). It is data only; `asap_planner` reads it from /// `--deployment-capabilities ` and shows it in the stage document. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(default, deny_unknown_fields)] diff --git a/crates/logical-asap-optimization/src/lib.rs b/crates/logical-asap-optimization/src/lib.rs index 213790d6d..3864772dc 100644 --- a/crates/logical-asap-optimization/src/lib.rs +++ b/crates/logical-asap-optimization/src/lib.rs @@ -89,7 +89,10 @@ pub use pass1::{ }; // Pass 2: its entry and its error. pub use pass1::memo_population::CandidatePopulationError; -pub use pass2::logical_memo_candidates::generate_logical_asap_memo_candidates; +pub use pass2::logical_memo_candidates::{ + generate_logical_asap_memo_candidates, pass2_after_local_window_composition, + pass2_local_window_composition, +}; pub use pass2::memo::state_groups::Pass2RuleError; // The CandidateLogicalASAPDAGs a memo stands for (developer tools' // reference pricing). diff --git a/crates/logical-asap-optimization/src/pass2/logical_memo_candidates.rs b/crates/logical-asap-optimization/src/pass2/logical_memo_candidates.rs index 834b451a6..678ba5c0d 100644 --- a/crates/logical-asap-optimization/src/pass2/logical_memo_candidates.rs +++ b/crates/logical-asap-optimization/src/pass2/logical_memo_candidates.rs @@ -22,9 +22,26 @@ pub fn generate_logical_asap_memo_candidates( pass1: PerComputationLogicalMemoCandidates, entries: &[QueryWorkloadEntry], ) -> Result { + pass2_after_local_window_composition(pass2_local_window_composition(pass1, entries)) +} + +/// Pass 2's rule 1 alone: Pass 1's memo with the local tumbling-window +/// candidates for every legal width (`asap_planner` prints this step). +pub fn pass2_local_window_composition( + pass1: PerComputationLogicalMemoCandidates, + entries: &[QueryWorkloadEntry], +) -> Memo { let mut memo = pass1.memo; // 1. Local window composition. add_local_window_composition(&mut memo, entries); + memo +} + +/// The rest of Pass 2 on [`pass2_local_window_composition`]'s memo: rules +/// 2–9 and `add_shared_summary_state_groups`. +pub fn pass2_after_local_window_composition( + mut memo: Memo, +) -> Result { apply_sharing_rules(&mut memo)?; // 10. One group per shared summary build, read by its reader groups. add_shared_summary_state_groups(&mut memo)?; diff --git a/crates/plan-selection/src/lib.rs b/crates/plan-selection/src/lib.rs index b0d89d35b..77100f7df 100644 --- a/crates/plan-selection/src/lib.rs +++ b/crates/plan-selection/src/lib.rs @@ -239,7 +239,7 @@ //! No memory limit applies outside cost model 3, where the cluster's memory //! is declared ([`ClusterLimits::memory_bytes`](asap_cost::ClusterLimits::memory_bytes)); without one, a selected //! plan whose peak memory is over `LARGE_PEAK_MEMORY_BYTES` (10^12) gets a -//! warning, which `stage_pipeline` reports ([`large_peak_memory_warning`]). +//! warning, which `asap_planner` reports ([`large_peak_memory_warning`]). #[cfg(test)] use asap_types::workload::RepeatedDemand; @@ -270,8 +270,8 @@ pub(crate) use selector::select_among_plans; /// The whole-workload reference pricer (feature `reference-pricer`): the /// selector Stage 3 used before the memo search, over whole physical plans -/// a developer tool enumerates (`stage_pipeline --exhaustive` and -/// `--price-enumerated`, `planner_fuzz`). Not part of Stage 3's interface. +/// a developer tool enumerates (`asap_devtools::memo_pricing`, +/// `planner_fuzz`). Not part of Stage 3's interface. #[cfg(feature = "reference-pricer")] pub mod reference_pricer { pub use crate::report::{Rejection, WholePlanSelection}; diff --git a/crates/planner/src/run.rs b/crates/planner/src/run.rs index ec86b39a4..5069ca1de 100644 --- a/crates/planner/src/run.rs +++ b/crates/planner/src/run.rs @@ -37,7 +37,7 @@ use crate::pass::{ /// How the facade searches. #[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] pub struct PlanSearchSettings { - /// Keep Pass 1's output for inspection (`stage_pipeline`, the DAG + /// Keep Pass 1's output for inspection (`asap_planner`, the DAG /// viewer): [`PlanningRun::per_computation_candidates`]. Default off: /// production planning copies no memo. pub inspect: bool, @@ -63,6 +63,9 @@ pub enum SelectionError { /// Stage 2 could not build the plan Stage 3 selected. #[error("building the selected plan: {0}")] Realization(PhysicalCandidateError), + /// The run stopped before plan selection (`asap_planner --run-until`). + #[error("plan selection did not run")] + NotRun, } /// What the facade did: each stage's counts and time. diff --git a/docs/design_docs/architecture/README.md b/docs/design_docs/architecture/README.md index 353169d4a..f714afc1f 100644 --- a/docs/design_docs/architecture/README.md +++ b/docs/design_docs/architecture/README.md @@ -50,7 +50,7 @@ target. | Plan selection | `plan-selection` | Stage 3: accuracy and latency checks, costing and selection; the planning models and the Stage 1 to Stage 3 driver | | Planner facade | `planner` | `e2e_plan`: Stage 0 followed by Stages 1–3 | | Executor | `executor` | Stage 4: runs a `PhysicalASAPDAG` over `asap_sketchlib` summaries; no stage crate depends on it | -| Developer inspection | `devtools` | `stage_pipeline` and the Stage Viewer documents, corpus coverage tools | +| Developer inspection | `devtools` | `asap_planner` and the Stage Viewer documents, corpus coverage tools | | End-to-end validation | `integration-tests` | Verify behavior across frontends, stages and execution | ## Inputs and outputs diff --git a/docs/design_docs/architecture/input-output-workflow.md b/docs/design_docs/architecture/input-output-workflow.md index 0e62400dc..400b73675 100644 --- a/docs/design_docs/architecture/input-output-workflow.md +++ b/docs/design_docs/architecture/input-output-workflow.md @@ -116,7 +116,7 @@ struct QueryWorkload { | Field | Required | Purpose | |---|---:|---| -| `language` | Yes | Source language shared by every entry: PromQL, a SQL dialect, DataFusion, or Elastic DSL. It selects the frontend. A workload in several languages is several `PlanningWorkload`s, one per language, which `stage_pipeline` plans together (repeated `--workload`). | +| `language` | Yes | Source language shared by every entry: PromQL, a SQL dialect, DataFusion, or Elastic DSL. It selects the frontend. A workload in several languages is several `PlanningWorkload`s, one per language, which `asap_planner` plans together (repeated `--workload`). | | `query_batch` | Optional | Finite one-time query entries. `None` means there is no batch portion. | | `repeating_queries` | Optional | Recurrent query entries. `None` means there is no repeating portion. | @@ -215,7 +215,7 @@ struct LabelSetDistinctValues { | `arrival` | Present; may be `Unknown` | Distinguishes data at rest, ingesting data, and mixed data. Ingestion-time builds are possible only for ingesting or mixed data. | | `rows_per_second` | With `Ingesting`/`Mixed` | Rows arriving per second over all sources, used to price scans and continuous maintenance. It must be finite and non-negative. A source with no `series_count` + `sample_interval` of its own is priced at its series share of it when every source declares `series_count`, else at all of it. | | `series_count` | Optional | Series in the source. Plan selection bounds a query's number of values `G`, and its input row count, by the series counts of the sources it reads (each branch of an `or` counted); unknown, they stay unbound. | -| `metric_type` | Optional | Declared Prometheus type (`PrometheusMetricType`): `counter`, `gauge`, `histogram`, `native_histogram` or `summary`; a counter proves samples non-negative. Never inferred from a name. A `histogram` or `summary` is keyed by its family name `foo` and supplies its series (`DataWorkload::series_facts`): `foo_bucket` (histogram) or `foo` (summary, per-series quantiles read as values), and the counters `foo_sum` and `foo_count`; any other name is looked up as is. The family entry's `metric_type`, `values_non_negative` and `sample_interval` apply to its series; counts, norms and bounds are read per series name. `histogram_quantile` reads float input as classic buckets, declared `histogram` or undeclared, and rejects a `native_histogram` or `summary`; any query reading a `native_histogram` is rejected. `rate`, `increase` or `irate` over a metric declared a gauge, and `sum`, `avg`, `quantile`, `stddev` or `stdvar` across a summary's quantile series, still plan, with a warning from the PromQL frontend (`metric_type_warnings`), which `stage_pipeline` reports. | +| `metric_type` | Optional | Declared Prometheus type (`PrometheusMetricType`): `counter`, `gauge`, `histogram`, `native_histogram` or `summary`; a counter proves samples non-negative. Never inferred from a name. A `histogram` or `summary` is keyed by its family name `foo` and supplies its series (`DataWorkload::series_facts`): `foo_bucket` (histogram) or `foo` (summary, per-series quantiles read as values), and the counters `foo_sum` and `foo_count`; any other name is looked up as is. The family entry's `metric_type`, `values_non_negative` and `sample_interval` apply to its series; counts, norms and bounds are read per series name. `histogram_quantile` reads float input as classic buckets, declared `histogram` or undeclared, and rejects a `native_histogram` or `summary`; any query reading a `native_histogram` is rejected. `rate`, `increase` or `irate` over a metric declared a gauge, and `sum`, `avg`, `quantile`, `stddev` or `stdvar` across a summary's quantile series, still plan, with a warning from the PromQL frontend (`metric_type_warnings`), which `asap_planner` reports. | | `values_non_negative` | Optional; default `false` | Declares every sample non-negative, for a gauge as well as a counter. It is the sign a relative-error bound needs to carry through `sum` and `avg` (a per-series DDSketch quantile under `avg by`, say); without it, and for a gauge, that bound is unknown. | | `sample_interval` | PromQL and MetricsQL; nonzero | How often each series of the metric is sampled. Bare instant selectors over it use it as their selection horizon; without it, such a query fails to lower. | | `stream_norms`, `max_distinct_per_evaluation` | Optional | Bounds that Count-Min/Count Sketch and HLL/UnivMon guarantees are bound with. | @@ -314,7 +314,7 @@ PlanningWorkload + frontend dependencies + PlanningModels |---|---|---| | Plan a workload end to end | `asap_planner::e2e_plan` | `PlanOutput` | | Plan already-lowered roots and inspect every stage | `asap_planner::run_planning` | `PlanningRun`, whose `selection` is the `Selection` | -| Inspect a workload's stages as a document | `stage_pipeline` (devtools) | The stage document the [Stage Viewer](../../user_guide_docs/stage-viewer.md) shows | +| Inspect a workload's stages as a document | `asap_planner` (devtools) | The stage document the [Stage Viewer](../../user_guide_docs/stage-viewer.md) shows | The integration associates each lowered root, by its position, with its workload entry (accuracy target, latency bound, recurrence). See the diff --git a/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md b/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md index 21d2422fb..3011064f3 100644 --- a/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md +++ b/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md @@ -9,7 +9,7 @@ ## 1. What is new -1. An **end-to-end function API** to lib users, giving them a one-function-call abstraction from a prepared input to the selected Post-ASAP DAG. +1. An **end-to-end function API** to lib users, giving them a one-function-call abstraction from a prepared input to the selected Post-ASAP DAG. 2. A **pluggable optimization pass**, which gives developers a replaceable optimization stage. What that buys: @@ -33,11 +33,11 @@ PlanningWorkload ──lowering──▶ ParsedWorkload ──OptimizationPass ## 2. The types -This section introduces three key interfacing types of the unified workflow. +This section introduces three key interfacing types of the unified workflow. The following diagram illustrates the flow of data through the unified workflow: -`e2e_plan` is the outer, light-blue box: a prepared workload goes in one end, the selected plan comes out the other. +`e2e_plan` is the outer, light-blue box: a prepared workload goes in one end, the selected plan comes out the other. Inside, the workflow is composed of 2 phases: A fixed lowering phase and a pluggable optimization phase. `OptimizationInput` is the intermediate type between the lowering and optimization phases. @@ -131,7 +131,7 @@ tracked in #580. `run_planning` runs these steps over the query-language frontends' roots (Stage 0) and returns `PlanningRun`: Pass 1's candidates, the Pass 2 and -Stage 2 outputs, the selection, the selected plan and a report; the `stage_pipeline` devtool shows them. +Stage 2 outputs, the selection, the selected plan and a report; the `asap_planner` devtool shows them. `ReplacementStrategy` remains a concept of the legacy candidate search, which no pass uses any more. diff --git a/docs/design_docs/proposals/planner-layering-example1-acceptance.md b/docs/design_docs/proposals/planner-layering-example1-acceptance.md index 176692367..c66cd3aa6 100644 --- a/docs/design_docs/proposals/planner-layering-example1-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example1-acceptance.md @@ -4,10 +4,10 @@ Audience: planner designers and the implementers of the planning stages. Source: [planner-layering.md](planner-layering.md), "Example 1: Aggregation over dimensions". Tests: `crates/integration-tests/tests/planner_layering_example1.rs` (helpers in `tests/planner_layering_common/`). Viewer fixture: -`tools/dag-viewer/examples/planner-layering-example1.json`, written by -`stage_pipeline --example planner-layering-1 --exhaustive ---enforce-accuracy-guarantees --max-displayed 100` (`stage_pipeline` ignores -accuracy guarantees without the flag). +`tools/dag-viewer/examples/planner-layering-example1.json`, written by the +former `stage_pipeline --example planner-layering-1 --exhaustive +--enforce-accuracy-guarantees --max-displayed 100` (a historical record; +`stage_pipeline` ignored accuracy guarantees without the flag). This spec was written by a test designer before the stages existed. A test whose expectation needs a feature that is not in the stack yet is diff --git a/docs/design_docs/proposals/planner-layering-example2-acceptance.md b/docs/design_docs/proposals/planner-layering-example2-acceptance.md index 0932addcc..cbb2214e7 100644 --- a/docs/design_docs/proposals/planner-layering-example2-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example2-acceptance.md @@ -38,12 +38,12 @@ structure, three statistics of one input over the same 1-min window: `distinct_over_time(flows_src_ip[1m])`, `entropy_over_time(flows_src_ip[1m])` and `l2_over_time(flows_src_ip[1m])`, with Q1–Q3's targets and a 15-s `sample_interval` on `flows_src_ip`. The -`stage_pipeline --example planner-layering-2 --enforce-accuracy-guarantees` document plans the SQL +`asap_planner --workload tools/dag-viewer/workloads/planner-layering/example2.json --sql-catalog-table --run-until stage3` document plans the SQL workload itself. ## Expected candidates per stage -| Stage | Doc | Planner (SQL workload, `stage_pipeline --example planner-layering-2 --enforce-accuracy-guarantees`) | +| Stage | Doc | Planner (SQL workload, `asap_planner --workload tools/dag-viewer/workloads/planner-layering/example2.json --sql-catalog-table --run-until stage3`) | |---|---|---| | Query-language frontends | One summary-free DAG: `Distinct`, `Entropy`, `L2` over `flows.src_ip`, last 1 m | Same. Q1 is a distinct count; Q2 is `Entropy` and Q3 is `L2` over the per-source counts, each with an exact population count for the empty-input NULL. | | Logical ASAP-aware optimization, Pass 1 | Each computation: exact, a specialized summary, UnivMon. 3 × 3 × 3 = **27** | Q1: exact, HLL, KMV, UnivMon (no Theta: `asap_sketchlib` has no Theta sketch). Q2, Q3: exact, UnivMon (no specialized entropy or norm summary). 4 × 2 × 2 = **16** | @@ -154,8 +154,10 @@ strictest size. ## Devtool -`stage_pipeline --example planner-layering-2 --enforce-accuracy-guarantees` -plans the SQL workload with the `flows(ts, src_ip)` catalog (without the -flag, `stage_pipeline` ignores accuracy guarantees, and none of the plans -below is rejected for its guarantee). The stage viewer has no tab for it; open the +`asap_planner --workload tools/dag-viewer/workloads/planner-layering/example2.json +--sql-catalog-table --run-until stage3 --format json`, where `` +is the `flows` table of `tools/dag-viewer/workloads/planner-layering/tables.json`, +plans the SQL workload with the `flows(ts, src_ip)` catalog (with +`--ignore-accuracy-guarantees`, none of the plans below is rejected for its +guarantee). The stage viewer has no tab for it; open the document with **Open stage document…**. diff --git a/docs/design_docs/proposals/planner-layering-example3-acceptance.md b/docs/design_docs/proposals/planner-layering-example3-acceptance.md index 3e885ea21..f6f253b9d 100644 --- a/docs/design_docs/proposals/planner-layering-example3-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example3-acceptance.md @@ -7,17 +7,19 @@ windows" and the Pass 2 section. Tests: `crates/integration-tests/tests/planner_layering_example3.rs` (helpers in `tests/planner_layering_common/`; the Example 3/4 workloads and the stage pipeline in `asap_devtools::{planner_layering_examples, planning_stages}`). -Devtool: `stage_pipeline --example planner-layering-3a` and -`planner-layering-3b`, with `--enforce-accuracy-guarantees` (without it, -`stage_pipeline` ignores accuracy guarantees and DDSketch is not rejected). Both run every candidate (2,187 physical for 3a: +Devtool: `asap_planner --workload +tools/dag-viewer/workloads/planner-layering/example3a.json --run-until stage3` +and the same with `example3b.json` (with `--ignore-accuracy-guarantees`, +DDSketch is not rejected). Both run every candidate (2,187 physical for 3a: 243 from Pass 1, each with 3 Pass 2 shared-input candidates, 972 logical, and each shared input also built once per evaluation) through plan -selection; `--max-displayed` (default 64) limits only how many the document -shows, so 3a selects P485: KLL for all five queries. +selection; in the former `stage_pipeline` (a historical record), +`--max-displayed` (default 64) limited only how many the document showed, +so 3a selects P485: KLL for all five queries. This spec was written by a test designer who does not implement the stages. Tests run the stages through `planner_layering_common::run_stages`, which -calls `run_planning_stages`, the pipeline `stage_pipeline` runs. A test +calls `run_planning_stages`, the pipeline `asap_planner` runs. A test that needs a missing feature is `#[ignore]`d with the PR that delivers it. ## Workload diff --git a/docs/design_docs/proposals/planner-layering-example4-acceptance.md b/docs/design_docs/proposals/planner-layering-example4-acceptance.md index 65b0b1a5f..ae9371317 100644 --- a/docs/design_docs/proposals/planner-layering-example4-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example4-acceptance.md @@ -159,14 +159,15 @@ it. ## Devtool -No `--example planner-layering-4`. Pattern A and B run as -`planner-layering-3a` and `3b`, and Pattern B's materialization choices -(B1, B2, and B3 with -`--deployment-capabilities '{"query_time_results_retained": true}'`) appear in -the `3b` document; `--cost-model` selects cost model 1, 2 -or 3 (with `--cluster-memory-bytes`, `--cluster-disk-bytes` and -`--cluster-objective` for cost model 3). A useful id still needs: - -* the variant (as given, monthly, at rest) selectable in `stage_pipeline`; +No Example 4 workload file. Pattern A and B run as the +`tools/dag-viewer/workloads/planner-layering/example3a.json` and +`example3b.json` workloads, and Pattern B's materialization choices +(B1, B2, and B3 with a `--deployment-capabilities` file holding +`{"query_time_results_retained": true}`) appear in the `3b` document; the +`cost_model` of the `--cost-inputs` file selects cost model 1, 2 +or 3 (whose cluster memory, disk and objective are fields of that +`cost_model`). A useful id still needs: + +* the variant (as given, monthly, at rest) selectable in `asap_planner`; * the viewer showing each materialized node's retention (it shows each node's materialization already). diff --git a/docs/design_docs/proposals/planner-layering.md b/docs/design_docs/proposals/planner-layering.md index de50a09e7..cb93a3acb 100644 --- a/docs/design_docs/proposals/planner-layering.md +++ b/docs/design_docs/proposals/planner-layering.md @@ -408,7 +408,7 @@ all; the memo has 259 candidates. `accuracy_budget_parts` 2 KLL), 336 `TumblingWindows` and 345 `SharedBuild`. G6 has 3: `Pass1`, `SummaryResult` and `SharedBuild`. -- Enumeration stops at `stage_pipeline`'s default limit of 1,000 candidate +- Enumeration stops at the former `stage_pipeline`'s default limit of 1,000 candidate DAGs (`--memo-enumeration-limit`), as it does after step 3. | After | Groups | Candidates for memo groups | Candidate DAGs | @@ -420,7 +420,7 @@ all; the memo has 259 candidates. These numbers are the production planner's, read from the stage-pipeline document `tools/dag-viewer/examples/memo-ratio-of-quantiles.json` (made by -`stage_pipeline` from `tools/dag-viewer/memo-examples/promql/ratio-of-quantiles-5m.json`, +the former `stage_pipeline` from `tools/dag-viewer/memo-examples/promql/ratio-of-quantiles-5m.json`, with accuracy guarantees enforced and the `aws-fargate-us-east-1` cost model). `crates/integration-tests/tests/memo_enumeration.rs` lowers the query as production does (with the series-identity column) and reads the memo from the @@ -581,7 +581,7 @@ nodes are kept whole, so `operator_node` indexes the output's candidates are attached to the memo groups, and records a `PrunedCandidate` like every other Pass 1 prune. -`stage_pipeline` writes both lists as `pruned_candidates` under +`asap_planner` writes both lists as `pruned_candidates` under `stage1_logical_asap_optimization` and `stage2_physical_asap_optimization`, and the viewer's Memo view lists a group's entries in its group panel. diff --git a/docs/develop_docs/asap-aware-mapping-architecture.md b/docs/develop_docs/asap-aware-mapping-architecture.md index fa83723aa..d638c7a36 100644 --- a/docs/develop_docs/asap-aware-mapping-architecture.md +++ b/docs/develop_docs/asap-aware-mapping-architecture.md @@ -26,7 +26,7 @@ in `asap-integration-tests` (`crate_graph`) checks these edges with | `asap-plan-selection` (`crates/plan-selection`) | Stage 3: `select_plan` over Stage 2's output; accuracy and latency checks, pricing candidates with `asap-cost` | | `asap-planner` (`crates/planner`) | The facade: `e2e_plan` runs Stage 0, then the optimization pass (`Stage1ToStage3Pass` by default), which orchestrates Stages 1–3 (`run_planning`); the planning models (`PlanningModels`) and the `OptimizationPass` trait | | `asap-executor` (`crates/executor`) | Stage 4, the reference executor: operators and summary kernels over `asap_sketchlib`; it runs a `PhysicalASAPDAG` | -| `asap-devtools` | `stage_pipeline` and the other inspection tools, the design-doc example workloads (`planner_layering_examples`) and the candidate labels | +| `asap-devtools` | `asap_planner` and the other inspection tools, the design-doc example workloads (`planner_layering_examples`) and the candidate labels | ```text asap-types ◄── frontends (Stage 0) @@ -110,8 +110,7 @@ The unwired maintained-population rule has been removed as well; no Stage 1 pass limits. It measures the valid ones and selects one under the cost model; ties keep enumeration order. -No stage caps the candidates it plans; `stage_pipeline` caps only what it -displays and refuses workloads over `--max-combinations`. +No stage caps the candidates it plans, and neither does `asap_planner`. ## Verification @@ -124,5 +123,6 @@ displays and refuses workloads over `--max-combinations`. the design document's examples through every stage; `reuse_rules.rs`, `deployment_capabilities.rs`, `materialization_e2e.rs` and `stage1_logical_asap_optimization.rs` cover single rules end to end. -- `crates/devtools/tests/stage_pipeline.rs` checks the stage document and the - committed viewer example. +- `crates/devtools/tests/asap_planner.rs` checks the stage document and the + committed viewer examples; `asap_planner_acceptance.rs` checks + `--run-until` and the refused flags. diff --git a/docs/develop_docs/end-to-end-accuracy-guarantees.md b/docs/develop_docs/end-to-end-accuracy-guarantees.md index 11788e753..4374708cf 100644 --- a/docs/develop_docs/end-to-end-accuracy-guarantees.md +++ b/docs/develop_docs/end-to-end-accuracy-guarantees.md @@ -43,7 +43,7 @@ The main implementation locations are: | Accuracy model and built-in propagation | `asap_accuracy` | | Candidate construction and legality filtering | `asap_aware_mapping::replacement` | | Parameter sizing hooks | `asap_aware_mapping::cost_model` | -| Guarantee and rejection export | The `stage_pipeline` devtool's document (`crates/devtools/src/bin/stage_pipeline.rs`) | +| Guarantee and rejection export | The `asap_planner` devtool's document (`crates/devtools/src/bin/asap_planner.rs`) | Read the sections below when changing one of those contracts. diff --git a/docs/develop_docs/extend-asap-aware-mapping.md b/docs/develop_docs/extend-asap-aware-mapping.md index cc41ad6f3..9c03a415e 100644 --- a/docs/develop_docs/extend-asap-aware-mapping.md +++ b/docs/develop_docs/extend-asap-aware-mapping.md @@ -102,11 +102,11 @@ tests in `planner_layering_example4.rs`. `DeploymentCapabilities`. Both reach the stages through `PlanningModels`; a stage function takes only the field it reads (Stage 2 takes `query_time_results_retained: bool`, not the whole struct). -- Expose it in `stage_pipeline` and in the +- Expose it in `asap_planner` and in the [Stage Viewer guide](../user_guide_docs/stage-viewer.md). Tests: a Stage 3 unit test that the check rejects exactly the candidates it -should, and a `crates/devtools/tests/stage_pipeline.rs` test for the flag. +should, and a `crates/devtools/tests/asap_planner.rs` test for the flag. ## 5. Add a query-language construct @@ -127,6 +127,8 @@ tests (`analyze_corpora`, `variant_coverage`) show the coverage change. - [ ] Unit tests beside the code and one end-to-end test through the stages. - [ ] The design document's examples still plan the same, or the change is explained: run the `planner_layering_example*` tests and - `stage_pipeline --example planner-layering-1 --enforce-accuracy-guarantees` - (and `-2`, `-3a`, `-3b`). + `asap_planner --workload + tools/dag-viewer/workloads/planner-layering/example1.json --run-until stage3` + (and `example2.json`, with `--sql-catalog-table` for each table in + `tables.json`, `example3a.json` and `example3b.json`). - [ ] The design, developer and user docs that name what changed are updated. diff --git a/docs/develop_docs/joint-candidate-workload-evaluation.md b/docs/develop_docs/joint-candidate-workload-evaluation.md index ae9fb7725..de827e8e4 100644 --- a/docs/develop_docs/joint-candidate-workload-evaluation.md +++ b/docs/develop_docs/joint-candidate-workload-evaluation.md @@ -20,7 +20,7 @@ The report separates local candidate generation, memo expansion, inventory/model The separate nine-example viewer suite also passed, including SQL rollup and the composed DDS range query (174.52 s total, concurrency two). Zero-budget and refinement-budget reports passed their focused checks. These examples supplement the thirty random mixed workloads. -To reproduce the thirty-workload collection, build `stage_pipeline`, freeze that executable, then run `python3 tools/validate_joint_workloads.py OUTPUT_DIRECTORY FROZEN_EXECUTABLE`. The runner uses the unchanged viewer manifests, enforced guarantees, Fargate prices, and a 90-second external deadline. Full regression and exact-head CI are separate release gates; successful workload coverage alone does not establish release readiness. +To reproduce the thirty-workload collection, build `asap_planner` (which replaced `stage_pipeline`), freeze that executable, then run `python3 tools/validate_joint_workloads.py OUTPUT_DIRECTORY FROZEN_EXECUTABLE`. The runner uses the unchanged viewer manifests, enforced guarantees, Fargate prices, and a 90-second external deadline. Full regression and exact-head CI are separate release gates; successful workload coverage alone does not establish release readiness. ## Regression coverage diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index fd3db4dfe..03f3a695c 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -297,8 +297,8 @@ let settings = CostInputs { let models = PlanningModels::builtin().with_cost_inputs(&settings); ``` -The `stage_pipeline` devtool runs the same stages and writes every stage's -candidates as one JSON document; see the +The `asap_planner` devtool runs the same stages and writes every stage's +output as text or, with `--format json`, as one JSON document; see the [Stage Viewer guide](../user_guide_docs/stage-viewer.md). ## Workload inputs and defaults diff --git a/docs/develop_docs/planner-snapshots.md b/docs/develop_docs/planner-snapshots.md index 1460b6238..04986939e 100644 --- a/docs/develop_docs/planner-snapshots.md +++ b/docs/develop_docs/planner-snapshots.md @@ -1,8 +1,8 @@ # Planner snapshots The planner has no reference answer to test against, so CI pins what it -produces. For each query in a corpus, the plan that `stage_pipeline` selects -(default search) is committed as a snapshot. The `Corpus snapshots` workflow +produces. For each query in a corpus, the plan that `asap_planner +--run-until stage3` selects is committed as a snapshot. The `Corpus snapshots` workflow checks them nightly on `main` (and on demand from the Actions tab, for any branch); a change to any of these plans fails it until the snapshots are regenerated and the change is explained. @@ -11,7 +11,7 @@ regenerated and the change is explained. `crates/devtools/tests/corpus_snapshots//`: -- `corpus.json`: the inputs shared by the corpus's queries (`stage_pipeline_args`, +- `corpus.json`: the inputs shared by the corpus's queries (`asap_planner_args`, the repeating-query entry and the `data_workload`), and the queries, each with a stable `id`. Each query is planned on its own. A query with a `skip_reason` entry (saying why) is skipped and has no snapshot. @@ -31,24 +31,23 @@ Only the plan, which must not change between runs: embedded search report, and the causes and reasons of the rejections. Floats are rounded to 12 significant digits. Everything about the search -except whether it proved the plan optimal is left out: candidate ids (such as -`pane-round-12-joint-plan-1`), the search report's counters, bounds and timings, the Stage 1 and Stage 2 -candidate lists, and the memo (`stage_pipeline` runs with `--no-memo`). These -vary from run to run because the search's tumbling-window-width rounds have wall-clock -budgets, so a different round can find the same plan. +except whether it proved the plan optimal is left out: the search report's +counters, bounds and timings, the Stage 1 and Stage 2 candidate lists, and +the memo. Timings vary from run to run, and the rest describes how the plan +was found, not the plan. ## Sketch plan snapshots The corpus queries carry no accuracy target, so their plans are exact. `crates/devtools/tests/sketch_plan_snapshots/` pins the plans of workloads -that select sketches: `manifest.json` lists each workload (a workload file or -built-in example, its deployment, and whether accuracy guarantees are +that select sketches: `manifest.json` lists each workload (a workload file, its deployment +capabilities and cost inputs files, and whether accuracy guarantees are enforced), and `.json` is its snapshot, holding the same fields as a corpus snapshot plus every query's text and the `search_status`. The test, `sketch_plans_match_snapshots`, is in the same `corpus_snapshots` binary and runs nightly with it; the two tests never run at once. Every workload must end -with status `optimal`: a search stopped by any budget may select another plan -on a slower machine, so any other status fails. Regenerate with: +with status `optimal`, unless the manifest gives it a +`deadline_reached_reason`; any other status fails. Regenerate with: ```sh UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored sketch_plans_match_snapshots @@ -57,7 +56,7 @@ UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots ## When the check fails The test names each changed query and its first differing line. CI uploads -the regenerated snapshots, with each one's full `stage_pipeline` document +the regenerated snapshots, with each one's full `asap_planner` document (`.document.json`, which [`tools/dag-viewer`](../../tools/dag-viewer/README.md) opens), as the `corpus-snapshots` artifact. @@ -72,16 +71,12 @@ the PR description. The test also fails, instead of recording a snapshot, in these cases: -- A query's search was ended by a wall-clock budget: its status is - `time_limit`, or planning used the whole 240 s limit. Its plan would then - depend on machine speed. Only the final tumbling-window-width round's status is checked: an - earlier round cut short by its own wall-clock budget is not reported. - A snapshot file has no matching checked query in `corpus.json`. -- `stage_pipeline` crashed. +- `asap_planner` failed. ## Adding queries Add the query to `corpus.json` with a new `id` and regenerate. Before you -commit, run the check twice, plus once on one core (`taskset -c 0`), and confirm -it passes each time. Keep each query well under the 240 s limit on one core -with a release build; otherwise give it a `skip_reason`. +commit, run the check twice and confirm it passes each time. Planning has no +time limit, so a query that takes minutes in a release build slows every +nightly run; give such a query a `skip_reason`. diff --git a/docs/develop_docs/replacement-explanations.md b/docs/develop_docs/replacement-explanations.md index d8b22082a..93e69f094 100644 --- a/docs/develop_docs/replacement-explanations.md +++ b/docs/develop_docs/replacement-explanations.md @@ -5,7 +5,7 @@ > and why each candidate was pruned or rejected, is now in the stages' > output: Pass 1's pruned candidates with their `PruneReason`, Stage 2's > pruned candidates, and Stage 3's `Rejection` reasons, all written by -> `stage_pipeline` (see the [Stage Viewer guide](../user_guide_docs/stage-viewer.md)). +> `asap_planner` (see the [Stage Viewer guide](../user_guide_docs/stage-viewer.md)). ASAP-aware mapping should make its discovered opportunities visible to users and other components. diff --git a/docs/user_guide_docs/README.md b/docs/user_guide_docs/README.md index 735345617..4cee670dd 100644 --- a/docs/user_guide_docs/README.md +++ b/docs/user_guide_docs/README.md @@ -4,5 +4,5 @@ User guides show supported commands, inputs, outputs, and ways to verify the result. - [Run and inspect a query](run-a-query.md) -- [Plan a workload with stage_pipeline](stage-pipeline.md) +- [Plan a workload with asap_planner](asap-planner.md) - [See how ASAPPlanner plans a workload: the Stage Viewer](stage-viewer.md) diff --git a/docs/user_guide_docs/asap-planner.md b/docs/user_guide_docs/asap-planner.md new file mode 100644 index 000000000..3b29c71f9 --- /dev/null +++ b/docs/user_guide_docs/asap-planner.md @@ -0,0 +1,131 @@ +# Plan a workload with asap_planner + +`asap_planner` runs the planner's stages on a workload, up to the stage you +choose, and prints what each stage produced: as text, or as a JSON document +for the [Stage Viewer](stage-viewer.md). It reports candidates, selection, +modeled costs and accuracy checks; it does not deploy or execute the plan. + +## Build and run + +Run from an up-to-date ASAPPlanner repository root. You need Rust/Cargo, +a C/C++ toolchain, CMake, and the libclang shared library. HiGHS is built as a +native dependency; its Rust bindings use libclang. + +On Ubuntu/Debian, install the native build dependencies with: + +```sh +sudo apt-get update +sudo apt-get install build-essential cmake libclang-dev +``` + +Run it through Cargo; a checkout does not put `asap_planner` on your `PATH`: + +```sh +cargo run --release -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/workloads/planner-layering/example1.json \ + --run-until stage3 +``` + +The `--` separates Cargo options from planner options. Release builds are +recommended: each stage runs to completion with no time limit, which can take +minutes in a debug build. + +## Options + +```text +asap_planner (--workload )... --run-until stage0|stage1_pass1|stage1_pass2|stage2|stage3 + [--format text|json] [--out ] [--sql-catalog-table ]... + [--cost-inputs ] [--deployment-capabilities ] + [--ignore-accuracy-guarantees] +``` + +| Option | Meaning | +| --- | --- | +| `--workload ` | A `PlanningWorkload` JSON file: the queries, their requirements and recurrence, and the data-source facts. Repeat it to plan several files together, including PromQL and SQL files; sources they share must have the same facts. Required. | +| `--run-until ` | The last stage to run. Every stage up to and including it is printed. Required. | +| `--format text\|json` | `text` (the default) for reading; `json` for an `asap-stage-pipeline/v1` document. | +| `--out ` | Write to this file instead of standard output. | +| `--sql-catalog-table ` | One SQL table a SQL workload reads, as `{"name", "columns": [{"name", "type", "nullable"}], "time_index"}`. Repeat it per table. | +| `--cost-inputs ` | The cost model and its prices (`CostInputs` JSON). The files in `tools/dag-viewer/cost-inputs/` are ready to use. Without it, the built-in usage-over-time model is used. Stage 3 prices only usage-over-time models (`aws-fargate-us-east-1.json` is one); a provisioned- or cluster-capacity model ends Stage 3 with an `UnsupportedCostModel` error. | +| `--deployment-capabilities ` | What the deployment can do, e.g. `{"raw_data_retained": true, "query_time_results_retained": false}`. Set it to match your deployment. | +| `--ignore-accuracy-guarantees` | Accept candidates without an accuracy guarantee. By default each query's accuracy requirement is enforced. | + +Any other option, or a missing required one, is a usage error (exit +status 2). An input the planner cannot read exits with status 1. So does a +run in which a stage failed (for example, Pass 1 rejected a query, or no +plan meets the requirements): the output is still written, with the error +under the stage that failed. + +## Stages + +| `--run-until` | Prints, after the stages before it | +| --- | --- | +| `stage0` | The frontends' query DAGs and their memo groups. | +| `stage1_pass1` | Stage 1 Pass 1: each computation's candidates, and the candidates pruned with their reason. | +| `stage1_pass2` | Stage 1 Pass 2: the memo after local window composition (a candidate for every legal tumbling-window width), then the memo after the sharing rules, which Stage 2 implements. | +| `stage2` | Stage 2: the physical candidates and the ones pruned. | +| `stage3` | Stage 3: the selected plan, its cost per node, each query's accepted accuracy and the search status. | + +Every stage sees every legal tumbling-window width at once, so the selected +plan is the best over all of them. + +## Plan a workload file + +The example below plans the checked-in +[range-heavy workload](../../tools/dag-viewer/memo-examples/promql/rangeheavy-t8-t5-1h.json) +with raw data retained and AWS Fargate prices: + +```sh +echo '{"raw_data_retained": true, "query_time_results_retained": false}' > capabilities.json +cargo run --release -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/memo-examples/promql/rangeheavy-t8-t5-1h.json \ + --deployment-capabilities capabilities.json \ + --cost-inputs tools/dag-viewer/cost-inputs/aws-fargate-us-east-1.json \ + --run-until stage3 +``` + +A SQL workload also needs its tables: + +```sh +cargo run --release -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/memo-examples/sql/rollupheavy-t11-service-endpoint-5m.json \ + $(python3 -c 'import json; [print("--sql-catalog-table", json.dumps(t, separators=(",", ":"))) for t in json.load(open("tools/dag-viewer/memo-examples/sql/tables.json"))]') \ + --run-until stage3 +``` + +See [Run and inspect a query](run-a-query.md) for workload, source-fact and +SQL examples. To try a single query, write a one-query workload file. + +## Inspect the result + +With `--format json`, open the document in the +[Stage Viewer](stage-viewer.md), or read Stage 3 directly: + +```sh +cargo run --release -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/workloads/planner-layering/example1.json \ + --run-until stage3 --format json --out plan.json +python3 -c 'import json; print(json.dumps(json.load(open("plan.json"))["stage3_plan_selection"]["search_report"], indent=2))' +``` + +Sections of stages that were not run are `null`. In +`stage3_plan_selection`, check `selected_dag`, `cost`, `search_report` and +any `error`: a written document alone does not mean planning succeeded. +A selected plan is proven optimal only when `search_report.status` is +`optimal`. Costs are model predictions, not measured execution times or a +cloud bill. + +## Troubleshooting + +- **`asap_planner: command not found`:** use `cargo run ... -- ...`, or the + binary's path after `cargo build --release -p asap-devtools --bin asap_planner`. +- **`unknown argument ...`:** the option does not exist. Earlier + `stage_pipeline` options such as `--promql`, `--example`, `--table`, + `--cost-model` and `--milp-time-limit-s` were removed: write a workload + file, use `--sql-catalog-table` and `--cost-inputs`. There is no time limit. +- **`--run-until is required`:** name the last stage to run. +- **Cannot read the workload:** paths are relative to your working + directory; an absolute path avoids ambiguity. +- **`Unable to find libclang`:** install `libclang-dev`, or point + `LIBCLANG_PATH` at the directory containing `libclang.so` (a directory, not + the file name), then re-run the same command. diff --git a/docs/user_guide_docs/run-a-query.md b/docs/user_guide_docs/run-a-query.md index eaa4b1a9e..666d2badb 100644 --- a/docs/user_guide_docs/run-a-query.md +++ b/docs/user_guide_docs/run-a-query.md @@ -14,9 +14,8 @@ tool on first use. | Command (`cargo run -p asap-devtools --bin … -- …`) | Input / options | Result | | --- | --- | --- | -| `stage_pipeline --promql "" --out plan.json` | One or more `--promql` / `--sql` queries, or `--example ` | Plans the workload through Stages 0–3 and writes the stage document the [Stage Viewer](stage-viewer.md) shows | -| `convert_rqe_atomic_costs ` | sketch-bench's per-operation cost table | Writes the measured table `stage_pipeline --measured-operation-costs` reads; skipped rows are listed on stderr | -| `show_logical_dag --sample-interval-ms 1000 queries.txt` | File path, or stdin when omitted | Prints each query's logical DAG (Pre-ASAP IR) | +| `asap_planner --workload --run-until ` | `PlanningWorkload` JSON files | Runs the planner's stages up to `` and prints each stage's output as text, or with `--format json` as the document the [Stage Viewer](stage-viewer.md) shows | +| `convert_rqe_atomic_costs ` | sketch-bench's per-operation cost table | Writes a measured table to set as `measured_operation_costs` in an `asap_planner --cost-inputs` file; skipped rows are listed on stderr | | `analyze_corpora --corpora --sample-interval-ms 1000 --out-dir ` | Repository PromQL corpora, output directory | Writes successful/error IR dumps and summary reports | | `analyze_corpora --sql-corpora --out-dir ` | Repository SQL corpora, output directory | Writes SQL corpus reports | | `variant_coverage --sample-interval-ms 1000` | Repository corpora | Reports Pre-ASAP IR variant coverage | @@ -24,79 +23,51 @@ tool on first use. ## Plan a workload -For build dependencies, workload-file commands, search budgets, and troubleshooting, -see [Plan a workload with stage_pipeline](stage-pipeline.md). +For build dependencies, every option and troubleshooting, see +[Plan a workload with asap_planner](asap-planner.md). -`stage_pipeline` runs the query-language frontends (Stage 0), logical +`asap_planner` runs the query-language frontends (Stage 0), logical ASAP-aware optimization (Stage 1), physical ASAP-aware optimization -(Stage 2) and plan selection (Stage 3), and writes one JSON document: +(Stage 2) and plan selection (Stage 3) on a workload file, and prints what +each stage produced: ```sh -cargo run -p asap-devtools --bin stage_pipeline -- \ - --promql "topk by (job) (10, sum_over_time(http_requests_total[1m]))" \ - --series-count 1000 --max-error 0.01 --failure-probability 0.001 \ - --enforce-accuracy-guarantees --out plan.json +cargo run --release -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/workloads/planner-layering/example1.json \ + --run-until stage3 ``` -Plan selection prices only from declared facts. `--series-count` (with -the sample interval) or `--rows-per-second` gives each source's arrival -rate; `--rows-held` bounds data at rest and a scan with no window (a SQL -query without a time predicate reads every row held). Without the fact a -scan needs, planning fails with an error that names the source and the -missing fact. - -Without `--enforce-accuracy-guarantees`, `stage_pipeline` ignores accuracy -guarantees: every output is accepted whatever its guarantee, and plans are -compared on cost alone. The document's top-level `accuracy_guarantees` says -`ignored` or `enforced`. The document's `stage3_plan_selection.selected_dag` -names the selected physical candidates, and `stage3_plan_selection.cost` -holds the selected plan's `cost_per_second`. For this query the selected plan -is the exact plan. The Count-Min top-k plan is not selected because the -command declares no metric type, so the samples are not proven non-negative, -and the CountSketch top-k plan because its output has no guarantee. Under -`--exhaustive`, `reference_pricing.pruned_candidates` gives each plan's `cause` -(`count_min_weights_not_proven_non_negative`, `no_accuracy_guarantee`) and -reason. -Open the document in the [Stage Viewer](stage-viewer.md) to see every -stage, or run a built-in example of the design document, such as -`--example planner-layering-1 --exhaustive --enforce-accuracy-guarantees`. The [stage_pipeline guide](stage-pipeline.md) describes more options. +The workload file holds the queries, their accuracy requirements and how +often they run, and the data-source facts plan selection prices from: each +source's series count and sample interval, or rows per second. Without the +fact a scan needs, planning fails with an error that names the source and +the missing fact. + +Each query's accuracy requirement is enforced unless you pass +`--ignore-accuracy-guarantees`. The output ends with Stage 3's selection: +the selected physical candidates, the plan's `cost_per_second` with each +node's charge, each query's accepted accuracy, and the search status +(`Optimal` when the plan is proven the cheapest). For this example the +selected plan is exact for both queries. Add `--format json --out plan.json` +and open the document in the [Stage Viewer](stage-viewer.md) to see every +stage. ## Inspect a query's logical DAG -### Create a query file - -Create a text file containing one query per line. Prefix each query with `sql>` or `promql>`. - -For example, `queries.txt`: - -```text -promql> quantile(0.99, rate(http_requests_total[5m])) -sql> SELECT service, COUNT(*) FROM metrics GROUP BY service -``` - -Blank lines and lines beginning with `#` are ignored. `show_logical_dag` -accepts `sql>` and `promql>`; MetricsQL is available through the library frontend. -SQL examples use the fixed catalog -`metrics(ts: Timestamp, service: Utf8, region: Utf8, latency: Float64, bytes: Int64)`. -For your own schema, provide a `SqlCatalog` through the library API, or -declare tables with `stage_pipeline --table`. - -### Show the logical DAG - -Run: +`--run-until stage0` prints each query's logical DAG (Pre-ASAP IR) as the +frontends lowered it, and its memo groups. It never picks a sketch: it shows +what a query means, with exact operators only. ```sh -cargo run -p asap-devtools --bin show_logical_dag -- --sample-interval-ms 1000 queries.txt +cargo run -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/workloads/planner-layering/example1.json \ + --run-until stage0 ``` -You can also provide the queries through stdin: - -```sh -cargo run -p asap-devtools --bin show_logical_dag -- --sample-interval-ms 1000 < queries.txt -``` +A SQL workload also needs the tables it reads, one `--sql-catalog-table` +per table; see [Plan a workload with asap_planner](asap-planner.md). -Each input line is followed by its debug IR or an `ERR:` message. The tool -never picks a sketch: it shows what a query means, with exact operators only. +### Analyze the corpora To dump and compare every PromQL corpus, run: @@ -158,8 +129,7 @@ cargo run -p asap-devtools --example canonical_examples - [Pre-ASAP IR reference](../develop_docs/pre-asap-ir.md) - [Post-ASAP IR reference](../design_docs/concepts/post-asap-ir.md) -PromQL commands require `--sample-interval-ms`, the nonzero sample interval in -milliseconds declared for every metric a query reads (`stage_pipeline` -defaults it to 15000). The examples use one second; supply the interval for -your data. `show_logical_dag` requires this option even for SQL-only input -files. +The corpus commands require `--sample-interval-ms`, the nonzero sample +interval in milliseconds of every metric a PromQL query reads. The examples +use one second; supply the interval for your data. A workload file declares +each metric's `sample_interval` in its data workload instead. diff --git a/docs/user_guide_docs/stage-pipeline.md b/docs/user_guide_docs/stage-pipeline.md deleted file mode 100644 index 5f79f103a..000000000 --- a/docs/user_guide_docs/stage-pipeline.md +++ /dev/null @@ -1,135 +0,0 @@ -# Plan a workload with stage_pipeline - -`stage_pipeline` plans queries through Stages 0–3 and writes a JSON document -for the [Stage Viewer](stage-viewer.md). It reports candidate selection, -modeled costs and accuracy checks; it does not deploy or execute the plan. - -## Build and run - -Run from an up-to-date ASAPPlanner repository root. You need Rust/Cargo, -a C/C++ toolchain, CMake, and the libclang shared library. HiGHS is built as a -native dependency; its Rust bindings use libclang. - -On Ubuntu/Debian, install the native build dependencies with: - -```sh -sudo apt-get update -sudo apt-get install build-essential cmake libclang-dev -``` - -Use Cargo to build and run the command; a checkout does not install -`stage_pipeline` on your shell's `PATH`: - -```sh -cargo run --release -p asap-devtools --bin stage_pipeline -- \ - --promql 'sum_over_time(requests[5m])' \ - --series-count 1000 --sample-interval-ms 15000 \ - --enforce-accuracy-guarantees \ - --out plan.json -``` - -The `--` separates Cargo options from planner options. `--out` is required. -Release builds are recommended for planning-performance measurements. Initial -compilation can take several minutes and is separate from planning time. - -Alternatively, build once and run the binary directly: - -```sh -cargo build --release -p asap-devtools --bin stage_pipeline -"${CARGO_TARGET_DIR:-target}/release/stage_pipeline" \ - --example planner-layering-1 --enforce-accuracy-guarantees --out example.json -``` - -## Plan a workload file - -Use a `PlanningWorkload` JSON file containing query requirements, recurrence -and data-source facts. The example below uses the checked-in -[range-heavy workload](../../tools/dag-viewer/memo-examples/promql/rangeheavy-t8-t5-1h.json). -Run it from the repository root; no input file needs to be created. - -```sh -cargo run --release -p asap-devtools --bin stage_pipeline -- \ - --workload tools/dag-viewer/memo-examples/promql/rangeheavy-t8-t5-1h.json \ - --deployment-capabilities '{"raw_data_retained":true,"query_time_results_retained":false}' \ - --cost-model aws-fargate-us-east-1 \ - --enforce-accuracy-guarantees \ - --out rangeheavy-plan.json -``` - -`stage_pipeline` has one search engine, `candidate-connections` (the -document's `search` field names it); there is no `--search` option. - -The deployment declaration allows retained raw data but does not allow query-time -results to be kept across evaluations. Set capabilities to match your deployment. -For file inputs, accuracy targets come from the workload file, not the inline -query flags `--max-error`, `--failure-probability`, or `--accuracy-metric`. - -`--workload` can repeat to jointly plan multiple files, including separate -PromQL and SQL workloads. Shared-source facts must agree; SQL inputs also need -appropriate `--table` declarations. See [Run and inspect a query](run-a-query.md) -for workload, source-fact, and SQL examples. - -## Search, accuracy, and costs - -| Option | Meaning | -| --- | --- | -| `--enforce-accuracy-guarantees` | Enforce the queries' accuracy requirements. Without it, this inspection tool ignores accuracy guarantees, although executability checks still apply. | -| `--cost-model aws-fargate-us-east-1` | Compare plans using the configured AWS Fargate pricing model. Other supported names are `usage-over-time`, `provisioned-capacity`, and `cluster-capacity`. | -| `--max-displayed 64` | Limit candidates shown in the document; this is not a search budget. | -| `--exhaustive --max-combinations 10000` | Enumerate up to this many memo choices of Pass 2's memo, implement each with Stage 2 and price every resulting plan with the whole-workload reference pricer; its selection, costs and rejections are under `reference_pricing`. It is a reference, not Stage 3's search. | -| `--cost-inputs cost_inputs.json` | Load the cost inputs (`CostInputs` JSON) instead of `--cost-model`, cluster flags, or `--measured-operation-costs`. | -| `--no-memo` | Leave out the `memo` section (the memo after each step and the enumerated candidate DAGs), for a smaller document. | -| `--price-enumerated 50` | Cost the first 50 enumerated candidate DAGs through Stage 2 and Stage 3, and flag any cheaper than the selected plan. Off by default. | - -For inline queries, supply the data facts needed for pricing: for example, -`--series-count` and `--sample-interval-ms`, or `--rows-per-second`. -`--rows-held` describes retained rows for scans that need that bound. -Workload files carry these facts in their data workload. Missing facts are not -interpreted as zero cost. - -## Inspect the result - -Open the generated JSON file in the [Stage Viewer](stage-viewer.md), or inspect -Stage 3 directly: - -```sh -python3 - <<'PY' -import json -with open('plan.json') as source: - document = json.load(source) -print('Format:', document['format']) -print(json.dumps(document['stage3_plan_selection'], indent=2)) -PY -``` - -The document format is `asap-stage-pipeline/v1`. Check `selected_dag`, -`cost`, `search_report` and any `error` in `stage3_plan_selection`, and the -counts and times in `planning_report`. A written JSON document -alone does not mean planning succeeded: `selected_dag` may be null. -The search status (`search_report.status`) of a selected plan is `optimal`: -the plan is optimal over the candidates Stage 2 generated. Costs are model predictions, not measured -execution times or an actual cloud bill. - -## Troubleshooting - -- **`stage_pipeline: command not found`:** use `cargo run ... -- ...`, or the - binary's explicit path after building. Building does not add it to `PATH`. -- **`unknown argument --search`:** the option was removed; drop it. The - candidate-connection search is the only search. For any other - `unknown argument`, check that your branch and binary are current. -- **`--out is required`:** supply an output JSON path; create its parent directory - first if needed. -- **Cannot read the workload:** paths are relative to your working directory; - an absolute input path avoids ambiguity. -- **`Unable to find libclang`:** install `libclang-dev`, or point - `LIBCLANG_PATH` at the directory containing an existing `libclang.so`: - - ```sh - export LIBCLANG_PATH=/path/to/llvm/lib - # Re-run the original cargo command; cargo clean is not needed. - ``` - - The value is a directory, not the library filename. For example, if libclang - is supplied by the Python `libclang` package, its directory may be - `.../site-packages/clang/native`. Confirm that `libclang.so` exists there. - CMake's unused-variable warnings are not the cause of this bindgen failure. diff --git a/docs/user_guide_docs/stage-viewer.md b/docs/user_guide_docs/stage-viewer.md index eb068e952..2d915fba7 100644 --- a/docs/user_guide_docs/stage-viewer.md +++ b/docs/user_guide_docs/stage-viewer.md @@ -4,7 +4,8 @@ The Stage Viewer is a web page that shows what ASAPPlanner does with one workload at each [planning stage](../design_docs/proposals/planner-layering.md#asapplanner-system-overview-planning-stages), and which plan it selects. It reads one JSON document per workload, written -by `stage_pipeline`. It does not run the plans. +by `asap_planner --run-until stage3 --format json`. It does not run the +plans. ## Start it @@ -14,14 +15,14 @@ From the repository root, with Rust/Cargo and Python 3 installed: python3 tools/dag-viewer/server.py ``` -The server builds `stage_pipeline` (a few minutes the first time), writes the -example documents and prints the address, `http://127.0.0.1:8000`. On a +The server builds `asap_planner` (a few minutes the first time), writes the +example documents into `tools/dag-viewer/out/` and prints the address, `http://127.0.0.1:8000`. On a remote machine, forward the port first: `ssh -L 8000:127.0.0.1:8000 `. | Option | Use it to | | --- | --- | | `--port 8765` | Serve on another port | -| `--skip-build` | Reuse an already-built `stage_pipeline` | +| `--skip-build` | Reuse an already-built `asap_planner` | | `--regenerate` | Rewrite the example documents after a planner change | ## Look at an example @@ -31,12 +32,10 @@ what the workload is. Start with: - **1 · dashboard panels**: two dashboard panels; the all-exact plan wins. +- **3a · historical p99 batch**: five p99 reports over 1 to 5 years, run + once. - **3b · live p99 panel**: the whole-window KLL beats exact and the tumbling windows. -- **Ratio of quantiles (memo after each step)**: `avg by (job) (quantile_over_time(0.5, data[5m]) / - quantile_over_time(0.5, data[5m]) + quantile_over_time(0.9, data[5m]))`. - This one shows the memo: 7 groups, and 7 × 7 = 49 complete DAGs after - Pass 1. `#example3b` in the address opens a tab directly. @@ -53,13 +52,11 @@ From top to bottom: | Data workload | The facts given about each source (series count, sample interval, distinct values per label, …) and the deployment capabilities. | | Stage 3 · plans by cost | The selected plan (`✓ selected`) and the plans it beat: valid but costlier, or `✗ invalid` with the reason. Click a plan to draw it in the lanes. | | Stage 0 lane | The queries as lowered by the frontends. Data flows bottom-up. | -| Enumerated DAGs | The complete candidate DAGs the memo stands for, one choice of candidate per group. The header counts them after each step. Each row names the groups where it differs from the selected plan. Click a row to draw that DAG in the Stage 1 lane; the Stage 2 lane is then empty, since an enumerated DAG has no physical plan. | | Stage 1 and Stage 2 lanes | The logical candidate, and the physical candidate with each node's timing and cost. | | Materialized views | What a deployment would install for the selected plan, one row per Stage 2 node. | | Memo | The memo after each step, one tab per step (Stage 0, Pass 1 and two Pass 2 tabs). Each box is a memo group, with its candidates inside. A double border with `✚ new` marks candidates that step added. On the last tab, the selected plan's candidates are green. A group with many candidates shows one count per origin; click a count to list them. | -The Enumerated DAGs and Memo panels appear only when the document has a -`memo` section. +The Memo panel appears only when the document has a `memo` section. In the lanes: @@ -75,44 +72,31 @@ click the background to go back. Drag to pan and scroll to zoom. ## Plan your own queries -**In the page:** click **Query editor**, enter SQL or PromQL queries (and -the tables SQL reads), set ε and the data facts, and click **Plan**. Or -click **Workload sets** and pick a row. +**In the page:** click **Open stage document or workload…** and pick a +`PlanningWorkload` JSON file; the server plans it through Stage 3 and shows +it. Or click **Workload sets** and pick a row. **From the command line:** ```sh -cargo run -p asap-devtools --bin stage_pipeline -- \ - --promql "topk by (job) (10, sum_over_time(http_requests_total[1m]))" \ - --series-count 1000 --max-error 0.01 --failure-probability 0.001 \ - --enforce-accuracy-guarantees --out plan.json +cargo run --release -p asap-devtools --bin asap_planner -- \ + --workload tools/dag-viewer/workloads/planner-layering/example1.json \ + --run-until stage3 --format json --out plan.json ``` Then open `plan.json` with **Open stage document or workload…**, or drop it -on the page. - -Useful options: - -- `--enforce-accuracy-guarantees`: reject plans that miss a query's accuracy - target. Without it, guarantees are ignored and plans are compared on cost - alone. The Stage 3 header says which applied. -- `--price-enumerated `: also cost the first n enumerated DAGs through - Stage 2 and Stage 3. Each row then shows its cost or why it is invalid. A - DAG cheaper than the selected plan turns red, which would point at a - planner bug. -- `--no-memo`: leave the memo out, for a smaller document. - -For more options and examples, see -[Plan a workload with stage_pipeline](stage-pipeline.md). +on the page. Each query's accuracy requirement is enforced unless you pass +`--ignore-accuracy-guarantees`; the Stage 3 header says which applied. For +every option, see [Plan a workload with asap_planner](asap-planner.md). ## When something goes wrong | Symptom | What to do | | --- | --- | -| `Could not load the planner output: …` | The document is from an older `stage_pipeline`; write it again. | +| `Could not load the planner output: …` | The document is from an older planner; write it again with `asap_planner`. | | `Planning failed` with a network error or `HTTP 501` | The page was opened without the server. Start `server.py` and open the address it prints. | | The examples don't change after a planner change | Restart the server with `--regenerate`. | -| No Memo or Enumerated DAGs panel | The document has no `memo` section (it was written with `--no-memo`, or by an older `stage_pipeline`). | +| No Memo panel | The document has no `memo` section (it was written by an older planner). | For the document format and the viewer's code, see [`tools/dag-viewer/README.md`](../../tools/dag-viewer/README.md). diff --git a/tools/dag-viewer/README.md b/tools/dag-viewer/README.md index 083600cbe..c6800ee1c 100644 --- a/tools/dag-viewer/README.md +++ b/tools/dag-viewer/README.md @@ -1,151 +1,93 @@ # ASAPPlanner Stage Viewer A browser view of what the planner does with one workload, read from an -`asap-stage-pipeline/v1` document written by the `stage_pipeline` devtool. -`stage_pipeline` uses the candidate-connection MILP, the same search used by -`plan_installable`. By default the document keeps the selected plan and its -validation, cost, and search reports. `--exhaustive` uses the same engine to -archive evaluated assignments within the configured time and evaluation -budgets; only `catalog_exhausted` establishes complete traversal. The document -shows: +`asap-stage-pipeline/v1` document written by the `asap_planner` devtool +(`asap_planner --workload --run-until stage3 --format json`). The +document shows: - **Stage 0**, the query-language frontends' DAG, with one root per query; -- **Stage 1**, local computation choices and reconstructed logical plans for - evaluated candidate selections; -- **Stage 2**, executable physical plans, their materialization assignments, - and any recorded physical-legality rejection; -- **Stage 3**, validation, configured objective values, the selected plan, - and bounded-search status. A feasible selected plan need not be a proven optimum. +- **Stage 1**, the logical candidates of the memo and the selected plan's + logical DAG (`L1`); +- **Stage 2**, the selected plan as an executable physical DAG (`P1`), with + each node's timing and materialization, and the physical candidates Stage 2 + pruned; +- **Stage 3**, the selected plan's cost, the accuracy it accepted per query, + its materialized views and the search report. To run it, see [RUNNING.md](RUNNING.md). ## The page - **Examples**: one tab per entry of `examples.json`: #509 Examples 1, 3a - and 3b, workload-01 of each ASAPQuery random workload set (below), and - six sharing examples, each one kind of sharing on the smallest subset - of a paper workload template (below). `#example3b` in the URL opens that - example. + and 3b. The local server plans each one the first time it starts. + `#example3b` in the URL opens that example. - **Workload sets** (button): one table per set listed in `workloads/index.json`, one row per workload with its query count, query labels, the queries (text, how often, accuracy) and the sources they - read. Clicking a row plans it with the local server (`/api/plan-workload`: - `stage_pipeline --workload`, default search, the set's - `deployment_capabilities`, `cost_model` and `accuracy_guarantees`, and - for a SQL workload the tables in the `tables.json` next to its file). - Without the server, a row - with a precomputed `document` opens it and the others say they need the - server. `#workload-sets` in the URL opens the section. + read. Clicking a row plans it with the local server, as the set says + (deployment capabilities, cost model, accuracy guarantees). Without the + server, the rows say they need it. `#workload-sets` in the URL opens the + section. - **Query workload** with each query's accuracy, latency, recurrence, predictability and range evaluation, and a line that counts the candidates at each stage. - **Data workload** beside it: the arrival and one row per declared source - with exactly the facts `SourceFacts` has (series count, metric type, + with exactly the facts it declares (series count, metric type, non-negative values, sample interval, stream norms, distinct bound, - distinct values per label, distinct values per label set) and - the queries whose text names it, plus the deployment capabilities the - run planned with. -- **Stage 3 · plans by cost**: every candidate the document shows. First the - selected plan, then the other valid ones cheapest first, then the - rejected ones with their reasons (or "not judged" when plan selection - failed with an error), then the Stage 2 pruned ones and the - Stage 1 candidates that did not compose, each with its reason. The header - says how many candidates Stage 1 enumerated, how many the document shows - when it caps them, and why Stage 1 or plan selection failed when it did. - Clicking a row shows it in the lanes. The header also names the search and - what it checked, why the workload was searched as one when it was - (`searched_as_one`, e.g. "searched as one: not in joint space") and the - classes of interchangeable queries that took one shared choice - (`interchangeable_queries`, e.g. "interchangeable queries: (p50, p90, - p99)"); under the default search the list has the selected plan - and the rejections Stage 3 recorded, which are not clickable. + distinct values per label, distinct values per label set) and the queries + whose text names it, plus the deployment capabilities the run planned + with. +- **Stage 3 · plans by cost**: the selected plan, then the Stage 2 pruned + candidates and the Stage 1 candidates that did not compose, each with its + reason. The header counts the candidates, names the search and its report + (status, cost lower bound, MILP size, times), says whether accuracy + guarantees were enforced or ignored, shows the run's warnings, and says + why Stage 1 or plan selection failed when it did. - **Three lanes**, stacked top to bottom so each is as wide as the page: - Stage 0, the chosen Stage 1 candidate, and its Stage 2 candidate - with each node's timing (also its fill: green at ingestion time, orange - at query time; the border keeps its category) and Stage 3 cost (`× n` when a node that is not - materialized is computed once per reader). Each lane is the whole - workload's DAG, so a node shared by several queries appears once. Nodes - are data sources, relational operators or summary operators; summary - builds print their configuration (for example `CmsWithHeap · depth 7 · - heap 100 · width 272`) and whether there is one per group, one per - series, or one shared instance. A standalone scalar query (`time()`, - `scalar(v)`, `1 + 2`) is a dashed `Scalar` node over the nodes it reads. - Query roots have a thick border. A node that several queries' roots reach - says `shared ×n`. A `SummaryTumblingWindowChain` is one node (`30 tumbling windows × 20 s`); clicking - it draws its tumbling windows, newest first, each a `TimeShift` by its offset (none - for the newest), a `TimeRange` of one tumbling window and the tumbling window's build with its - window (`window t−40 s … t−20 s`), and clicking a tumbling window folds them back. - This is display only: the document keeps the chain. -- **Enumerated DAGs**, between the Stage 0 and Stage 1 lanes, shown only - when the document has `memo.enumeration`: the complete candidate DAGs the - memo stands for. The header counts them after each memo step (`Stage 0: 1 · - Pass 1: 49 · Pass 2: ≥ 1000`; `≥` and the limit when counting stopped), says how - many of the enumerated step's DAGs the document lists, and, when they were - priced, how many are cheaper than the selected plan. Each row names the - groups whose candidate differs from the selected plan's choice - (`G7 → #2 CountSketch+heap (pass1)`: group, candidate index, its label and - origin when the step's memo lists the candidate; a group a choice does not - name takes candidate 0). The selected choice is over the last memo step; - when an earlier step was enumerated, the header says so and only each - row's own groups are compared. A row also shows `accuracy budget parts > 1` - when a chosen candidate is sized for a part of its budget, and, when priced, its cost and - validity (`✗ invalid` with the cause). A row cheaper than the selected - plan is red; the selected choice says `✓ selected`. Clicking a row draws - its DAG in the Stage 1 lane; the document holds every listed choice's - composed DAG, so the page composes nothing. Such a DAG has no Stage 2 - plan, so the Stage 2 lane is cleared. Its roots are its root groups, in - increasing group id; the lane draws each as the root of the queries in its - group's `roots_of`, or labels it `root group #n` when that cannot be - matched (the enumerated step's memo is not in the document). When the enumerated step is earlier than the last - one, no row is marked selected. + Stage 0, the Stage 1 candidate, and its Stage 2 plan with each node's + timing (also its fill: green at ingestion time, orange at query time) and + Stage 3 cost (`× n` when a node that is not materialized is computed once + per reader). Each lane is the whole workload's DAG, so a node shared by + several queries appears once and says `shared ×n`. Summary builds print + their configuration (for example `CmsWithHeap · depth 7 · heap 100 · + width 272`) and whether there is one per group, one per series, or one + shared instance. A standalone scalar query (`time()`, `scalar(v)`, + `1 + 2`) is a dashed `Scalar` node over the nodes it reads. Query roots + have a thick border. A `SummaryTumblingWindowChain` is one node (`6 + tumbling windows × 10 s`); clicking it draws its tumbling windows, newest + first, each with its window (`window t−20 s … t−10 s`), and clicking a + tumbling window folds them back. - **Highlighting**: the **highlight query** menu dims, in every lane, what that query's root does not reach. Clicking a node dims everything but - what it reads and what reads it, directly or through other nodes; clicking - the background goes back to the query's highlight. + what it reads and what reads it; clicking the background goes back to the + query's highlight. - **Materialized views** under the Stage 2 lane: the selected plan's - materialized views as `plan_installable` gives them, one row per Stage 2 node: its - coverage definition, operator and summary parameters, tumbling windows (`30 × 20 s` - for a tumbling-window chain), how it is refreshed, its retention, the Stage 3 cost and the - queries that read it. A chain's row lists its installed tumbling windows (node id, - tumbling window, window, retention) under **installed**. Clicking a row selects its - node in the Stage 2 lane. Stage 3 predicts no - per-node state size, so the table has none. The lane's header also shows - the shown candidate's Stage 3 usage. -- **Search summary**, under the materialized views: candidate-connection reports - show the incumbent cost, available lower bound and gap, refinement count, - and solver status. Historical documents may also contain a legacy memo-plan - view of shared units. Rendering that archived format does not enable a second - planner implementation; new documents use the candidate-connection report. -- **Memo**, when the document has a `memo` section: the memo after each - step, one tab per step (`Stage 0`, `Stage 1, Pass 1`, `Stage 1, Pass 2`; - a step production did not run is not there), opening on - the last. Each memo group is a box `G ·