feat(native): report SQL test and scenario errors natively (#1046)

Scenario extraction, scope relationship scans and macro-test CTE scans
return Python's exact scanner and classification errors from one native
port of Python's top-level CTE scanner, under the adapter's lexical rules.
Scenario source checks no longer fall back to Python on a reference
error. Only Polyglot steps and non-ASCII classification still run in
Python.

Author: kvlonge

Cargo.lock

@@ -11,11 +11,11 @@ that can be tested and benchmarked with `cargo test` alone.
 | `sqlbuild-sqltext` | Lexical SQL text without polyglot: comment, quote and parenthesis scanning, the rules quote policy, model header tokenization and matching, static variable substitution, static reference extraction, and the general reference scan with table-function call arguments under each adapter's lexical rules. |
 | `sqlbuild-config` | Configuration file reading without Python: `tomllib`-compatible TOML, PyYAML `safe_load`-compatible YAML 1.1, and typed project and local config readers for the fields discovery needs. Errors carry their kind and position; YAML outside the supported subset is an `Unsupported` error. |
 | `sqlbuild-discovery` | Native project discovery and the only implementation of the model, SQL test, scenario, source and schema file collections and the declaration layout: the shared directory walk with Python's glob and sort semantics, file reading with Python's newline handling and read errors, and parsing of authored files with Python's exact discovery messages. Results are plain data the Python discovery facade materialises. |
-| `sqlbuild-scopes` | Declaration scopes: the scope index with Python's record orders and diagnostics, declaration visibility, relationship grants, the scope lookup groups, and the dialect-aware scan of SQL tests and scenarios for their expected models. Anything it cannot reproduce exactly defers to Python. |
+| `sqlbuild-scopes` | Declaration scopes: the scope index with Python's record orders and diagnostics, declaration visibility, relationship grants, the scope lookup groups, and the dialect-aware scan of SQL tests and scenarios for their expected models and top-level CTEs, with Python's scanner errors. Anything it cannot reproduce exactly defers to Python. |
 | `sqlbuild-model-config` | Model configuration: MODEL header columns and audits, `${...}` template expansion with its environment and context reads, and the template and macro presence scans over authored config values, all read through a trait over the caller's values. Anything it cannot reproduce exactly defers to Python, which also raises every model config error. |
 | `sqlbuild-render` | Native rendering: the macro call scanner (a byte-for-byte port of Python's), splicing of rendered calls with code-point spans, and the in-compile memo of recorded macro calls and their replayable events, which can carry results across compiles through the shared native store. Anything it cannot reproduce exactly defers to Python. |
 | `sqlbuild-attachments` | Compile attachments for tests, audits, sources, functions, scenarios and seeds: attached generic audit argument merge, raw and quoted argument rendering, and severity and run scope resolution. Anything it cannot reproduce exactly defers to Python, which also raises every attachment error. |
-| `sqlbuild-analysis` | SQL analysis over polyglot: SQL tokens, query analysis, the binding catalog, semantic validation and usage, column references, type normalization, and SQL-test extraction, planning and rendering. Type normalization defers to Python for anything it cannot reproduce exactly. |
+| `sqlbuild-analysis` | SQL analysis over polyglot: SQL tokens, query analysis, the binding catalog, semantic validation and usage, column references, type normalization, and SQL-test and scenario extraction, planning and rendering. Type normalization defers to Python for anything it cannot reproduce exactly. |
 | `sqlbuild-rules` | Built-in rules and the rules engine, the custom-rule host, SQL lint, quality checks and formatting, rules configuration and the request models. It also owns the build identity script. |
 | `sqlbuild-python` | The only PyO3 crate: the `_native` module, its Python classes and functions, conversions from Python objects, and the process allocator. |
 
@@ -31,7 +31,7 @@ sqlbuild-python
 ```
 
 `sqlbuild-config` and `sqlbuild-model-config` do not depend on the crates below them today, and
-`sqlbuild-scopes` uses only `sqlbuild-sqltext`; their place in the order fixes which crates may
+`sqlbuild-scopes` uses only `sqlbuild-sqltext` and `sqlbuild-core`; their place in the order fixes which crates may
 use them. The JSON emitter
 lives in `sqlbuild-core` so that any layer can produce text that must equal Python's
 `json.dumps` output.

crates/README.md

@@ -21,6 +21,7 @@ serde.workspace = true
 serde_json.workspace = true
 sha2.workspace = true
 sqlbuild-core.workspace = true
+sqlbuild-scopes.workspace = true
 sqlbuild-sqltext.workspace = true
 
 [lints]

crates/sqlbuild-analysis/Cargo.toml

@@ -69,15 +69,6 @@ struct TestRequest {
     authored: bool,
 }
 
-/// How strictly top-level CTE names are read.
-#[derive(Clone, Copy, Debug, PartialEq, Eq)]
-pub(crate) enum NameRule {
-    /// Unquoted ASCII identifiers only; anything else is an error naming the CTE.
-    Plain,
-    /// Quoted and Unicode identifiers are read, and the caller decides what to do with them.
-    Lenient,
-}
-
 /// One extraction error, with help and the offending text where the author can act on it.
 #[derive(Debug, PartialEq, Eq)]
 pub(crate) struct Failure {
@@ -198,8 +189,7 @@ fn extract_test(test: &TestRequest, syntax: &LexicalSyntax) -> Result<Classified
     if test.authored {
         return authored_ctes(test, syntax);
     }
-    let (ctes, _) =
-        extract_ctes_with_quoting(&test.sql, &test.file_label, syntax, NameRule::Plain)?;
+    let ctes: Vec<Cte> = scan_ctes(&test.sql, &test.file_label, syntax)?.ctes;
     let scope = TestScope {
         file: &test.file_label,
         syntax,
@@ -214,7 +204,7 @@ fn extract_test(test: &TestRequest, syntax: &LexicalSyntax) -> Result<Classified
 
 /// The authored block's CTEs, or the located error of a CTE name; other errors read as no CTEs.
 fn authored_ctes(test: &TestRequest, syntax: &LexicalSyntax) -> Result<Classified, Failure> {
-    match scan_ctes(&test.sql, &test.file_label, syntax, NameRule::Plain) {
+    match scan_ctes(&test.sql, &test.file_label, syntax) {
         Ok(scanned) => Ok(Classified::Authored {
             ctes: scanned
                 .ctes
@@ -228,31 +218,14 @@ fn authored_ctes(test: &TestRequest, syntax: &LexicalSyntax) -> Result<Classifie
     }
 }
 
-/// The top-level CTEs and whether any CTE name was a quoted identifier.
-pub(crate) fn extract_ctes_with_quoting(
-    sql: &str,
-    file: &str,
-    syntax: &LexicalSyntax,
-    rule: NameRule,
-) -> Result<(Vec<Cte>, bool), Failure> {
-    scan_ctes(sql, file, syntax, rule).map(|scanned| (scanned.ctes, scanned.quoted))
-}
-
 /// The top-level CTEs of one test, with the code-point offset of each stripped body.
 struct ScannedCtes {
     ctes: Vec<Cte>,
-    quoted: bool,
     body_starts: Vec<usize>,
 }
 
-fn scan_ctes(
-    sql: &str,
-    file: &str,
-    syntax: &LexicalSyntax,
-    rule: NameRule,
-) -> Result<ScannedCtes, Failure> {
+fn scan_ctes(sql: &str, file: &str, syntax: &LexicalSyntax) -> Result<ScannedCtes, Failure> {
     let mut body_starts: Vec<usize> = Vec::new();
-    let mut quoted = false;
     let mut index = skip_ignorable(sql, 0, syntax)?;
     index = consume_keyword(sql, index, "WITH").ok_or_else(|| {
         format!("SQL test '{file}' must declare mock CTEs and one __expected__<model> CTE in a top-level WITH clause")
@@ -264,12 +237,7 @@ fn scan_ctes(
     let mut ctes: Vec<Cte> = Vec::new();
     let mut seen: HashSet<String> = HashSet::new();
     loop {
-        quoted |= matches!(byte_at(sql, index), Some(b'"' | b'`'));
-        let (name, end) = match rule {
-            NameRule::Plain => read_cte_name(sql, index, file, syntax)?,
-            NameRule::Lenient => read_identifier(sql, index, syntax)
-                .ok_or_else(|| format!("SQL test '{file}' expected a CTE name"))?,
-        };
+        let (name, end) = read_cte_name(sql, index, file, syntax)?;
         if !seen.insert(name.clone()) {
             return Err(format!("SQL test '{file}' defines duplicate CTE '{name}'").into());
         }
@@ -304,11 +272,7 @@ fn scan_ctes(
             "SQL test '{file}' must end after its CTEs; only an optional ceremonial top-level `SELECT 1` may follow them"
         ).into());
     }
-    Ok(ScannedCtes {
-        ctes,
-        quoted,
-        body_starts,
-    })
+    Ok(ScannedCtes { ctes, body_starts })
 }
 
 /// Read a top-level CTE name, which must be an unquoted ASCII identifier.

crates/sqlbuild-analysis/src/compiler/_helpers/sql_tests/extraction.rs

@@ -1,17 +1,21 @@
-//! Python's `extract_sql_scenario_ctes` for scenarios its top-level scanner reads exactly.
+//! Python's `extract_sql_scenario_ctes`: the exact top-level scan, then classification and errors.
+
+use std::collections::{HashMap, HashSet};
 
 use serde::Serialize;
+use sqlbuild_scopes::relationship_names::main::top_level_ctes::scan_top_level_ctes;
+use sqlbuild_scopes::relationship_names::models::{RelationshipSource, TopLevelCtes};
+use sqlbuild_sqltext::sql_scan::models::LexicalSyntax;
 
-use crate::compiler::_helpers::sql_tests::extraction::{
-    Cte, NameRule, extract_ctes_with_quoting, generic_syntax, validate_independence,
-};
+type Cte = (String, String);
 
-/// Characters whose Python case mapping reaches ASCII, which Python's keyword match would accept.
-const CASE_MAPPED_TO_ASCII: [char; 20] = [
-    '\u{df}', '\u{130}', '\u{131}', '\u{149}', '\u{17f}', '\u{1f0}', '\u{1e96}', '\u{1e97}',
-    '\u{1e98}', '\u{1e99}', '\u{1e9a}', '\u{1e9e}', '\u{212a}', '\u{fb00}', '\u{fb01}', '\u{fb02}',
-    '\u{fb03}', '\u{fb04}', '\u{fb05}', '\u{fb06}',
-];
+const SOURCE_PREFIX: &str = "__source__";
+const REF_PREFIX: &str = "__ref__";
+const SEED_PREFIX: &str = "__seed__";
+const DBT_REF_PREFIX: &str = "__dbt_ref__";
+const EXPECTED_PREFIX: &str = "__expected__";
+const ASSERT_PREFIX: &str = "__assert__";
+const MACRO_PREFIX: &str = "__macro__";
 
 #[derive(Debug, Default, Serialize)]
 #[serde(rename_all = "camelCase")]
@@ -27,77 +31,169 @@ struct ClassifiedScenario {
     assertion_names: Vec<String>,
 }
 
-pub(crate) fn extract_scenario_json(sql: &str, file: &str) -> Result<Option<String>, String> {
-    if sql.chars().any(|character| {
-        matches!(character, '\u{1c}'..='\u{1f}') || CASE_MAPPED_TO_ASCII.contains(&character)
-    }) {
-        return Ok(None);
+/// One scenario's scanner error, or its classification after any Polyglot independence check.
+#[derive(Debug, Default, Serialize)]
+#[serde(rename_all = "camelCase")]
+struct ScenarioOutcome {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    scan_error: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    independence: Option<Vec<Cte>>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    error: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    scenario: Option<ClassifiedScenario>,
+}
+
+pub(crate) fn extract_scenario_json(
+    sql: &str,
+    file: &str,
+    syntax: &LexicalSyntax,
+) -> Result<Option<String>, String> {
+    let outcome: ScenarioOutcome =
+        match scan_top_level_ctes(sql, file, RelationshipSource::Scenario, syntax) {
+            TopLevelCtes::Deferred => return Ok(None),
+            TopLevelCtes::Failed(message) => ScenarioOutcome {
+                scan_error: Some(message),
+                ..ScenarioOutcome::default()
+            },
+            TopLevelCtes::Scanned(ctes) => {
+                let independence: Option<Vec<Cte>> =
+                    python_checks_independence(&ctes).then(|| ctes.clone());
+                match classify(ctes, file) {
+                    Ok(scenario) => ScenarioOutcome {
+                        independence,
+                        scenario: Some(scenario),
+                        ..ScenarioOutcome::default()
+                    },
+                    Err(message) => ScenarioOutcome {
+                        independence,
+                        error: Some(message),
+                        ..ScenarioOutcome::default()
+                    },
+                }
+            }
+        };
+    serde_json::to_string(&outcome)
+        .map(Some)
+        .map_err(|error| error.to_string())
+}
+
+/// Whether Python's independence check parses a body; when it does not, the check passes.
+fn python_checks_independence(ctes: &[Cte]) -> bool {
+    let mut names_by_key: HashMap<String, &str> = HashMap::new();
+    for (name, _) in ctes {
+        names_by_key.insert(name.to_ascii_lowercase(), name);
     }
-    let syntax = generic_syntax();
-    let Ok((ctes, quoted)) = extract_ctes_with_quoting(sql, file, &syntax, NameRule::Lenient)
-    else {
-        return Ok(None);
+    let keys_with_prefix = |prefix: &str| -> HashSet<&String> {
+        names_by_key
+            .iter()
+            .filter(|(_, name)| name.starts_with(prefix))
+            .map(|(key, _)| key)
+            .collect()
     };
-    if quoted || !ctes.iter().all(|cte| is_plain_identifier(&cte.0)) {
-        return Ok(None);
+    let expected: HashSet<&String> = keys_with_prefix(EXPECTED_PREFIX);
+    let assertions: HashSet<&String> = keys_with_prefix(ASSERT_PREFIX);
+    for (name, body) in ctes {
+        let key: String = name.to_ascii_lowercase();
+        let prohibited: &str = if expected.contains(&key) {
+            ASSERT_PREFIX
+        } else if assertions.contains(&key) {
+            EXPECTED_PREFIX
+        } else {
+            continue;
+        };
+        if folded_body_may_contain(body, prohibited) {
+            return true;
+        }
     }
-    if validate_independence(&ctes, file, &syntax).is_err() {
-        return Ok(None);
+    if expected.is_empty() || assertions.is_empty() {
+        return false;
     }
-    let Some(scenario) = classify(ctes) else {
-        return Ok(None);
-    };
-    serde_json::to_string(&scenario)
-        .map(Some)
-        .map_err(|error| error.to_string())
+    for (_, body) in ctes {
+        for key in names_by_key.keys() {
+            if folded_body_may_contain(body, key) {
+                return true;
+            }
+        }
+    }
+    false
 }
 
-/// Python's `_read_identifier` reads only `[A-Za-z_][A-Za-z0-9_]*` the same way.
-fn is_plain_identifier(name: &str) -> bool {
-    let mut bytes = name.bytes();
-    bytes
-        .next()
-        .is_some_and(|first| first.is_ascii_alphabetic() || first == b'_')
-        && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
+/// Whether the body's Python `casefold` may contain the text; non-ASCII may fold onto ASCII.
+fn folded_body_may_contain(body: &str, lower_ascii: &str) -> bool {
+    !body.is_ascii() || body.to_ascii_lowercase().contains(lower_ascii)
 }
 
-fn classify(ctes: Vec<Cte>) -> Option<ClassifiedScenario> {
+fn classify(ctes: Vec<Cte>, file: &str) -> Result<ClassifiedScenario, String> {
     let mut scenario: ClassifiedScenario = ClassifiedScenario::default();
     for cte in ctes {
-        let name: String = cte.0.clone();
-        if let Some(value) = name.strip_prefix("__source__") {
-            scenario.source_fixtures.push(required(value)?);
+        let name: &str = &cte.0;
+        if let Some(value) = name.strip_prefix(SOURCE_PREFIX) {
+            scenario
+                .source_fixtures
+                .push(required(value, "__source__<source>", file)?);
             scenario.authored.push(cte);
-        } else if let Some(value) = name.strip_prefix("__ref__") {
-            scenario.ref_fixtures.push(required(value)?);
+        } else if let Some(value) = name.strip_prefix(REF_PREFIX) {
+            scenario
+                .ref_fixtures
+                .push(required(value, "__ref__<model>", file)?);
             scenario.authored.push(cte);
-        } else if let Some(value) = name.strip_prefix("__seed__") {
-            scenario.seed_fixtures.push(required(value)?);
+        } else if let Some(value) = name.strip_prefix(SEED_PREFIX) {
+            scenario
+                .seed_fixtures
+                .push(required(value, "__seed__<seed>", file)?);
             scenario.authored.push(cte);
-        } else if let Some(value) = name.strip_prefix("__dbt_ref__") {
-            scenario.dbt_ref_fixtures.push(required(value)?);
+        } else if let Some(value) = name.strip_prefix(DBT_REF_PREFIX) {
+            scenario.dbt_ref_fixtures.push(required(
+                value,
+                "__dbt_ref__<model> or __dbt_ref__<package>__<model>",
+                file,
+            )?);
             scenario.authored.push(cte);
-        } else if let Some(value) = name.strip_prefix("__expected__") {
-            scenario.expected_models.push(required(value)?);
+        } else if let Some(value) = name.strip_prefix(EXPECTED_PREFIX) {
+            scenario
+                .expected_models
+                .push(required(value, "__expected__<model>", file)?);
             scenario.expected.push(cte);
-        } else if let Some(value) = name.strip_prefix("__assert__") {
-            scenario.assertion_names.push(required(value)?);
+        } else if let Some(value) = name.strip_prefix(ASSERT_PREFIX) {
+            scenario
+                .assertion_names
+                .push(required(value, "__assert__<assertion>", file)?);
             scenario.assertions.push(cte);
-        } else if name.starts_with("__macro__") {
-            return None;
+        } else if name.starts_with(MACRO_PREFIX) {
+            return Err(format!(
+                "SQL scenario '{file}' does not support macro mock CTE '{name}'. Scenarios run \
+                 real project macros; use SQL unit tests for macro mocks."
+            ));
         } else {
             scenario.authored.push(cte);
         }
     }
-    let has_fixture: bool = !(scenario.source_fixtures.is_empty()
+    if scenario.source_fixtures.is_empty()
         && scenario.ref_fixtures.is_empty()
         && scenario.seed_fixtures.is_empty()
-        && scenario.dbt_ref_fixtures.is_empty());
-    let has_check: bool =
-        !(scenario.expected_models.is_empty() && scenario.assertion_names.is_empty());
-    (has_fixture && has_check).then_some(scenario)
+        && scenario.dbt_ref_fixtures.is_empty()
+    {
+        return Err(format!(
+            "SQL scenario '{file}' must define at least one __source__*, __ref__*, __seed__*, or \
+             __dbt_ref__* fixture CTE"
+        ));
+    }
+    if scenario.expected_models.is_empty() && scenario.assertion_names.is_empty() {
+        return Err(format!(
+            "SQL scenario '{file}' must define at least one __expected__<model> or \
+             __assert__<assertion> CTE"
+        ));
+    }
+    Ok(scenario)
 }
 
-fn required(value: &str) -> Option<String> {
-    (!value.is_empty()).then(|| value.to_owned())
+fn required(value: &str, label: &str, file: &str) -> Result<String, String> {
+    if value.is_empty() {
+        return Err(format!(
+            "SQL scenario '{file}' must use {label} to identify a target"
+        ));
+    }
+    Ok(value.to_owned())
 }