Skip to content

chore: remove the Polyglot equivalence check #175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kvlonge merged 2 commits into from
Nov 11, 2026
Merged

chore: remove the Polyglot equivalence check #175

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e913118
chore: remove the Polyglot equivalence check
kvlonge Nov 11, 2026
bbd150f
ci: stop requiring the removed exact check's test binary
kvlonge Nov 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ jobs:
with:
workspaces: fuzz
shared-key: fuzz
- name: Seed the corpus with the exact check's statements
- name: Seed the corpus with the corpora's statements
run: python3 -m scripts.seed_fuzz_corpus --out fuzz/corpus/tolerant
- name: Fuzz
working-directory: fuzz
Expand Down
49 changes: 0 additions & 49 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 0 additions & 12 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ members = [
"crates/sqlquack-compare",
"crates/sqlquack-evidence",
"crates/sqlquack",
"tools/sqlquack-polyglot-check",
"tools/sqlquack-coverage-check",
"crates/sqlquack-syntax-postgres",
"crates/sqlquack-lint",
Expand Down Expand Up @@ -45,17 +44,6 @@ anstream = "1.0"
arrow = { version = "60", default-features = false, features = ["ipc"] }
clap = { version = "4.5", features = ["derive"] }
duckdb = "=1.20000.0"
polyglot-sql = { git = "https://github.com/sqlstack-io/polyglot.git", rev = "7d5fe2ccce7b3cbd1413a2823560f8dfc63ecc2e", default-features = false, features = [
"dialect-snowflake",
"dialect-duckdb",
"dialect-bigquery",
"dialect-databricks",
"dialect-postgresql",
"dialect-sqlite",
"generate",
"semantic",
"stacker",
] }
proc-macro2 = "1"
quote = "1"
rayon = "1"
Expand Down
33 changes: 15 additions & 18 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ PYTHON_TYPED := scripts tests/unit/scripts python/src $(PYTHON_API_TESTS)
PYTHON_TESTS := tests/unit/scripts $(PYTHON_API_TESTS)
endif

.PHONY: distribution-notices format rust-lint python-check fensu syntax-coverage depth-probe depth-probe-debug instruction-counts instruction-baseline gate-build polyglot-guard engine-free bundled-extensions dialect-builds generate-cases generated-cases generate-syntax-cases syntax-cases dialect-cases corpus-rerecord generate-referee-cases referee-cases referee-pending translation-files translation-results help pending check-syntax record-syntax check-convert record-convert sql-sizes sql-size-baseline sql-benchmark check-ci check-merge test test-rust test-python python-module python-api python-wheel verify-recorded verify-changed replay-scope test-crate check-pr-metadata evidence-clone evidence-refresh evidence-pin evidence-pin-check clutter-guard polyglot-duckshop census-duckshop fuzz-smoke gate gate-parallel census duckshop-compiled duckshop-measure
.PHONY: distribution-notices format rust-lint python-check fensu syntax-coverage depth-probe depth-probe-debug instruction-counts instruction-baseline gate-build engine-free bundled-extensions dialect-builds generate-cases generated-cases generate-syntax-cases syntax-cases dialect-cases corpus-rerecord corpus-verdicts corpus-verdicts-baseline generate-referee-cases referee-cases referee-pending translation-files translation-results help pending check-syntax record-syntax check-convert record-convert sql-sizes sql-size-baseline sql-benchmark check-ci check-merge test test-rust test-python python-module python-api python-wheel verify-recorded verify-changed replay-scope test-crate check-pr-metadata evidence-clone evidence-refresh evidence-pin evidence-pin-check clutter-guard census-duckshop fuzz-smoke gate gate-parallel census duckshop-compiled duckshop-measure

format:
cargo fmt --all
Expand Down Expand Up @@ -116,10 +116,6 @@ evidence-pin-check:
clutter-guard:
uv run python -m scripts.check_clutter_guard

# Fails when a crate other than the dev-only exact check reaches polyglot at runtime.
polyglot-guard:
uv run python -m scripts.check_polyglot_guard

# Fails when the conversion pipeline crate, which builds without DuckDB's engine, reaches it.
engine-free:
@tree="$$(cargo tree --locked -p sqlquack-convert -e normal --prefix none)" || exit 1; \
Expand Down Expand Up @@ -170,6 +166,15 @@ generate-cases:
corpus-rerecord:
SQLQUACK_RERECORD=1 cargo test --locked -p sqlquack --lib everyday_corpus

# Checks SQLQuack's parse verdict on every statement of the corpora under corpora/polyglot-check
# against coverage/syntax/corpus-verdicts.json, and fails on any change.
corpus-verdicts: gate-build
$(GATE_DIR)/syntax-coverage corpus-verdicts

# Records the verdicts corpus-verdicts checks. Run it in the change that moves one, and commit it.
corpus-verdicts-baseline: gate-build
$(GATE_DIR)/syntax-coverage corpus-verdicts --update

# Fails when a generated case file differs from what its generator writes.
generated-cases:
uv run python -m scripts.generate_cases --check $(GENERATED_CASE_ARGS)
Expand Down Expand Up @@ -220,7 +225,7 @@ referee-pending:
uv run python -m scripts.write_referee_pending $(REFEREE_ARGS)

# The static gate, the first step of `gate`, and the Python API's tests.
check-ci: rust-lint python-check fensu syntax-coverage depth-probe instruction-counts evidence-pin-check clutter-guard polyglot-guard engine-free bundled-extensions dialect-builds generated-cases syntax-cases dialect-cases referee-cases translation-files python-api distribution-notices replay-scope ## The static gate CI runs.
check-ci: rust-lint python-check fensu syntax-coverage depth-probe instruction-counts corpus-verdicts evidence-pin-check clutter-guard engine-free bundled-extensions dialect-builds generated-cases syntax-cases dialect-cases referee-cases translation-files python-api distribution-notices replay-scope ## The static gate CI runs.

# Each distribution ships the licence and third-party notices; its copies must match the root files.
distribution-notices:
Expand Down Expand Up @@ -484,14 +489,6 @@ duckshop-compiled:
(cd $(DUCKSHOP_DIR)/duckshop-10k && $(abspath $(DUCKSHOP_DIR)/venv/bin/sqb) compile); \
echo "$(SQLBUILD_REF) $(SQLBUILD_VERSION)" > $(DUCKSHOP_DIR)/.ref

# The exact parser check over each compiled DuckShop project in DUCKSHOP_COMPILED.
polyglot-duckshop:
@for compiled in $(DUCKSHOP_COMPILED); do test -d "$$compiled" || \
{ echo "polyglot-duckshop: $$compiled is missing; run make duckshop-compiled" >&2; exit 1; }; done
for compiled in $(DUCKSHOP_COMPILED); do \
cargo run --locked --release -p sqlquack-polyglot-check -- directory "$$compiled" --dialect snowflake || exit 1; \
done

# Converting each compiled DuckShop project, with chained models and a diagnostic for each blocked
# file, must finish within DUCKSHOP_CENSUS_SECONDS. DuckShop 10K takes about 25 s on a development
# machine; 300 s is about twelve times that, room for slower CI runners, so only a pathological
Expand Down Expand Up @@ -537,7 +534,7 @@ duckshop-measure: duckshop-compiled
--sqlquack "$$target/release/sqlquack" --library "$$library" \
--work "$$out/work" --out "$$out/snowflake.json"

# Fuzzes each of FUZZ_TARGETS from the exact check's statements for FUZZ_SECONDS: tolerant parsing,
# Fuzzes each of FUZZ_TARGETS from the corpora's statements for FUZZ_SECONDS: tolerant parsing,
# and strict parsing through to rendered diagnostics.
fuzz-smoke:
uv run python -m scripts.seed_fuzz_corpus --out $(FUZZ_CORPUS)
Expand All @@ -546,8 +543,8 @@ fuzz-smoke:
-max_total_time=$(FUZZ_SECONDS) -max_len=4096 -timeout=10 -rss_limit_mb=2048) || exit 1; \
done

# Everything CI runs: the static gate, the tests, the recorded-case replay, the exact check and the
# timed conversion on DuckShop, and the fuzz smoke.
# Everything CI runs: the static gate, the tests, the recorded-case replay, the timed conversion on
# DuckShop, and the fuzz smoke.
# After the static gate, the independent targets run in parallel, each target's output kept
# together; any failure fails the gate. The fuzz smoke and then the SQL timing benchmark run alone
# afterwards, so each gets the machine to itself.
Expand All @@ -556,7 +553,7 @@ gate: check-ci $(if $(filter command line,$(origin DUCKSHOP_COMPILED)),,duckshop
$(MAKE) fuzz-smoke
$(MAKE) sql-benchmark

gate-parallel: test-rust test-python verify-recorded polyglot-duckshop census-duckshop
gate-parallel: test-rust test-python verify-recorded census-duckshop

# Example: make census CENSUS_ARGS="path/to/sql --json"
census:
Expand Down
4 changes: 2 additions & 2 deletions THIRD_PARTY_NOTICES
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ answer from Snowflake itself. sqlglot's expected DuckDB output is kept in the `s
field as reference side data only: it is never asserted, does not count toward verification and
is not part of the case hash.

The exact-check corpus corpora/polyglot-check/sqlglot.json holds the SQL of sqlglot's test
The corpus corpora/polyglot-check/sqlglot.json holds the SQL of sqlglot's test
fixtures at v30.14.0 (tests/fixtures/identity.sql, tests/test_parser.py and tests/dialects/),
as extracted by polyglot's fixture tooling. polyglot, below, is itself inspired by sqlglot and
ships sqlglot's licence. corpora/polyglot-check/sqlglot_postgres.json holds the SQL of
Expand Down Expand Up @@ -60,7 +60,7 @@ polyglot
SQLQuack's tokenizer, parser and syntax tree are a modified fork of polyglot
(https://github.com/tobilg/polyglot), as forked at https://github.com/sqlstack-io/polyglot.
The derived code is in crates/sqlquack-syntax-snowflake, which now holds most of it,
crates/sqlquack-syntax and crates/sqlquack-derive. The exact-check corpus
crates/sqlquack-syntax and crates/sqlquack-derive. The corpus
corpora/polyglot-check/polyglot_tests.json holds the SQL inputs of polyglot's tests.
SQLQuack's DuckDB result-type rules, in crates/sqlquack-semantics/src/duckdb/profile/_helpers/typing.rs,
are ported from polyglot's engine type rules, and the fixture
Expand Down
2 changes: 1 addition & 1 deletion corpora/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ Inputs of checks that never ship. Their outcomes are not recorded here.

| Directory | Read by |
|---|---|
| `polyglot-check/` | `tools/sqlquack-polyglot-check`, which proves that `sqlquack-syntax` parses exactly as polyglot does, and the fuzz corpus seed (`scripts/fuzz_corpus`). `expected_differences.toml` lists the reviewed differences. The `*.NOTICE` files and `THIRD_PARTY_NOTICES` give the sources' licences. |
| `polyglot-check/` | `syntax-coverage corpus-verdicts` (`make corpus-verdicts`), against SQLQuack's verdicts recorded in `coverage/syntax/corpus-verdicts.json`, and the fuzz corpus seed (`scripts/fuzz_corpus`). The `*.NOTICE` files and `THIRD_PARTY_NOTICES` give the sources' licences. |
| `duckdb-core/` | The DuckDB parser's corpus tests in `crates/sqlquack-syntax-duckdb`; see its README. |
| `databricks-core/` | The acceptance corpus for the Databricks parser, each statement tagged with the catalogue forms it uses; no test reads it yet. See its README. |
Loading
Loading