Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,14 @@ Documentation edits usually do release, on purpose: `make skills` copies the doc
installed skill stays in step with the docs. A homepage or styling change touches only `website/`
and does not.

Merge pull requests one at a time, each with its branch up to date with `main`. The release
workflow only trusts a release base whose tree passed Verify on the pull request that produced
it. If two pull requests merge back to back, `main` holds a combination no pull request verified,
and the release workflow stops before it refreshes the release pull request's lockfiles. A later
run that leaves the release pull request unchanged does not repair it. To recover, close the
release pull request, then merge the next up-to-date pull request; Release Please opens a fresh
release pull request from the verified tree.

## One-time setup

A repository administrator does these once, in a browser:
Expand Down
Binary file modified .github/demos/check-fix.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/sqlclean-logo-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/sqlclean-logo-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
53 changes: 28 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,44 @@
<p align="center">
<img src="https://raw.githubusercontent.com/sqlstack-io/sqlclean/main/.github/sqlclean-logo.png" alt="SQLClean" width="100%">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/sqlstack-io/sqlclean/main/.github/sqlclean-logo-dark.png">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/sqlstack-io/sqlclean/main/.github/sqlclean-logo-light.png">
<img src="https://raw.githubusercontent.com/sqlstack-io/sqlclean/main/.github/sqlclean-logo-light.png" alt="SQLClean" width="100%">
</picture>
</p>

<h2 align="center">The fastest SQL linter.</h2>

<p align="center">
<strong>50× faster than sqruff. Lint 10,000 SQL files in under a second.</strong> <a href="https://sqlclean.io/docs/benchmarks/">Benchmarks</a>
</p>

<p align="center">
Catch SQL mistakes, apply safe fixes, and format consistently. Add your own rules in Python.
</p>

<p align="center">
<br>
<a href="https://sqlclean.io">Website</a> ·
<a href="https://sqlclean.io/docs/quickstart/">Get started</a> ·
<a href="https://sqlclean.io/docs/">Docs</a> ·
<a href="https://sqlclean.io/docs/benchmarks/">Benchmarks</a>
</p>

Catch SQL mistakes, apply safe fixes, and format consistently. Add your own rules in Python.
SQLClean parses and checks SQL in Rust, and runs your Python Rules on statements it has already
parsed.

```bash
uv tool install sqlclean # or: pip install sqlclean
```

<p align="center">
<img src="https://raw.githubusercontent.com/sqlstack-io/sqlclean/main/.github/demos/check-fix.gif" alt="sqlclean check reports three findings, then sqlclean fix applies two fixes and formats the file" width="100%">
<img src="https://raw.githubusercontent.com/sqlstack-io/sqlclean/main/.github/demos/check-fix.gif" alt="sqlclean check reports an unused CTE, a temporary table from a custom Rule and a comparison with NULL; sqlclean fix removes the CTE and formats the file; sqlclean fix --unsafe-fixes rewrites = NULL to IS NULL" width="100%">
</p>

`sqlclean check` reports a bare `JOIN`, a `COUNT(1)` and a temporary table from a custom Rule.
`sqlclean fix` applies the two fixes and formats the file; the custom Rule's finding has no fix,
so it stays.
`sqlclean check` reports an unused CTE, a temporary table from a custom Rule and a `= NULL`
comparison that is never true. `sqlclean fix` applies the safe fix, removing the CTE, and formats
the file. It leaves `= NULL` alone because rewriting it to `IS NULL` changes which rows match;
`sqlclean fix --unsafe-fixes` applies that fix. The custom Rule's finding has no fix, so it stays.

## Speed

Linting the same 10,282 Snowflake SQL files on 4 CPUs, with no cache:

| Command | Wall time | vs `sqlclean check` |
| Command | Wall time | Time relative to `sqlclean check` |
|---|---:|---:|
| `sqlclean check` (default Rules) | **0.81 s** | 1× |
| `sqlclean check --select SQC` (all Rules) | **1.33 s** | 1.6× |
Expand All @@ -49,6 +52,18 @@ The tools don't check the same things, so see the
[benchmarks](https://sqlclean.io/docs/benchmarks/) for coverage, the corpus, the method and how
to run it yourself.

## Quick start

```bash
sqlclean check # report findings in every .sql file under the directory
sqlclean fix # apply safe fixes, then format
sqlclean format # rewrite files in the canonical layout
```

Add a `sqlclean.toml` to choose a dialect, files or Rules. See the
[quickstart](https://sqlclean.io/docs/quickstart/) and
[configuration](https://sqlclean.io/docs/configuration/).

## Correct and safe

- **Rules for SQL that runs but is wrong.** Built-in Rules flag comparisons with `NULL`, `NOT IN`
Expand Down Expand Up @@ -88,18 +103,6 @@ Point `[rules] custom = ["sql_rules"]` in `sqlclean.toml` at it. See
[custom Rules](https://sqlclean.io/docs/custom-rules/overview/) and
[testing custom Rules](https://sqlclean.io/docs/custom-rules/testing/).

## Quick start

```bash
sqlclean check # report findings in every .sql file under the directory
sqlclean fix # apply safe fixes, then format
sqlclean format # rewrite files in the canonical layout
```

Add a `sqlclean.toml` to choose a dialect, files or Rules. See the
[quickstart](https://sqlclean.io/docs/quickstart/) and
[configuration](https://sqlclean.io/docs/configuration/).

## Integrations

```yaml
Expand Down
2 changes: 1 addition & 1 deletion website/recordings/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ scripts are the source; the rendered GIF, MP4 and PNG files go to `out/`, which

| Demo | Shows |
|------|-------|
| `check-fix` | `sqlclean check` reports built-in and custom findings, and `sqlclean fix` fixes and formats. |
| `check-fix` | `sqlclean check` reports built-in and custom findings, `sqlclean fix` applies the safe fix and formats, and `sqlclean fix --unsafe-fixes` applies the unsafe one. |

The README uses `check-fix`, copied into `.github/demos/`.

Expand Down
35 changes: 11 additions & 24 deletions website/recordings/prepare.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,25 @@ mkdir -p "$dir"
cd "$dir"
case "$demo" in
check-fix)
# One messy query: a bare JOIN, COUNT(1) and a temporary table that a custom Rule forbids.
# One query with a NULL comparison, an unused CTE and a temporary table that a custom Rule forbids.
mkdir -p models sql_rules
cat > sqlclean.toml <<'TOML'
dialect = "duckdb"

[rules]
select = ["SQC025", "XSQC"]
custom = ["sql_rules"]
TOML
cat > models/orders.sql <<'SQL'
select o.order_id, c.name, count(1) as items
from orders o join tmp_customers c on o.customer_id = c.customer_id
group by o.order_id, c.name
with refunds as (
select order_id from payments where status = 'refunded'
)
select o.order_id, c.name, o.total
from orders o
join tmp_customers c on o.customer_id = c.customer_id
where o.shipped_at = null
SQL
cat > sql_rules/style.py <<'PY'
from sqlclean import FixSafety, RuleContext, Statement, rule
cat > sql_rules/naming.py <<'PY'
from sqlclean import Finding, RuleContext, Statement, rule


@rule(
Expand All @@ -33,28 +36,12 @@ from sqlclean import FixSafety, RuleContext, Statement, rule
remediation="Read from the published table instead.",
enabled_by_default=True,
)
def temporary_table(*, statement: Statement, ctx: RuleContext) -> list:
def temporary_table(*, statement: Statement, ctx: RuleContext) -> list[Finding]:
return [
ctx.finding(subject=statement, node=table)
for table in statement.tables
if table.name.startswith("tmp_")
]


@rule(
code="XSQCSTYLE001",
message="COUNT(1) should be COUNT(*)",
remediation="Write COUNT(*).",
fixable=FixSafety.SAFE,
enabled_by_default=True,
)
def count_one(*, statement: Statement, ctx: RuleContext) -> list:
counts = {f.end + 1 for f in statement.functions if f.name.upper() == "COUNT"}
return [
ctx.finding(subject=statement, node=literal, fix=[ctx.replace(node=literal, text="*")])
for literal in statement.literals
if literal.text == "1" and literal.start in counts
]
PY
;;
*)
Expand Down
26 changes: 22 additions & 4 deletions website/recordings/tapes/check-fix.tape
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Source tapes/common.tape
Set Height 820
Set Height 880
Output out/check-fix.gif
Output out/check-fix.mp4
Hide
Expand All @@ -14,8 +14,26 @@ Type "sqlclean check"
Sleep 300ms
Enter
Wait@60s /\$\s*$/
Sleep 3s
Type "clear && sqlclean fix"
Sleep 5s
Hide
Type "clear"
Enter
Sleep 300ms
Show
Type "sqlclean fix"
Sleep 300ms
Enter
Wait@60s /\$\s*$/
Sleep 1500ms
Type "cat models/orders.sql"
Enter
Sleep 5s
Hide
Type "clear"
Enter
Sleep 300ms
Show
Type "sqlclean fix --unsafe-fixes"
Sleep 300ms
Enter
Wait@60s /\$\s*$/
Expand All @@ -24,4 +42,4 @@ Type "cat models/orders.sql"
Enter
Sleep 1s
Screenshot out/check-fix.png
Sleep 8s
Sleep 5s
Loading