_{by SQLStack}

Streaming pipelines for ClickHouse, in typed SQL.

Change your SQL, see exactly what will rebuild, and replay it from retained stream history.
Website · Get started · Docs · Demo

SQLTide brings dbt-style authoring and deployment to streaming. Models compile into ClickHouse tables and materialized views, every build shows its plan first, and a change to your logic is rebuilt from retained Kafka history.

uv tool install sqltide        # or: pip install sqltide

Key features

• Typed SQL models. Explicit output types, __source() and __ref() dependencies, and offline compilation with no warehouse connection.
• Replay-aware builds. Affected models are reconstructed from retained stream history.
• Plans before changes. sqt plan is read-only, and sqt build replans against current warehouse state before it runs.
• Staged deployments. Build virtually, then audit, promote, and roll back.
• Quality built in. SQL tests, audits, sensors, and SQLClean rules and formatting.
• A UI for operating it. Sources, lag, lineage, plans, runs, and quality in sqt dev, with trusted-proxy or password authentication for shared installations.

Example

A source and a model:

sources:
  - name: orders
    kind: kafka
    broker_list: kafka:9092
    topic: source.orders
    replay_boundary:
      mode: offsets

MODEL (
  engine "MergeTree()",
  order_by ["order_id", "_replay_partition", "_replay_offset"],
);

SELECT
  order_id::String AS order_id,
  _replay_partition::Int32 AS _replay_partition,
  _replay_offset::Int64 AS _replay_offset,
  _replay_timestamp::DateTime64(3) AS _replay_timestamp
FROM __source("orders")

Quick start

sqt discover    # inspect authored resources
sqt compile     # validate offline
sqt plan        # read-only warehouse plan
sqt build       # confirm and execute
sqt test
sqt audit
sqt dev         # local UI at 127.0.0.1:8000

The orders demo runs the whole workflow locally with Redpanda and ClickHouse. See the quickstart to start your own project.

Documentation

• Models and pipelines
• Replay and lineage
• Deployments
• Destructive operations
• Project configuration
• SQL Rules and formatting
• Changelog and contributing

License

Apache-2.0