Skip to content

fix(snowflake): create state tables permanent with maximum time travel - #964

Merged
kvlonge merged 2 commits into
mainfrom
fix/snowflake-permanent-state-tables
Oct 6, 2026
Merged

kvlonge merged 2 commits into
mainfrom
fix/snowflake-permanent-state-tables

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Why

On Snowflake, SQLBuild's _sqlbuild_* state tables were created TRANSIENT with no retention setting, so they had at most one day of time travel and no fail-safe; the microbatch state table was created permanent through a separate path. Losing state history means losing run, audit, freshness and fingerprint records (CHI-517).

Changes

  • Every Snowflake state table is created CREATE TABLE IF NOT EXISTS … DATA_RETENTION_TIME_IN_DAYS = 90 (the Enterprise maximum). If the account rejects 90 (Standard edition allows 1), that statement is retried once with 1, remembered per database and schema; the successful retry reports as one successful statement.
  • Retention is set only at creation: no extra statements on later runs, and existing state tables are never altered or converted.
  • One state-table DDL path for all tables; the transient flag is removed. Other warehouses' DDL is unchanged. Docs describe Snowflake state tables.

Verification

  • Rendered DDL for all state tables on every adapter; fallback, no-retry and per-schema tests; lifecycle events and CLI progress through the real Snowflake cursor; offline Snowflake build for Enterprise-like and Standard-like accounts; DuckDB state E2E paths.
  • One independent review plus follow-up; make test, make check-ci.

@kvlonge
kvlonge enabled auto-merge (squash) October 6, 2026 08:22
@kvlonge
kvlonge merged commit 1293e5d into main Oct 6, 2026
57 of 59 checks passed
@kvlonge
kvlonge deleted the fix/snowflake-permanent-state-tables branch October 6, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant