Skip to content

perf: focused agent skill and batched Snowflake retention inspection - #798

Merged
kvlonge merged 3 commits into
mainfrom
docs/focused-sqlbuild-skill
Sep 23, 2026
Merged

kvlonge merged 3 commits into
mainfrom
docs/focused-sqlbuild-skill

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Why

The packaged agent skill was one generated 15.7k-line SKILL.md (the whole docs site), too large for agents to load usefully, and it did not steer them towards the features that make changes safe to verify. Separately, Snowflake planning with managed time_travel_retention issued one INFORMATION_SCHEMA.TABLES query per table, serially; on a ~390-model project that was about 97% of plan generation.

Changes

  • Hand-written SKILL.md (163 lines) plus task guides in references/ (verifying changes, exploring a project, incremental models, testing, macros and declarations) that push sqb diff and raw-query diff, sqb scope, sqb lineage, microbatching, audits and tests.
  • Generator writes one reference file per docs page under references/docs/, with relative links, per-page contents lists, a grouped CONTENTS.md, and pruning of stale generated pages.
  • sqb skills installs the whole directory: collision check before any write, stale generated files pruned, user files kept, maintenance compares the whole directory.
  • Snowflake list_relations now reads retention_time; planning uses it from the warehouse snapshot for the destination relation only, falling back to inspection otherwise.
  • New batched inspect_retentions (one query per schema on Snowflake) used by build-time reconciliation; a relation is re-inspected only after this pass changed it.

Verification

  • Focused unit/e2e tests for installer, skill content guards (length, links, commands, case collisions), generator, Snowflake batching, planner snapshot use and identity check, executor batching.
  • make test (7007 passed) and make check-ci.
  • Real Snowflake dev-schema sqb plan with retention managed: plan generation 228.2s -> 2.8s, identical 703 retention entries and plan.

@kvlonge
kvlonge enabled auto-merge (squash) September 23, 2026 21:21
@kvlonge
kvlonge merged commit 2a5a54e into main Sep 23, 2026
28 checks passed
@kvlonge
kvlonge deleted the docs/focused-sqlbuild-skill branch September 23, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant