diff --git a/PLAN.md b/PLAN.md index ae604e4..50edbfc 100644 --- a/PLAN.md +++ b/PLAN.md @@ -5478,6 +5478,38 @@ never looks like a fact. fixture the plan goes from 102 786 to 114 483 prompt tokens (+11%) and from 36 650 to 41 670 completion tokens (+14%). +**As built — the store (2026-10-27).** `INSIGHTS_DB_VERSION = 2`. Migration 1 +creates `insight_word (id, entry)`, indexed on `word` and on `id`. `entry` is +the word entry as JSON text, without its `meaning`; `word`, `concern`, `sense` +and `confidence` are generated columns over it. The decisions: + +- **JSON text with generated columns, as `insight` has.** These strings come + from a model or a side-car, and a plain column cannot hold U+0000 (the binding + reads it back cut short). The round-trip property's words are binary strings, + and they import whole. `in` rides in `entry`: it is a reserved word in SQL. +- **No `meaning`.** A sentence per entry was a third of the table at Broadleaf's + scale. An answer that shows it reads the block, by the M16b rule this + section's table cites. +- **Backfilled after all.** This section said there was nothing to backfill, + but a build that asked prompt 2 while its store was still at version 1 (#35) + writes words into version-1 blocks. The migration projects them. +- **Written per record**, in the unit's transaction: first the replaced + record's rows are deleted, then the record is written, then its rows. An + import deletes the whole table and rebuilds it from the blocks. There is one + row per entry, a repeated word included: the projection says what the block + says. +- **A reader still refuses an older store** (M16c). The refusal now names the + free way out: any `explain` on the store migrates it, `--dry-run` included. So + `serve`'s panel shows nothing on a version-1 store until then. +- **Tested from a real file.** `packages/insights/test/fixtures/store-v1.sql` + is the `.dump` of a store the version-1 build wrote (6 records, 2 failures, a + run, 2 blocks with words), kept with the export that build wrote. +- **Scale, on a synthetic side-car** with Broadleaf's counts and this section's + word estimates (28 206 records, 131 190 entries): the import takes 1.5 s and + the export is byte-identical. The projection is 22 MB of a 91 MB store, + `WHERE word = 'order'` takes 0.5 ms, and the migration's backfill 0.6 s. The + real Broadleaf side-car is owed. + ### 20.4 Milestone split **M19a — the lexicon, the split, the artifact** (closes the profile half of #19) @@ -5540,10 +5572,11 @@ never looks like a fact. `@codegraph/vocabulary` (types only, through the env); the module prompt quoting its types' entries through a `RecordSource.words(id)` projection; the word list in the fingerprint inputs. -- [ ] The store at version 2: migration 1 creating `insight_word`, written +- [x] The store at version 2: migration 1 creating `insight_word`, written per record in the unit's transaction and rebuilt on `--import`; the migration tests of §20.3; the round-trip property unchanged and green - on every existing side-car. + on every existing side-car. *(The real Broadleaf side-car is owed; + measured on a synthetic one at its scale.)* - [ ] `--terms N`; `--dry-run`/`--estimate` reflecting the longer completions (`COMPLETION_TOKENS_PER_BLOCK` re-measured); a reader's answer for a record explained under prompt `"1"`; on the fake client diff --git a/docs/cli.md b/docs/cli.md index a7ef3fa..2fcebdb 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -336,7 +336,9 @@ invocation: side-car carries it). The command writes nothing: the store is opened as a reader, a missing one is a usage error rather than a file that appears, and one written by a newer codegraph — or a SQLite file that is not an insights store — -is named and left alone. +is named and left alone. One an older build wrote is refused and left alone too: +any `codegraph explain` on it migrates it, `--dry-run` included, which calls no +model. ### `scm` Mines a repository's git history into a deterministic `history.jsonl`: diff --git a/docs/insights.md b/docs/insights.md index b0b78bd..e46d7fb 100644 --- a/docs/insights.md +++ b/docs/insights.md @@ -188,6 +188,23 @@ id would come back shorter); and the tables are **rowid tables** (`WITHOUT ROWID` keeps kilobyte-wide rows in interior pages: 96 MB against 57 MB on Broadleaf, for no faster read). +**Version 2: the words, projected (M19c).** `insight_word` holds one row per +entry of a block's `words`: the entry as JSON text, without its `meaning` (a +sentence, read from the block when an answer shows it), with `word`, `concern`, +`sense` and `confidence` as generated columns over it, indexed on `word` and +on `id`. "Every entity that defines `order`" is then an index lookup, where it +would otherwise scan every block. A record's rows are written with it, in the +same transaction, after the rows of the record it replaces are deleted, and +`--import` rebuilds the table from the blocks. Nothing is exported from it: the +block stays the one copy that counts, so `decode → import → export` is +untouched. Migration 1 creates the table and projects the words a version-1 +block already holds. A build that asked prompt 2 while its store was still at +version 1 wrote some. It is the first rung of the ladder to run on a real file, +and its tests start from one the version-1 build wrote +(`packages/insights/test/fixtures/store-v1.sql`): the records, failures, ledger +and export stamp come through intact, the export is the same bytes, a reader +refuses the file untouched, and a migration that fails leaves it at version 1. + A `run` table is the one thing the store holds that the side-car does not: one row per run — when, which models and provider, counts, usage, why it aborted. The trailer only ever knew the last run. It is declared losable: delete the @@ -233,7 +250,9 @@ Broadleaf, the graph the rest. **The store's readers (M16c).** A reader gets the store with `{ readOnly: true }`: no migration, and SQLite itself refuses every write (`PRAGMA -query_only`). It is handed an ordinary connection rather than a read-only file +query_only`). So a store an older build wrote is refused rather than read; the +refusal says that any `codegraph explain` on it migrates it, `--dry-run` +included, which calls no model. It is handed an ordinary connection rather than a read-only file handle, because the latter — the first attempt — may not clean up after itself and stranded `-wal`/`-shm` beside the model. What a reader asks never reads a block: `query()` returns ROWS (envelope, description, and the same diff --git a/packages/cli/src/args.ts b/packages/cli/src/args.ts index d73feae..c68c252 100644 --- a/packages/cli/src/args.ts +++ b/packages/cli/src/args.ts @@ -515,7 +515,7 @@ export const SERVE_SPEC: CommandSpec = { type: "string", describe: "The insights store behind the page's explanation panel (what 'codegraph explain' wrote). " + - "Default: .insights.db beside the first model; with none, the page simply shows no explanations.", + "Default: .insights.db beside the first model; with none, or one an older codegraph wrote that no 'codegraph explain' has migrated yet, the page shows no explanations.", placeholder: "FILE", }, ], diff --git a/packages/cli/src/insight-route.ts b/packages/cli/src/insight-route.ts index 6c9e012..842f2cf 100644 --- a/packages/cli/src/insight-route.ts +++ b/packages/cli/src/insight-route.ts @@ -43,7 +43,8 @@ export function insightLookup(storePath: string): InsightLookup { try { store = sqliteInsightsStore(db, { readOnly: true }); } catch { - // Not ours, or from a newer build: nothing this page can show. `codegraph insights` says why. + // Not ours, from a newer build, or from an older one no explain has migrated yet: nothing this page can show. + // `codegraph insights` says why. db.close(); } return store; diff --git a/packages/cli/test/explain.test.ts b/packages/cli/test/explain.test.ts index f4843a9..bdd1e2a 100644 --- a/packages/cli/test/explain.test.ts +++ b/packages/cli/test/explain.test.ts @@ -172,6 +172,25 @@ describe("explain --dry-run", () => { expect(codeB).toBe(EXIT.OK); }); + it("migrates a store an older build wrote, as a reader's refusal says it does, and calls nothing", async () => { + const dir = mkdtempSync(join(tmpdir(), "explain-v1-")); + try { + const path = join(dir, "model.insights.db"); + const v1 = loadSqlite().open(path); + v1.exec(readFileSync(fileURLToPath(new URL("../../insights/test/fixtures/store-v1.sql", import.meta.url)), "utf8")); + v1.close(); + const seam: ExplainSeam = { ...realSeam(), env: {}, clientFor: () => { throw new Error("a dry run asks for no client"); } }; + expect(await explainCommand(options(["--out", join(dir, "model.insights.jsonl"), "--dry-run"]), captureIo(), seam)).toBe(EXIT.OK); + const after = loadSqlite().open(path); + expect(after.prepare("PRAGMA user_version").get()).toEqual({ user_version: 2 }); + expect(after.prepare("SELECT count(*) AS n FROM insight").get()).toEqual({ n: 6 }); + after.close(); + expect(readdirSync(dir)).toEqual(["model.insights.db"]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + it("--json prints one machine-readable plan whose operations precede their type", async () => { const io = captureIo(); await explainCommand(options(["--dry-run", "--json"]), io, seamWith({}).seam); diff --git a/packages/cli/test/insights.test.ts b/packages/cli/test/insights.test.ts index 1453e8e..ac9c545 100644 --- a/packages/cli/test/insights.test.ts +++ b/packages/cli/test/insights.test.ts @@ -1,6 +1,7 @@ import { mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { fileURLToPath } from "node:url"; import { afterAll, beforeAll, describe, expect, it } from "vitest"; import { loadSqlite } from "@codegraph/analyzer"; import { @@ -218,4 +219,21 @@ describe("codegraph insights: no store, or not ours", () => { rmSync(other); } }); + + it("a store an older build wrote is refused, with the way to migrate it that calls no model, and left alone", () => { + const older = join(dir, "older.insights.db"); + const v1 = loadSqlite().open(older); + v1.exec(readFileSync(fileURLToPath(new URL("../../insights/test/fixtures/store-v1.sql", import.meta.url)), "utf8")); + v1.close(); + const before = readFileSync(older); + const io = captureIo(); + try { + expect(runSync(["insights", model, "--store", older], io)).toBe(EXIT.USAGE); + expect(io.stderr()).toMatch(/at version 1 and this build reads 2/u); + expect(io.stderr()).toMatch(/'codegraph explain' on it does, '--dry-run' included/u); + expect(readFileSync(older).equals(before)).toBe(true); + } finally { + rmSync(older); + } + }); }); diff --git a/packages/insights/src/store-sqlite.ts b/packages/insights/src/store-sqlite.ts index 065602e..345a5d3 100644 --- a/packages/insights/src/store-sqlite.ts +++ b/packages/insights/src/store-sqlite.ts @@ -52,6 +52,14 @@ import { * would come back SHORTER, silently. So a column string holding U+0000 is * refused at write; everything a model or a provider worded (block, reason, * metadata) is stored as JSON text, where both are escapes and survive. + * + * A RECORD'S WORDS ARE A PROJECTION (version 2, M19c). `insight_word` holds one + * row per entry of a block's `words`: the entry as JSON text, less its + * `meaning` (a sentence, read from the block when an answer shows it), with + * generated columns over it, so "every entity that defines `order`" is an + * index lookup rather than a scan of every block. It is written with its + * record, in the same transaction, and rebuilt by an import: the block stays + * the one copy that counts, and nothing is exported from the projection. */ /** Index i takes a store from version i to i+1. Append; never edit a shipped entry. */ @@ -108,6 +116,23 @@ CREATE TABLE failure ( prompt_tokens INTEGER, completion_tokens INTEGER, cost REAL, run_id INTEGER REFERENCES run(id) ); +`, + // A version-1 block can hold words already (a build that asked prompt 2 before its store was version 2): projected here. + ` +CREATE TABLE insight_word ( + id TEXT NOT NULL REFERENCES insight(id), + entry TEXT NOT NULL, + word TEXT GENERATED ALWAYS AS (entry ->> '$.word') VIRTUAL, + concern TEXT GENERATED ALWAYS AS (entry ->> '$.concern') VIRTUAL, + sense TEXT GENERATED ALWAYS AS (entry ->> '$.sense') VIRTUAL, + confidence REAL GENERATED ALWAYS AS (entry ->> '$.confidence') VIRTUAL +); +CREATE INDEX insight_word_id ON insight_word(id); +CREATE INDEX insight_word_word ON insight_word(word); + +INSERT INTO insight_word (id, entry) + SELECT insight.id, json_remove(entry.value, '$.meaning') FROM insight, json_each(insight.block, '$.words') AS entry + ORDER BY insight.rowid, entry.key; `, ]; @@ -147,7 +172,9 @@ function migrate(db: SqliteDatabase, readOnly: boolean): void { // A reader brings nothing up to date: an empty file is not a store, and an older one waits for a run to migrate it. if (tables === 0) throw new InsightsStoreError("this file is not an insights store (it is empty)"); if (version < INSIGHTS_DB_VERSION) { - throw new InsightsStoreError(`this insights store is at version ${version} and this build reads ${INSIGHTS_DB_VERSION}; any 'codegraph explain' run on it migrates it`); + throw new InsightsStoreError( + `this insights store is at version ${version} and this build reads ${INSIGHTS_DB_VERSION}: a reader never migrates it, and any 'codegraph explain' on it does, '--dry-run' included, which calls no model`, + ); } } if (version > INSIGHTS_DB_VERSION) { @@ -242,6 +269,8 @@ export function sqliteInsightsStore(db: SqliteDatabase, options: StoreOpenOption const insertInsight = db.prepare(`INSERT OR REPLACE INTO insight (${INSIGHT_COLUMNS}, run_id) VALUES (${"?, ".repeat(18)}?)`); const insertFailure = db.prepare(`INSERT OR REPLACE INTO failure (${FAILURE_COLUMNS}, run_id) VALUES (${"?, ".repeat(14)}?)`); const deleteFailure = db.prepare("DELETE FROM failure WHERE id = ?"); + const deleteWords = db.prepare("DELETE FROM insight_word WHERE id = ?"); + const insertWord = db.prepare("INSERT INTO insight_word (id, entry) VALUES (?, ?)"); const getMeta = db.prepare("SELECT value FROM meta WHERE key = ?"); const setMeta = db.prepare("INSERT OR REPLACE INTO meta (key, value) VALUES (?, ?)"); const dropMeta = db.prepare("DELETE FROM meta WHERE key = ?"); @@ -263,9 +292,11 @@ export function sqliteInsightsStore(db: SqliteDatabase, options: StoreOpenOption return typeof value === "string" ? (JSON.parse(value) as unknown) : undefined; }; - /** Validated BEFORE the first bind, so a bad member aborts its unit with nothing written. */ + /** Validated BEFORE the first bind, so a bad member aborts its unit with nothing written. Its words go with it. */ const writeRecord = (input: InsightRecord, run: number | null): void => { const r = InsightRecord.parse(input); + // First: the replaced record's words must not outlive it, and the row they reference is about to be replaced. + deleteWords.run(r.id); insertInsight.run( nullable(r.id), LEVEL_RANK[r.level], nullable(r.kind), nullable(r.name), nullable(r.file), nullable(r.key?.lang), nullable(r.key?.module), nullable(r.key?.symbol), nullable(r.key?.disambiguator), @@ -276,6 +307,7 @@ export function sqliteInsightsStore(db: SqliteDatabase, options: StoreOpenOption JSON.stringify(r.block), run, ); + for (const { meaning: _meaning, ...listed } of r.block.words ?? []) insertWord.run(r.id, JSON.stringify(listed)); }; const writeFailure = (input: FailureRecord, run: number | null): void => { @@ -485,7 +517,7 @@ export function sqliteInsightsStore(db: SqliteDatabase, options: StoreOpenOption importFile(file: InsightsFile) { transaction(() => { - db.exec("DELETE FROM insight; DELETE FROM failure;"); + db.exec("DELETE FROM insight_word; DELETE FROM insight; DELETE FROM failure;"); for (const record of file.records) writeRecord(record, null); for (const failure of file.failures) writeFailure(failure, null); setMeta.run("header", JSON.stringify(InsightsHeader.parse(file.header))); diff --git a/packages/insights/src/store.ts b/packages/insights/src/store.ts index d547aa8..dd34f8d 100644 --- a/packages/insights/src/store.ts +++ b/packages/insights/src/store.ts @@ -18,8 +18,8 @@ import type { InsightsFile } from "./sidecar.js"; * does not carry, and it is declared losable. */ -/** Bumped with every migration appended to the adapter's ladder — never regenerated. */ -export const INSIGHTS_DB_VERSION = 1; +/** Bumped with every migration appended to the adapter's ladder — never regenerated. 2: `insight_word` (M19c). */ +export const INSIGHTS_DB_VERSION = 2; /** `PRAGMA application_id`: "CGI1". A SQLite file that is not ours is refused, not migrated. */ export const INSIGHTS_APPLICATION_ID = 0x43474931; diff --git a/packages/insights/test/fixtures/store-v1.insights.jsonl b/packages/insights/test/fixtures/store-v1.insights.jsonl new file mode 100644 index 0000000..0f033d0 --- /dev/null +++ b/packages/insights/test/fixtures/store-v1.insights.jsonl @@ -0,0 +1,10 @@ +{"t":"header","kind":"codegraph.insights/1","generatedBy":"@codegraph/insights","promptVersion":"2","metamodel":"specy.domain/3","models":{"leaf":"openai/gpt-5.6-luna","rollup":"openai/gpt-5.6-luna"},"provider":"openrouter","depth":1,"source":{"paths":["model.jsonl"],"langs":["java"],"view":{"name":"all","filters":[]}}} +{"t":"i","id":"java:com.acme.order/Basket.add(com.acme.order.Basket.Line)","key":{"lang":"java","module":"com.acme.order","symbol":"Basket.add","disambiguator":"(com.acme.order.Basket.Line)"},"level":"operation","kind":"method","scc":["java:com.acme.order/Basket.add(com.acme.order.Basket.Line)"],"origin":"llm","block":{"name":"add","description":"Adds a line — « déjà » 注文 📦.","safe":true,"idempotent":true,"owner":"entity","handlesCommand":null,"emits":[],"preconditions":[],"postconditions":[],"invariantsEnforced":[],"usesSpi":[],"domainTerms":["order"],"confidence":0.8},"fingerprint":"1111111111111111111111111111111111111111111111111111111111111111","model":"openai/gpt-5.6-luna","usage":{"promptTokens":1400,"completionTokens":450,"cost":0.0004}} +{"t":"i","id":"java:com.acme.order/Order.reference()","key":{"lang":"java","module":"com.acme.order","symbol":"Order.reference","disambiguator":"()"},"level":"operation","kind":"method","name":"reference","file":"com/acme/order/Order.java","origin":"template","block":{"name":"reference","description":"Returns the reference.","safe":true,"idempotent":true,"owner":"entity","handlesCommand":null,"emits":[],"preconditions":[],"postconditions":[],"invariantsEnforced":[],"usesSpi":[],"domainTerms":["order"],"confidence":0.8},"fingerprint":"1111111111111111111111111111111111111111111111111111111111111111"} +{"t":"i","id":"java:com.acme.order/Order.total()","key":{"lang":"java","module":"com.acme.order","symbol":"Order.total","disambiguator":"()"},"level":"operation","kind":"method","name":"total","file":"com/acme/order/Order.java","origin":"llm","block":{"name":"total","description":"Sums the order's lines.","safe":true,"idempotent":true,"owner":"entity","handlesCommand":null,"emits":[],"preconditions":[],"postconditions":[],"invariantsEnforced":[],"usesSpi":[],"domainTerms":["order"],"confidence":0.8},"fingerprint":"1111111111111111111111111111111111111111111111111111111111111111","model":"openai/gpt-5.6-luna","usage":{"promptTokens":1400,"completionTokens":450,"cost":0.0004}} +{"t":"i","id":"java:com.acme.order/Basket","key":{"lang":"java","module":"com.acme.order","symbol":"Basket"},"level":"type","kind":"class","name":"Basket","origin":"llm","block":{"name":"Basket","description":"A customer's order.","concept":"entity","eventKind":null,"interfaceRole":null,"aggregateRoot":false,"containedIn":null,"syncPattern":null,"identity":"reference","fields":[],"invariants":[],"stateMachine":null,"relatesTo":[],"exposes":[],"dependsOn":[],"domainTerms":["order"],"confidence":0.9},"fingerprint":"1111111111111111111111111111111111111111111111111111111111111111","model":"openai/gpt-5.6-luna","usage":{"promptTokens":1400,"completionTokens":450,"cost":0.0004}} +{"t":"i","id":"java:com.acme.order/Order","key":{"lang":"java","module":"com.acme.order","symbol":"Order"},"level":"type","kind":"class","name":"Order","file":"com/acme/order/Order.java","origin":"llm","block":{"name":"Order","description":"A customer's order.","concept":"aggregate","eventKind":null,"interfaceRole":null,"aggregateRoot":true,"containedIn":null,"syncPattern":null,"identity":"reference","fields":[],"invariants":[],"stateMachine":null,"relatesTo":[],"exposes":[],"dependsOn":[],"domainTerms":["order"],"words":[{"word":"order","in":["Order"],"concern":"domain","sense":"a customer's request","meaning":"What a customer asked to buy.","confidence":0.9},{"word":"reference","in":["reference"],"concern":"domain","sense":"the order's identity","meaning":"How an order is told apart.","confidence":0.7}],"confidence":0.9},"fingerprint":"2222222222222222222222222222222222222222222222222222222222222222","model":"openai/gpt-5.6-luna","usage":{"promptTokens":1400,"completionTokens":450,"cost":0.0004}} +{"t":"i","id":"java:com.acme.order","key":{"lang":"java","module":"com.acme.order","symbol":"com.acme.order"},"level":"module","kind":"package","name":"com.acme.order","origin":"llm","block":{"name":"Ordering","description":"Takes orders.","apis":[],"spis":[],"dependsOn":[],"concepts":[{"name":"Order","concept":"aggregate"}],"boundedContextHint":{"name":"Ordering","rationale":"its own words"},"sharedKernelHint":null,"ubiquitousLanguage":[],"words":[{"word":"order","in":["com.acme.order"],"concern":"domain","sense":"the ordering business","meaning":"The package that takes orders.","confidence":0.8}],"glossary":[{"word":"order","concern":"domain","sense":"a customer's request","meaning":"What a customer asked to buy.","polysemous":false,"senses":[]}],"confidence":0.6},"fingerprint":"2222222222222222222222222222222222222222222222222222222222222222","model":"openai/gpt-5.6-luna","usage":{"promptTokens":1400,"completionTokens":450,"cost":0.0004}} +{"t":"f","id":"java:com.acme.order.adapter/LedgerAdapter.post(java.lang.Object)","level":"operation","members":["java:com.acme.order.adapter/LedgerAdapter.post(java.lang.Object)"],"model":"openai/gpt-5.6-luna","reason":{"kind":"provider","message":"no credits","status":402,"retryable":false},"attempts":1,"calls":1} +{"t":"f","id":"java:com.acme.order.legacy/List","key":{"lang":"java","module":"com.acme.order.legacy","symbol":"List"},"level":"type","members":["java:com.acme.order.legacy/List","java:com.acme.order.legacy/Node"],"model":"openai/gpt-5.6-luna","reason":{"kind":"invalid-answer","message":"missing `fields`"},"attempts":2,"calls":2,"usage":{"promptTokens":3000,"completionTokens":900}} +{"t":"eof","counts":{"records":6,"llm":2,"template":0,"reused":4,"failed":2},"usage":{"promptTokens":3200,"completionTokens":1100,"cost":0.0009},"generatedAt":"2026-10-27T10:05:00.000Z"} diff --git a/packages/insights/test/fixtures/store-v1.sql b/packages/insights/test/fixtures/store-v1.sql new file mode 100644 index 0000000..e22e3ba --- /dev/null +++ b/packages/insights/test/fixtures/store-v1.sql @@ -0,0 +1,66 @@ +PRAGMA foreign_keys=OFF; +BEGIN TRANSACTION; +CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL) WITHOUT ROWID; +INSERT INTO meta VALUES('eof','{"t":"eof","counts":{"records":6,"llm":2,"template":0,"reused":4,"failed":2},"usage":{"promptTokens":3200,"completionTokens":1100,"cost":0.0009},"generatedAt":"2026-10-27T10:05:00.000Z"}'); +INSERT INTO meta VALUES('exported','{"path":"model.insights.jsonl","size":4321,"mtimeMs":1760950800000}'); +INSERT INTO meta VALUES('header','{"t":"header","kind":"codegraph.insights/1","generatedBy":"@codegraph/insights","promptVersion":"2","metamodel":"specy.domain/3","models":{"leaf":"openai/gpt-5.6-luna","rollup":"openai/gpt-5.6-luna"},"provider":"openrouter","depth":1,"source":{"paths":["model.jsonl"],"langs":["java"],"view":{"name":"all","filters":[]}}}'); +CREATE TABLE run ( + id INTEGER PRIMARY KEY, + started_at TEXT NOT NULL, + finished_at TEXT, + pid INTEGER, + provider TEXT, + leaf TEXT NOT NULL, + rollup TEXT NOT NULL, + depth INTEGER NOT NULL, + llm INTEGER, template INTEGER, reused INTEGER, failed INTEGER, calls INTEGER, + prompt_tokens INTEGER, completion_tokens INTEGER, cost REAL, + aborted TEXT +); +INSERT INTO run VALUES(1,'2026-10-27T10:00:00.000Z','2026-10-27T10:05:00.000Z',4242,'openrouter','openai/gpt-5.6-luna','openai/gpt-5.6-luna',1,2,0,4,2,2,3200,1100,0.0009,NULL); +CREATE TABLE insight ( + id TEXT PRIMARY KEY, + level INTEGER NOT NULL, + kind TEXT NOT NULL, + name TEXT, + file TEXT, + key_lang TEXT, key_module TEXT, key_symbol TEXT, key_disambiguator TEXT, + scc TEXT, + origin TEXT NOT NULL, + fingerprint TEXT NOT NULL, + model TEXT, + prompt_tokens INTEGER, completion_tokens INTEGER, cost REAL, + metadata TEXT, + block TEXT NOT NULL, + concept TEXT GENERATED ALWAYS AS (block ->> '$.concept') VIRTUAL, + confidence REAL GENERATED ALWAYS AS (block ->> '$.confidence') VIRTUAL, + run_id INTEGER REFERENCES run(id) +); +INSERT INTO insight VALUES('java:com.acme.order/Basket.add(com.acme.order.Basket.Line)',0,'method',NULL,NULL,'java','com.acme.order','Basket.add','(com.acme.order.Basket.Line)','["java:com.acme.order/Basket.add(com.acme.order.Basket.Line)"]','llm','1111111111111111111111111111111111111111111111111111111111111111','openai/gpt-5.6-luna',1400,450,0.0004,NULL,'{"name":"add","description":"Adds a line — « déjà » 注文 📦.","safe":true,"idempotent":true,"owner":"entity","handlesCommand":null,"emits":[],"preconditions":[],"postconditions":[],"invariantsEnforced":[],"usesSpi":[],"domainTerms":["order"],"confidence":0.8}',NULL); +INSERT INTO insight VALUES('java:com.acme.order/Order.reference()',0,'method','reference','com/acme/order/Order.java','java','com.acme.order','Order.reference','()',NULL,'template','1111111111111111111111111111111111111111111111111111111111111111',NULL,NULL,NULL,NULL,NULL,'{"name":"reference","description":"Returns the reference.","safe":true,"idempotent":true,"owner":"entity","handlesCommand":null,"emits":[],"preconditions":[],"postconditions":[],"invariantsEnforced":[],"usesSpi":[],"domainTerms":["order"],"confidence":0.8}',NULL); +INSERT INTO insight VALUES('java:com.acme.order/Order.total()',0,'method','total','com/acme/order/Order.java','java','com.acme.order','Order.total','()',NULL,'llm','1111111111111111111111111111111111111111111111111111111111111111','openai/gpt-5.6-luna',1400,450,0.0004,NULL,'{"name":"total","description":"Sums the order''s lines.","safe":true,"idempotent":true,"owner":"entity","handlesCommand":null,"emits":[],"preconditions":[],"postconditions":[],"invariantsEnforced":[],"usesSpi":[],"domainTerms":["order"],"confidence":0.8}',NULL); +INSERT INTO insight VALUES('java:com.acme.order/Basket',1,'class','Basket',NULL,'java','com.acme.order','Basket',NULL,NULL,'llm','1111111111111111111111111111111111111111111111111111111111111111','openai/gpt-5.6-luna',1400,450,0.0004,NULL,'{"name":"Basket","description":"A customer''s order.","concept":"entity","eventKind":null,"interfaceRole":null,"aggregateRoot":false,"containedIn":null,"syncPattern":null,"identity":"reference","fields":[],"invariants":[],"stateMachine":null,"relatesTo":[],"exposes":[],"dependsOn":[],"domainTerms":["order"],"confidence":0.9}',NULL); +INSERT INTO insight VALUES('java:com.acme.order/Order',1,'class','Order','com/acme/order/Order.java','java','com.acme.order','Order',NULL,NULL,'llm','2222222222222222222222222222222222222222222222222222222222222222','openai/gpt-5.6-luna',1400,450,0.0004,NULL,'{"name":"Order","description":"A customer''s order.","concept":"aggregate","eventKind":null,"interfaceRole":null,"aggregateRoot":true,"containedIn":null,"syncPattern":null,"identity":"reference","fields":[],"invariants":[],"stateMachine":null,"relatesTo":[],"exposes":[],"dependsOn":[],"domainTerms":["order"],"words":[{"word":"order","in":["Order"],"concern":"domain","sense":"a customer''s request","meaning":"What a customer asked to buy.","confidence":0.9},{"word":"reference","in":["reference"],"concern":"domain","sense":"the order''s identity","meaning":"How an order is told apart.","confidence":0.7}],"confidence":0.9}',1); +INSERT INTO insight VALUES('java:com.acme.order',2,'package','com.acme.order',NULL,'java','com.acme.order','com.acme.order',NULL,NULL,'llm','2222222222222222222222222222222222222222222222222222222222222222','openai/gpt-5.6-luna',1400,450,0.0004,NULL,'{"name":"Ordering","description":"Takes orders.","apis":[],"spis":[],"dependsOn":[],"concepts":[{"name":"Order","concept":"aggregate"}],"boundedContextHint":{"name":"Ordering","rationale":"its own words"},"sharedKernelHint":null,"ubiquitousLanguage":[],"words":[{"word":"order","in":["com.acme.order"],"concern":"domain","sense":"the ordering business","meaning":"The package that takes orders.","confidence":0.8}],"glossary":[{"word":"order","concern":"domain","sense":"a customer''s request","meaning":"What a customer asked to buy.","polysemous":false,"senses":[]}],"confidence":0.6}',1); +CREATE TABLE failure ( + id TEXT PRIMARY KEY, + level INTEGER NOT NULL, + members TEXT NOT NULL, + key_lang TEXT, key_module TEXT, key_symbol TEXT, key_disambiguator TEXT, + model TEXT NOT NULL, + reason TEXT NOT NULL, + reason_kind TEXT GENERATED ALWAYS AS (reason ->> '$.kind') VIRTUAL, + status INTEGER GENERATED ALWAYS AS (reason ->> '$.status') VIRTUAL, + attempts INTEGER NOT NULL, + calls INTEGER NOT NULL, + prompt_tokens INTEGER, completion_tokens INTEGER, cost REAL, + run_id INTEGER REFERENCES run(id) +); +INSERT INTO failure VALUES('java:com.acme.order.adapter/LedgerAdapter.post(java.lang.Object)',0,'["java:com.acme.order.adapter/LedgerAdapter.post(java.lang.Object)"]',NULL,NULL,NULL,NULL,'openai/gpt-5.6-luna','{"kind":"provider","message":"no credits","status":402,"retryable":false}',1,1,NULL,NULL,NULL,1); +INSERT INTO failure VALUES('java:com.acme.order.legacy/List',1,'["java:com.acme.order.legacy/List","java:com.acme.order.legacy/Node"]','java','com.acme.order.legacy','List',NULL,'openai/gpt-5.6-luna','{"kind":"invalid-answer","message":"missing `fields`"}',2,2,3000,900,NULL,1); +CREATE INDEX insight_concept ON insight(concept) WHERE concept IS NOT NULL; +CREATE INDEX insight_run ON insight(run_id) WHERE run_id IS NOT NULL; +COMMIT; +-- '.dump' carries neither: the version-1 build stamped the file with both. +PRAGMA application_id = 1128745265; +PRAGMA user_version = 1; diff --git a/packages/insights/test/fixtures/write-store-v1.mjs b/packages/insights/test/fixtures/write-store-v1.mjs new file mode 100644 index 0000000..68298c4 --- /dev/null +++ b/packages/insights/test/fixtures/write-store-v1.mjs @@ -0,0 +1,82 @@ +// The version-1 store of store.test.ts's migration tests, written by the VERSION-1 build's own code +// (packages/insights/dist built at eb0fab4, the last commit whose store is version 1) to stand for the files +// users hold. store-v1.sql is `sqlite3 store-v1.insights.db .dump` plus the two PRAGMAs a dump does not +// carry; store-v1.insights.jsonl is the export that build wrote. Never regenerate them: this script stays +// to say what they hold, and it refuses any other build. +// Usage, from such a build: node write-store-v1.mjs +import { writeFileSync, rmSync } from "node:fs"; +import { join } from "node:path"; +const [repo, outDir] = process.argv.slice(2); +const { sqliteInsightsStore, decodeInsights, encodeInsightsToString, INSIGHTS_DB_VERSION, INSIGHTS_KIND, INSIGHTS_GENERATOR, INSIGHTS_METAMODEL } = + await import(join(repo, "packages/insights/dist/index.js")); +const { loadSqlite } = await import(join(repo, "packages/analyzer/dist/index.js")); +if (INSIGHTS_DB_VERSION !== 1) throw new Error(`this is not the version-1 build (${INSIGHTS_DB_VERSION})`); + +const header = (promptVersion, provider) => ({ + t: "header", kind: INSIGHTS_KIND, generatedBy: INSIGHTS_GENERATOR, promptVersion, metamodel: INSIGHTS_METAMODEL, + models: { leaf: "openai/gpt-5.6-luna", rollup: "openai/gpt-5.6-luna" }, provider, depth: 1, + source: { paths: ["model.jsonl"], langs: ["java"], view: { name: "all", filters: [] } }, +}); +const OPERATION = { + name: "total", description: "Sums the order's lines.", safe: true, idempotent: true, owner: "entity", handlesCommand: null, + emits: [], preconditions: [], postconditions: [], invariantsEnforced: [], usesSpi: [], domainTerms: ["order"], confidence: 0.8, +}; +const TYPE = { + name: "Order", description: "A customer's order.", concept: "aggregate", eventKind: null, interfaceRole: null, aggregateRoot: true, + containedIn: null, syncPattern: null, identity: "reference", fields: [], invariants: [], stateMachine: null, relatesTo: [], + exposes: [], dependsOn: [], domainTerms: ["order"], confidence: 0.9, +}; +const MODULE = { + name: "Ordering", description: "Takes orders.", apis: [], spis: [], dependsOn: [], concepts: [{ name: "Order", concept: "aggregate" }], boundedContextHint: { name: "Ordering", rationale: "its own words" }, + sharedKernelHint: null, ubiquitousLanguage: [], confidence: 0.6, +}; +const key = (module, symbol, disambiguator) => ({ lang: "java", module, symbol, ...(disambiguator === undefined ? {} : { disambiguator }) }); +const fp = (c) => c.repeat(64); +const usage = { promptTokens: 1400, completionTokens: 450, cost: 0.0004 }; +const rec = (id, level, kind, block, extra) => ({ t: "i", id, level, kind, origin: "llm", block, fingerprint: fp("1"), model: "openai/gpt-5.6-luna", usage, ...extra }); + +// Explained under prompt 1: no words anywhere. +const prompt1 = [ + rec("java:com.acme.order/Order.total()", "operation", "method", OPERATION, { key: key("com.acme.order", "Order.total", "()"), name: "total", file: "com/acme/order/Order.java" }), + rec("java:com.acme.order/Order.reference()", "operation", "method", { ...OPERATION, name: "reference", description: "Returns the reference." }, + { origin: "template", model: undefined, usage: undefined, key: key("com.acme.order", "Order.reference", "()"), name: "reference", file: "com/acme/order/Order.java" }), + rec("java:com.acme.order/Basket.add(com.acme.order.Basket.Line)", "operation", "method", { ...OPERATION, name: "add", description: "Adds a line — « déjà » 注文 \u{1F4E6}." }, + { key: key("com.acme.order", "Basket.add", "(com.acme.order.Basket.Line)"), scc: ["java:com.acme.order/Basket.add(com.acme.order.Basket.Line)"] }), + rec("java:com.acme.order/Order", "type", "class", TYPE, { key: key("com.acme.order", "Order"), name: "Order", file: "com/acme/order/Order.java", metadata: { note: "kept" } }), + rec("java:com.acme.order/Basket", "type", "class", { ...TYPE, name: "Basket", concept: "entity", aggregateRoot: false }, { key: key("com.acme.order", "Basket"), name: "Basket" }), + rec("java:com.acme.order", "module", "package", MODULE, { key: key("com.acme.order", "com.acme.order"), name: "com.acme.order" }), +]; +const failures = [ + { t: "f", id: "java:com.acme.order.adapter/LedgerAdapter.post(java.lang.Object)", level: "operation", members: ["java:com.acme.order.adapter/LedgerAdapter.post(java.lang.Object)"], + model: "openai/gpt-5.6-luna", reason: { kind: "provider", message: "no credits", status: 402, retryable: false }, attempts: 1, calls: 1 }, + { t: "f", id: "java:com.acme.order.legacy/List", level: "type", members: ["java:com.acme.order.legacy/List", "java:com.acme.order.legacy/Node"], key: key("com.acme.order.legacy", "List"), + model: "openai/gpt-5.6-luna", reason: { kind: "invalid-answer", message: "missing `fields`" }, attempts: 2, calls: 2, usage: { promptTokens: 3000, completionTokens: 900 } }, +]; +const eof1 = { t: "eof", counts: { records: 6, llm: 5, template: 1, reused: 0, failed: 2 }, usage: { promptTokens: 7000, completionTokens: 2250, cost: 0.002 }, generatedAt: "2026-10-20T09:00:00.000Z" }; + +const path = join(outDir, "store-v1.insights.db"); +rmSync(path, { force: true }); +const store = sqliteInsightsStore(loadSqlite().open(path)); +store.importFile(decodeInsights(encodeInsightsToString(header("1", "openrouter"), prompt1, eof1, failures))); +store.markExported({ path: "model.insights.jsonl", size: 4321, mtimeMs: 1760950800000 }); + +// Then a run of a build that already asks prompt 2 but still writes version 1 (PR #35): two units with words. +const run = store.beginRun({ header: header("2", "openrouter"), startedAt: "2026-10-27T10:00:00.000Z", pid: 4242 }); +const words = [ + { word: "order", in: ["Order"], concern: "domain", sense: "a customer's request", meaning: "What a customer asked to buy.", confidence: 0.9 }, + { word: "reference", in: ["reference"], concern: "domain", sense: "the order's identity", meaning: "How an order is told apart.", confidence: 0.7 }, +]; +store.putUnit(run, "java:com.acme.order/Order", [rec("java:com.acme.order/Order", "type", "class", { ...TYPE, words }, { key: key("com.acme.order", "Order"), name: "Order", file: "com/acme/order/Order.java", fingerprint: fp("2") })]); +store.putUnit(run, "java:com.acme.order", [rec("java:com.acme.order", "module", "package", { + ...MODULE, + words: [{ word: "order", in: ["com.acme.order"], concern: "domain", sense: "the ordering business", meaning: "The package that takes orders.", confidence: 0.8 }], + glossary: [{ word: "order", concern: "domain", sense: "a customer's request", meaning: "What a customer asked to buy.", polysemous: false, senses: [] }], +}, { key: key("com.acme.order", "com.acme.order"), name: "com.acme.order", fingerprint: fp("2") })]); +store.finishRun(run, { + eof: { t: "eof", counts: { records: 6, llm: 2, template: 0, reused: 4, failed: 2 }, usage: { promptTokens: 3200, completionTokens: 1100, cost: 0.0009 }, generatedAt: "2026-10-27T10:05:00.000Z" }, + failures, calls: 2, +}); +const sidecar = `${[...store.export()].join("\n")}\n`; +store.close(); +writeFileSync(join(outDir, "store-v1.insights.jsonl"), sidecar); +console.log(`wrote ${path} (store version ${INSIGHTS_DB_VERSION})`); diff --git a/packages/insights/test/store.test.ts b/packages/insights/test/store.test.ts index 04501ae..d854b29 100644 --- a/packages/insights/test/store.test.ts +++ b/packages/insights/test/store.test.ts @@ -1,9 +1,10 @@ import { mkdtempSync, readFileSync, readdirSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { fileURLToPath } from "node:url"; import fc from "fast-check"; import { afterEach, describe, expect, it } from "vitest"; -import { loadSqlite } from "@codegraph/analyzer"; +import { loadSqlite, type SqliteDatabase } from "@codegraph/analyzer"; import { INSIGHTS_GENERATOR, INSIGHTS_KIND, @@ -16,6 +17,7 @@ import { type ModuleBlock, type OperationBlock, type TypeBlock, + type WordEntry, } from "../src/schema.js"; import { decodeInsights, encodeInsightsToString } from "../src/sidecar.js"; import { INSIGHTS_APPLICATION_ID, INSIGHTS_DB_VERSION, INSIGHT_ANSWER_KIND, InsightsStoreError, answerInsight, insightsStorePathFor } from "../src/store.js"; @@ -578,6 +580,138 @@ describe("never disposable: a store is migrated or refused, never rebuilt", () = }); }); +/* ── version 2 (M19c): a record's words, projected into insight_word ── */ + +/** What `insight_word` holds, per id, in row order: each entry as JSON decodes it. */ +function projected(db: SqliteDatabase): Map { + const out = new Map(); + for (const row of db.prepare("SELECT id, entry FROM insight_word ORDER BY rowid").all()) { + const id = row["id"] as string; + out.set(id, [...(out.get(id) ?? []), JSON.parse(row["entry"] as string) as unknown]); + } + return out; +} + +/** An entry as the projection lists it: its `meaning` stays in the block. */ +const listed = ({ meaning: _meaning, ...entry }: WordEntry): Omit => entry; +/** What it must hold: each record's `words`, in block order; a record without any has no row. */ +const wordsOf = (records: readonly InsightRecord[]): Map => + new Map(records.filter((r) => (r.block.words ?? []).length > 0).map((r) => [r.id, (r.block.words ?? []).map(listed)])); + +const ORDER: WordEntry = { word: "order", in: ["Order"], concern: "domain", sense: "a request", meaning: "What a customer asked for.", confidence: 0.9 }; +const LINES: WordEntry = { word: "lines", in: ["lines"], concern: "domain", sense: "its items", meaning: "What the order holds.", confidence: 0.7 }; + +describe("version 2 (M19c): insight_word, a projection of every record's words", () => { + it("is every record's words in block order, for any side-car; an import rebuilds it whole", () => { + fc.assert( + fc.property(fileArb, fileArb, (first, second) => { + const db = loadSqlite().open(":memory:"); + const store = sqliteInsightsStore(db); + // Compared with the DECODED file, as the round trip is: JSON has no -0. + store.importFile(decodeInsights(first)); + expect(projected(db)).toEqual(wordsOf(decodeInsights(first).records)); + store.importFile(decodeInsights(second)); + expect(projected(db)).toEqual(wordsOf(decodeInsights(second).records)); + store.close(); + }), + { numRuns: 60 }, + ); + }); + + it("a put replaces a record's words with its new ones, in the unit's transaction", () => { + const db = loadSqlite().open(":memory:"); + const store = sqliteInsightsStore(db); + const run = store.beginRun({ header: HEADER, startedAt: "t" }); + const order = (words?: WordEntry[]): InsightRecord => ({ ...type("t/Order"), block: { ...TYPE, ...(words === undefined ? {} : { words }) } }) as InsightRecord; + store.putUnit(run, "t/Order", [order([ORDER, LINES])]); + expect(projected(db)).toEqual(new Map([["t/Order", [listed(ORDER), listed(LINES)]]])); + store.putUnit(run, "t/Order", [order([LINES])]); + expect(projected(db)).toEqual(new Map([["t/Order", [listed(LINES)]]])); + // Explained again without words (a template): no row is left to say otherwise. + store.putUnit(run, "t/Order", [order()]); + expect(projected(db)).toEqual(new Map()); + // A unit refused whole writes none of its members' words. + store.putUnit(run, "t/Order", [order([ORDER])]); + const bad = { ...op("o/c"), fingerprint: "short" } as InsightRecord; + expect(() => store.putUnit(run, "t/Basket", [{ ...type("t/Basket"), block: { ...TYPE, words: [LINES] } } as InsightRecord, bad])).toThrow(); + expect(projected(db)).toEqual(new Map([["t/Order", [listed(ORDER)]]])); + store.close(); + }); + + /* + * THE FIRST RUNG OF THE LADDER TO RUN ON A REAL FILE. fixtures/store-v1.sql is a store the version-1 build + * wrote (write-store-v1.mjs says what it holds): a side-car imported under prompt 1, then a run of a build + * that already asked prompt 2's words while its store was still version 1 — so two blocks hold words. + */ + describe("from version 1", () => { + const V1_SQL = fileURLToPath(new URL("./fixtures/store-v1.sql", import.meta.url)); + const V1_EXPORT = readFileSync(fileURLToPath(new URL("./fixtures/store-v1.insights.jsonl", import.meta.url)), "utf8"); + let dir: string | undefined; + afterEach(() => { + if (dir !== undefined) rmSync(dir, { recursive: true, force: true }); + dir = undefined; + }); + const v1Store = (): string => { + dir = mkdtempSync(join(tmpdir(), "insights-v1-")); + const path = join(dir, "model.insights.db"); + const db = loadSqlite().open(path); + db.exec(readFileSync(V1_SQL, "utf8")); + db.close(); + return path; + }; + /** Opens as the CLI does: a refused open closes its connection. */ + const open = (path: string, readOnly = false) => { + const db = loadSqlite().open(path); + try { + return { db, store: sqliteInsightsStore(db, { readOnly }) }; + } catch (error) { + db.close(); + throw error; + } + }; + + it("migrates with its records, failures, ledger and stamps intact, and exports the bytes the version-1 build did", () => { + const path = v1Store(); + const { db, store } = open(path); + expect(db.prepare("PRAGMA user_version").get()).toEqual({ user_version: 2 }); + expect(store.stats()).toMatchObject({ records: 6, failures: 2, byLevel: { operation: 3, type: 2, module: 1 }, byOrigin: { llm: 5, template: 1 } }); + expect(store.runs()).toMatchObject([{ id: 1, startedAt: "2026-10-27T10:00:00.000Z", finishedAt: "2026-10-27T10:05:00.000Z", counts: { llm: 2, calls: 2 } }]); + expect(store.exported()).toEqual({ path: "model.insights.jsonl", size: 4321, mtimeMs: 1760950800000 }); + expect(exportOf(store)).toBe(V1_EXPORT); + // The words already in its blocks are projected: a version-1 block CAN hold them. + expect(projected(db)).toEqual(wordsOf(decodeInsights(V1_EXPORT).records)); + expect(projected(db).size).toBe(2); + store.close(); + // Migrated once: reopened, it is version 2 and the same store. + const again = open(path); + expect(exportOf(again.store)).toBe(V1_EXPORT); + expect(projected(again.db).size).toBe(2); + again.store.close(); + expect(readdirSync(dir!)).toEqual(["model.insights.db"]); + }); + + it("a reader refuses it, names the way to migrate without a call, and leaves its bytes alone", () => { + const path = v1Store(); + const before = readFileSync(path); + expect(() => open(path, true)).toThrow(/at version 1 and this build reads 2.*--dry-run/u); + expect(readFileSync(path).equals(before)).toBe(true); + }); + + it("a migration that fails leaves it at version 1 with every record", () => { + const path = v1Store(); + // Any failure mid-migration will do: here, the table migration 1 creates is already there. + const db = loadSqlite().open(path); + db.exec("CREATE TABLE insight_word (stray INTEGER)"); + db.close(); + expect(() => open(path)).toThrow(/from version 1 failed; it was left at version 1/u); + const after = loadSqlite().open(path); + expect(after.prepare("PRAGMA user_version").get()).toEqual({ user_version: 1 }); + expect(after.prepare("SELECT count(*) AS n FROM insight").get()).toEqual({ n: 6 }); + after.close(); + }); + }); +}); + describe("insightsStorePathFor", () => { it("sits beside the side-car it exports", () => { expect(insightsStorePathFor("/x/model.insights.jsonl")).toBe("/x/model.insights.db"); diff --git a/website/content/docs/reference/cli/serve.md b/website/content/docs/reference/cli/serve.md index 784c6a6..a15a78a 100644 --- a/website/content/docs/reference/cli/serve.md +++ b/website/content/docs/reference/cli/serve.md @@ -37,7 +37,7 @@ codegraph serve --app [--data-dir DIR] [--extractors FILE] [--port N] [--host AD | `--internal-only` | Drop stub (external) entities and every edge touching one. | — | | `--declared-only` | Keep only `declared` facts; drop derived and dynamic-candidate edges. | — | | `--no-cache` | Read the model.jsonl directly; never build or reuse a sibling model.db. | — | -| `--insights FILE` | The insights store behind the page's explanation panel (what 'codegraph explain' wrote). With none, the page simply shows no explanations. | `.insights.db` beside the first model | +| `--insights FILE` | The insights store behind the page's explanation panel (what 'codegraph explain' wrote). With none, or one an older codegraph wrote that no 'codegraph explain' has migrated yet, the page shows no explanations. | `.insights.db` beside the first model | | `--app` | The desktop app's daemon: no model on argv; bind 127.0.0.1:0 under a per-launch token, print one JSON line {port, token} on stdout, open folders through POST /jobs, exit when stdin closes. | — | | `--data-dir DIR` | With --app: where models, their model.db caches, the page's artifacts and the recents live. | the OS application-data directory, `codegraph/` under it | | `--extractors FILE` | With --app: the extractor registry, a JSON list of { name, path, extensions[], launch?, env? } — the daemon runs an entry, never a language. | — (only model.jsonl files open) |