feat(insights): the word readers (M19c)

PLAN.md

@@ -5510,6 +5510,38 @@ and `confidence` are generated columns over it. The decisions:
   `WHERE word = 'order'` takes 0.5 ms, and the migration's backfill 0.6 s. The
   real Broadleaf side-car is owed.
 
+**As built — the store's readers (2026-10-27).** `codegraph insights --word`,
+`--glossary`, `--polysemy` and `--concerns`, over `glossary.ts` in
+`@codegraph/insights` and four projections of the port (`wordHits`,
+`glossaryEntries`, `concernRows`, `withoutWords`). The decisions:
+
+- **The module comes from the natural key.** `insights` opens the store alone,
+  so a record is placed by its key: core renders the module's id from `lang`
+  and `module`, groups compare through `naturalKeyIndex`, and the id is never
+  parsed. "Grouped module → type → operation" is by level within each module:
+  an operation is not nested under its type, which only the model knows.
+- **Tab-separated, like the command's other lists**, the module first so the
+  grouping survives `sort`. A module's glossary entry heads its group. The
+  entries show no `meaning`: the projection does not hold it, and `--glossary`
+  and `--id` read it from the block.
+- **Polysemy across modules** is two entries that differ in concern or in
+  sense, case and spacing aside: the wording varies from call to call, and an
+  exact comparison would list every word two modules define. The most senses
+  first.
+- **Concerns** count distinct words per concern, so a word two units of one
+  module judged differently counts under both.
+- **The prompt-1 answer** (`wordsAbsence`): a record a model wrote without
+  `words` is prompt 1's, one no model wrote is a template's. `--id` says which,
+  `--glossary` on such a module says why it has none, and the summary and every
+  word answer count what they cannot see.
+- **Lookup is exact for any string.** The `word` index holds what `->>` made of
+  the word; a lone surrogate's bytes are not the binding's, so such a word is
+  looked for by a scan, and JS equality decides either way.
+- **Measured** through the CLI on the synthetic store at Broadleaf's scale:
+  `--word order` (549 entries, 19 glossary entries) 0.46 s, `--polysemy`
+  0.48 s, `--concerns` 0.54 s, Node's start included. The re-explained
+  Broadleaf store is owed, as above.
+
 ### 20.4 Milestone split
 
 **M19a — the lexicon, the split, the artifact** (closes the profile half of #19)
@@ -5582,11 +5614,13 @@ and `confidence` are generated columns over it. The decisions:
       answer for a record explained under prompt `"1"`; on the fake client
       and live on the Java fixture; the §20.3 estimate replaced by the
       Broadleaf measurement. *(In: `--terms`, the estimate's per-word
-      figures, the fake-client run. Owed: the measurement, live runs, the
-      prompt-1 answer.)*
+      figures, the fake-client run, the prompt-1 answer. Owed: the
+      measurement, live runs.)*
 - [ ] Readers: `insights --word`, `--glossary`, `--polysemy`, `--concerns`;
       `vocabulary --drift-insights`; `/glossary.json`; the entries in the
-      Vocabulary tab and in the Navigate panel as inferences.
+      Vocabulary tab and in the Navigate panel as inferences. *(In: the four
+      `insights` readers. Owed: `--drift-insights`, `/glossary.json`, the tab
+      and the panel.)*
 - [ ] `docs/vocabulary.md` (the idioms per profile, the classes, the
       weights, the artifact, the three layers, the readers, the drift
       reports); README names the feature; `docs/cli.md` the command and the

docs/cli.md

@@ -87,6 +87,7 @@ codegraph explain  [model.jsonl] [--src DIR] [--out FILE] [--dry-run]
 
 codegraph insights [model.jsonl] [--store FILE] [--json]
                    [--id ID | --failures | --runs |
+                    --word WORD | --glossary MODULE_ID | --polysemy | --concerns |
                     --level operation|type|module --concept NAME
                     --min-confidence X --max-confidence X --limit N]
 
@@ -327,13 +328,36 @@ invocation:
 - `--id ID` — one explanation whole: envelope, description, block. An id that
   was asked for and **failed** is answered with the reason, exit `3`; an id the
   store has never heard of is a usage error;
+- `--word WORD` — every explained entity whose names spell the word (looked up
+  lowercase, as `vocabulary` splits names), grouped by module: one
+  tab-separated line per entry — the module, `glossary` or the entity's level,
+  the id, the concern, the confidence, the sense, and the identifiers the word
+  is in. Each module's own glossary entry comes first (its last cell lists the
+  types' senses when the module flagged the word polysemous), then the module,
+  its types, its operations. A record is placed by its natural key; one
+  without a key is under `-`. The meaning is not shown: `--glossary` and `--id`
+  show it;
+- `--glossary MODULE_ID` — a module's glossary whole, one line per word: the
+  word, the concern, sense and meaning the module reconciled from its types,
+  and the types' senses where they disagree;
+- `--polysemy` — the words a module flagged, and the words two modules'
+  glossaries define differently (in concern or sense, case and spacing aside):
+  every module's entry for each, side by side, the most senses first. No
+  model call judges them across the corpus: a human does;
+- `--concerns` — per module: how many distinct words its records define, and
+  how many the model called `domain`, `technical` and `ambiguous` (a word two
+  units judged differently counts under both), the most-worded module first;
 - `--failures` — the units still owed, with the reason each failed;
 - `--runs` — the ledger, oldest first: when, which models, what each run
   explained and spent.
 
 `--json` makes each answer one kind-tagged document (`codegraph.insightsSummary/1`,
-`…List/1`, `…Failures/1`, `…Runs/1`; `--id --json` is the record exactly as the
-side-car carries it). The command writes nothing: the store is opened as a
+`…List/1`, `…Word/1`, `…Glossary/1`, `…Polysemy/1`, `…Concerns/1`, `…Failures/1`,
+`…Runs/1`; `--id --json` is the record exactly as the side-car carries it).
+A record a model explained under prompt 1 defines no words, so the word
+answers cannot see it: they say on stderr how many such records there are,
+the summary counts them, `--id` says so on the record, and `--glossary` on
+such a module says why it has none. The next `explain` re-explains them. The command writes nothing: the store is opened as a
 reader, a missing one is a usage error rather than a file that appears, and one
 written by a newer codegraph — or a SQLite file that is not an insights store —
 is named and left alone. One an older build wrote is refused and left alone too:

docs/insights.md

@@ -279,6 +279,26 @@ everything else on that page is extracted from source, and this is not.
 Operations are not shown: a navigator member node carries no entity id, and
 ids are never constructed.
 
+**The words' readers (M19c).** `glossary.ts` rolls the records' `words` and
+the modules' `glossary` up, over four more projections of the port:
+`wordHits(word)` (the `insight_word` index, joined to each record's envelope),
+`glossaryEntries(word?)` (each module block's `glossary`, through `json_each`
+over the module records alone), `concernRows()` (each module, word and concern
+once) and `withoutWords()`. A record is placed in its module by its natural
+key: core renders the module's id from the key's `lang` and `module`, and the
+groups are compared through `naturalKeyIndex`. The id is never parsed; a record
+without a key is placed nowhere, and still answered. The index holds what
+`->>` made of a word, which for a lone surrogate (no identifier spells one; a
+hand-edited side-car might) is not what the binding sends, so such a word is
+looked for by a scan. Either way, what is returned is what equals the word
+once parsed back from JSON. A record a model wrote under prompt 1 has no
+`words`: `wordsAbsence(record)` says so (`prompt-1`, or `templated` for a record
+no model wrote), and every report counts those records, since it cannot see
+them. `codegraph insights --word`, `--glossary`, `--polysemy` and `--concerns`
+print them (cli.md). On the synthetic store of Broadleaf's size,
+`--word order` prints its 549 entries and 19 glossary entries in under half a
+second, Node's start included.
+
 `@codegraph/insights` stays pure: `store.ts` is a port, and `store-sqlite.ts`
 is handed an OPEN database, so the path, the file and `node:sqlite` itself
 (one import site, in the analyzer's `store/sqlite.ts`) stay with the CLI.

packages/cli/src/args.ts

@@ -758,7 +758,7 @@ export type InsightLevel = (typeof INSIGHT_LEVELS)[number];
  */
 export const INSIGHTS_SPEC: CommandSpec = {
   name: "insights",
-  summary: "Query what `explain` has written: a summary, a list, one explanation, the run ledger.",
+  summary: "Query what `explain` has written: a summary, a list, one explanation, the words and their glossary, the run ledger.",
   positional: {
     name: "model.jsonl",
     describe: "The model whose insights store (<model>.insights.db, beside it) to read. The model itself is not opened.",
@@ -812,6 +812,35 @@ export const INSIGHTS_SPEC: CommandSpec = {
       placeholder: "N",
       integer: true,
     },
+    {
+      name: "word",
+      type: "string",
+      describe:
+        "List every explained entity whose names spell this word (split as 'codegraph vocabulary' splits names: lowercase), " +
+        "by module: the module's glossary entry, then the module, its types, its operations — each with the concern, " +
+        "confidence and sense the model gave the word there, and the identifiers it is in. One index lookup.",
+      placeholder: "WORD",
+    },
+    {
+      name: "glossary",
+      type: "string",
+      describe:
+        "Print a module's glossary whole: each word its types spell, with the concern, sense and meaning the module " +
+        "reconciled from them, and the types' own senses where they disagree.",
+      placeholder: "MODULE_ID",
+    },
+    {
+      name: "polysemy",
+      type: "boolean",
+      describe:
+        "List the words a module flagged as meaning different things to its types, and the words two modules' glossaries " +
+        "define differently: every module's entry, side by side, the most senses first.",
+    },
+    {
+      name: "concerns",
+      type: "boolean",
+      describe: "Per module: how many distinct words its records define, and how many the model judged domain, technical, ambiguous.",
+    },
     {
       name: "failures",
       type: "boolean",
@@ -1314,7 +1343,7 @@ export interface ExplainOptions extends ModelInputOptions, ViewOptions, CacheOpt
 }
 
 /** What `insights` prints: exactly one of these, decided by the flags. */
-export type InsightsMode = "summary" | "list" | "one" | "failures" | "runs";
+export type InsightsMode = "summary" | "list" | "one" | "word" | "glossary" | "polysemy" | "concerns" | "failures" | "runs";
 
 export interface InsightsOptions {
   /** The model path the store sits beside; never opened. */
@@ -1323,6 +1352,10 @@ export interface InsightsOptions {
   readonly store: string | undefined;
   readonly mode: InsightsMode;
   readonly id: string | undefined;
+  /** `--word WORD`, looked up lowercase. */
+  readonly word: string | undefined;
+  /** `--glossary MODULE_ID`. */
+  readonly glossary: string | undefined;
   readonly level: InsightLevel | undefined;
   readonly concept: string | undefined;
   readonly minConfidence: number | undefined;
@@ -2013,18 +2046,30 @@ export function parseInvocation(argv: readonly string[]): Invocation {
     }
     case "insights": {
       const id = stringOf(values, "id");
+      const word = stringOf(values, "word");
+      const glossary = stringOf(values, "glossary");
       const filters = ["level", "concept", "min-confidence", "max-confidence", "limit"].filter((name) => stringOf(values, name) !== undefined);
-      // One question per invocation: the four shapes print different things.
-      const asked = [id === undefined ? undefined : "--id", flagOf(values, "failures") ? "--failures" : undefined, flagOf(values, "runs") ? "--runs" : undefined, filters.length === 0 ? undefined : `--${filters[0] ?? ""}`].filter(
-        (flag): flag is string => flag !== undefined,
-      );
+      // One question per invocation: each shape prints a different thing.
+      const questions: readonly (readonly [InsightsMode, string | undefined])[] = [
+        ["one", id === undefined ? undefined : "--id"],
+        ["word", word === undefined ? undefined : "--word"],
+        ["glossary", glossary === undefined ? undefined : "--glossary"],
+        ["polysemy", flagOf(values, "polysemy") ? "--polysemy" : undefined],
+        ["concerns", flagOf(values, "concerns") ? "--concerns" : undefined],
+        ["failures", flagOf(values, "failures") ? "--failures" : undefined],
+        ["runs", flagOf(values, "runs") ? "--runs" : undefined],
+        ["list", filters[0] === undefined ? undefined : `--${filters[0]}`],
+      ];
+      const asked = questions.filter(([, flag]) => flag !== undefined);
       if (asked.length > 1) {
         throw new UsageError(
-          `${asked.join(" and ")} ask different questions`,
-          "Pass one of: --id ID (one explanation), a list filter (--level, --concept, --min/--max-confidence), --failures, --runs — or none, for the summary.",
+          `${asked.map(([, flag]) => flag).join(" and ")} ask different questions`,
+          "Pass one of: --id ID (one explanation), a list filter (--level, --concept, --min/--max-confidence), --word WORD, " +
+            "--glossary MODULE_ID, --polysemy, --concerns, --failures, --runs — or none, for the summary.",
         );
       }
-      const mode: InsightsMode = id !== undefined ? "one" : flagOf(values, "failures") ? "failures" : flagOf(values, "runs") ? "runs" : filters.length > 0 ? "list" : "summary";
+      if (word?.trim() === "") throw new UsageError("--word needs a word", "A word as 'codegraph vocabulary' lists it, e.g. --word order.");
+      const mode: InsightsMode = asked[0]?.[0] ?? "summary";
       return {
         kind: "run",
         command: "insights",
@@ -2033,6 +2078,8 @@ export function parseInvocation(argv: readonly string[]): Invocation {
           store: stringOf(values, "store"),
           mode,
           id,
+          word,
+          glossary,
           level: stringOf(values, "level") as InsightLevel | undefined,
           concept: stringOf(values, "concept"),
           minConfidence: confidenceOf(values, "min-confidence"),