feat(insights): explain defines the names' words (M19c)

PLAN.md

@@ -5443,6 +5443,41 @@ and the Navigate explanation panel renders `words` and `glossary` through
 the same generic `blockFacts` it renders everything else with: an inference
 never looks like a fact.
 
+**As built — layer 2 (2026-10-27).** As designed, with these decisions:
+
+- **The own name's words only.** A qualifier's words are its container's,
+  asked where they are the own name: `com.acme.order` is asked for `order`,
+  `AcmeOrder::OrderLine` for `order, line`. `definitionSpeller` in
+  `@codegraph/vocabulary` (`define.ts`) splits and classifies exactly as the
+  cloud does and keeps `domain` and `technical`; it takes no technical list,
+  since that list only moves a word between two classes that are both asked.
+- **A port, not an import.** `@codegraph/insights` imports nothing from the
+  vocabulary, not even types: the context env takes a `Speller`
+  (`words(id)`, `ranked(moduleId)`) and the CLI hands in `definitionSpeller`
+  and `rankedWords` over the run's `buildVocabulary`, built on first ask.
+- **Kept on receipt.** The words are given: an entry for a word the prompt
+  did not list is dropped, a word given twice keeps its first entry, and
+  `in` is restated from the split, a fact the prompt stated. A module's
+  glossary is filtered the same way. An answer without `words` (or a
+  module's without `glossary`) gets the repair re-ask, then a failure.
+- **Templated records carry no `words`**: no model judged them. The
+  definition of done's "every record carries `words`" holds for every
+  record a model wrote.
+- **The glossary's candidates** are the words its types' prompts asked
+  (their names and fields), ordered by the module's vocabulary row order (an
+  unranked word after, by text) and cut at `--terms`. The module prompt
+  shows, per word, each type's `concern`, `sense` and `meaning`, or "no type
+  defined it", read through `RecordSource.words(id)` (a `block -> '$.words'`
+  projection in the store) only when the prompt renders.
+- **Fingerprints** take, per member, the identifiers and words asked and a
+  module's glossary words; never what a type said of them. The same
+  `--terms` reuses every module, and another redoes exactly the modules whose
+  cut moved.
+- **The estimate** adds 30 completion tokens per word and 40 per glossary
+  entry, this section's figures, until a run measures them. On the Java
+  fixture the plan goes from 102 786 to 114 483 prompt tokens (+11%) and
+  from 36 650 to 41 670 completion tokens (+14%).
+
 ### 20.4 Milestone split
 
 **M19a — the lexicon, the split, the artifact** (closes the profile half of #19)
@@ -5499,7 +5534,7 @@ never looks like a fact.
 
 **M19c — the glossary: words in the explanation**
 
-- [ ] `words` on the three blocks and `glossary` on the module block,
+- [x] `words` on the three blocks and `glossary` on the module block,
       `.optional()` in the record schema and required on the wire;
       `PROMPT_VERSION = "2"`; the *Words to define* prompt section fed by
       `@codegraph/vocabulary` (types only, through the env); the module
@@ -5513,7 +5548,9 @@ never looks like a fact.
       completions (`COMPLETION_TOKENS_PER_BLOCK` re-measured); a reader's
       answer for a record explained under prompt `"1"`; on the fake client
       and live on the Java fixture; the §20.3 estimate replaced by the
-      Broadleaf measurement.
+      Broadleaf measurement. *(In: `--terms`, the estimate's per-word
+      figures, the fake-client run. Owed: the measurement, live runs, the
+      prompt-1 answer.)*
 - [ ] Readers: `insights --word`, `--glossary`, `--polysemy`, `--concerns`;
       `vocabulary --drift-insights`; `/glossary.json`; the entries in the
       Vocabulary tab and in the Navigate panel as inferences.

docs/cli.md

@@ -81,7 +81,7 @@ codegraph explain  [model.jsonl] [--src DIR] [--out FILE] [--dry-run]
                    [--model SLUG] [--rollup-model SLUG] [--depth N]
                    [--max-calls N] [--max-tokens N] [--scope IDS]
                    [--concurrency N]
-                   [--max-lines N] [--max-scc N] [--force] [--retry-failed]
+                   [--max-lines N] [--max-scc N] [--terms N] [--force] [--retry-failed]
                    [--export | --import FILE] [--yes] [--json]
                    [--framework spring] [--internal-only] [--declared-only]
 
@@ -303,7 +303,11 @@ interactive terminal it is required. A unit that could not be explained exits
 side-car; `--retry-failed` redoes just those units and their direct
 dependents. A `401`/`402`/`403` aborts the run instead of failing every
 remaining unit the same way; `--max-tokens N` caps what one answer may cost,
-which on a prepaid account is what keeps a low balance from refusing every call. Design record: [insights.md](insights.md).
+which on a prepaid account is what keeps a low balance from refusing every call.
+Each unit is also asked for the meaning of the words its names spell (`words`:
+its own name, an operation's parameters, a type's fields, split as `vocabulary`
+splits them), and a module reconciles what its types said into a `glossary`
+of its heaviest `--terms N` words (default 60). Design record: [insights.md](insights.md).
 
 ### `insights`
 Asks the insights store what `explain` has written — and opens nothing else:

docs/insights.md

@@ -103,6 +103,27 @@ The system message carries the Specy definitions, one line per concept, and
 the task per level. A cycle asks for one entry per member id. A cycle larger
 than `--max-scc` is chunked: each chunk sees the others' signatures only.
 
+**The words of the names (prompt 2, PLAN.md §20.3).** Every unit is also asked
+what the words of its names mean in it: a section *Words to define* lists, per
+identifier, the words of its own name and of an operation's parameters or a
+type's fields, split by `@codegraph/vocabulary` (injected as a `Speller`: the
+insights package runs no splitter). Short words and keywords are not asked; the
+deterministic class is withheld, so the judgement — `domain`, `technical` or
+`ambiguous` — is the model's. The answer is one `words` entry per word, with
+its `sense` and its `meaning` in this unit. The words are given, not chosen: on
+receipt an entry for a word not listed is dropped, a repeat keeps the first,
+and `in` is restated from the split. A module is also shown what its types
+said of the words they spell — read from the store through a `words(id)`
+projection, as summaries are, and only when its prompt renders — heaviest
+first by the vocabulary's module weights, cut at `--terms N`, and answers a
+`glossary` that reconciles them, flagging a word its types disagree on as
+`polysemous`. Templated records carry no `words`.
+
+`PROMPT_VERSION` is `"2"`. A record explained under prompt 1 keeps decoding and
+re-encoding to its own bytes (`words` and `glossary` are optional in the record
+schema) and stays in the store until a run replaces it; the answer schema
+requires both, and an answer without them gets the repair re-ask.
+
 ## IN-4 · The side-car and its determinism
 
     {"t":"header","kind":"codegraph.insights/1",…,"metamodel":"specy.domain/3",…}
@@ -247,8 +268,11 @@ is handed an OPEN database, so the path, the file and `node:sqlite` itself
 
 Every record carries a sha256 over what it was computed from: the prompt
 version, the model slug, the level, the source slices, the comments, the
-signatures, a digest of the facts shown, the **fingerprints of the units it
-depended on**, and the ids of dependencies that had no record. Merkle-style:
+signatures, a digest of the facts shown, the words its prompt asks to define
+and a module's glossary words (so a lexicon correction re-explains exactly the
+units whose words it changed, and another `--terms` exactly the modules whose
+cut moved), the **fingerprints of the units it depended on**, and the ids of
+dependencies that had no record. Merkle-style:
 change one leaf's source and exactly its transitive dependents and containers
 change; nothing else does. The explanation text is deliberately **not**
 hashed — a non-deterministic answer must never cascade re-runs.

packages/cli/src/args.ts

@@ -658,6 +658,14 @@ export const EXPLAIN_SPEC: CommandSpec = {
       integer: true,
       defaultValue: "12",
     },
+    {
+      name: "terms",
+      type: "string",
+      describe: "Words a module's glossary reconciles: the heaviest N its types define.",
+      placeholder: "N",
+      integer: true,
+      defaultValue: "60",
+    },
     {
       name: "scope",
       type: "string",
@@ -1282,6 +1290,8 @@ export interface ExplainOptions extends ModelInputOptions, ViewOptions, CacheOpt
   readonly concurrency: number;
   readonly maxLines: number;
   readonly maxScc: number;
+  /** `--terms N`: a module's glossary keeps its heaviest N words. */
+  readonly terms: number;
   /** `--scope a,b`; empty = everything. */
   readonly scope: readonly string[];
   readonly framework: string | undefined;
@@ -1983,6 +1993,7 @@ export function parseInvocation(argv: readonly string[]): Invocation {
           concurrency: integerOf(values, "concurrency") ?? 4,
           maxLines: integerOf(values, "max-lines") ?? 200,
           maxScc: integerOf(values, "max-scc") ?? 12,
+          terms: integerOf(values, "terms") ?? 60,
           scope: metricList(stringOf(values, "scope")),
           framework: stringOf(values, "framework"),
           ...viewOf(values),

packages/cli/src/commands/explain.ts

@@ -4,6 +4,9 @@ import { createInterface } from "node:readline/promises";
 import { ModelBuilder, readModelRecordsSync } from "@codegraph/core";
 import { FRAMEWORK_PROFILES, buildDomainFacts, folderFor, loadSqlite } from "@codegraph/analyzer";
 import {
+  COMPLETION_TOKENS_PER_BLOCK,
+  COMPLETION_TOKENS_PER_GLOSSARY_ENTRY,
+  COMPLETION_TOKENS_PER_WORD,
   INSIGHTS_GENERATOR,
   INSIGHTS_KIND,
   INSIGHTS_METAMODEL,
@@ -39,6 +42,7 @@ import {
   type Unit,
 } from "@codegraph/insights";
 import { clientFromEnv, resolveProvider, type LlmClient, type Provider } from "@codegraph/llm";
+import { buildVocabulary, definitionSpeller, rankedWords, type VocabularyArtifact } from "@codegraph/vocabulary";
 import type { ExplainOptions } from "../args.js";
 import { EXIT, UsageError, type ExitCode } from "../exit.js";
 import { errLine, errLines, outLine, type IoSink } from "../io.js";
@@ -218,7 +222,10 @@ export async function explainCommand(
 
     const srcRoot = resolveSourceRoot(options, graph.union.models[0]?.root, units, seam);
     const reader = createSourceReader((relative) => seam.fs.readFile(join(srcRoot, relative)));
-    const env: PlanEnv = { graph, facts, units, reader, unitOf: walk.unitOf };
+    // The words a prompt asks to define are the cloud's (PLAN.md §20.3); the module weights are counted on first ask.
+    let vocabulary: VocabularyArtifact | undefined;
+    const speller = { words: definitionSpeller(graph), ranked: (id: string) => rankedWords((vocabulary ??= buildVocabulary(graph, { view })), id) };
+    const env: PlanEnv = { graph, facts, units, reader, unitOf: walk.unitOf, speller };
 
     const journal = `${out}.journal`;
     const loaded = loadExisting(out, storePath, journal, seam, io);
@@ -246,6 +253,7 @@ export async function explainCommand(
       force: options.force,
       maxCalls: options.maxCalls,
       inScope: options.retryFailed ? retryScope(previousFailures, walk) : scopePredicate(options.scope, graph, units),
+      terms: options.terms,
     });
 
     if (!source.clean) {
@@ -300,7 +308,7 @@ export async function explainCommand(
     const book: StoreBook = loaded.storeBook ?? storeBook(writing);
     book.begin(run);
 
-    const result = await executeRun(plan, env, book, complete, { maxScc: options.maxScc, depth: options.depth, maxLines: options.maxLines, previousFailures, ...(keyOf === undefined ? {} : { keyOf }) }, {
+    const result = await executeRun(plan, env, book, complete, { maxScc: options.maxScc, depth: options.depth, maxLines: options.maxLines, terms: options.terms, previousFailures, ...(keyOf === undefined ? {} : { keyOf }) }, {
       concurrency: options.concurrency,
       onFailure: (failure) => writing.fail(run, failure),
       onStep: (event) => {
@@ -760,7 +768,7 @@ function printEstimate(plan: RunPlan, options: ExplainOptions, io: IoSink): void
         ...(cost === undefined ? {} : { cost }),
         assumptions: {
           promptTokens: "characters of the rendered prompts / 4",
-          completionTokens: "measured block averages: operation 450, type 650, module 900 tokens, per block asked for",
+          completionTokens: `measured block averages: ${outputAssumption()}`,
           excluded: "repair re-asks, rate-limit retries",
         },
       }),
@@ -793,11 +801,17 @@ function estimateLines(plan: RunPlan, options: ExplainOptions): string[] {
   } else {
     lines.push("  cost: pass --price-in USD --price-out USD (per million tokens) for a figure");
   }
-  lines.push("  input ≈ rendered prompts at 4 characters per token; output ≈ measured block averages (operation 450, type 650, module 900) per block asked for.");
+  lines.push(`  input ≈ rendered prompts at 4 characters per token; output ≈ measured block averages: ${outputAssumption()}.`);
   lines.push("  not included: repair re-asks, rate-limit retries. Records already in the side-car are reused, not re-sent.");
   return lines;
 }
 
+/** What the output figure assumes, from the constants it is computed with. */
+function outputAssumption(): string {
+  const { operation, type, module } = COMPLETION_TOKENS_PER_BLOCK;
+  return `operation ${operation}, type ${type}, module ${module} tokens per block asked for, plus an estimated ${COMPLETION_TOKENS_PER_WORD} per word and ${COMPLETION_TOKENS_PER_GLOSSARY_ENTRY} per glossary entry asked to define`;
+}
+
 /**
  * The gate before spending: the estimate on stderr, then a question. A run
  * that plans no call, or `--yes`, passes silently; no terminal and no `--yes`