[jgrodziski]

First step of milestone M19c in PLAN.md §20 (word cloud / glossary, #6). Stacked on #33 (itself on #32, #31, #30, #29, #28 and #27; #26 is merged): review those first. This PR's diff is the eight commits after #33, plus the merge that brings #27's test-generator fix up the stack.

explain now asks each unit what the words of its names mean, and asks each package to reconcile its classes' answers into a glossary. The words come from the same splitter as the word cloud, so the explanations and the cloud talk about the same words.

What a prompt asks (packages/insights)

• Words to define. Each member of a unit lists its identifiers with their words: an operation its name and parameters, a type its name and fields, a module its name. On the Java fixture, Order is asked for order, max, lines, channel and trail. Only words the vocabulary classes domain or technical are listed. The class itself is not shown to the model.
• Every block gains words. Each entry has word, in (the identifiers it appears in), concern (domain, technical or ambiguous), sense, meaning and confidence.
• A module block also gains glossary. The prompt shows the words its types spell, heaviest first by vocabulary weight, each with what the types' records said about it. Each glossary entry gives concern, sense and meaning; when the types disagree, polysemous is set and each sense is listed under senses. --terms N (default 60) keeps the heaviest N words.
• What is kept of an answer.
  • A word the prompt did not list is dropped.
  • A repeated word keeps its first entry.
  • in is rewritten from the actual split, never taken from the model.
  • A glossary entry for an unlisted word is dropped.
• Fingerprints. The words asked join a unit's inputs. A rerun with the same --terms makes no call; a different value redoes only modules.
• The store. RecordSource.words(id) reads a record's words; the SQLite store projects block -> '$.words' without decoding the block.

Prompt version 2

PROMPT_VERSION moves from 1 to 2, so the next run on an existing store redoes every explanation. Version-1 records still decode, and decode → import → export still gives the same bytes: words and glossary are optional on a stored block. An answer is held to the new shape, so a new answer without words gets the usual one re-ask, then a failure.

vocabulary and the CLI

• @codegraph/vocabulary.
  • definitionSpeller(graph) gives an entity's own-name words, domain and technical only, in order and deduplicated.
  • rankedWords(artifact, scope) gives a scope's words in weight order.
• The CLI hands both to the walk as a Speller, so insights never imports vocabulary. The vocabulary is built only when a module's glossary needs its ranking.
• --terms 0 is a usage error (exit 2).
• The estimate. It counts 30 completion tokens per word and 40 per glossary entry asked, and its footer now says so. On the Java fixture the estimate grows by 11% in input and 14% in output, to 114 483 tokens in and 41 670 out ($0.0365 at $0.10/M in, $0.60/M out).

Docs

• docs/insights.md (IN-3, and the IN-5 fingerprint inputs) and docs/cli.md.
• On the website:
  • the explain reference gains --terms;
  • the insights.jsonl reference gains the words and glossary tables, WORD_CONCERNS, and the version-1 exception;
  • the dry-run and estimate figures are refreshed in the tutorial, the cost how-to and Discover step 5.
• PLAN §20.3 gets an "as built" paragraph; §20.4's first M19c item is checked.

Worth a look in review:

• Own-name words only. com.acme.order is asked only for order. A word of a name's prefix is asked where that prefix is the container's own name, so it is defined once, by the unit it names.
• No words on templated records. Getters, setters and the other templated units make no call, so nothing defines their words. PLAN's "every record carries words" therefore holds only for records a model wrote.
• The per-word figures are PLAN's guesses. They are not measured yet; that needs a live run.
• Version 2 costs a full rerun on any existing store. That was the point of versioning the prompt, but it is money.

The test-generator fix (merged from #27)

test(insights): build record arbitraries once. On #32, CI timed out on the store's decode → import → export round-trip property (5.3 s against Vitest's 5 s). Building a random record took most of the test's time: each one built a fresh regex-based generator for its fingerprint. The generators are now built once. The round trip drops from about 650 to 180 ms locally, and the projection properties from about 400 to 60 ms. A store that drops metadata on read, or projects the wrong JSON path, still fails them.

Verification

• pnpm -r build and typecheck are green, eslint is clean on every changed file, pnpm run gen:schemas leaves schemas/ unchanged, and the Hugo site builds.
• Tests:
  • cli 1,003, vocabulary 332, insights 136;
  • core 544, viz 143, navigator-ui 105, city 94, extractor-typescript 66, scm 56, navigator 40, llm 34;
  • analyzer passes 613 of 614: the Node 26 node:sqlite warning test noted on feat(core): language lexicons on profiles (M19a) #26.
• Deliberate mutations. I broke the code 30 times, one change at a time; the tests caught 28. Of the two they missed:
  • one showed a definitionSpeller parameter did nothing, so I removed it;
  • one showed nothing tested that --terms changes a module's fingerprint, so I added the reuse test.
• The single executable. ./build.sh --ts --sea passes its smoke checks.
• No live run yet. Every test uses the fake client.

Still owed for M19c:

• A live run on the Java fixture (68 calls; about six cents under version 1), and measuring the per-word cost from it.
• The store's word index (PR 10).
• The readers (insights --word/--glossary/--polysemy, vocabulary --drift-insights, /glossary.json, the navigator entries), and what a reader says about a version-1 record (PR 11).

Part of #6.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	codegraph	eb0fab4	Commit Preview URL Branch Preview URL	Oct 27 2026, 03:04 PM