diff --git a/PLAN.md b/PLAN.md index 28b7801..94d93de 100644 --- a/PLAN.md +++ b/PLAN.md @@ -5542,6 +5542,50 @@ and `confidence` are generated columns over it. The decisions: 0.48 s, `--concerns` 0.54 s, Node's start included. The re-explained Broadleaf store is owed, as above. +**As built — `vocabulary --drift-insights` (2026-10-28).** `concernDrift` and +`businessJudged` in `@codegraph/vocabulary` (`concern-drift.ts`), fed by the +store's `wordHits()`, which with no word now returns every entry. The decisions: + +- **Neither package imports the other.** The vocabulary package takes + `WordJudgement`s in its own shape and restates the three concerns + (`JUDGED_CONCERNS`), pinned equal to the insights block's by a CLI test; the + command maps the store's rows into them. +- **The class is the list's, per entity.** Each judgement's word is classed as + `buildVocabulary` would class it in that entity's language (read from the + union) under the list in effect, `--technical` included. So a word judged in + two languages may be two rows (`select`: Java `domain`, Go `keyword`). +- **Two lists.** The technical words a model judged domain somewhere, and the + domain words it judged technical, ranked by the judgements against, ties by + word; `--top N` cuts each (40). `ambiguous` is counted and contradicts + neither class; a `keyword` or `short` word (a lexicon corrected since) is + counted and contradicts nothing. +- **Kept as the vocabulary keeps.** A record of an id the model does not hold + (explained against another version of it) or the view does not keep is left + out and counted on stderr, as are prompt-1 records. +- **The surest five.** A word's judgements against are ordered by confidence, + then by id; the text shows five per word and says how many more, `--json` + every one. On Broadleaf every judgement in full was 6 868 lines, each padded + to the longest operation id (≈ 600 characters), so the entity now comes last, + unpadded: `getSectionKey in java:…AdminFieldController.getSectionKey(…)`. +- **The business list against both** (`--drift FILE` beside it): each term's + words with the classes the corpus says them under and the model's concerns + over every entity; a term drifts on a word the list does not call domain + (`list`) or a model judged technical somewhere (`model`). +- **A judgement never sets the exit code.** It is an inference; `--drift` + keeps its own exit 3 for a term the corpus never says (`said` is the + corpus-level drift's answer, pinned by a test). +- **The store** is beside the first model, or `--insights FILE` (serve's flag + name); missing or unmigrated, a usage error before the model is read. + `--scope`, `--level`, `--class` and `--svg` with it, and `--insights` + without it, are usage errors. `--json` is `report: "drift-insights"` in + `codegraph.vocabularyReport/1`. +- **Measured** on a store built over Broadleaf's own ids (29 098 records, the + words `definitionSpeller` asks plus parameters and attributes: 109 208 + entries, 12 % judged against the list): 4.1 s end to end against 3.2 s for + the plain report on the same machine (an x86-64 Linux box, slower than the + M1 of §20.1), at the same peak memory (≈ 0.5 GB). A store a model wrote is + owed, as above. + ### 20.4 Milestone split **M19a — the lexicon, the split, the artifact** (closes the profile half of #19) @@ -5619,7 +5663,7 @@ and `confidence` are generated columns over it. The decisions: - [ ] Readers: `insights --word`, `--glossary`, `--polysemy`, `--concerns`; `vocabulary --drift-insights`; `/glossary.json`; the entries in the Vocabulary tab and in the Navigate panel as inferences. *(In: the four - `insights` readers. Owed: `--drift-insights`, `/glossary.json`, the tab + `insights` readers, `--drift-insights`. Owed: `/glossary.json`, the tab and the panel.)* - [ ] `docs/vocabulary.md` (the idioms per profile, the classes, the weights, the artifact, the three layers, the readers, the drift diff --git a/docs/cli.md b/docs/cli.md index 97589da..db6efc7 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -69,6 +69,7 @@ codegraph navigator [model.jsonl] [--out FILE] codegraph vocabulary [model.jsonl] [--level corpus|module|type] [--scope ID] [--top N] [--class domain|technical|keyword|short|all] [--weights KEY=N,…] [--technical FILE] [--drift FILE] + [--drift-insights [--insights FILE]] [--out FILE] [--svg FILE] [--json] [--internal-only] [--declared-only] @@ -254,7 +255,21 @@ one block per scope, ranked by distinct terms (the cognitive-load report); `--weights` and `--technical FILE` change the scheme, and the header says which one counted. `--drift FILE` compares with a glossary, one business term per line: the terms no scope of `--level` says (exit `3`), the ones it says -and where, and the domain words no term contains. `--out FILE` writes the +and where, and the domain words no term contains. `--drift-insights` sets the +classes against what `explain` judged (M19c): it reads the insights store beside +the first model (`--insights FILE` names another), classes each word a model +defined as the list does in that entity's language, and lists the technical +words a model called domain somewhere, then the domain words it called +technical, the most judgements against first (`--top N` per list, 40 by +default), each with the judgements against it, surest first: confidence, +sense, and the identifiers in which entity (five per word, and how many more: +`--json` lists every one). With `--drift FILE` it adds the business terms against both: each +word's class and the model's concerns, and why the term drifts (`list`, +`model`). The judgements are inferences and never set the exit code; `--drift` +still exits `3` on a term the corpus never says. `--scope`, `--level`, +`--class` and `--svg` do not apply to it; records of entities the model or the +view does not have, and records explained under prompt 1, are counted on stderr. +`--json` gives `report: "drift-insights"`. `--out FILE` writes the `vocabulary.json` artifact (`codegraph.vocabulary/1`) beside the report. `--svg FILE` draws the cloud of the one scope reported — the corpus, or `--scope`'s — from the words `--class` and `--top` pick: size on the square diff --git a/docs/insights.md b/docs/insights.md index 072e1f1..b02e0f5 100644 --- a/docs/insights.md +++ b/docs/insights.md @@ -264,7 +264,8 @@ id: *explained* (the record), *failed* (the unit's failure — matched by member so every member of a failed cycle is answered), or *unknown*. "No explanation" is two different facts, and only one of them is silence. -Two readers exist. `codegraph insights` (cli.md) opens the store and nothing +Two readers answer about records (a third, `vocabulary --drift-insights`, +about words: below). `codegraph insights` (cli.md) opens the store and nothing else — not the model, not `explain.ts`, whose module graph is the provider SDK. And `codegraph serve` answers `/insight.json?id=…`, its one route that is not an artifact: tens of megabytes of prose do not belong in `navigator.json`, so a @@ -281,7 +282,8 @@ ids are never constructed. **The words' readers (M19c).** `glossary.ts` rolls the records' `words` and the modules' `glossary` up, over four more projections of the port: -`wordHits(word)` (the `insight_word` index, joined to each record's envelope), +`wordHits(word?)` (the `insight_word` index, joined to each record's envelope; +every entry with no word), `glossaryEntries(word?)` (each module block's `glossary`, through `json_each` over the module records alone), `concernRows()` (each module, word and concern once) and `withoutWords()`. A record is placed in its module by its natural @@ -299,6 +301,14 @@ print them (cli.md). On the synthetic store of Broadleaf's size, `--word order` prints its 549 entries and 19 glossary entries in under half a second, Node's start included. +One reader opens the store BESIDE the model: `codegraph vocabulary +--drift-insights` (cli.md). Only the model says which class the deterministic +rules give a word in an entity (the entity's language is the model's), so that +comparison cannot live in `insights`, which never opens the model. It reads +every entry through `wordHits()`, hands them to `@codegraph/vocabulary` in that +package's own shape (neither package imports the other), and sets the model's +concern beside the list's class, word by word, with the entities. + `@codegraph/insights` stays pure: `store.ts` is a port, and `store-sqlite.ts` is handed an OPEN database, so the path, the file and `node:sqlite` itself (one import site, in the analyzer's `store/sqlite.ts`) stay with the CLI. diff --git a/packages/cli/src/args.ts b/packages/cli/src/args.ts index 14d3965..f29d350 100644 --- a/packages/cli/src/args.ts +++ b/packages/cli/src/args.ts @@ -435,6 +435,19 @@ export const VOCABULARY_SPEC: CommandSpec = { "Compare with a glossary, one business term per line: the terms no scope of --level says (exit 3), and the domain words no term contains.", placeholder: "FILE", }, + { + name: "drift-insights", + type: "boolean", + describe: + "Compare the words' classes with what 'codegraph explain' judged them: the technical words a model called domain somewhere, " + + "and the reverse, with the entities; with --drift, the business terms against both.", + }, + { + name: "insights", + type: "string", + describe: "The insights store --drift-insights reads. Default: .insights.db beside the first model.", + placeholder: "FILE", + }, { name: "out", type: "string", @@ -1278,6 +1291,10 @@ export interface VocabularyOptions extends ModelInputOptions, ViewOptions, Cache readonly technical: string | undefined; /** `--drift FILE`: the glossary to compare with. */ readonly drift: string | undefined; + /** `--drift-insights`: the classes against the concerns a model judged, from the insights store. */ + readonly driftInsights: boolean; + /** `--insights FILE`: that store; undefined = beside the first model. */ + readonly insights: string | undefined; /** `--out FILE`: the artifact, beside the report. */ readonly out: string | undefined; /** `--svg FILE`: the cloud of the one scope reported, beside the report. */ @@ -2205,6 +2222,26 @@ function vocabularyOf(values: ParsedValues): Omit { } const svg = stringOf(values, "svg"); const level = stringOf(values, "level"); + const driftInsights = flagOf(values, "drift-insights"); + const insights = stringOf(values, "insights"); + if (insights !== undefined && !driftInsights) { + throw new UsageError("--insights does not apply without --drift-insights", "--insights names the store --drift-insights reads."); + } + if (driftInsights) { + const read = [ + [scope !== undefined, "--scope"], + [level !== undefined, "--level"], + [stringOf(values, "class") !== undefined, "--class"], + [svg !== undefined, "--svg"], + ] as const; + for (const [given, flag] of read) { + if (!given) continue; + throw new UsageError( + `${flag} does not apply with --drift-insights`, + "--drift-insights compares every word a model judged, over the corpus, with the class the list gives it.", + ); + } + } const many = drift !== undefined ? "--drift" : level !== undefined && level !== "corpus" ? `--level ${level}` : undefined; if (svg !== undefined && many !== undefined) { throw new UsageError(`--svg does not apply with ${many}`, "--svg draws one scope's cloud: the corpus, or the one --scope names."); @@ -2217,6 +2254,8 @@ function vocabularyOf(values: ParsedValues): Omit { weights: weightsOf(stringOf(values, "weights")), technical: stringOf(values, "technical"), drift, + driftInsights, + insights, out: stringOf(values, "out"), svg, json: flagOf(values, "json"), diff --git a/packages/cli/src/commands/insights.ts b/packages/cli/src/commands/insights.ts index 989916c..8f633ec 100644 --- a/packages/cli/src/commands/insights.ts +++ b/packages/cli/src/commands/insights.ts @@ -1,12 +1,8 @@ -import { existsSync } from "node:fs"; -import { loadSqlite } from "@codegraph/analyzer"; import { - InsightsStoreError, WORD_CONCERNS, answerInsight, concernsReport, polysemyReport, - sqliteInsightsStore, wordReport, wordsAbsence, type FailureRecord, @@ -19,7 +15,7 @@ import { import type { InsightsOptions } from "../args.js"; import { EXIT, UsageError, type ExitCode } from "../exit.js"; import { errLine, outLine, type IoSink } from "../io.js"; -import { storePathForModel } from "../insights-paths.js"; +import { openInsightsReader, storePathForModel } from "../insights-paths.js"; /** * `codegraph insights [model.jsonl] [--id ID | --level … --concept … | --failures | --runs] [--json]` @@ -42,7 +38,7 @@ import { storePathForModel } from "../insights-paths.js"; */ export function insightsCommand(options: InsightsOptions, io: IoSink): ExitCode { const path = options.store ?? storePathForModel(options.model); - const store = openReader(path); + const store = openInsightsReader(path, "--store"); try { switch (options.mode) { case "summary": @@ -69,24 +65,6 @@ export function insightsCommand(options: InsightsOptions, io: IoSink): ExitCode } } -function openReader(path: string): InsightsStore { - // Checked first: opening a path that does not exist would CREATE it, and asking must not leave a file behind. - if (!existsSync(path)) { - throw new UsageError( - `no insights store at ${path}`, - "'codegraph explain ' creates it (and imports a side-car it finds there). Pass --store FILE if it is elsewhere.", - ); - } - const db = loadSqlite().open(path); - try { - return sqliteInsightsStore(db, { readOnly: true }); - } catch (error) { - db.close(); - if (!(error instanceof InsightsStoreError)) throw error; - throw new UsageError(`${path}: ${error.message}`, "Pass --store FILE to read another store.", { cause: error }); - } -} - const plural = (count: number, noun: string, many = `${noun}s`): string => `${count} ${count === 1 ? noun : many}`; const usd = (cost: number | undefined): string => (cost === undefined ? "" : `, $${cost.toFixed(4)}`); @@ -212,7 +190,7 @@ function failureReason(failure: FailureRecord): string { } /** What every word answer is blind to, on stderr, where a pipe does not read it. */ -function noteWithoutWords(count: number, io: IoSink): void { +export function noteWithoutWords(count: number, io: IoSink): void { if (count === 0) return; errLine(io, `note: ${plural(count, "record")} a model wrote under prompt 1 define${count === 1 ? "s" : ""} no words; the next 'codegraph explain' re-explains ${count === 1 ? "it" : "them"}.`); } diff --git a/packages/cli/src/commands/vocabulary.ts b/packages/cli/src/commands/vocabulary.ts index 81f2b50..12a298d 100644 --- a/packages/cli/src/commands/vocabulary.ts +++ b/packages/cli/src/commands/vocabulary.ts @@ -1,30 +1,42 @@ import { Buffer } from "node:buffer"; import { readFileSync } from "node:fs"; import { type CodeGraph, type ViewDescriptor } from "@codegraph/analyzer"; +import type { InsightsStore } from "@codegraph/insights"; import { DEFAULT_WEIGHTS, + JUDGED_CONCERNS, WEIGHT_KEYS, buildVocabulary, + businessJudged, + concernDrift, operationScope, parseBusinessTerms, parseTechnicalWords, vocabularyCloudSvg, vocabularyDrift, vocabularyToJsonString, + type BusinessTermJudged, type BusinessTerms, + type ConcernCounts, + type ConcernDrift, type DriftReport, + type JudgedWord, type OperationScope, type ScopeLevel, + type TermClass, type VocabularyArtifact, type VocabularyOptions as CountOptions, type VocabularyScope, type Weights, + type WordJudgement, } from "@codegraph/vocabulary"; import type { VocabularyOptions } from "../args.js"; import { EXIT, UsageError, type ExitCode } from "../exit.js"; +import { openInsightsReader, storePathForModel } from "../insights-paths.js"; import { errLine, errLines, outLines, type IoSink } from "../io.js"; import { openAnalysis } from "../source.js"; import { resolveView, viewLabel } from "../view.js"; +import { noteWithoutWords } from "./insights.js"; /** * `codegraph vocabulary `: the words the model's names are @@ -41,16 +53,20 @@ import { resolveView, viewLabel } from "../view.js"; * * `--drift` exits 3 when a business term is said by no scope of the level, * as `analyze --report cycles` does when a cycle exists: the tool worked, and - * the answer is bad news a CI job may gate on. + * the answer is bad news a CI job may gate on. `--drift-insights` sets what a + * model judged beside the list, and a judgement is an inference: it never + * makes a finding, so only `--drift`'s own facts set the exit code there. */ export function vocabularyCommand(options: VocabularyOptions, io: IoSink): ExitCode { - // The flag files first: a bad one must fail before the model is read. + // The flag files first, the store with them: a bad one must fail before the model is read. const technical = options.technical === undefined ? undefined : technicalOf(options.technical); const glossary = options.drift === undefined ? undefined : glossaryOf(options.drift); const weights: Weights = { ...DEFAULT_WEIGHTS, ...options.weights }; + const judges = options.driftInsights ? judgesOf(options.insights ?? storePathForModel(options.models[0] ?? "")) : undefined; - const source = openAnalysis(options.models, options, io); + let source: ReturnType | undefined; try { + source = openAnalysis(options.models, options, io); const graph = source.graph(); const view = resolveView(options); const counting: CountOptions = { view, weights, ...(technical === undefined ? {} : { technical: technical.words }) }; @@ -65,12 +81,21 @@ export function vocabularyCommand(options: VocabularyOptions, io: IoSink): ExitC technicalFile: options.technical, }; let code: ExitCode = source.clean ? EXIT.OK : EXIT.FINDINGS; - if (glossary !== undefined) { + if (judges !== undefined) { + const report = concernDrift(graph, judgementsOf(judges.store), counting); + const business = glossary === undefined ? undefined : businessJudged(artifact, glossary.terms, report); + errLines(io, glossary === undefined ? [] : wordlessWarnings(glossary)); + outLines(io, judgedReport(report, header, judges, options, glossary, business)); + errLines(io, leftOutNotes(report)); + noteWithoutWords(judges.store.withoutWords(), io); + const unsaid = business?.filter((term) => !term.said).length ?? 0; + if (unsaid > 0) { + errLine(io, `${plural(unsaid, "business term")} ${neverSays("corpus")} — exiting 3 (findings).`); + code = EXIT.FINDINGS; + } + } else if (glossary !== undefined) { const report = vocabularyDrift(artifact, glossary.terms, { level: options.level }); - errLines( - io, - glossary.wordless.map(({ line, text }) => `warning: line ${line} of ${glossary.path} names no word, skipped: ${text}`), - ); + errLines(io, wordlessWarnings(glossary)); outLines(io, driftReport(artifact, report, header, options, glossary)); const unsaid = report.business.filter((term) => term.scopes.length === 0).length; if (unsaid > 0) { @@ -101,7 +126,8 @@ export function vocabularyCommand(options: VocabularyOptions, io: IoSink): ExitC } return code; } finally { - source.close(); + source?.close(); + judges?.store.close(); } } @@ -149,6 +175,10 @@ function glossaryOf(path: string): Glossary { return { path, ...parsed }; } +function wordlessWarnings(glossary: Glossary): readonly string[] { + return glossary.wordless.map(({ line, text }) => `warning: line ${line} of ${glossary.path} names no word, skipped: ${text}`); +} + /* ------------------------------------------------------------- the scopes */ /** A stored scope, or an operation's: what a block prints. */ @@ -218,7 +248,7 @@ interface Header { readonly technicalFile: string | undefined; } -function headerLines(report: "words" | "drift", header: Header, where: string): readonly string[] { +function headerLines(report: "words" | "drift" | "drift-insights", header: Header, where: string): readonly string[] { const list = header.technicalFile === undefined ? "" : ` from ${header.technicalFile}`; return [ `codegraph vocabulary — ${report}`, @@ -477,6 +507,155 @@ function driftReport( return lines; } +/* --------------------------------------------------- drift-insights report */ + +/** The insights store `--drift-insights` reads, and where it is: the report names it. */ +interface Judges { + readonly path: string; + readonly store: InsightsStore; +} + +function judgesOf(path: string): Judges { + return { path, store: openInsightsReader(path, "--insights") }; +} + +/** Every word a model defined, as the vocabulary package takes it: the store's rows, never its blocks. */ +function* judgementsOf(store: InsightsStore): Iterable { + for (const hit of store.wordHits()) { + const { word, concern, sense, confidence, in: inside } = hit.entry; + yield { id: hit.id, word, concern, sense, confidence, in: inside }; + } +} + +/** `domain 3 · technical 1`: the concerns judged, in the block's order, the ones never judged left out. */ +function concernsCell(concerns: ConcernCounts): string { + const judged = JUDGED_CONCERNS.filter((concern) => concerns[concern] > 0); + return judged.length === 0 ? "not judged" : judged.map((concern) => `${concern} ${concerns[concern]}`).join(" · "); +} + +function judgedReport( + report: ConcernDrift, + header: Header, + judges: Judges, + options: VocabularyOptions, + glossary: Glossary | undefined, + business: readonly BusinessTermJudged[] | undefined, +): readonly string[] { + const top = options.top ?? 40; + const sections = [ + { key: "technicalJudgedDomain", of: "technical", against: "domain", words: report.technicalJudgedDomain }, + { key: "domainJudgedTechnical", of: "domain", against: "technical", words: report.domainJudgedTechnical }, + ] as const; + const judgedOf = (termClass: TermClass): number => report.words.filter((w) => w.class === termClass).length; + const models = judges.store.header()?.models; + + if (options.json) { + return [ + JSON.stringify({ + kind: REPORT_KIND, + report: "drift-insights", + models: header.models, + view: header.view, + weights: header.weights, + technical: header.technical, + insights: { + path: judges.path, + models: models ?? null, + withoutWords: judges.store.withoutWords(), + leftOut: report.leftOut, + }, + compared: report.compared, + ...Object.fromEntries( + sections.map(({ key, of, words }) => [ + key, + { judged: judgedOf(of), total: words.length, shown: Math.min(top, words.length), top, words: words.slice(0, top) }, + ]), + ), + ...(glossary === undefined || business === undefined + ? {} + : { glossary: { path: glossary.path, terms: glossary.terms.length, wordless: glossary.wordless }, business }), + }), + ]; + } + + const judgedBy = models === undefined ? "" : `, judged by ${models.leaf} (operations), ${models.rollup} (types, modules)`; + const lines = [...headerLines("drift-insights", header, `insights: ${judges.path}${judgedBy}`)]; + if (glossary !== undefined) lines.push(`glossary: ${glossary.path}, ${plural(glossary.terms.length, "business term")}`); + lines.push( + "", + report.compared.entries === 0 + ? "compared: no word entry — no explained entity of this model defines a word" + : `compared: ${plural(report.compared.entries, "word entry", "word entries")} from ` + + plural(report.compared.entities, "explained entity", "explained entities"), + ); + + for (const { of, against, words } of sections) { + const judged = judgedOf(of); + const shown = words.slice(0, top); + const title = `${of} by the list, judged ${against}`; + lines.push(""); + if (judged === 0) lines.push(`${title}: no ${of} word judged`); + else if (words.length === 0) lines.push(`${title}: none of ${plural(judged, `${of} word`)}`); + else { + const cut = shown.length === words.length ? "" : ` (${shown.length} shown, --top ${top})`; + lines.push(`${title}: ${words.length} of ${plural(judged, `${of} word`)}${cut}, the most judgements against first`); + lines.push(...judgedBlocks(shown)); + } + } + + if (business !== undefined) { + const drifting = business.filter((term) => term.drift).length; + lines.push( + "", + `business terms against both: ${drifting} of ${business.length} drift, a word the list or a model does not call domain`, + "", + ); + const cells = business.flatMap((term) => + term.words.map((w, i) => [ + i === 0 ? term.text : "", + w.word, + w.classes.length === 0 ? "not said" : w.classes.join(", "), + concernsCell(w.concerns), + w.drift.join(", "), + ]), + ); + lines.push(...table(["TERM", "WORD", "LIST", "MODEL", "DRIFT"], cells, 5)); + } + return lines; +} + +/** The judgements against a word a text report shows, its surest; `--json` carries every one. */ +const AGAINST_SHOWN = 5; + +/** + * A word and its judgements, then the surest ones its class contradicts: + * confidence, the sense quoted, then "identifiers in entity", unpadded — an + * operation's id carries its signature and a type's identifiers its fields, + * both too long to pad a column to. + */ +function judgedBlocks(words: readonly JudgedWord[]): readonly string[] { + const shown = words.flatMap((w) => w.against.slice(0, AGAINST_SHOWN)); + const senseWidth = shown.reduce((width, a) => Math.max(width, JSON.stringify(a.sense).length), 0); + return words.flatMap((w) => [ + ` ${w.word} ${concernsCell(w.concerns)}`, + ...w.against + .slice(0, AGAINST_SHOWN) + .map((a) => ` ${a.confidence.toFixed(2)} ${pad(JSON.stringify(a.sense), senseWidth)} ${a.in.join(", ")} in ${a.id}`), + ...(w.against.length > AGAINST_SHOWN ? [` … and ${w.against.length - AGAINST_SHOWN} more (--json lists every one)`] : []), + ]); +} + +/** What the comparison could not see, on stderr: records the model cannot place, or the view drops. */ +function leftOutNotes(report: ConcernDrift): readonly string[] { + const { unknown, notKept } = report.leftOut; + return [ + ...(unknown === 0 + ? [] + : [`note: ${plural(unknown, "explained record names", "explained records name")} no entity of this model (explained against another version of it): left out.`]), + ...(notKept === 0 ? [] : [`note: ${plural(notKept, "explained record is", "explained records are")} outside the view: left out.`]), + ]; +} + /* ---------------------------------------------------------------- shared */ /** A left-aligned text column (the first `left` ones), the numbers right-aligned. */ diff --git a/packages/cli/src/insights-paths.ts b/packages/cli/src/insights-paths.ts index a87bead..6299faa 100644 --- a/packages/cli/src/insights-paths.ts +++ b/packages/cli/src/insights-paths.ts @@ -1,4 +1,7 @@ -import { insightsStorePathFor } from "@codegraph/insights"; +import { existsSync } from "node:fs"; +import { loadSqlite } from "@codegraph/analyzer"; +import { InsightsStoreError, insightsStorePathFor, sqliteInsightsStore, type InsightsStore } from "@codegraph/insights"; +import { UsageError } from "./exit.js"; /** * WHERE A MODEL'S INSIGHTS LIVE. `explain` writes them and `insights` reads @@ -15,3 +18,26 @@ export function sidecarPathFor(modelPath: string): string { export function storePathForModel(modelPath: string): string { return insightsStorePathFor(sidecarPathFor(modelPath)); } + +/** + * A store opened as a READER (`query_only`), or a usage error: a missing one + * is not created, and one this build must not touch is named and left alone. + * `flag` is the one that names another store (`--store FILE`). + */ +export function openInsightsReader(path: string, flag: string): InsightsStore { + // Checked first: opening a path that does not exist would CREATE it, and asking must not leave a file behind. + if (!existsSync(path)) { + throw new UsageError( + `no insights store at ${path}`, + `'codegraph explain ' creates it (and imports a side-car it finds there). Pass ${flag} FILE if it is elsewhere.`, + ); + } + const db = loadSqlite().open(path); + try { + return sqliteInsightsStore(db, { readOnly: true }); + } catch (error) { + db.close(); + if (!(error instanceof InsightsStoreError)) throw error; + throw new UsageError(`${path}: ${error.message}`, `Pass ${flag} FILE to read another store.`, { cause: error }); + } +} diff --git a/packages/cli/test/args.test.ts b/packages/cli/test/args.test.ts index d569656..a897681 100644 --- a/packages/cli/test/args.test.ts +++ b/packages/cli/test/args.test.ts @@ -311,6 +311,8 @@ describe("vocabulary", () => { weights: {}, technical: undefined, drift: undefined, + driftInsights: false, + insights: undefined, out: undefined, svg: undefined, json: false, @@ -346,6 +348,9 @@ describe("vocabulary", () => { expect(parse(["vocabulary", "a.json", "--svg", "cloud.svg", "--level", "corpus"])).toMatchObject({ options: { svg: "cloud.svg", level: "corpus" }, }); + expect(parse(["vocabulary", "a.json", "--drift-insights", "--insights", "s.db", "--drift", "g.txt", "--top", "3"])).toMatchObject({ + options: { driftInsights: true, insights: "s.db", drift: "g.txt", top: 3 }, + }); }); it("offers the package's levels and classes, not a copy", () => { @@ -391,6 +396,19 @@ describe("vocabulary", () => { } }); + // The list against the model is word by word over the corpus: no scope, level, class or cloud picks its words. + it("refuses the flags --drift-insights would leave unread, and a store with nothing to read it", () => { + for (const [argv, message] of [ + [["--drift-insights", "--scope", "x"], "--scope does not apply with --drift-insights"], + [["--drift-insights", "--level", "corpus"], "--level does not apply with --drift-insights"], + [["--drift-insights", "--class", "all"], "--class does not apply with --drift-insights"], + [["--drift-insights", "--svg", "c.svg"], "--svg does not apply with --drift-insights"], + [["--insights", "s.db"], "--insights does not apply without --drift-insights"], + ] as const) { + expect(usageErrorFor(["vocabulary", "a.json", ...argv]).message, argv.join(" ")).toBe(message); + } + }); + // A cloud is one scope's: the corpus, or the one --scope names. it("refuses --svg with a report of many scopes", () => { for (const [argv, message] of [ diff --git a/packages/cli/test/cache.test.ts b/packages/cli/test/cache.test.ts index f7c3dfd..bd3274b 100644 --- a/packages/cli/test/cache.test.ts +++ b/packages/cli/test/cache.test.ts @@ -263,6 +263,8 @@ describe("vocabulary through the store", () => { weights: {}, technical: undefined, drift: undefined, + driftInsights: false, + insights: undefined, out: undefined, svg: undefined, json: true, diff --git a/packages/cli/test/e2e-exit-codes.test.ts b/packages/cli/test/e2e-exit-codes.test.ts index b6b8328..8c33beb 100644 --- a/packages/cli/test/e2e-exit-codes.test.ts +++ b/packages/cli/test/e2e-exit-codes.test.ts @@ -130,6 +130,9 @@ const USAGE_CASES: readonly Case[] = [ { why: "an unknown role to weigh", argv: ["vocabulary", FIXTURE, "--weights", "class=8"], expected: EXIT.USAGE }, // Raised inside the command, once the model says the id is not there. { why: "an unknown scope", argv: ["vocabulary", FIXTURE, "--scope", "java:nowhere/Nothing"], expected: EXIT.USAGE }, + // The fixture has no store beside it: refused before the model is read. + { why: "words to compare with no insights store", argv: ["vocabulary", FIXTURE, "--drift-insights"], expected: EXIT.USAGE }, + { why: "a store and nothing to read it", argv: ["vocabulary", FIXTURE, "--insights", "model.insights.db"], expected: EXIT.USAGE }, ]; const FINDING_CASES: readonly Case[] = [ diff --git a/packages/cli/test/vocabulary.test.ts b/packages/cli/test/vocabulary.test.ts index 75cddec..ddf1bbd 100644 --- a/packages/cli/test/vocabulary.test.ts +++ b/packages/cli/test/vocabulary.test.ts @@ -1,10 +1,30 @@ -import { mkdtempSync, writeFileSync } from "node:fs"; +import { copyFileSync, mkdtempSync, readdirSync, readFileSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { readModelFileSync } from "@codegraph/core"; -import { buildGraph, loadModels } from "@codegraph/analyzer"; -import { buildVocabulary, operationScope, vocabularyCloudSvg, vocabularyToJsonString } from "@codegraph/vocabulary"; +import { buildGraph, loadSqlite, loadModels } from "@codegraph/analyzer"; +import { + INSIGHTS_GENERATOR, + INSIGHTS_KIND, + INSIGHTS_METAMODEL, + PROMPT_VERSION, + WORD_CONCERNS, + decodeInsights, + encodeInsightsToString, + sqliteInsightsStore, + type InsightRecord, + type InsightsHeader, + type WordConcern, + type WordEntry, +} from "@codegraph/insights"; +import { + JUDGED_CONCERNS, + buildVocabulary, + operationScope, + vocabularyCloudSvg, + vocabularyToJsonString, +} from "@codegraph/vocabulary"; import { describe, expect, it } from "vitest"; import type { VocabularyOptions } from "../src/args.js"; import { vocabularyCommand } from "../src/commands/vocabulary.js"; @@ -31,6 +51,8 @@ function options(overrides: Partial = {}): VocabularyOptions weights: {}, technical: undefined, drift: undefined, + driftInsights: false, + insights: undefined, out: undefined, svg: undefined, json: false, @@ -372,6 +394,211 @@ describe("vocabulary: --drift", () => { }); }); +const said = (word: string, concern: WordConcern, sense: string, inside: string[], confidence = 0.8): WordEntry => + ({ word, in: inside, concern, sense, meaning: `${sense}, here.`, confidence }); + +/** A record as 'codegraph explain' writes one, carrying these words; none is a prompt-1 record. */ +function record(id: string, level: "module" | "type" | "operation", words: WordEntry[] | undefined): InsightRecord { + const block = + level === "module" + ? { name: "M", description: "A module.", apis: [], spis: [], dependsOn: [], concepts: [], boundedContextHint: null, sharedKernelHint: null, ubiquitousLanguage: [], confidence: 0.6, glossary: [] } + : level === "type" + ? { name: "T", description: "A type.", concept: "entity", eventKind: null, interfaceRole: null, aggregateRoot: false, containedIn: null, syncPattern: null, identity: null, fields: [], invariants: [], stateMachine: null, relatesTo: [], exposes: [], dependsOn: [], domainTerms: [], confidence: 0.7 } + : { name: "op", description: "Bills.", safe: false, idempotent: false, owner: "domainService", handlesCommand: null, emits: [], preconditions: [], postconditions: [], invariantsEnforced: [], usesSpi: [], domainTerms: [], confidence: 0.7 }; + const kind = level === "module" ? "package" : level === "type" ? "class" : "method"; + return { t: "i", id, level, kind, origin: "llm", model: "strong/model", fingerprint: "a".repeat(64), block: { ...block, ...(words === undefined ? {} : { words }) } } as InsightRecord; +} + +/** An insights store in `dir`, holding these records. */ +function storeWith(dir: string, records: readonly InsightRecord[]): string { + const header: InsightsHeader = { + t: "header", kind: INSIGHTS_KIND, generatedBy: INSIGHTS_GENERATOR, promptVersion: PROMPT_VERSION, metamodel: INSIGHTS_METAMODEL, + models: { leaf: "cheap/model", rollup: "strong/model" }, provider: "openrouter", depth: 1, + source: { paths: ["model.jsonl"], langs: ["java"], view: { name: "all", filters: [] } }, + }; + const path = join(dir, "model.insights.db"); + const store = sqliteInsightsStore(loadSqlite().open(path)); + const eof = { t: "eof" as const, counts: { records: records.length, llm: records.length, template: 0, reused: 0, failed: 0 }, usage: { promptTokens: 0, completionTokens: 0 }, generatedAt: "2026-10-28T10:00:00.000Z" }; + store.importFile(decodeInsights(encodeInsightsToString(header, records, eof))); + store.close(); + return path; +} + +/** + * A store over the fixture's own ids, as 'codegraph explain' would leave it: + * `service` judged domain in OrderService (the list calls it technical), + * `trail` judged technical in LedgerAdapter (the list calls it domain), a + * record of an entity the model does not have, and one explained under prompt 1. + */ +function storeOf(dir: string): string { + return storeWith(dir, [ + record("java:com.acme.order/OrderService.bill(com.acme.order.Order)", "operation", [said("bill", "domain", "to invoice", ["bill"])]), + record("java:com.acme.order.adapter/LedgerAdapter", "type", [ + said("ledger", "domain", "the books", ["LedgerAdapter"]), + said("adapter", "technical", "a port's implementation", ["LedgerAdapter"]), + said("trail", "technical", "an audit log", ["trail"], 0.7), + ]), + record("java:com.acme.order/Basket", "type", undefined), + record("java:com.acme.order/Order", "type", [said("order", "domain", "a customer's request", ["Order"]), said("channel", "domain", "a sales channel", ["channel"])]), + record("java:com.acme.order/OrderService", "type", [ + said("order", "domain", "a customer's request", ["OrderService"]), + said("service", "domain", "after-sales care", ["OrderService"], 0.6), + said("ledger", "domain", "the books", ["ledger"]), + ]), + record("java:com.acme.order/Refund", "type", [said("refund", "domain", "money back", ["Refund"])]), + record("java:com.acme.order", "module", [said("order", "domain", "the ordering business", ["com.acme.order"])]), + ]); +} + +describe("vocabulary: --drift-insights, the list against what a model judged", () => { + const store = storeOf(mkdtempSync(join(tmpdir(), "codegraph-cli-vocabulary-store-"))); + const judged = (overrides: Partial = {}) => vocabularyTo({ driftInsights: true, insights: store, ...overrides }); + + it("lists the technical words a model called domain somewhere, then the reverse, each with the entities that did", () => { + const { io, code } = judged(); + const lines = io.stdoutLines(); + expect(lines[0]).toBe("codegraph vocabulary — drift-insights"); + expect(lines[2]).toBe(`insights: ${store}, judged by cheap/model (operations), strong/model (types, modules)`); + const report = lines.slice(lines.indexOf("") + 1); + expect(report).toEqual([ + "compared: 10 word entries from 5 explained entities", + "", + "technical by the list, judged domain: 1 of 2 technical words, the most judgements against first", + " service domain 1", + ` 0.60 "after-sales care" OrderService in java:com.acme.order/OrderService`, + "", + "domain by the list, judged technical: 1 of 5 domain words, the most judgements against first", + " trail technical 1", + ` 0.70 "an audit log" trail in java:com.acme.order.adapter/LedgerAdapter`, + ]); + // The judgements are inferences: they never make a finding. + expect(code).toBe(EXIT.OK); + expect(io.stderr()).toContain("note: 1 record a model wrote under prompt 1 defines no words; the next 'codegraph explain' re-explains it."); + expect(io.stderr()).toContain("note: 1 explained record names no entity of this model (explained against another version of it): left out."); + }); + + it("shows a word's five surest judgements against it, and says how many more --json holds", () => { + const types = ["Basket", "Batch", "Channel", "Money", "Notifications", "Order", "OrderService"]; + const crowded = storeWith( + mkdtempSync(join(tmpdir(), "codegraph-cli-vocabulary-crowded-")), + // The surest last by name: what is shown is decided by confidence, not by id. + types.map((name, i) => record(`java:com.acme.order/${name}`, "type", [said("adapter", "domain", "a go-between", [name], 0.3 + i / 10)])), + ); + const lines = vocabularyTo({ driftInsights: true, insights: crowded }).io.stdoutLines(); + const from = lines.indexOf(" adapter domain 7"); + expect(lines.slice(from + 1, from + 7).map((line) => line.split(/ {2,}/u).at(-1))).toEqual([ + ...types.slice(2).reverse().map((name) => `${name} in java:com.acme.order/${name}`), + "… and 2 more (--json lists every one)", + ]); + const json = JSON.parse(vocabularyTo({ driftInsights: true, insights: crowded, json: true }).io.stdout()) as { + technicalJudgedDomain: { words: { against: unknown[] }[] }; + }; + expect(json.technicalJudgedDomain.words[0]?.against).toHaveLength(7); + }); + + it("--top cuts each list, and says so", () => { + const lines = judged({ technical: file("tech.txt", "order\nchannel\nledger\n"), top: 1 }).io.stdoutLines(); + // The file replaces the shared list whole: `adapter` and `service` are domain words under it. + expect(lines).toContain("technical by the list, judged domain: 3 of 3 technical words (1 shown, --top 1), the most judgements against first"); + expect(lines).toContain("domain by the list, judged technical: 2 of 4 domain words (1 shown, --top 1), the most judgements against first"); + expect(lines.filter((line) => /^ {2}\S/u.test(line))).toEqual([" order domain 3", " adapter technical 1"]); + }); + + it("over a store whose records define no word, says there is nothing to compare", () => { + const dir = mkdtempSync(join(tmpdir(), "codegraph-cli-vocabulary-empty-")); + const empty = join(dir, "model.insights.db"); + sqliteInsightsStore(loadSqlite().open(empty)).close(); + const { io, code } = vocabularyTo({ driftInsights: true, insights: empty }); + expect(code).toBe(EXIT.OK); + const lines = io.stdoutLines(); + expect(lines).toContain("compared: no word entry — no explained entity of this model defines a word"); + expect(lines).toContain("technical by the list, judged domain: no technical word judged"); + expect(lines).toContain("domain by the list, judged technical: no domain word judged"); + }); + + it("classifies under the list --technical gives", () => { + const lines = judged({ technical: file("tech.txt", "trail\nadapter\n") }).io.stdoutLines(); + expect(lines).toContain("technical by the list, judged domain: none of 2 technical words"); + expect(lines).toContain("domain by the list, judged technical: none of 5 domain words"); + }); + + it("with --drift, sets the business terms against both, and still exits 3 on a term the corpus never says", () => { + const glossary = file("glossary.txt", "Order Service\nLedger\nCustomer\n"); + const { io, code } = judged({ drift: glossary }); + const lines = io.stdoutLines(); + expect(lines).toContain(`glossary: ${glossary}, 3 business terms`); + const from = lines.indexOf("business terms against both: 1 of 3 drift, a word the list or a model does not call domain"); + expect(from).toBeGreaterThan(0); + expect(lines.slice(from + 1).map((line) => line.trimEnd().split(/ {2,}/u))).toEqual([ + [""], + ["", "TERM", "WORD", "LIST", "MODEL", "DRIFT"], + ["", "Order Service", "order", "domain", "domain 3"], + ["", "service", "technical", "domain 1", "list"], + ["", "Ledger", "ledger", "domain", "domain 2"], + ["", "Customer", "customer", "not said", "not judged"], + ]); + expect(code).toBe(EXIT.FINDINGS); + expect(io.stderr()).toContain("1 business term the corpus never says — exiting 3 (findings)."); + }); + + it("says the same thing as one JSON object", () => { + const glossary = file("glossary.txt", "Order Service\n"); + const json = JSON.parse(judged({ drift: glossary, json: true, top: 1 }).io.stdout()) as Record; + expect(json).toMatchObject({ + kind: "codegraph.vocabularyReport/1", + report: "drift-insights", + insights: { path: store, models: { leaf: "cheap/model", rollup: "strong/model" }, withoutWords: 1, leftOut: { unknown: 1, notKept: 0 } }, + compared: { entries: 10, entities: 5 }, + technicalJudgedDomain: { judged: 2, total: 1, shown: 1, top: 1 }, + domainJudgedTechnical: { judged: 5, total: 1, shown: 1, top: 1 }, + glossary: { path: glossary, terms: 1, wordless: [] }, + business: [ + { + text: "Order Service", + said: true, + drift: true, + words: [ + { word: "order", classes: ["domain"], concerns: { domain: 3, technical: 0, ambiguous: 0 }, drift: [] }, + { word: "service", classes: ["technical"], concerns: { domain: 1, technical: 0, ambiguous: 0 }, drift: ["list"] }, + ], + }, + ], + }); + expect((json["technicalJudgedDomain"] as { words: unknown[] }).words).toEqual([ + { + word: "service", + class: "technical", + concerns: { domain: 1, technical: 0, ambiguous: 0 }, + against: [{ id: "java:com.acme.order/OrderService", word: "service", concern: "domain", sense: "after-sales care", confidence: 0.6, in: ["OrderService"] }], + }, + ]); + }); + + it("reads the store beside the first model by default, and writes nothing there", () => { + const dir = mkdtempSync(join(tmpdir(), "codegraph-cli-vocabulary-beside-")); + const model = join(dir, "model.jsonl"); + copyFileSync(FIXTURE, model); + const bytes = readFileSync(storeOf(dir)); + const before = readdirSync(dir).sort(); + const { io, code } = vocabularyTo({ models: [model], driftInsights: true }); + expect(code).toBe(EXIT.OK); + expect(io.stdoutLines()[2]).toMatch(new RegExp(`^insights: {2}${join(dir, "model.insights.db").replace(/[.*+?^${}()|[\]\\]/gu, "\\$&")},`, "u")); + expect(readdirSync(dir).sort()).toEqual(before); + expect(readFileSync(join(dir, "model.insights.db")).equals(bytes)).toBe(true); + }); + + it("refuses a store that is not there, naming the flag that points at another", () => { + const missing = join(mkdtempSync(join(tmpdir(), "codegraph-cli-vocabulary-none-")), "model.insights.db"); + const error = usageErrorOf({ driftInsights: true, insights: missing }); + expect(error.message).toBe(`no insights store at ${missing}`); + expect(error.hint).toContain("--insights FILE"); + }); + + it("knows the concerns a model may judge: the insights block's, restated", () => { + expect(JUDGED_CONCERNS).toEqual(WORD_CONCERNS); + }); +}); + describe("vocabulary: --out", () => { it("writes the artifact beside the report, and confirms on stderr", () => { const path = join(mkdtempSync(join(tmpdir(), "codegraph-cli-vocabulary-")), "vocabulary.json"); diff --git a/packages/insights/src/store-sqlite.ts b/packages/insights/src/store-sqlite.ts index f95968e..1f1f631 100644 --- a/packages/insights/src/store-sqlite.ts +++ b/packages/insights/src/store-sqlite.ts @@ -465,12 +465,12 @@ export function sqliteInsightsStore(db: SqliteDatabase, options: StoreOpenOption "SELECT w.entry AS entry, i.id AS id, i.level AS level, i.kind AS kind, i.name AS name, i.model AS model, i.key_lang, i.key_module, i.key_symbol, i.key_disambiguator FROM insight_word w JOIN insight i ON i.id = w.id"; const hitsOfWord = db.prepare(`${HIT_SQL} WHERE w.word = ? ORDER BY w.rowid`); const everyHit = db.prepare(`${HIT_SQL} ORDER BY w.rowid`); - const wordHits = (word: string): WordHit[] => { - const rows = /\p{Surrogate}/u.test(word) ? everyHit.iterate() : hitsOfWord.iterate(word); + const wordHits = (word?: string): WordHit[] => { + const rows = word === undefined || /\p{Surrogate}/u.test(word) ? everyHit.iterate() : hitsOfWord.iterate(word); const hits: WordHit[] = []; for (const row of rows) { const entry = JSON.parse(row["entry"] as string) as ListedWord; - if (entry.word !== word) continue; + if (word !== undefined && entry.word !== word) continue; const level = LEVELS[Number(row["level"])] as Level; hits.push({ id: row["id"] as string, level, kind: row["kind"] as string, name: text(row, "name"), key: keyOf(row), model: text(row, "model"), entry }); } diff --git a/packages/insights/src/store.ts b/packages/insights/src/store.ts index eafe047..08206df 100644 --- a/packages/insights/src/store.ts +++ b/packages/insights/src/store.ts @@ -189,8 +189,8 @@ export interface InsightsStore { /** Rows matching a reader's filter, in side-car order. */ query(query: InsightQuery): InsightRow[]; stats(): InsightStats; - /** Every record's entry for one word, in side-car order then block order: an index lookup (M19c). */ - wordHits(word: string): WordHit[]; + /** Every record's words, in side-car order then block order; `word` keeps that word's, an index lookup (M19c). */ + wordHits(word?: string): WordHit[]; /** The module records' glossary entries, in side-car order then block order; `word` keeps that word's. */ glossaryEntries(word?: string): GlossaryHit[]; /** Each (module, word, concern) the records' words hold, once; in no promised order. */ diff --git a/packages/insights/test/store.test.ts b/packages/insights/test/store.test.ts index 172a6bf..53b4821 100644 --- a/packages/insights/test/store.test.ts +++ b/packages/insights/test/store.test.ts @@ -718,12 +718,13 @@ describe("the word readers' projections (M19c): what the blocks say, never more" const hit = (r: InsightRecord, entry: WordEntry) => ({ id: r.id, level: r.level, kind: r.kind, name: r.name, key: r.key, model: r.model, entry: listed(entry) }); const asSet = (rows: readonly T[]): string[] => rows.map((row) => JSON.stringify(row)).sort(); - it("for any side-car, exact on any string: one word's entries, the glossaries, each concern once, the records without words", () => { + it("for any side-car, exact on any string: every entry or one word's, the glossaries, each concern once, the records without words", () => { fc.assert( fc.property(fileArb, (file) => { const store = memory(); const { records } = decodeInsights(file); store.importFile(decodeInsights(file)); + expect(store.wordHits()).toEqual(records.flatMap((r) => (r.block.words ?? []).map((w) => hit(r, w)))); const spelled = new Set(records.flatMap((r) => (r.block.words ?? []).map((w) => w.word))); // Side-car order, then block order. The words are any strings, a lone surrogate (which the index cannot match) included. for (const word of [...spelled, "\u{1F4E6}never"]) { @@ -752,6 +753,7 @@ describe("the word readers' projections (M19c): what the blocks say, never more" store.putUnit(run, "t/A", [{ ...type("t/A"), block: { ...TYPE, words: [ORDER] } } as InsightRecord]); store.putUnit(run, "o/z", [op("o/z", { block: { ...OPERATION, words: [ORDER] } })]); expect(store.wordHits("order").map((h) => h.id)).toEqual(["o/z", "t/A", "t/B", "m/a", "m/b"]); + expect(store.wordHits().map((h) => h.id)).toEqual(["o/z", "t/A", "t/B", "m/a", "m/b"]); expect(store.glossaryEntries().map((g) => g.id)).toEqual(["m/a", "m/b"]); store.close(); }); diff --git a/packages/vocabulary/src/concern-drift.ts b/packages/vocabulary/src/concern-drift.ts new file mode 100644 index 0000000..156c9b0 --- /dev/null +++ b/packages/vocabulary/src/concern-drift.ts @@ -0,0 +1,213 @@ +import { getProfile, type EntityId } from "@codegraph/core"; +import { compareIds, identityView, type CodeGraph, type View } from "@codegraph/analyzer"; +import type { VocabularyArtifact } from "./artifact.js"; +import { DEFAULT_TECHNICAL, TERM_CLASSES, createClassifier, type Classify, type TermClass } from "./classify.js"; +import type { BusinessTerm } from "./drift.js"; +import { languagesOf, technicalDigest } from "./reading.js"; + +/** + * THE LIST AGAINST THE MODEL (PLAN.md §20.3, layer 3): the class the + * deterministic rules give a word, beside the concern a model judged it to + * have in each explained entity whose names spell it. A technical word judged + * domain is a technical word in a domain position (`service` in + * `CustomerServiceTicket`); a domain word judged technical is a candidate for + * the list. Neither is a fact: both sides are reported, nothing is decided. + * + * The judgements come from the insights store, which this package never + * imports: the caller hands them over in this file's shape. + */ + +/** What a model may call a word: the insights block's concerns, restated (a CLI test pins the two equal). */ +export const JUDGED_CONCERNS = ["domain", "technical", "ambiguous"] as const; +export type JudgedConcern = (typeof JUDGED_CONCERNS)[number]; + +/** One entry of an explained entity's `words`: the model's judgement of one word in that entity. */ +export interface WordJudgement { + readonly id: EntityId; + readonly word: string; + readonly concern: JudgedConcern; + readonly sense: string; + readonly confidence: number; + /** The entity's identifiers the word is in. */ + readonly in: readonly string[]; +} + +export type ConcernCounts = Readonly>; + +/** A word under one class: a class is decided per language, so a word judged in two may be two rows. */ +export interface JudgedWord { + readonly word: string; + readonly class: TermClass; + readonly concerns: ConcernCounts; + /** The judgements the class contradicts — domain for a technical word, technical for a domain one; most confident first. */ + readonly against: readonly WordJudgement[]; +} + +export interface ConcernDrift { + /** By word in code-unit order, then class. */ + readonly words: readonly JudgedWord[]; + /** The list says technical, a model said domain: the most judgements against first, ties by word. */ + readonly technicalJudgedDomain: readonly JudgedWord[]; + /** The list says domain, a model said technical; ranked likewise. */ + readonly domainJudgedTechnical: readonly JudgedWord[]; + readonly compared: { readonly entries: number; readonly entities: number }; + /** Records left out, by distinct id: no entity of this model (explained against another version), or none the view keeps. */ + readonly leftOut: { readonly unknown: number; readonly notKept: number }; + /** The list the classes came from, as an artifact states it. */ + readonly technical: VocabularyArtifact["technical"]; +} + +export interface ConcernDriftOptions { + readonly view?: View; + /** Replaces the shared technical list whole, as `buildVocabulary`'s option does. */ + readonly technical?: readonly string[]; +} + +/** + * Compare every judgement of a kept entity with the class the list gives its + * word in that entity's language — read from the union, never from its id. + * Kept is the vocabulary's rule: the view's verdict on a non-stub entity. + */ +export function concernDrift( + graph: CodeGraph, + judgements: Iterable, + options: ConcernDriftOptions = {}, +): ConcernDrift { + const view = options.view ?? identityView; + const technical = options.technical ?? DEFAULT_TECHNICAL; + const langOf = languagesOf(graph); + const classifiers = new Map(); + const classOf = (id: EntityId, word: string): TermClass => { + const lang = langOf.get(id) ?? ""; + let classify = classifiers.get(lang); + if (classify === undefined) { + classify = createClassifier(getProfile(lang)?.lexicon, technical); + classifiers.set(lang, classify); + } + return classify(word); + }; + + const rows = new Map; against: WordJudgement[] }>(); + const unknown = new Set(); + const notKept = new Set(); + const entities = new Set(); + let entries = 0; + for (const judgement of judgements) { + const entity = graph.entity(judgement.id); + if (entity === undefined) { + unknown.add(judgement.id); + continue; + } + if (graph.isStub(judgement.id) || !view.entity(entity, graph)) { + notKept.add(judgement.id); + continue; + } + const termClass = classOf(judgement.id, judgement.word); + const key = `${termClass}\u0000${judgement.word}`; + let row = rows.get(key); + if (row === undefined) { + row = { word: judgement.word, class: termClass, concerns: { domain: 0, technical: 0, ambiguous: 0 }, against: [] }; + rows.set(key, row); + } + row.concerns[judgement.concern] += 1; + if (judgement.concern === OPPOSITE[termClass]) row.against.push(judgement); + entities.add(judgement.id); + entries += 1; + } + + const words: JudgedWord[] = [...rows.values()] + .map((row) => ({ ...row, against: row.against.sort(byJudgement) })) + .sort((a, b) => compareIds(a.word, b.word) || TERM_CLASSES.indexOf(a.class) - TERM_CLASSES.indexOf(b.class)); + const contradicted = (termClass: TermClass): JudgedWord[] => + words + .filter((w) => w.class === termClass && w.against.length > 0) + .sort((a, b) => b.against.length - a.against.length || compareIds(a.word, b.word)); + return { + words, + technicalJudgedDomain: contradicted("technical"), + domainJudgedTechnical: contradicted("domain"), + compared: { entries, entities: entities.size }, + leftOut: { unknown: unknown.size, notKept: notKept.size }, + technical: technicalDigest(technical), + }; +} + +/** The concern a class is contradicted by; the classes a definition is never asked for have none. */ +const OPPOSITE: Readonly> = { + domain: "technical", + technical: "domain", + keyword: undefined, + short: undefined, +}; + +/** The model's surest first, the telling ones; then by entity and what was said, whatever order the store answered in. */ +function byJudgement(a: WordJudgement, b: WordJudgement): number { + return ( + b.confidence - a.confidence || + compareIds(a.id, b.id) || + compareIds(a.sense, b.sense) || + compareIds(a.in.join("\u0000"), b.in.join("\u0000")) + ); +} + +/** Why a business word drifts: the list does not call it domain, or a model judged it technical somewhere. */ +export type BusinessDrift = "list" | "model"; + +export interface BusinessWordJudged { + readonly word: string; + /** The classes the corpus says it under; none when no kept name says it. */ + readonly classes: readonly TermClass[]; + /** Over every entity that judged it, whatever its class there. */ + readonly concerns: ConcernCounts; + /** Empty when both call it domain, or say nothing of it. */ + readonly drift: readonly BusinessDrift[]; +} + +export interface BusinessTermJudged { + readonly text: string; + readonly words: readonly BusinessWordJudged[]; + /** Every word of it said by the corpus: `vocabularyDrift`'s answer at the corpus level. */ + readonly said: boolean; + /** A word of it drifts. */ + readonly drift: boolean; +} + +/** + * The business list against both (PLAN.md §20.3): a business term is a domain + * concept by definition, so each of its words is shown with what the list and + * the model call it. The artifact must be counted under the drift's own list. + */ +export function businessJudged( + artifact: VocabularyArtifact, + terms: readonly BusinessTerm[], + drift: ConcernDrift, +): readonly BusinessTermJudged[] { + if (artifact.technical.sha256 !== drift.technical.sha256) { + throw new Error("businessJudged: the artifact was counted under another technical list than the drift"); + } + const classes = new Map(); + for (const term of artifact.terms) classes.set(term.text, [...(classes.get(term.text) ?? []), term.class]); + const concerns = new Map>(); + for (const w of drift.words) { + const sum = concerns.get(w.word) ?? { domain: 0, technical: 0, ambiguous: 0 }; + for (const concern of JUDGED_CONCERNS) sum[concern] += w.concerns[concern]; + concerns.set(w.word, sum); + } + return terms.map((term): BusinessTermJudged => { + const words = term.words.map((word): BusinessWordJudged => { + const said = classes.get(word) ?? []; + const judged = concerns.get(word) ?? { domain: 0, technical: 0, ambiguous: 0 }; + const drift: BusinessDrift[] = [ + ...(said.some((c) => c !== "domain") ? (["list"] as const) : []), + ...(judged.technical > 0 ? (["model"] as const) : []), + ]; + return { word, classes: said, concerns: judged, drift }; + }); + return { + text: term.text, + words, + said: words.every((w) => w.classes.length > 0), + drift: words.some((w) => w.drift.length > 0), + }; + }); +} diff --git a/packages/vocabulary/src/index.ts b/packages/vocabulary/src/index.ts index 58ef528..2b41e0b 100644 --- a/packages/vocabulary/src/index.ts +++ b/packages/vocabulary/src/index.ts @@ -16,3 +16,4 @@ export * from "./carriers.js"; export * from "./define.js"; export * from "./cloud.js"; export * from "./svg.js"; +export * from "./concern-drift.js"; diff --git a/packages/vocabulary/test/concern-drift.test.ts b/packages/vocabulary/test/concern-drift.test.ts new file mode 100644 index 0000000..8eb9f32 --- /dev/null +++ b/packages/vocabulary/test/concern-drift.test.ts @@ -0,0 +1,216 @@ +import { makeView } from "@codegraph/analyzer"; +import fc from "fast-check"; +import { describe, expect, it } from "vitest"; +import { buildVocabulary } from "../src/aggregate.js"; +import { + JUDGED_CONCERNS, + businessJudged, + concernDrift, + type ConcernDrift, + type WordJudgement, +} from "../src/concern-drift.js"; +import { parseBusinessTerms, vocabularyDrift } from "../src/drift.js"; +import { cls, graphOf, pkg, structural } from "./toy.js"; + +/** + * The list against the model (PLAN.md §20.3, layer 3): what the deterministic + * class says of a word, beside what the model judged it to be in each entity + * whose names spell it. The model's judgements are inputs here, as the store + * would hand them over; nothing is asked. + */ + +const order = "java:com.acme.order"; +const Order = `${order}/Order`; +const Service = `${order}/OrderService`; +const String_ = "java:java.lang/String"; +const corpus = [ + pkg(order, "com.acme.order"), + cls(Order, "Order", order), + structural(`${Order}#sortOrder`, "attribute", "sortOrder", Order), + cls(Service, "OrderService", order), + pkg("java:java.lang", "java.lang", undefined, true), + cls(String_, "String", "java:java.lang", true), +]; +const graph = graphOf([{ lang: "java", entities: corpus }]); + +const judged = (id: string, word: string, concern: WordJudgement["concern"], sense: string, inside: readonly string[] = [word]): WordJudgement => ({ + id, + word, + concern, + sense, + confidence: 0.8, + in: inside, +}); + +const JUDGEMENTS: readonly WordJudgement[] = [ + judged(order, "order", "domain", "the ordering business", ["com.acme.order"]), + judged(Order, "order", "domain", "a customer's request", ["Order"]), + judged(Order, "sort", "technical", "arranging", ["sortOrder"]), + judged(Service, "order", "domain", "a customer's request", ["OrderService"]), + judged(Service, "service", "domain", "after-sales care", ["OrderService"]), + judged(`${Order}.ship()`, "ship", "domain", "send it", ["ship"]), + judged(String_, "string", "technical", "text"), +]; + +/** `[word, class, domain, technical, ambiguous, against ids]` per judged word. */ +const rows = (words: ConcernDrift["words"]) => + words.map((w) => [w.word, w.class, ...JUDGED_CONCERNS.map((c) => w.concerns[c]), w.against.map((a) => a.id)]); + +describe("concernDrift: the list's class beside the model's concern, word by word", () => { + it("counts every judgement of a kept entity under its word and the class the list gives it there", () => { + const report = concernDrift(graph, JUDGEMENTS); + expect(rows(report.words)).toEqual([ + ["order", "domain", 3, 0, 0, []], + ["service", "technical", 1, 0, 0, [Service]], + ["sort", "domain", 0, 1, 0, [Order]], + ]); + expect(report.compared).toEqual({ entries: 5, entities: 3 }); + }); + + it("leaves out, and counts, records of no entity of this model and of entities no kept view holds", () => { + // `ship()` is not in the model (explained against another version of it); a stub never speaks. + expect(concernDrift(graph, JUDGEMENTS).leftOut).toEqual({ unknown: 1, notKept: 1 }); + const internal = concernDrift(graph, [judged(order, "order", "domain", "x")], { + view: makeView("none", () => false, () => false), + }); + expect(internal.compared).toEqual({ entries: 0, entities: 0 }); + expect(internal.leftOut).toEqual({ unknown: 0, notKept: 1 }); + }); + + it("lists a technical word the model judged domain somewhere, and a domain word it judged technical, with the entities that did", () => { + const report = concernDrift(graph, JUDGEMENTS); + expect(report.technicalJudgedDomain.map((w) => [w.word, w.against.map((a) => [a.id, a.sense, a.in])])).toEqual([ + ["service", [[Service, "after-sales care", ["OrderService"]]]], + ]); + expect(report.domainJudgedTechnical.map((w) => [w.word, w.against.map((a) => [a.id, a.sense, a.in])])).toEqual([ + ["sort", [[Order, "arranging", ["sortOrder"]]]], + ]); + }); + + it("lists the judgements against a class most confident first, then by entity", () => { + const sure = { ...judged(Service, "sort", "technical", "arranging"), confidence: 0.9 }; + const unsure = { ...judged(order, "sort", "technical", "arranging"), confidence: 0.3 }; + const tied = { ...judged(Order, "sort", "technical", "arranging"), confidence: 0.3 }; + const [sort] = concernDrift(graph, [unsure, tied, sure]).domainJudgedTechnical; + expect(sort?.against.map((a) => a.id)).toEqual([Service, order, Order]); + }); + + it("an ambiguous judgement is counted, and is against neither class", () => { + const report = concernDrift(graph, [judged(Service, "service", "ambiguous", "both")]); + expect(rows(report.words)).toEqual([["service", "technical", 0, 0, 1, []]]); + expect(report.technicalJudgedDomain).toEqual([]); + }); + + it("classifies under the list it is given: a word moved onto it changes sections", () => { + const report = concernDrift(graph, JUDGEMENTS, { technical: ["order"] }); + expect(rows(report.words)).toEqual([ + ["order", "technical", 3, 0, 0, [order, Order, Service]], + ["service", "domain", 1, 0, 0, []], + ["sort", "domain", 0, 1, 0, [Order]], + ]); + expect(report.technicalJudgedDomain.map((w) => w.word)).toEqual(["order"]); + expect(report.technical).toEqual(buildVocabulary(graph, { technical: ["order"] }).technical); + }); + + it("decides a class per entity's language: Go reserves `select`, Java does not", () => { + const go = "go:shop"; + const mixed = graphOf([ + { lang: "java", entities: [pkg(order, "com.acme.order"), cls(`${order}/Select`, "Select", order)] }, + { lang: "go", entities: [pkg(go, "shop"), cls(`${go}/selectAll`, "selectAll", go)] }, + ]); + const report = concernDrift(mixed, [ + judged(`${order}/Select`, "select", "technical", "a query"), + judged(`${go}/selectAll`, "select", "technical", "a query"), + ]); + expect(rows(report.words)).toEqual([ + ["select", "domain", 0, 1, 0, [`${order}/Select`]], + ["select", "keyword", 0, 1, 0, []], + ]); + }); + + it("ranks each list by the entries against it, most first, ties by word", () => { + const report = concernDrift(graph, [ + judged(Order, "basket", "technical", "a container"), + judged(Order, "sort", "technical", "arranging"), + judged(Service, "sort", "technical", "arranging"), + ]); + expect(report.domainJudgedTechnical.map((w) => [w.word, w.against.length])).toEqual([ + ["sort", 2], + ["basket", 1], + ]); + }); + + const ids = [order, Order, Service, `${Order}#sortOrder`, String_, `${order}/Gone`]; + const judgement = fc.record({ + id: fc.constantFrom(...ids), + word: fc.constantFrom("order", "service", "sort", "line", "type"), + concern: fc.constantFrom(...JUDGED_CONCERNS), + sense: fc.constantFrom("a", "b"), + confidence: fc.constantFrom(0.2, 0.9), + in: fc.array(fc.constantFrom("Order", "sortOrder"), { maxLength: 2 }), + }); + + it("conserves: every judgement is compared once under its own word, or left out, whatever the order", () => { + fc.assert( + fc.property( + fc.array(judgement, { maxLength: 30 }).chain((all) => fc.tuple(fc.constant(all), fc.shuffledSubarray(all, { minLength: all.length }))), + ([all, shuffled]) => { + const report = concernDrift(graph, all); + expect(concernDrift(graph, shuffled)).toEqual(report); + const kept = all.filter((j) => graph.entity(j.id) !== undefined && !graph.isStub(j.id)); + expect(report.compared.entries).toBe(kept.length); + for (const w of report.words) { + const mine = kept.filter((j) => j.word === w.word); + for (const c of JUDGED_CONCERNS) expect(w.concerns[c]).toBe(mine.filter((j) => j.concern === c).length); + const opposite = w.class === "technical" ? "domain" : w.class === "domain" ? "technical" : undefined; + expect(w.against.length).toBe(mine.filter((j) => j.concern === opposite).length); + } + expect(report.technicalJudgedDomain).toEqual(report.words.filter((w) => w.class === "technical" && w.concerns.domain > 0).sort((a, b) => b.against.length - a.against.length || (a.word < b.word ? -1 : 1))); + expect(report.domainJudgedTechnical.every((w) => w.class === "domain" && w.against.length > 0)).toBe(true); + }, + ), + { numRuns: 200 }, + ); + }); +}); + +describe("businessJudged: the business list against both", () => { + const artifact = buildVocabulary(graph); + const report = concernDrift(graph, JUDGEMENTS); + const terms = (...lines: string[]) => parseBusinessTerms(lines.join("\n")).terms; + + it("gives each business word the classes the corpus says it under and the model's judgements of it", () => { + expect(businessJudged(artifact, terms("Order", "Order Service", "Customer"), report)).toEqual([ + { text: "Order", words: [{ word: "order", classes: ["domain"], concerns: { domain: 3, technical: 0, ambiguous: 0 }, drift: [] }], said: true, drift: false }, + { + text: "Order Service", + words: [ + { word: "order", classes: ["domain"], concerns: { domain: 3, technical: 0, ambiguous: 0 }, drift: [] }, + { word: "service", classes: ["technical"], concerns: { domain: 1, technical: 0, ambiguous: 0 }, drift: ["list"] }, + ], + said: true, + drift: true, + }, + { text: "Customer", words: [{ word: "customer", classes: [], concerns: { domain: 0, technical: 0, ambiguous: 0 }, drift: [] }], said: false, drift: false }, + ]); + }); + + it("drifts on a word the model judged technical somewhere, though the list calls it domain; on both, for both reasons", () => { + const [sortOrder] = businessJudged(artifact, terms("Sort Order"), report); + expect(sortOrder?.words.map((w) => [w.word, w.drift])).toEqual([["sort", ["model"]], ["order", []]]); + expect(sortOrder?.drift).toBe(true); + const both = concernDrift(graph, [judged(Service, "service", "technical", "a layer")]); + expect(businessJudged(artifact, terms("Service"), both)[0]?.words[0]?.drift).toEqual(["list", "model"]); + }); + + it("says a term is said exactly when the corpus-level drift does", () => { + const glossary = terms("Order", "Sort Order", "Order Customer", "service", "Ship"); + const said = vocabularyDrift(artifact, glossary).business.map((t) => t.scopes.length > 0); + expect(businessJudged(artifact, glossary, report).map((t) => t.said)).toEqual(said); + expect(said).toEqual([true, true, false, true, false]); + }); + + it("refuses an artifact counted under another technical list", () => { + expect(() => businessJudged(buildVocabulary(graph, { technical: ["order"] }), terms("Order"), report)).toThrow(/another technical list/u); + }); +}); diff --git a/website/content/docs/reference/cli/vocabulary.md b/website/content/docs/reference/cli/vocabulary.md index c16f6cb..ef7a8b5 100644 --- a/website/content/docs/reference/cli/vocabulary.md +++ b/website/content/docs/reference/cli/vocabulary.md @@ -12,8 +12,9 @@ Every `TNamed` entity's name is split into words (`OrderLineRepository` → `ord ``` codegraph vocabulary [model.jsonl...] [--level ] [--scope ID] [--top N] [--class ] [--weights KEY=N,…] - [--technical FILE] [--drift FILE] [--out FILE] [--svg FILE] - [--json] [--internal-only] [--declared-only] [--no-cache] + [--technical FILE] [--drift FILE] [--drift-insights [--insights FILE]] + [--out FILE] [--svg FILE] [--json] [--internal-only] [--declared-only] + [--no-cache] ``` ## Arguments @@ -33,6 +34,8 @@ codegraph vocabulary [model.jsonl...] [--level ] [--scope ID | `--weights KEY=N,…` | Override weights, e.g. `type=6,qualifier=0`. Keys: `module`, `type`, `operation`, `attribute`, `parameter`, `local`, `other`, `qualifier`. | — | | `--technical FILE` | Replace the shared technical words with this file's, one word per line; each language's own still apply. | — | | `--drift FILE` | Compare with a glossary, one business term per line: the terms no scope of `--level` says (exit 3), and the domain words no term contains. | — | +| `--drift-insights` | Compare the words' classes with what [`explain`](/docs/reference/cli/explain/) judged them: the technical words a model called domain somewhere, and the reverse, with the entities; with `--drift`, the business terms against both. | — | +| `--insights FILE` | The insights store `--drift-insights` reads. | `.insights.db` beside the first model | | `--out FILE` | Also write the [`vocabulary.json`](/docs/reference/artifacts/vocabulary-json/) artifact to this file. | — | | `--svg FILE` | Also draw one scope's cloud to this SVG file: the corpus, or the `--scope` one; `--class` and `--top` pick its words. | — | | `--json` | Print the same information as a machine-readable JSON object on stdout. | — | @@ -49,6 +52,17 @@ A type's words are those of everything that folds into it, a module's likewise, The glossary is a plain text file, one business term per line; blank lines and `#` comments are skipped. Each term is split by the same universal rules, so `Order Line` meets `order` and `line`. A term is *said* by a scope when every one of its words is, whatever class the code gives the word, so `--level module` asks for all of them in one module. Nothing is stemmed: `lines` is not `line`. The report lists the terms no scope says with their missing words, the terms it says and where, and the `domain` words the code says that no term contains. +## Against the model + +The classes are rules: a word is `technical` because a list says so, whatever the name it sits in. [`explain`](/docs/reference/cli/explain/) asks a model, for each entity it explains, what each word of its names means *there*, and whether it is about the domain, about the machinery, or ambiguous. `--drift-insights` sets the two side by side. It reads the insights store beside the first model (`--insights FILE` names another) and classes each word a model defined as the list does in that entity's language, under the `--technical` list if one is given. Then it lists: + +- the **technical words a model called domain** somewhere: `service` in `CustomerServiceTicket`, a technical word in a domain position; +- the **domain words a model called technical** somewhere: candidates for the technical list, or homonyms (`order` in `sortOrder`). + +Each word comes with how often each concern was judged, then the judgements the class contradicts, the model's surest first: its confidence, the sense it gave, and the identifiers the word is in, in which entity. The text shows five per word and says how many more there are; `--json` lists every one. The words with the most judgements against come first, and `--top N` cuts each list (40 by default). With `--drift FILE`, the business terms follow: each of their words with the classes the corpus says it under and the model's judgements, and what drifts: `list` when the list does not call it domain, `model` when a model judged it technical somewhere. + +The judgements are inferences, so they never set the exit code; `--drift` still exits `3` on a term the corpus never says. Records the comparison cannot use are counted on stderr: an entity the model no longer has (explained against another version of it) or the view drops, and records explained under prompt 1, before words were asked. `--scope`, `--level`, `--class` and `--svg` do not apply. `codegraph insights --word WORD` shows every judgement of one word. + ## Cloud `--svg FILE` draws the cloud of the one scope the report shows, the corpus or the one `--scope` names, from the words `--class` and `--top` pick. A word's size follows the square root of its weight, so its area reads as the weight, and its colour says its class and nothing else: `domain` in ink, `technical` muted, `keyword` and `short` faint, with a dark palette for a dark page. Words are placed heaviest first, each at the first free point of a spiral out from the centre. The legend states the scope, the counts, the weights, the view and the technical list. The font is monospace, so the bytes are the same on every run and any machine. @@ -63,7 +77,7 @@ One word across the whole city is the city metric `vocabulary:`: `codegrap `0` ok · `1` internal error (a bug) · `2` usage error · `3` findings. -`3` when the load was not clean, or with `--drift` when a business term is said by no scope of the level. An unknown `--scope` id, an unknown `--weights` key or a `--technical` line that is not one word is a usage error, and so is `--svg` with `--level module|type` or `--drift`: a cloud is one scope's. +`3` when the load was not clean, or with `--drift` when a business term is said by no scope of the level. A model's judgement never makes a finding. An unknown `--scope` id, an unknown `--weights` key or a `--technical` line that is not one word is a usage error, and so is `--svg` with `--level module|type` or `--drift`: a cloud is one scope's. With `--drift-insights`, a missing store, or one an older codegraph wrote that no `explain` has migrated yet, is a usage error, and so is `--insights` without it. ## Example @@ -115,6 +129,40 @@ code only: 4 of 71 domain words no business term contains (--top 4), by weight d 1 business term no module says — exiting 3 (findings). ``` +Against a store whose records judged `service` domain in `OrderService`, and `trail` technical in `LedgerAdapter` and `Order`: + +```console +$ printf 'Order Line\nService Level\nLedger\n' > glossary.txt +$ codegraph vocabulary model.jsonl --drift-insights --drift glossary.txt +… +insights: model.insights.db, judged by cheap/model (operations), strong/model (types, modules) +… +glossary: glossary.txt, 3 business terms + +compared: 10 word entries from 4 explained entities + +technical by the list, judged domain: 1 of 2 technical words, the most judgements against first + service domain 1 + 0.60 "after-sales care" OrderService in java:com.acme.order/OrderService + +domain by the list, judged technical: 2 of 4 domain words, the most judgements against first + trail technical 2 + 0.70 "an audit log" trail in java:com.acme.order.adapter/LedgerAdapter + 0.65 "a change log" trail in java:com.acme.order/Order + order domain 2 · technical 1 + 0.40 "the argument's type" order in java:com.acme.order/OrderService.bill(com.acme.order.Order) + +business terms against both: 2 of 3 drift, a word the list or a model does not call domain + + TERM WORD LIST MODEL DRIFT + Order Line order domain domain 2 · technical 1 model + line domain not judged + Service Level service technical domain 1 list + level not said not judged + Ledger ledger domain domain 2 +1 business term the corpus never says — exiting 3 (findings). +``` + ## See also -[vocabulary.json](/docs/reference/artifacts/vocabulary-json/) · [`profiles`](/docs/reference/cli/profiles/) · [CLI conventions](/docs/reference/cli/) +[vocabulary.json](/docs/reference/artifacts/vocabulary-json/) · [`profiles`](/docs/reference/cli/profiles/) · [`explain`](/docs/reference/cli/explain/) · [CLI conventions](/docs/reference/cli/) diff --git a/website/content/docs/reference/exit-codes.md b/website/content/docs/reference/exit-codes.md index 7ca9be3..7a46a66 100644 --- a/website/content/docs/reference/exit-codes.md +++ b/website/content/docs/reference/exit-codes.md @@ -49,7 +49,7 @@ Valid options for 'codegraph city': --height METRIC, --height-scale