Move the perf-scaling and adding-a-word paragraphs out of root AGENTS.md

.github/workflows/python-package.yml

@@ -31,7 +31,9 @@ jobs:
       # stops reporting a line once it is recorded: a line tracer
       # (sys.settrace, its core below 3.14) is called on every line
       # every time, and cost CI's 3.12 job most of its ~12 minutes on
-      # the property grids in tests/v2/test_properties.py. The package
+      # the property grids in tests/v2/test_properties.py (measured
+      # 2026-09-29: the three #397 grids took 139s under the settrace core
+      # on 3.12, 24s under sys.monitoring, 17s with no coverage). The package
       # has no version-conditional code, so one interpreter's report
       # stands for all of them.
       COVERAGE_ON: ${{ matrix.python-version == '3.14' }}

docs/design/decisions/vocabulary-exclusions.md

@@ -62,7 +62,7 @@ Excluded (TITLES):
 - The per-word refusals are data since #750 (2026-10-19): each word refused a title reading — the Indic ঠাকুর/ठाकुर, कुमारी/কুমারী, बेगम/বেগম, शेख/শেখ, आचार्य and राजा/रानी (#343, #344), and #606's ông/bà, thiru, puan, pan, rodina, gróf, cik, paron, tikin, Latin janab, marshal and justice, #662's malam (Derek's decline) and bay, #660's shehu, and #789's wm (2026-10-22) — is an `excluded` entry in its language's file with its reason, and with its counts as name readings where they were taken. `uv run python tools/vocab/query.py --excluded title` lists them. A contributor who adds one back meets the reason as a build error.
 - shehu's counts do not settle a graf-class weighing, and must not be read as if they did: its one leading-position figure, the Wikidata given-name count on its entry, is a lower bound (Shagari's own entry has no given-name statement), and its other counts are family-name bearers, a position C-i does not ask about. The evidence that Shehu leads a name is not a count. It is Shagari himself, and CLDR's Hausa sample data, whose six renderings carrying the word all file it as the given name (`grep '"given": "Shehu"' tools/differential/cldr_fields.jsonl`). It arrived with an early bulk import (1a51555f) and was never put to C-i before #660. Whoever re-argues it counts Hausa given-name use first; the accepted cost is the honorific use, `Shehu Abubakar` reading given Shehu as any untitled name does, and the reading belongs in an opt-in pack if anyone asks for it. malam is the opposite case: CLDR's Hausa data files Malam as the title (`grep '"title": "Malam"' tools/differential/cldr_fields.jsonl`), so its refusal is a decline on the counts, not a shehu-style removal.
 - The class-level half of the Indic refusals stays here: the divergence on शेख/শেখ is deliberate and is the sri/shri case run backwards — the Latin sheikh cluster SHIPS as given-name titles for the Arabic addressing form, while the Indic spellings name the family. shehu's removal moved three CLDR corpus names (`Shehu Muhammed`, `Muhammed, Shehu`, `Modibbo, Shehu Bello`), all three now agreeing with CLDR on every role (moves: `uv run python tools/differential/snapshot.py --since v2.4.1`).
-- The trailing-position rule that must NOT be adopted, and 2026-09-08 says what the line is: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Dean is deliberately absent" gotcha; this is the trailing half, and it shadows the family name rather than the given (#316). What #316 settled is that the prohibition is on BARE words: "Mary Jane King" still reads family King, while "John Smith Prof." reads title Prof. **The second half of that argument is RETRACTED 2026-09-09**, in review of the docs commit: "a period-marked trailing word being one nobody writes as a surname" claimed the period keeps the collision out of the trailing slot, and it does not. Measured on the branch tree — "Mary Jane King." reads title "King.", given Mary, family Jane, the family name shadowed exactly as the bare reading would have shadowed it, and "John Smith Judge." reads title "Judge." The reach is the vocabulary rather than a handful: of the 746 titles in no suffix set, 627 read as a trailing title once a period is written behind them, and every one of the 119 that do not is kept out by the abbreviation SHAPE — a digit, a hyphen, an apostrophe, or a script whose letters carry combining marks (rules/H2 records that half) — no plain ASCII-letter title missing at all. So king, judge, bishop, baron and sheriff ARE reachable in the trailing slot; what the prohibition above buys is the bare spelling and nothing more. The period is a WRITING convention the rule can read, not evidence about the word, and reading it is an ACCEPTED COST under the premise that the input is a name (rules.md H Background): someone who ends a name with a period-marked word has written an abbreviation. rules/H5 carries it as an Accepted clause with "Mary Jane King." as the executable example. (Since 2026-10-26 `king` is a title only inside a title phrase, decisions/H9, #752 phase 2j: "Mary Jane King." reads family "King." and the surnames this paragraph names are pope, judge, bishop, baron and sheriff, rules/H5's example "Mary Jane Pope.".) Recompute the reach with `L = Parser().lexicon; ws = [w for w in L.titles - L.suffix_acronyms - L.suffix_words if " " not in w]; len([w for w in ws if parse("Mary Jane %s." % w).title])` against `len(ws)` — 627 of 746 on 2026-09-09 — the digits move as the vocabulary grows and neither argument does. See #H5. The bare prohibition has one exception since #835 (decisions/H8): a listed salutation ending the given part after a family comma is the title, the comma having marked the surname's place and the salutations borne as names being excluded.
+- The trailing-position rule that must NOT be adopted, and 2026-09-08 says what the line is: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Titles permanently shadow first names" gotcha, whose example is Dean; this is the trailing half, and it shadows the family name rather than the given (#316). What #316 settled is that the prohibition is on BARE words: "Mary Jane King" still reads family King, while "John Smith Prof." reads title Prof. **The second half of that argument is RETRACTED 2026-09-09**, in review of the docs commit: "a period-marked trailing word being one nobody writes as a surname" claimed the period keeps the collision out of the trailing slot, and it does not. Measured on the branch tree — "Mary Jane King." reads title "King.", given Mary, family Jane, the family name shadowed exactly as the bare reading would have shadowed it, and "John Smith Judge." reads title "Judge." The reach is the vocabulary rather than a handful: of the 746 titles in no suffix set, 627 read as a trailing title once a period is written behind them, and every one of the 119 that do not is kept out by the abbreviation SHAPE — a digit, a hyphen, an apostrophe, or a script whose letters carry combining marks (rules/H2 records that half) — no plain ASCII-letter title missing at all. So king, judge, bishop, baron and sheriff ARE reachable in the trailing slot; what the prohibition above buys is the bare spelling and nothing more. The period is a WRITING convention the rule can read, not evidence about the word, and reading it is an ACCEPTED COST under the premise that the input is a name (rules.md H Background): someone who ends a name with a period-marked word has written an abbreviation. rules/H5 carries it as an Accepted clause with "Mary Jane King." as the executable example. (Since 2026-10-26 `king` is a title only inside a title phrase, decisions/H9, #752 phase 2j: "Mary Jane King." reads family "King." and the surnames this paragraph names are pope, judge, bishop, baron and sheriff, rules/H5's example "Mary Jane Pope.".) Recompute the reach with `L = Parser().lexicon; ws = [w for w in L.titles - L.suffix_acronyms - L.suffix_words if " " not in w]; len([w for w in ws if parse("Mary Jane %s." % w).title])` against `len(ws)` — 627 of 746 on 2026-09-09 — the digits move as the vocabulary grows and neither argument does. See #H5. The bare prohibition has one exception since #835 (decisions/H8): a listed salutation ending the given part after a family comma is the title, the comma having marked the surname's place and the salutations borne as names being excluded.
 
 Excluded (Policy.script_orders defaults): Script.KATAKANA is deliberately absent — a pure-katakana token is predominantly a transcribed foreign name kept in its source order, so nothing defaults on it (rule W4's boundary). Noted 2026-08-15: of the three Script-keyed axes, this is the one with no force-a-decision guard (mechanisms.md#FORCE-A-DECISION-TABLE), so a new Script member silently gets no order.
 

nameparser/_pipeline/AGENTS.md

@@ -2,6 +2,10 @@
 
 Loaded when a session reads or edits a pipeline stage. The root AGENTS.md carries the 2.0 conventions every stage follows (one stage per module, the `Consumes:`/`Produces:`/`Reads:` header, the frame budget, checkable stage-attribution claims); this file carries what only work inside the stages needs. The contract behind the ambiguity rules below is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE, and rules/A1 is its observable side.
 
+## A new inner loop needs a scaling guard
+
+A loop that grows with the input -- per token, per member, per pair -- needs a row in `tests/v2/test_benchmark.py`, whose module docstring says which table; the absolute-time tests there use constant-size input and are blind to a complexity regression.
+
 ## Ambiguity reports
 
 ### Emit at the decision site

nameparser/config/capitalization.py

@@ -3,7 +3,8 @@
 # `cased` property of the word's reading, one per word, and
 # tools/vocab/build.py refuses one that does not spell its word's letters
 # and digits (#459) before Lexicon ever sees it. Still a plain dict, the
-# one vocabulary constant not frozen (AGENTS.md, "Configuration layer").
+# one vocabulary constant not frozen (AGENTS.md's vocabulary and
+# configuration-layer section, "Neither dict was frozen").
 from nameparser.config import _compiled
 
 CAPITALIZATION_EXCEPTIONS = _compiled.CAPITALIZATION_EXCEPTIONS

nameparser/vocab/AGENTS.md

@@ -0,0 +1,31 @@
+# Adding a word to nameparser/vocab/
+
+Loaded when a session reads a vocabulary TOML file. The steps for adding a word are in `nameparser/vocab/README.md`, the reference for anyone editing these files; this file holds the traps those steps have run into, filed under the README step each belongs to. Each names the PR that found it.
+
+## Step 1: its other readings
+
+First check the word's OTHER readings: `uv run python tools/vocab/query.py --word <word>` lists every reading and exclusion of it in every file, which is where the overlaps between categories now show. Real overlaps exist: `st`/`freiherr` ∈ `PARTICLES` ∩ `TITLES`, and `do`/`mc`/`vd` ∈ `PARTICLES` ∩ `SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PARTICLES` ∩ `BOUND_GIVEN_NAMES` (position-dependent: leading token → given-name join, mid-name → family-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; `abd`, kept out of `BOUND_GIVEN_NAMES` until decisions/P5 found the credential collision cost nothing at either end).
+
+## Step 2: real names, not memory
+
+Then check the word against real names, not memory: `uv run python tools/name_usage.py <word>` counts its bearers as a given name and as a family name (US Census surnames, SSA given names, Wikidata with countries of citizenship), and gives its given-name frequency band per country from Jörg Michael's `nam_dict.txt` (54 countries and an "other countries" column, GFDL, downloaded into the cache and never committed) — the one source that sees a given name outside the US and Wikidata's lower bound, such as Korean `In` (band A in Korea) or Vietnamese `Bay` (6 in Vietnam). A band is a frequency class, not a count, and the dictionary is a selection, so a miss there (`Shehu`) proves nothing; and because a truncated source reads as a run of false zeros, a pinned download is checked against its sha256 on every load from the cache, never against a floor on what it holds — #775's first fix refused fewer than 40,000 names and passed a `nam_dict.txt` missing only its last line (`Zyta`). Pass a name as it is written (`McDonald`): Wikidata labels match exactly, and no rule rebuilds an internal capital from `mcdonald`. A word in a non-Latin script is counted by its romanization too, since the Census, SSA and nam_dict hold only Latin names, and that count weighs only where its bearers write the script: #835 vetted its salutations by their own spellings and its review found `बाबू` flagged while `babu` is a US surname (1,496) and an Indian given-name element (Ram Babu), where the romanization of `মি`, `Mi`, counts Korean and Chinese bearers who never write Bengali. It reports and does not decide — read the counts against the position the set's claim acts on. For a suffix it also has the other side: `--fetch-nppes` counts the US healthcare providers who write the word as a credential (NPPES, the 4.8 million credentialed providers in the pinned August 2026 file, so blind to law, accounting and honours), and `--rank N --unlisted` lists the commonest credentials the vocabulary misses, with where each lands in `John Smith, X` -- the list #797 came from. Where a local corpus is fetched (`tools/differential/local_corpora.py`, mechanisms.md#NAME-DATA-JOBS), a row says how many of its records (for ClinicalTrials.gov, officials) write the word, and how many after a comma: ClinicalTrials.gov's study officials show where a credential actually stands in a name as typed.
+
+## Step 2, for a glued word: does any name, or any longer word, end in it?
+
+A word that would peel GLUED (`GLUED_HONORIFICS`) asks a different question, whether any name ENDS in it, and for a CJK word the tool answers that too: an ends-with row from a cached index of Wikidata's CJK name labels (built on first use; 33 minutes on 2026-10-17), plus the names the word ends across the family/given join (阿部 + 長 for 部長). Read the tool's hits by item before counting them, because labels include transcriptions and mislabels: #707's one hit for 二世 was 厄德二世, the given name Odo with a regnal numeral in its Chinese label, borne by 109 Western people named Odo. Then ask a third question the tool cannot: does a common longer word end in it? The peel matches the end of a token, spaced or glued, so a listed tail splits every unlisted compound that ends in it — #707 vetted 議員, 社長 and 医学博士 against names and its review found 衆議院議員, 副社長 and 獣医学博士 cut at the tail (`田中 副社長` → given 副); no corpus held a compound office word until #707's rules examples added two, so the snapshot was silent about the class. List the common compounds whole or record the split as a cost.
+
+## Step 3: the predicates that read a member's spelling
+
+Check the predicates that test a member's SPELLING rather than its membership: they encode what the set's members happened to look like when they were written. `_pieces.starts_a_credential_run` implemented the credential run's "Latin" (rules/S8 now, part of S2 until #682 split it) as `isascii()`, exact until #661 added `mlađi`, which would have read `mlađi Jones` differently from `stariji Jones`; the design-docs review caught it, after a first draft had credited the test to the wrong clause of S2 and proposed deleting it on a mutation that moved nothing because nothing pinned it. (#679 then dropped the script test on purpose, after measuring what it protected.) Before adding the first member of a new script, or the first accented one, grep the set's consumers for `isascii`, `isalpha` and script tests, and read the rule each one implements. The same goes for the first member holding PUNCTUATION: #700 added `d'` and `az-`, the parse read them right and every snapshot move was an intended one, and `capitalized()` still wrote `D' Ivoi`, because case repair splits a token at its apostrophe or hyphen and asks the vocabulary of the stem. The snapshot records both case views since #826, but only for the names the corpora hold, so parse such a member's names through `capitalized()` and `capitalized(force=True)` in each case before calling it done.
+
+## Step 4: record the vetting on the word
+
+Record what the vetting found ON the word, not in a comment or a decisions sentence: `note`, `issue`, a `given`/`family` reading with dated counts, and for a word you decided to keep out an `excluded` entry with its reason, so the next contributor who adds it meets the reason as a build error (decisions/vocabulary-model). A name reading is not only provenance on a word that has a credential, particle or title reading: it IS the clash, and derives the marker (an ambiguous acronym, a particle that may be given, since 2026-10-25, a generation or agnomen that may be a name, rules/S11, since phase 2h the same day, so given-name counts recorded on `junior` would gate it, and a title that may be a given name, rules/H9, since phase 2i), so counts kept as evidence AGAINST marking such a word go in the clashing reading's `note` instead, as `ld`'s do.
+
+## Step 6: the user docs' doctests
+
+The user docs use real words as examples of an UNLISTED one, and listing the word breaks the example: #797 listed `dphil`, and docs/customize.rst's capitalization example (`parse("jane smith dphil").capitalized()` giving `Dphil`) failed in all five CI build jobs while `uv run pytest` passed, because `sphinx-build -b doctest docs` is a separate step. Run it after adding a word, and grep `docs/*.rst` for the word; the example now uses `dclinpsy`, a credential left unlisted on purpose, so list it only after moving the example.
+
+## When a word's reading moves, or a new derivation reads old readings
+
+A word whose reading moves may be the worked example of the class it is leaving: `king` was the rules' and tests' title-that-is-also-a-surname across 35 files when #901 gave it a given reading, and five tests went vacuous before the review found them. Grep rules, decisions Standing lines, the RFCs, AGENTS.md, docs/*.rst, code comments and tests for the word before measuring, and replace it with a stand-in checked to read exactly as the word read on master in every example (`pope` there). A new derivation reads readings other work recorded for another purpose, so list what it would mark before shipping it: phase 2i's first build found thirteen salutations whose given readings #835 had kept as H8's evidence (`mr` among them, so `Mr Smith` would have read given 'Mr'), and moved their counts into notes.