diff --git a/docs/design/decisions/C15-comma-with-nothing-before-it.md b/docs/design/decisions/C15-comma-with-nothing-before-it.md index 2f48840d8..3c8fbe4ef 100644 --- a/docs/design/decisions/C15-comma-with-nothing-before-it.md +++ b/docs/design/decisions/C15-comma-with-nothing-before-it.md @@ -4,12 +4,13 @@ - A comma with nothing before it is a family comma naming no surname, the part after it read as the given part after a family comma reads, held by an invariant over two inputs rather than by a sweep shape: `, X` reads as `Smith, X` reads, less the surname, its fields and ambiguity kinds alike; a particle P6 attaches is then the family alone (`, Jane van` → family 'van') — 2026-10-27 #892. - H1 is the one exception: a title and one name word, or a bound pair standing for it, read as the family (`, Dr. John` → family 'John', `, Dr. abdul Rahman` → family 'abdul Rahman'), the comma naming no surname to decide the field by, as a comma with nothing but titles after it names no given part — 2026-10-27 (Derek). +- No code is written for the empty head alone: P6 attaches behind it as behind any family comma, ahead of H1, so a particle ending the name leaves H1 two name words and the exception does not apply (`, Dr. Jones vd` → given 'Jones', family 'vd') — 2026-10-27 #892 (Derek). -Open: whether the code written for the empty head alone (#890's floor for `, Dr. fils`, the P6 attachment site in post_rules) can go, the question #892's fourth point asks; the invariant answers it by mutation, one site at a time. +Open: none in this file. ## Log -- 2026-10-27 #892 — A COMMA WITH NOTHING BEFORE IT IS A FAMILY COMMA NAMING NO SURNAME. The shape had no rule: clauses in rules/H8, rules/O6 and rules/M2 and review findings in decisions/C1, S10, A1 and H5 each decided it locally, and nameparser/AGENTS.md made every comma change sweep it. Real names almost never take it: 10 of about 6 million local-corpus names (FEC 4, tse 3, PSC 2, tw-donors 1, the last opening with a fullwidth comma, `，邱詩涵`, which reads as `, 邱詩涵` does), the issue's count of 2026-10-25 missing the fullwidth one; recounted 2026-10-27 by scanning each local corpus's names for one opening, after spaces, with `,` or `，`. The alternative, dropping the comma so `, John Smith` reads as `John Smith`, was declined: the shape's one source is an empty surname field in listing data (#613's review, `, PhD` and `, Jr.`), and reading it as a whole name contradicts that source. +- 2026-10-27 #892 (its `vd` and `mc` bullet superseded by 2026-10-27 #892 (Derek), below) — A COMMA WITH NOTHING BEFORE IT IS A FAMILY COMMA NAMING NO SURNAME. The shape had no rule: clauses in rules/H8, rules/O6 and rules/M2 and review findings in decisions/C1, S10, A1 and H5 each decided it locally, and nameparser/AGENTS.md made every comma change sweep it. Real names almost never take it: 10 of about 6 million local-corpus names (FEC 4, tse 3, PSC 2, tw-donors 1, the last opening with a fullwidth comma, `，邱詩涵`, which reads as `, 邱詩涵` does), the issue's count of 2026-10-25 missing the fullwidth one; recounted 2026-10-27 by scanning each local corpus's names for one opening, after spaces, with `,` or `，`. The alternative, dropping the comma so `, John Smith` reads as `John Smith`, was declined: the shape's one source is an empty surname field in listing data (#613's review, `, PhD` and `, Jr.`), and reading it as a whole name contradicts that source. - Measured on master (8a65ee18) before any change, over the grid `tests/v2/test_properties.py::test_an_empty_head_reads_as_its_family_comma_twin` builds (10 title runs × 32 bodies × 11 trailing words, 3,519 given parts under each of three orders): every given part reads as its twin but 1,075 per order, all of them H1's, so the rule landed with no code change. A first comparison's eight differing given parts holding `MA`, among them `, MA`, `, (Bob) MA`, `, J. MA` and `, JOHN SMITH MA`, were the twin's doing, not the empty head's: a surname written `Smith` brings a case contrast the given part lacks, so `Smith, MA` reads suffix 'MA' by its capitals (rules/S6) while `, MA` and `SMITH, MA` both read given 'MA'. The test writes the twin's surname in the given part's case for that reason, and rules/C15's Accepted bullet records the reading. - The particle-and-suffix words `vd` and `mc` ending the name behind H1's family reading (`, Dr. Jones vd`) are the exception's knock-on, not a second one: the twin attaches the word to its surname (rules/P6), the empty head has no family to attach it to once H1 has made the lone word the family, and it reads as the suffix. Behind a given-name title H1 reads no family and P6 attaches as in the twin (`, Sir Jones vd` → family 'vd'). - The exception is stated by its result, not by the comma-less name: `, Jane Mrs` reads family 'Jane' where `Jane Mrs` reads family 'Mrs' (a bare salutation is a title only after a family comma, rules/H8), and `, Dr. abdul Rahman` reads family 'abdul Rahman' where `Dr. abdul Rahman` reads given 'abdul', family 'Rahman'. A first draft compared against the comma-less name and the design-docs review found it false on 493 of the 1,075 H1 rows per order. @@ -17,3 +18,5 @@ Open: whether the code written for the empty head alone (#890's floor for `, Dr. - Recompute: `uv run pytest -n0 "tests/v2/test_properties.py::test_an_empty_head_reads_as_its_family_comma_twin"`; its docstring records the negative control. - 2026-10-27 (Derek) — H1 STAYS THE ONE EXCEPTION, stated as the comma deciding nothing. "Keep H1 as the one exception, stated as a comma with nothing before it not deciding the field." `, Dr. John` keeps family 'John', where its twin `Smith, Dr. John` reads given 'John': rules/H1 already says a comma with nothing but titles after it names no given part and so does not decide the field, and a comma with nothing before it names no family, the mirror case. Following the invariant instead would have read given 'John' for every such row, 1,075 per order on the grid; rules/H8's Accepted bullet records the trailing spellings' family reading (`, Jane Mrs`, and the period-marked one since H5, #316). + +- 2026-10-27 #892 (Derek) — THE EMPTY HEAD'S OWN CODE IS GONE, the question #892's fourth point asked. SUPERSEDES: the `vd` and `mc` bullet of 2026-10-27 #892, which read `, Dr. Jones vd` as suffix 'vd'. A search of the pipeline for code that reads an empty part before the comma found one site: P6's second attachment site, in post_rules after H1 and M4, with the guard in assign's comma site that kept the attachment from acting there (decisions/P6, #613). Switching both off moved 112 given parts per order on the invariant grid, every one a particle-and-suffix word `vd` or `mc` ending the name behind a title that addresses by surname (`, Dr. Jones vd`, `, Mr Jones mc`, `, Jones vd Mrs`), each from family 'Jones', suffix 'vd' to given 'Jones', family 'vd' with a `suffix-or-name` report, and moved them onto the twin: no row failed the invariant, and H1's rows fell from 1,075 to 963 per order. Off the grid the PR review found two more classes, each onto its twin too: the same tail behind a third comma part (`, Dr. Jones vd, PhD` → family 'vd', suffix 'PhD', where it read suffix 'vd, PhD'), and a never-given particle opening the part with no name word behind it (`, de PhD vd` → given 'de', suffix 'PhD vd', as `Smith, de PhD vd` and `, de vd` read, where the removed site, which lacked the comma site's P1 guard, read family 'vd', suffix 'PhD', reported). An ambiguous particle there already read so (`, Dr. Jones van` → family 'van'), and the site had kept the comma-less name's reading (`Dr. Jones vd`). Removed, with "ok, remove it". #890's floor for `, Dr. fils` is not the empty head's: it is rules/C4's count of name words before the comma, which a one-word head meets too (`Sir Alexandre, fils`). None of the 10 real names with the shape holds a trailing particle. diff --git a/docs/design/decisions/P6-trailing-orphan-particle.md b/docs/design/decisions/P6-trailing-orphan-particle.md index c71d38d02..4a6047ad5 100644 --- a/docs/design/decisions/P6-trailing-orphan-particle.md +++ b/docs/design/decisions/P6-trailing-orphan-particle.md @@ -12,7 +12,8 @@ Examples read under the default order unless a line names another. - The attachment reports the fork it decides, in the kind naming the reading it overrode: `suffix-or-name` where the run had been read as a post-nominal (`Berg, Jan vd`), `particle-or-given` where it holds an ambiguous particle (`Nguyen, Thi Van`, `Beethoven, Ludwig van`), and nothing for a never-given particle (`Jong, Piet de`) or a run read as name words with no ambiguous member (`Berg, Jan de vd`); where a caller's word reaches both arms the suffix arm wins — 2026-08-29 #405. - Without a comma the attachment fires under FAMILY_FIRST alone, where the run ENDS the name and stands in a MIDDLE, with no vocabulary test and no re-layout, the family needing a base of its own (`Beethoven Ludwig van` family-first → family 'van Beethoven'; `van Berg Jan de` family-first → family 'van'); under the default order the positional reading stands (`Jong Anke de` → family 'de'), and the given position under a family-first order is no P1 site (`Ménil de` family-first → given 'de') — 2026-08-30 #467. - A member of the ambiguous credential class written in capitals in a mixed-case name is the credential and the attachment stands down (`Doe, John DO` → suffix 'DO'), where `Doe, John Do` and the one-case `SMITH, JOHN DO` still attach; the condition reads the ambiguous tag, not the suffix role, so `vd` and `mc` stay in the run — 2026-09-18 #531. -- The comma site is decided in assign, right after the walk over the given part, asking P1's condition itself so `Smith, de Mesnil van` stays the fold's (family 'Smith de Mesnil van'); behind an EMPTY part before the comma the attachment stays in post_rules after H1 and M4 (`, Mr. Jones vd` → title 'Mr.', family 'Jones', suffix 'vd'), and the no-comma site stays there too — 2026-10-06 #613. +- The comma site is decided in assign, right after the walk over the given part, asking P1's condition itself so `Smith, de Mesnil van` stays the fold's (family 'Smith de Mesnil van'); the no-comma site stays in post_rules — 2026-10-06 #613. +- The comma site is assign's behind an EMPTY part before the comma too, ahead of H1, which then finds a family and stands down (`, Mr. Jones vd` → title 'Mr.', given 'Jones', family 'vd', as `Smith, Mr. Jones vd` reads given 'Jones', family 'vd Smith') — 2026-10-27 #892 (Derek). - The elided particles and the hyphen-attached Arabic articles, a closed list in each apostrophe spelling, are never-given particle vocabulary, and the rule is unchanged: `Ivoi, Paul d'` → family "d' Ivoi", `Rashid, Harun al-` → family 'al- Rashid'; a glued form is one word; and case repair asks the whole token the particle question first — 2026-10-17 #700. - The comma site and the no-comma family-first site are held to each other by tests/v2/test_order_correspondence.py, which a change breaking either side should expect to fail first — 2026-10-20 #761. @@ -79,7 +80,7 @@ Decided 2026-08-19 (was Open: [#380](https://github.com/derek73/python-nameparse - 2026-09-18 #531 (cross-amendment) — THE PRECEDENCE OVER S2 GAINS ONE EXCEPTION, AND THE RUN WALK GAINS ONE STAND-DOWN CONDITION. An all-caps member of the AMBIGUOUS credential class, in a name written in more than one case, reads as the credential and this attachment stands down: `Doe, John DO` is suffix `DO` where `Doe, John Do` and `SMITH, JOHN DO` still attach. The argument, the accepted cost (`SMITH, JOHN DO` paired with `NASCIMENTO, EDSON ARANTES DO`) and the two carve-outs are in decisions/S5's 2026-09-18 `do` bullet; what belongs here is the CONDITION, which is narrowed to the ambiguous tag rather than to the suffix role: a suffix-roled single-token piece carrying that tag is not part of the run, while `vd` and `mc` — unambiguous suffix vocabulary, also particles, also suffix-roled — stay inside it and are unaffected. Keying on the role instead would have stood the attachment down for them too, and keying on nothing would have let it silently override the slot's reading (`Doe, John DO` read family `DO Doe` with the condition absent, verified). The 2026-08-18 entry's "assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma" is unchanged for `vd` and `mc` and now has this one exception for `do`. - 2026-10-02 (Derek), #573 — A word of both the particle and the unambiguous suffix vocabulary reads alike in every case at three sites, P6's third exception among them. The record moved to decisions/P9 with that exception (2026-10-19). -- 2026-10-06 #613 — THE COMMA SITE IS DECIDED IN THE PASS THAT READS THE GIVEN PART. #611 found P6 showing the decisive sign smaller: assign held back from #602's run exactly the particle tail post_rules would attach a stage later, by asking `particle_tail` over roles that meant "not yet read" there and "suffix" in post_rules. The attachment and its third exception (`_inside_a_credential_run`) moved into assign, right after the given-part walk, and `particle_tail` is asked ONCE, at the moment the walk reaches #602's run: the pieces in front of the run hold the walk's roles and the run's words none, the state in which its answer equals the one over the finished roles (argued piece by piece in its docstring). post_rules ran P1's leading fold before P6, so assign asks P1's condition itself (`is_lone_never_given_particle` on the given piece), which is what keeps `Smith, de Mesnil van` the fold's. H1 and M4, the other rules ahead of the old site, are gated on there being no family, which a family comma always has. The no-comma site (#467) stays in post_rules, since it runs over the order P1's family-first redistribution leaves. NOTHING MOVES, measured 2026-10-06 against the commit before it (PR #616): a 103,786-text P6 grid (ten family-comma heads behind which zero to three of seventeen particle, credential and name words, with and without a trailing `, Jr.`, under four policies, comparing every token's role and tag, every report in order and `initials()`) and the #613 harness's 80,423 texts are identical, the gate byte-identical at all five baselines and the suite unchanged. ONE SHAPE DID MOVE, and the PR review found it on a grid with an EMPTY part before the comma, a head no measurement above held: there no family is named, H1 and M4 are not gated off, and they had made the given word the family before the old site attached -- `, Mr. Jones vd` read title 'Mr.', family 'Jones', suffix 'vd', and assign attaching first read given 'Jones', family 'vd' (177 texts on that grid). Reading P6 strictly instead (no family named, no attachment) moved 1,302 texts against master, `, Jan van` among them, so the attachment behind an empty family part stays in post_rules, after H1 and M4, through the same helper; that grid now differs from b547c247 only in C1's decided classes. Frames drop on every family-comma name, post_rules' block having paid a second `particle_tail` and a generator-driven search for the given word on every one, tail or not, where assign now attaches only behind a non-empty tail: `Smith, John` 169 → 160, `Smith, John, Jr.` 213 → 204, `Doe, Jane nee Smith` 302 → 293, a no-comma name unchanged. +- 2026-10-06 #613 (its site behind an empty family part superseded by 2026-10-27 #892, below) — THE COMMA SITE IS DECIDED IN THE PASS THAT READS THE GIVEN PART. #611 found P6 showing the decisive sign smaller: assign held back from #602's run exactly the particle tail post_rules would attach a stage later, by asking `particle_tail` over roles that meant "not yet read" there and "suffix" in post_rules. The attachment and its third exception (`_inside_a_credential_run`) moved into assign, right after the given-part walk, and `particle_tail` is asked ONCE, at the moment the walk reaches #602's run: the pieces in front of the run hold the walk's roles and the run's words none, the state in which its answer equals the one over the finished roles (argued piece by piece in its docstring). post_rules ran P1's leading fold before P6, so assign asks P1's condition itself (`is_lone_never_given_particle` on the given piece), which is what keeps `Smith, de Mesnil van` the fold's. H1 and M4, the other rules ahead of the old site, are gated on there being no family, which a family comma always has. The no-comma site (#467) stays in post_rules, since it runs over the order P1's family-first redistribution leaves. NOTHING MOVES, measured 2026-10-06 against the commit before it (PR #616): a 103,786-text P6 grid (ten family-comma heads behind which zero to three of seventeen particle, credential and name words, with and without a trailing `, Jr.`, under four policies, comparing every token's role and tag, every report in order and `initials()`) and the #613 harness's 80,423 texts are identical, the gate byte-identical at all five baselines and the suite unchanged. ONE SHAPE DID MOVE, and the PR review found it on a grid with an EMPTY part before the comma, a head no measurement above held: there no family is named, H1 and M4 are not gated off, and they had made the given word the family before the old site attached -- `, Mr. Jones vd` read title 'Mr.', family 'Jones', suffix 'vd', and assign attaching first read given 'Jones', family 'vd' (177 texts on that grid). Reading P6 strictly instead (no family named, no attachment) moved 1,302 texts against master, `, Jan van` among them, so the attachment behind an empty family part stays in post_rules, after H1 and M4, through the same helper; that grid now differs from b547c247 only in C1's decided classes. Frames drop on every family-comma name, post_rules' block having paid a second `particle_tail` and a generator-driven search for the given word on every one, tail or not, where assign now attaches only behind a non-empty tail: `Smith, John` 169 → 160, `Smith, John, Jr.` 213 → 204, `Doe, Jane nee Smith` 302 → 293, a no-comma name unchanged. - 2026-10-17 #700 — THE ELIDED PARTICLE AND THE HYPHEN-ATTACHED ARABIC ARTICLE ARE PARTICLE VOCABULARY, NEVER-GIVEN, AND THE RULE IS UNCHANGED. Library catalogs file `Paul d'Ivoi` as `Ivoi, Paul d'` and `Ḥusnī az-Zaʿīm` as `Zaʿīm, Ḥusnī az-`, the particle split off with its mark still on it; both read as a middle name (1.4.0 and 2.4 alike) because neither written form was in `PARTICLES`. P6 asks only the particle tag, so the fix is the vocabulary alone: no code moved. Three decisions, each taken by Derek on the #700 session's options. ELIDED SET: `d'`, `l'`, `dell'`, each also in its U+2019 spelling, and `de’` and `’t` beside the existing `de'` and `'t` (`’t` added in the PR review, a gap the test review found), since nothing folds the typographic apostrophe to the ASCII one; particles.py asserts at import that every elided entry has both spellings (a vocabulary build check since 2026-10-18, decisions/vocabulary-model). ARTICLES: a CLOSED list — `al-`, `ul-`, the article assimilated to each of the fourteen sun letters in ASCII and DMG spellings (`ar-`, `az-`, `aš-`, `ash-`, `aṭ-`, …; lām's being `al-` itself), and the Egyptian `el-` with its assimilations in ASCII spellings only (`es-`, `esh-`, `ed-`, …; no `eš-` or `eṣ-`) — all never-given, where the bare `al` stays ambiguous (Al Gore): written apart with its hyphen, the word is the article and nothing else. RENDERING: the attached family joins with a space like every particle, `d' Ivoi` and `al- Rashid`. Declined: a shape rule (a vowel, consonants, a hyphen) in place of the list. It would need a new predicate and the bare-word collision check the issue named, for a population the list already covers; the corpora attest seven article spellings (`al- az- aš- ar- es- as- aṭ-`), all in it — recompute: every whitespace- or comma-bounded token of `tools/differential/corpus*.jsonl` matching `\w{1,4}-` at a name's end. Not decided here: the GLUED forms (`Paul d'Ivoi`, `Harun al-Rashid`) stay one word with no particle split, out of #700's scope by its own statement. Hyphen-minus only: U+2010 and the non-breaking hyphen are not listed. The Scottish Gaelic `a’` (`Mac a’ Ghobhainn`, in corpus_cldr.jsonl) is mid-name and not an index-form particle, and was not added. Nor are the elisions no corpus holds (`dall'`, `nell'`, `degl'`): the list is closed, so such a spelling reads as a middle name until it is listed. @@ -87,3 +88,4 @@ Decided 2026-08-19 (was Open: [#380](https://github.com/derek73/python-nameparse Measured 2026-10-17 against the parent commit `b4a004c3`: 39 corpus names move, every one the same shape — a trailing `d'` or article leaves the middle name and joins the family (38 in corpus_gnd.jsonl, `Orval, Pierre d'` in corpus_conventions.jsonl), and no ambiguity kind changes. Recompute: `uv run python tools/differential/snapshot.py --since b4a004c3`. That is evidence about the corpus as much as the change: these are the comma-bearing names ending in one of the new forms, and GND's index forms are where they occur. - 2026-10-19 — RULE SPLIT: the third exception (a lone word of both the particle and the unambiguous suffix vocabulary inside a run of post-nominals) left rules/P6 for rules/P9, one rule per piece of evidence (docs/design/AGENTS.md), with rules/P2's clause on the same word before a family comma; #573 decided both. The #573 entry moved to decisions/P9 and a cross-reference stands in its place. Where an entry here says P6's third exception or `_inside_a_credential_run` (the #613 entry), that piece is rules/P9's now. - 2026-10-20 #761 (skeleton migration) — three passages of the block-format statement left the rule as argument or history rather than statement. The draft that tested the family-first slot alone and fired on 52 default-order names and 366 FAMILY_FIRST_GIVEN_LAST middles, and the FAMILY_FIRST line for `Nguyen Thi Van` removed rather than updated (#470), are the 2026-08-30 #467 entry above. The third is moved here: the comma site and the no-comma family-first site are checked against each other directly, as a pair, rather than only against their own examples — tests/v2/test_order_correspondence.py generates both writings and asserts they parse alike, plus a negative-control sweep pinning the disagreeing set the precedence over S2 names, so a change that breaks one side of that pair should expect that test, not the rule file, to say so first (decisions/two-input-invariants). +- 2026-10-27 #892 (Derek) — THE EMPTY-HEAD SITE IS GONE. SUPERSEDES: the 2026-10-06 #613 entry's keeping the attachment behind an empty family part in post_rules, after H1 and M4. rules/C15 reads a comma with nothing before it as a family comma naming no surname, so the attachment behind it is the comma site's, in assign, ahead of H1 and M4. #613 had kept a second site in post_rules for that head, after H1 and M4, so that `, Mr. Jones vd` kept the reading it had before #613 (family 'Jones', suffix 'vd'); that reading is the comma-less name's (`Mr. Jones vd`), not the twin's, and decisions/C15 holds the empty head to its twin. Removing it moves the particle-and-suffix words `vd` and `mc` ending the name behind a title that addresses by surname, 112 given parts per order on C15's invariant grid and the same tail behind a third comma part (`, Dr. Jones vd, PhD`), each now given 'Jones', family 'vd', with a `suffix-or-name` report, where an ambiguous particle already read so (`, Dr. Jones van` → family 'van'). A never-given particle opening the part with no name word behind it now meets the comma site's P1 guard, which the removed site lacked, and keeps the tail a suffix as its twin does (`, de PhD vd` → given 'de', suffix 'PhD vd'). The S13 shape decisions/S13 left as found, `, Jones (MA) vd`, reads its twin's nickname 'MA' with it, the particle being family before S13's pass reads what follows the clause. diff --git a/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md b/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md index 70200e310..9722abb4e 100644 --- a/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md +++ b/docs/design/decisions/S13-bracketed-member-settled-by-its-writing.md @@ -7,6 +7,8 @@ - Only a clause after the name is settled, behind its last name word: one in a name with no word of its own keeps the nickname (`Dr. (MA)`, `(M.A)`, where the review of the one-pass commit found both settled), and one with a name word after it keeps the nickname however it is written (`Jeffrey (JD) Bricken`, `Jean 'JD' Smith`, `Smith (MA), John`), while a suffix, a title or a maiden clause after it does not count, however it is written (`John Smith (MA), PhD`, `John Smith (MA) Ph. D.`, `John Smith (MA) Dr.`), asked of the roles the parse read rather than of the following words' tags — 2026-10-26 #810. - Where the rest of the name then reads otherwise than beside a bracketed unambiguous credential in the same place, the member's twin, the difference is an accepted cost, recorded in rules/S13's Accepted bullets and counted by tests/v2/test_parser.py's census (`Jack (MA)` → family 'Jack', where `Jack (PhD)` reads given 'Jack') — 2026-10-26 #810. +Elsewhere: P6's attachment after a comma with nothing before it, which the #810 entry's second review left putting a name word behind a settled clause, runs ahead of this rule's pass since decisions/P6's 2026-10-27 #892 entry, so `, Jones (MA) vd` reads nickname 'MA', family 'vd', as its twin `Smith, Jones (MA) vd` does. + ## Log - **2026-10-26 #810 (Derek, 2026-10-23; the single signal and the one pass Derek, 2026-10-26) — A BRACKETED MEMBER OF THE AMBIGUOUS CLASS AFTER THE NAME, SETTLED BY ITS WRITING, IS THE SUFFIX, READ IN ONE PASS.** Carries out item 16 of decisions/vocabulary-model's 2026-10-23 #752 entry, as phase 2l of #752. SUPERSEDES: in decisions/ma-do, the 2026-07-17 entry's side effect, in part (a parenthesized `(MA)` or `(DO)` no longer escaping to suffix), for a member after the name whose writing settles it. #810 asked whether `John Smith (M.A)` should read as `John Smith M.A` does, and found `(M.A)`, `"M.A"` and `(MA)` all reading as a nickname where 1.4.0 read the bracketed ones as a suffix; 2.0.0 through 2.6.0 read each as a nickname (measured 2026-10-26 on the wheels, `John Smith (MA)`, `John Smith (M.A)`, `Smith, John (MA)`). The escape had refused every member since `ma` joined the class (decisions/ma-do, 2026-07-17), and #796's 33 new members widened its reach. diff --git a/docs/design/rules/C15-comma-with-nothing-before-it.md b/docs/design/rules/C15-comma-with-nothing-before-it.md index 4c47e6a67..0acee1f99 100644 --- a/docs/design/rules/C15-comma-with-nothing-before-it.md +++ b/docs/design/rules/C15-comma-with-nothing-before-it.md @@ -19,20 +19,21 @@ **Except:** -- **A title and one name word** (H1): the comma names no surname, so it does not decide the field of that word, or of a bound pair standing for it, and the title's lone name word reads as the family. A particle that is also a suffix word, ending the name behind a title that addresses by surname, then has no family to attach to (P6) and reads as the suffix. +- **A title and one name word** (H1): the comma names no surname, so it does not decide the field of that word, or of a bound pair standing for it, and the title's lone name word reads as the family. ``` ", Dr. John" → family="John" · boundary ", Jane Mrs" → family="Jane" ", Dr. abdul Rahman" → family="abdul Rahman" - ", Dr. Jones vd" → suffix="vd" ``` **Precedence:** -- **A particle ending the name** (P6) attaches to the family after this comma as after any family comma, and with no surname written the family is that particle alone: `, Jane van` reads family `van`. P6 wins because the particle is in the part after the comma, which this rule reads as any given part is read. +- **A particle ending the name** (P6) attaches to the family after this comma as after any family comma, and with no surname written the family is that particle alone: `, Jane van` reads family `van`. P6 wins because the particle is in the part after the comma, which this rule reads as any given part is read, and it wins over H1's exception too, leaving H1 a given word and a family rather than one name word. ``` ", Jane van" → family="van" ", Sir Jones vd" → family="vd" + ", Dr. Jones vd" → given="Jones" + ", Dr. Jones vd" → family="vd" ``` **Accepted:** diff --git a/docs/design/rules/P6-trailing-orphan-particle.md b/docs/design/rules/P6-trailing-orphan-particle.md index 449ca7e54..e19e14c33 100644 --- a/docs/design/rules/P6-trailing-orphan-particle.md +++ b/docs/design/rules/P6-trailing-orphan-particle.md @@ -26,6 +26,12 @@ "Ménil de" family-first → given="de" · boundary ``` - **Nothing is asked about the WORD**, and that is what lets one rule read both traditions: the same vocabulary that spells the Dutch tussenvoegsel spells the Vietnamese given name, and the declared order separates them where a comma cannot — one order, not both, as an Accepted bullet below records. +- **A family comma with nothing before it** (C15) names no surname but marks where one goes, so the attachment fires there as after any family comma and the particle is the family alone. It fires ahead of H1, which then finds a given word beside a family rather than one name word. + ``` + ", Jane van" → family="van" + ", Dr. Jones vd" → given="Jones" + ", Dr. Jones vd" → family="vd" + ``` - **Where nothing has named the family, the particle is read where it stands**, and a particle CAN be a given name there: the never-given vocabulary supplies the reading position leaves open, and forbids nothing (parse("de") reports given "de", and so does every other word in that set). - **What reports.** After a family comma, a particle that is also of the AMBIGUOUS credential class (`do`), where neither S6's capitals nor S7's company nor a run a settled member starts reaches it (below), attaches, and the report gives it this rule's particle fork rather than S4's credential one. The fork is reported, in the kind naming the reading this attachment OVERRODE — what the run was read as before it fired, not what its words are. Where the run was read as a post-nominal, which is the S2 reading this rule outranks, the report is suffix-or-name. Otherwise, where the run holds an ambiguous particle, the reading overridden is that word as a name word, and the report is particle-or-given. Where the attachment overrode neither — a never-given particle read as a name word, which it could never have been — nothing was decided and nothing is reported. So after a family comma `do`, which sits in the AMBIGUOUS acronym half and was already read as a name word, reports particle-or-given while vd and mc report suffix-or-name, and a run of several words read as name words reports on its ambiguous member if it has one and otherwise reports nothing, whatever its words could have been read as standing alone. Both are pinned in tests/v2/cases.py. ``` @@ -108,5 +114,5 @@ ``` --- -**Signal:** position: a particle ending the name where a family comma or a family-first reading named the family +**Signal:** position: a particle ending the name where a family comma or a family-first reading marked the family's place **history:** decisions/P6 · **implemented:** nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py diff --git a/nameparser/_pipeline/AGENTS.md b/nameparser/_pipeline/AGENTS.md index 9c03b957e..346521893 100644 --- a/nameparser/_pipeline/AGENTS.md +++ b/nameparser/_pipeline/AGENTS.md @@ -26,7 +26,7 @@ The decision site also has the token index and the detail text in hand, which th ### A fork decided in several stages needs an emitter in each -**If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece, in `_group` when something shifts it off the name's leading piece and the prefix chain claims it, and wherever P6's attachment takes a trailing particle into the family -- in `_assign` after a comma that names a family (since #613), in `post_rules` at the end of a name read family-first (#467) and after a comma with nothing before it, where H1 and M4 must read the name first -- so every one of them reports; for two years only the first did. +**If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece, in `_group` when something shifts it off the name's leading piece and the prefix chain claims it, and wherever P6's attachment takes a trailing particle into the family -- in `_assign` after a family comma (since #613, and behind an empty part before the comma since #892), in `post_rules` at the end of a name read family-first (#467) -- so every one of them reports; for two years only the first did. What can still do the shifting is narrow, and #367 is why: a plain title no longer can (`Dr. Van Johnson` reads as `Van Johnson` does and reports from `_assign`), so the `_group` emitter needs a word that is BOTH a title and a particle — measured, `TITLES ∩ particles_ambiguous` is `{freiherr, st}` in the default vocabulary (`do` left TITLES in #296's audit; the particle-line block in decisions/vocabulary-exclusions records the before and after), plus any overlap a caller's config creates — standing ahead of the chained particle as the LEADING NAME word. Titles may precede it, so `Dr. St van Johnson` reaches the emitter and `St van Johnson` does too; a given name may not, so `Jan Freiherr von Richthofen` does not reach it while `Freiherr von Richthofen` and `Dr. Freiherr von Richthofen` do. Two shapes that look like they should reach it and do NOT, both measured by stepping `STAGES` and watching where `ambiguities` grows: `Dr. Do van Johnson` and `Do St Johnson` report from `assign`, not `group`, because `do` is no longer a title and so stays the leading name piece assign reports on — a both-vocabulary word CHAINED (`Jan St Johnson`) reports nothing at all. When checking whether that emitter is dead, a both-vocabulary word in the leading name position is the thing to look for, and the answer is that it is not dead. @@ -45,3 +45,5 @@ And work a report does must not walk the name: the given-name title test, asked ### Testing an emitter Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful. + +**Folding one site of a rule into another moves every input the two sites' GUARDS disagree on, not only the shape the fold was for.** Compare the guards line by line before measuring: #916 removed P6's empty-head site in post_rules for assign's comma site and measured its moves on C15's grid, which held a title-and-`vd` class only; the review found a second class off the grid (`, de PhD vd`), the comma site's P1 guard (`is_lone_never_given_particle`) being one the removed site never had. diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 7255dc427..ea188f9fc 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -829,7 +829,10 @@ def _inside_a_credential_run(seg: Sequence[Sequence[int]], and 'Doe, Jane PhD vd' keeps family 'vd Doe'. A title between it and the post-nominal in front is transparent, as it is to every trailing reading (H5): 'DOE, JANE PHD PROF. VD MA' reads as 'DOE, - JANE PHD VD MA' does. (Behind it no title is skipped: a title + JANE PHD VD MA' does. The run is never empty: the caller asks + only where `k < end`, and over an empty range the test below would + pass vacuously and report a fork on the word behind (#613's + review, ', Jane PhD do MA'). (Behind it no title is skipped: a title there leaves the word a name in assign, so nothing reaches here.) The test is the ROLES assign gave, not S7's company query: the @@ -850,12 +853,7 @@ def suffix_read(q: int) -> bool: def is_title(q: int) -> bool: return all(tokens[i].role is Role.TITLE for i in seg[q]) - # an empty run is no run: the walk stops short of a member already - # read as the credential ('Doe, Jane PhD do MA'), and there is - # nothing to decline. assign's caller never passes one, but - # post_rules' site behind an empty family part can (', Jane PhD do - # MA' reported a bogus third fork without this, #613's PR review) - if k == end or not all(suffix_read(q) for q in range(k, end)): + if not all(suffix_read(q) for q in range(k, end)): return False front = k - 1 while front > given_at and is_title(front): @@ -1796,19 +1794,21 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: else suffix_reading(tokens[i2])))) if sticky_from == len(pieces): k6, end6 = particle_tail(pieces, tokens) - # Only where the comma named a family: with nothing before - # it, H1 and M4 decide first and post_rules attaches after - # them. GIVEN ahead of the tail, which is what P6 says - # ("provided at least one given word remains"): the given is - # piece - # `n`, so a tail starting there leaves none ('Nguyen, Van'). - # And not where P1's fold will take the part: a never-given - # particle opening it folds every given and middle word - # into the family in post_rules, so no given word remains - # for this one either ('Smith, de Mesnil van' keeps - # 'van' where the fold puts it, not hoisted in front of - # 'Smith', the 2026-08-18 defect). - if (n < k6 < end6 and fam_pieces + # Behind every family comma, one with nothing before it + # too, which then makes the particle the whole family + # (', Dr. Jones vd'; decisions/C15, #892). GIVEN ahead of + # the tail -- rules/P6: "at least one given word must + # remain" -- the given is piece `n`, so a tail starting + # there leaves none ('Nguyen, Van'). + # And not where a never-given particle opens the part: P1's + # fold takes every given and middle word into the family in + # post_rules, so no given word remains for this one either + # ('Smith, de Mesnil van' keeps 'van' where the fold puts + # it, not hoisted in front of 'Smith', the 2026-08-18 + # defect); with no name word behind it to fold, the tail + # stays the suffix, behind an empty head as behind its twin + # (', de PhD vd' as 'Smith, de PhD vd', #892). + if (n < k6 < end6 and not is_lone_never_given_particle(pieces[n], tokens)): _attach_particle_tail(pieces, tokens, n, k6, end6, diff --git a/nameparser/_pipeline/_pieces.py b/nameparser/_pipeline/_pieces.py index b9e7630b5..a95067e07 100644 --- a/nameparser/_pipeline/_pieces.py +++ b/nameparser/_pipeline/_pieces.py @@ -1596,13 +1596,13 @@ def particle_tail(seg: Sequence[Sequence[int]], past, since `vd` arrives suffix-roled and is the run. Read off ROLES, asked at the moment assign's walk reaches #602's - run (or, with no run, once the walk has placed every piece; and by - post_rules' attachment behind an empty family part): every word in front of the run holds the role the walk gave - it, and the run's words hold none yet -- no name role, and a class - member not yet read as anything, the run being what will read it - as the credential. That is why the answer equals the one this - would give over the finished roles, which is what the attachment - needs. `lo == hi` where there is no such run. + run (or, with no run, once the walk has placed every piece): every + word in front of the run holds the role the walk gave it, and the + run's words hold none yet -- no name role, and a class member not + yet read as anything, the run being what will read it as the + credential. That is why the answer equals the one this would give + over the finished roles, which is what the attachment needs. + `lo == hi` where there is no such run. The single-token tests are inline (the common piece is one token), so the walk pays no frame per piece; it runs on every family-comma diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 34cf28d8e..080ce0ffe 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -12,9 +12,9 @@ a nickname), the stable "joined" tag on a post-nominal continuing the entry before it, and the ambiguity P6's attachment reports at the end of a name read -family-first (#467) and after a comma with nothing before it -- after a -family comma that names a family, that attachment is assign's since -#613. +family-first (#467) -- after a family comma, that attachment is +assign's since #613, and behind an empty part before the comma too +since #892. Reads: Policy.patronymic_rules, Policy.two_family_names, Policy.name_order (O6 stands down where the script's order differs from the declared one), Policy.middle_as_family, @@ -23,12 +23,12 @@ Lexicon.suffix_acronyms_ambiguous (a dotted member in a clause, which carries no tag there, S13). -Implements rules S13, H1, M4, P1, P6 (its no-comma site, and after a -comma naming no family), O1, O2, O6, O3 and R1 of docs/design/rules/; -each is cited at its code below, and H1/P1/O1/O2's history lives in -docs/design/decisions/. `suffix_entries` is the R1 entry pass as a -state-in/state-out function, for Parser.revise to run over a -forced-role sub-parse (#511); `post_rules` runs the same worker last. +Implements rules S13, H1, M4, P1, P6 (its no-comma site), O1, O2, O6, +O3 and R1 of docs/design/rules/; each is cited at its code below, and +H1/P1/O1/O2's history lives in docs/design/decisions/. +`suffix_entries` is the R1 entry pass as a state-in/state-out +function, for Parser.revise to run over a forced-role sub-parse +(#511); `post_rules` runs the same worker last. """ from __future__ import annotations @@ -37,10 +37,10 @@ from nameparser._lexicon import _normalize, _run_addresses_by_given from nameparser._pipeline._assign import ( - _attach_particle_tail, _name_alt, _name_pieces, _name_positions, + _name_alt, _name_pieces, _name_positions, ) from nameparser._pipeline._pieces import ( - is_lone_never_given_particle, particle_tail, settled_by_case, + is_lone_never_given_particle, settled_by_case, ) from nameparser._pipeline._state import ( AMBIGUOUS_ACRONYM_TAG, NAME_ROLES, @@ -777,26 +777,6 @@ def post_rules(state: ParseState) -> ParseState: middles = _idx(tokens, Role.MIDDLE) families = _idx(tokens, Role.FAMILY) - # rules/P6: "a particle ending the name attaches to that family - # name" -- after a family comma with NOTHING before it. assign takes - # the attachment wherever the comma names a family (#613); where it - # names none, H1 and M4 above may make the given word the family - # first, being gated on there being no family, and the attachment - # has to read the name they left -- ', Mr. Jones vd' keeps family - # 'Jones', suffix 'vd', as it read before #613 moved the site - # (caught in PR review: assign's attachment ran ahead of H1 and read - # given 'Jones', family 'vd'). The same helper, at the stage order - # the old site had, so the reading is the old one exactly. - if (state.structure is Structure.FAMILY_COMMA - and len(state.pieces) > 1 and not state.pieces[0]): - seg = state.pieces[1] - k, end = particle_tail(seg, tokens) - given_at = next((q for q in range(k) if any( - tokens[i].role is Role.GIVEN for i in seg[q])), None) - if given_at is not None: - _attach_particle_tail(seg, tokens, given_at, k, end, - ambiguities) - if state.policy.two_family_names: _take_second_family_name(state, tokens) diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 177fa44fb..3caaf9d47 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2870,9 +2870,9 @@ def _check_cjk_shape_purity(self) -> None: Case("an_elided_particle_after_an_empty_head", ", Paul d'", {"given": "Paul", "family": "d'"}, classification="fix(#700)", - notes="nothing before the comma names a family, so P6's " - "attachment has nothing to join and the particle is the " - "family by position, as ', Paul de' reads. 1.4.0 through " + notes="a family comma naming no surname (rules/C15): P6 " + "attaches the particle as behind any family comma, and it " + "is the family alone, as ', Paul de' reads. 1.4.0 through " "2.4.2 middle \"d'\" with no family"), Case("an_elided_particle_in_its_typographic_spelling_t", "Hooft, Gerard ’t", @@ -6535,8 +6535,8 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=("particle-or-given",), classification="fix(#623)", notes="the empty head (nameparser/AGENTS.md's comma sweep): the bound " - "holds there too, and P6's attachment, read in post_rules " - "behind an empty family part, takes 'van' as the family; " + "holds there too, and P6's attachment takes 'van' as the " + "family, as behind any family comma (rules/C15); " "1.4.0 and 2.3.0 read middle 'van Prof.'"), Case("the_chain_runs_again_where_its_merges_free_the_acronym", "Smith, Freiherr van Berg X.Y.Z.", @@ -8085,16 +8085,57 @@ def _check_cjk_shape_purity(self) -> None: ambiguities=(), classification="fix(#613)", notes="the same exit read ', Jr.' title 'Jr.' (PR review)"), - Case("no_family_named_lets_the_title_rule_decide_before_p6", + Case("empty_head_attaches_the_particle_before_the_title_rule", ", Mr. Jones vd", - {"title": "Mr.", "family": "Jones", "suffix": "vd"}, + {"title": "Mr.", "given": "Jones", "family": "vd"}, + ambiguities=("suffix-or-name",), + classification="fix(#892)", + notes="a comma with nothing before it is a family comma naming " + "no surname (rules/C15), so P6 attaches the particle as " + "behind any family comma, ahead of H1, which then finds " + "a family: the twin 'Smith, Mr. Jones vd' reads given " + "'Jones', family 'vd Smith'. #613 had kept the " + "comma-less reading, family 'Jones', suffix 'vd', with " + "a second P6 site in post_rules (decisions/P6)"), + Case("empty_head_attaches_the_particle_before_a_suffix", + ", Dr. Jones vd Jr.", + {"title": "Dr.", "given": "Jones", "family": "vd", + "suffix": "Jr."}, + ambiguities=("suffix-or-name",), + classification="fix(#892)", + notes="the walk looks past the trailing suffix to the particle " + "(rules/P6), behind an empty head as behind its twin " + "'Smith, Dr. Jones vd Jr.'; #613's post_rules site read " + "family 'Jones', suffix 'vd Jr.'"), + Case("empty_head_attaches_the_particle_before_a_suffix_part", + ", Dr. Jones vd, PhD", + {"title": "Dr.", "given": "Jones", "family": "vd", + "suffix": "PhD"}, + ambiguities=("suffix-or-name",), + classification="fix(#892)", + notes="a third comma part, which C15's invariant grid does not " + "write; #613's post_rules site read family 'Jones', " + "suffix \"vd, PhD\""), + Case("empty_head_attaches_the_particle_family_first", + ", Jane van Jr.", + {"given": "Jane", "family": "van", "suffix": "Jr."}, + ambiguities=("particle-or-given",), + policy=Policy(name_order=FAMILY_FIRST), + classification="fix(#892)", + notes="pinned field by field because C15's invariant compares " + "the empty head with its twin, and the two now share " + "assign's one attachment site, so a defect there would " + "move both; master read the same"), + Case("empty_head_never_given_particle_keeps_the_suffix_tail", + ", de PhD vd", + {"given": "de", "suffix": "PhD vd"}, ambiguities=(), - classification="fix(#613)", - notes="a comma with nothing before it names no family, so H1 " - "reads a title and one name word first and P6 attaches " - "after it, in post_rules; assign attaching ahead of H1 " - "read given 'Jones', family 'vd' (PR review, 177 texts " - "of that shape on the empty-head grid)"), + classification="fix(#892)", + notes="a never-given particle opening the part declines the " + "attachment (P1's guard at assign's site), as in the twin " + "'Smith, de PhD vd' and in ', de vd'; #613's post_rules " + "site, which had no such guard, read family 'vd', suffix " + "'PhD', reported (#916 review)"), Case("particles_side_by_side_in_a_credential_part_stay_credentials", "De La Cruz, DO DO DO", {"family": "De La Cruz", "suffix": "DO DO DO"}, @@ -8197,16 +8238,15 @@ def _check_cjk_shape_purity(self) -> None: "and a name word and reads a suffix comma, so the name " "divides as it would with no comma. b547c247 kept " "'김민준' whole, H1 then making it the family"), - Case("an_empty_family_part_declines_no_empty_particle_run", + Case("an_empty_head_credential_run_reports_once", ", Jane PhD do MA", {"given": "Jane", "suffix": "PhD do MA"}, ambiguities=("suffix-or-name",), classification="fix(#613)", - notes="post_rules' P6 site behind an empty family part can find " - "an EMPTY particle run (the walk stops short of 'do', read " - "as the credential), and the third exception must decline " - "nothing there; without its empty-run guard it reported a " - "third fork on 'MA' (#613's PR simplification pass). One " + notes="the walk stops short of 'do', read as the credential, so " + "P6 finds no particle run to attach (#613's PR " + "simplification pass found a bogus third fork on 'MA' from " + "post_rules' empty-head site, removed in #892). One " "report since #785, the capitals settling 'MA' (rules/S6); " "b547c247 gave two"), Case("a_glued_honorific_in_a_postnominal_part_stays_glued", diff --git a/tests/v2/pipeline/test_state.py b/tests/v2/pipeline/test_state.py index d455fe78b..e0edb2f4c 100644 --- a/tests/v2/pipeline/test_state.py +++ b/tests/v2/pipeline/test_state.py @@ -324,8 +324,8 @@ def test_stage_field_ownership() -> None: # comma marks the particles it joins to the family # vocab:folded-middle, for the family view's prepend order "assign": {"role", "tags"}, - # post_rules also tags: P6's attachments it still makes (the - # no-comma site, and after a comma with nothing before it) and + # post_rules also tags: P6's attachment it still makes (the + # no-comma site; behind an empty head it is assign's since #892) and # the middle_as_family fold mark folded tokens # vocab:folded-middle for the family view's prepend order, R2 # and R3 write their unjoined marks, and R1's entry pass marks diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index 34c247c6d..4847d7c8a 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -15,7 +15,7 @@ "C12": {"words": 316, "then_words": 107, "because_sentences": 2, "except": 2, "precedence": 1, "accepted": 2, "accepted_words": 59}, "C13": {"words": 259, "then_words": 74, "because_sentences": 1, "except": 1, "precedence": 1, "accepted": 0, "accepted_words": 0}, "C14": {"words": 231, "then_words": 111, "because_sentences": 3, "except": 1, "precedence": 0, "accepted": 0, "accepted_words": 0}, - "C15": {"words": 253, "then_words": 31, "because_sentences": 3, "except": 1, "precedence": 1, "accepted": 2, "accepted_words": 101}, + "C15": {"words": 242, "then_words": 31, "because_sentences": 3, "except": 1, "precedence": 1, "accepted": 2, "accepted_words": 101}, "D1": {"words": 73, "then_words": 26, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "D2": {"words": 65, "then_words": 37, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "H1": {"words": 463, "then_words": 306, "because_sentences": 3, "except": 1, "precedence": 0, "accepted": 4, "accepted_words": 370}, @@ -45,7 +45,7 @@ "P3": {"words": 639, "then_words": 598, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P4": {"words": 190, "then_words": 82, "because_sentences": 1, "except": 1, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P5": {"words": 717, "then_words": 399, "because_sentences": 1, "except": 2, "precedence": 3, "accepted": 3, "accepted_words": 198}, - "P6": {"words": 1034, "then_words": 651, "because_sentences": 2, "except": 2, "precedence": 5, "accepted": 6, "accepted_words": 459}, + "P6": {"words": 1086, "then_words": 703, "because_sentences": 2, "except": 2, "precedence": 5, "accepted": 6, "accepted_words": 459}, "P7": {"words": 138, "then_words": 64, "because_sentences": 3, "except": 0, "precedence": 0, "accepted": 1, "accepted_words": 28}, "P8": {"words": 515, "then_words": 450, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 2, "accepted_words": 309}, "P9": {"words": 405, "then_words": 265, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 0, "accepted_words": 0}, diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 3a1507ca8..9e0eca57d 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -291,10 +291,15 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # reported (+6); 1384 -> 1390 the same day, its review round's rows # for a leading period and a name of no word (+6); 1390 -> 1393 the # same day, #752 phase 2j's 'Nasreen Sultana, MBBS', the credential - # behind two name words reported suffix-or-name under three orders. - assert claims == 1393, ( - f"the field sweep checks {claims} claims, recorded as 1393 on " - f"2026-10-26") + # behind two name words reported suffix-or-name under three orders; + # 1393 -> 1396 on 2026-10-27, #892's ', Mr. Jones vd' reporting the + # suffix-or-name P6's attachment decides, under three orders; 1396 + # -> 1403 the same day, the #916 review's rows (', Dr. Jones vd + # Jr.' and ', Dr. Jones vd, PhD' under three orders, and ', Jane + # van Jr.' under FAMILY_FIRST). + assert claims == 1403, ( + f"the field sweep checks {claims} claims, recorded as 1403 on " + f"2026-10-27") #: The kinds that report a fork between two readings of a word, so carry @@ -624,8 +629,10 @@ def test_the_alternative_sweep_sees_the_fields_it_checks() -> None: # connective whose stand-in is an initial too, so its field is # pinned in _EXPECTED_ALTERNATIVES instead; 299 -> 302 the same day, # master's 'Nasreen Sultana, MBBS' row (#903) under each order, a - # comma credential run's report, whose stand-in the run reads too - assert (checked, skipped) == (436, 302), (checked, skipped) + # comma credential run's report, whose stand-in the run reads too; + # 436 -> 437 on 2026-10-27, #892's ', Jane van Jr.' row under + # FAMILY_FIRST, a particle-or-given report P6's attachment decides + assert (checked, skipped) == (437, 302), (checked, skipped) def test_every_case_name_is_unique() -> None: diff --git a/tests/v2/test_facade_cases.py b/tests/v2/test_facade_cases.py index 7e00799d4..354048ed9 100644 --- a/tests/v2/test_facade_cases.py +++ b/tests/v2/test_facade_cases.py @@ -131,6 +131,9 @@ # #700: P6's family-first site for the elided particle, under a # declared order v1 has no spelling for "an_elided_particle_ending_a_family_first_name", + # #892: P6's attachment behind an empty head under a declared + # family-first order, which v1 cannot express + "empty_head_attaches_the_particle_family_first", "family_comma_no_name_word_family_first_given_last", "family_comma_no_name_word_family_first_plain", "leading_never_given_particle_two_leftovers_family_first", diff --git a/tests/v2/test_properties.py b/tests/v2/test_properties.py index 165dd20bb..a16e218a2 100644 --- a/tests/v2/test_properties.py +++ b/tests/v2/test_properties.py @@ -287,10 +287,11 @@ def test_the_comma_agreement_exceptions_are_all_still_exceptions( _EMPTY_HEAD_TAILS = ("", "Jr.", "PhD", "Dr.", "Mrs", "III", "V", "MA", "Ma", "fils", "X.Y.Z.") #: Rows rules/C15's one exception takes, H1 reading the title's lone -#: name word as the family (', Dr. John'): 1,075 under each of the -#: three orders. A count, so the exception cannot grow unseen -#: (measured 2026-10-27). -_EMPTY_HEAD_H1_ROWS = 3225 +#: name word as the family (', Dr. John'), under each of the three +#: orders. A count per order, so the exception cannot grow unseen in +#: one of them (measured 2026-10-27; 1,075 each before #892's +#: empty-head P6 site went). +_EMPTY_HEAD_H1_ROWS = 963 def _empty_head_view(name: ParsedName, drop_first: bool = False @@ -333,10 +334,10 @@ def test_an_empty_head_reads_as_its_family_comma_twin() -> None: The one exception is H1's, asserted rather than skipped: where the twin reads a title and name words in the given field alone, the empty head reads them as the family (', Dr. John' -> family - 'John'), the comma naming no surname to decide by. A particle that - is also a suffix word, which the twin attached to its surname - (rules/P6), then has no family to attach to and is the suffix - (', Dr. Jones vd': family 'Jones', suffix 'vd'). + 'John'), the comma naming no surname to decide by. Only where the + twin, less its surname, reads no family: a particle P6 attaches + gives H1 a family, so it stands down (', Dr. Jones vd' -> given + 'Jones', family 'vd'). This replaces the empty head as a shape every comma sweep must hold (nameparser/AGENTS.md): a change that breaks it gets a generic fix @@ -349,13 +350,14 @@ def test_an_empty_head_reads_as_its_family_comma_twin() -> None: 1,098 rows, ', PhD' reading given 'PhD' among them. """ failures: list[str] = [] - h1_rows = 0 + h1_rows: dict[str, int] = {} rows = 0 for label, order in (("given-first", GIVEN_FIRST), ("family-first", FAMILY_FIRST), ("family-first-given-last", FAMILY_FIRST_GIVEN_LAST)): parser = Parser(policy=Policy(name_order=order)) + h1_rows[label] = 0 for parts in itertools.product(_EMPTY_HEAD_TITLES, _EMPTY_HEAD_BODIES, _EMPTY_HEAD_TAILS): @@ -372,21 +374,10 @@ def test_an_empty_head_reads_as_its_family_comma_twin() -> None: continue fields, kinds = twin if (fields["title"] and fields["given"] - and not fields["middle"]): + and not fields["middle"] and not fields["family"]): moved = dict(fields, given="", family=fields["given"]) - if fields["family"]: - # the tail P6 attached to the twin's surname, a - # particle that is also a suffix word: with no - # family to attach to it is the suffix, and the - # twin's report on the attachment goes with it - moved["suffix"] = " ".join( - w for w in (fields["family"], fields["suffix"]) - if w) - kinds = list(kinds) - if "SUFFIX_OR_NAME" in kinds: - kinds.remove("SUFFIX_OR_NAME") if empty == (moved, kinds): - h1_rows += 1 + h1_rows[label] += 1 continue failures.append( f"{label}: {', ' + given_part!r} reads {empty} but " @@ -395,7 +386,7 @@ def test_an_empty_head_reads_as_its_family_comma_twin() -> None: assert not failures, ( f"{len(failures)} given part(s) disagree:\n" + "\n".join(failures[:15])) - assert h1_rows == _EMPTY_HEAD_H1_ROWS, ( + assert set(h1_rows.values()) == {_EMPTY_HEAD_H1_ROWS}, ( f"H1's exception took {h1_rows} rows, recorded " f"{_EMPTY_HEAD_H1_ROWS}: read the rows it gained or lost " f"before re-recording") diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index f073020d0..06fe55c0c 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -15,7 +15,7 @@ {"name": ", Ana García López", "given": "Ana", "middle": "García López", "initials": "A. G. L."} {"name": ", Dean", "given": "Dean", "initials": "D."} {"name": ", Dr. John", "title": "Dr.", "family": "John", "initials": "J."} -{"name": ", Dr. Jones vd", "title": "Dr.", "family": "Jones", "suffix": "vd", "initials": "J.", "capitalized_force": "Dr. Jones VD"} +{"name": ", Dr. Jones vd", "title": "Dr.", "given": "Jones", "family": "vd", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (vocabulary)"], "initials": "J. v.", "capitalized_force": "Dr. Jones Vd"} {"name": ", Dr. abdul Rahman", "title": "Dr.", "family": "abdul Rahman", "initials": "a. R.", "capitalized_force": "Dr. Abdul Rahman"} {"name": ", Dr. fils", "title": "Dr.", "family": "fils", "initials": "f.", "capitalized_force": "Dr. Fils"} {"name": ", Jane", "given": "Jane", "initials": "J."}