Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -975,6 +975,19 @@ an initial written alone never joins the family:
>>> two_names.parse("Dr. Juan Pérez de la Vega Jr.").family
'Pérez de la Vega'

``surname2`` names the second of the two, which ``family`` keeps, so a
caller addressing or filing by the first takes it off; a name the
switch reads as one family has none:

.. doctest::

>>> ana.surname2
'López'
>>> ana.family.removesuffix(" " + ana.surname2)
'García'
>>> two_names.parse("Ana María García-López").surname2
''

A family that already holds two names keeps what it has, whether a
hyphen or ``y`` joins them (``"Ana María García-López"``, ``"José
Ortega y Gasset"``) or a particle chain took them both: a compound
Expand Down
4 changes: 3 additions & 1 deletion docs/design/decisions/O6-two-family-names.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Standing

- Two family names are an opt-in `Policy` switch, `two_family_names`, off by default, and not a locale pack: the convention is Spain's and most of Latin America's but not Argentina's, and given-first it is the same string shape as a name with a middle name, so no default can read it — 2026-10-24 #754.
- Both family names stay in `family` through 2.x; a second-surname field is the RFC's Phase 4 output model (docs/design/vocabulary-model-rfc.md, "The rules a pack can choose"), and a pack may later switch this rule on — 2026-10-24 #754.
- Both family names stay in `family` through 2.x; a second-surname field is the RFC's Phase 4 output model (docs/design/vocabulary-model-rfc.md, "The rules a pack can choose"), and a pack may later switch this rule on — 2026-10-24 #754; landed as the `surname2` view, `family` keeping both, 2026-10-29 #754, sub-phase 4d.
- The family's words are counted as P1's fold counts them, a connective join and a hyphen counting as their words, so a family already holding two keeps what it has; the word taken must count as one by the same count and be no initial written alone — 2026-10-24 #754.
- A name its script (W4) reads in an order other than the declared `name_order` is left alone; where the two agree it is read like any other — 2026-10-24 #754.
- Under a family-first order the first two name words are the family, the remainder laid out as the positions after a family — 2026-10-24 #754.
Expand Down Expand Up @@ -57,3 +57,5 @@
| PH | 20,000 | 94.7% → 54.9% | 265 | 8,234 | 8,024 |

Measured on master at 598167f4 over the 2025 Public Data File's extract fetched 2026-10-27. Recompute: `local_agreement.read` per record of `local_corpora.load("orcid-researchers")` with `FIELDED["orcid-researchers"]`, under `Parser()` and under `Parser(policy=Policy(two_family_names=True))` with `_title_words` of the default lexicon, grouped by the record's `country`; a record is fixed where it agrees only with the switch, broken where it agrees only without it, and the last column counts the broken records whose `family` box holds one word. A corpus capped at 20,000 records a country shows each country's convention, not how many people write it.

- 2026-10-29 #754, sub-phase 4d — THE TWO UNITS THIS RULE READS REACH A VIEW: `surname2` holds the one written second, `family` keeping both as before, and the rule marks that unit's words where it takes its word; nothing it reads or declines moved. The view, the mark and their measurement are rules/R10 and decisions/R10's 2026-10-29 entry.
19 changes: 19 additions & 0 deletions docs/design/decisions/R10-second-surname-view.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# R10 — the second-surname view

## Standing

- `ParsedName.surname2` holds the family's second name unit where rules/O6 read the family as two, and is empty otherwise; it is a view, not a role, so `family`, `surnames`, `as_dict()`, `replace()`'s fields and the repr are unchanged, item 5 of decisions/vocabulary-model's 2026-10-28 #754 output-model entry — 2026-10-29 #754, sub-phase 4d.
- O6's mark is a private tag that assemble drops, carried out on `Reading.second_surname`, a field `Token.reading`'s readings gained and that is not compared, where the RFC had a stable tag: a published tag would make an O6 parse unequal to its own 2.x pickle — 2026-10-29 #754, sub-phase 4d.

Elsewhere: why the second surname is a view of `family` rather than a role, the ORCID measurements behind it, and the `surnames` redefinition declined, decisions/vocabulary-model's 2026-10-28 #754 entry, item 5; how O6 counts a family's words and which names it declines, decisions/O6.

Open: a view for the first unit and its name, `surname` standing close to the `surnames` that exists, with the per-unit particles and base CLDR's `surname-prefix` and `surname-core` would need (the RFC's render-pattern table and Left open); whether a family a connective or a hyphen joins (`Ortega y Gasset`, `García-López`) should read as two units here though O6 counts it as holding two already.

## Log

- 2026-10-29 #754, sub-phase 4d — IMPLEMENTED item 5 of decisions/vocabulary-model's 2026-10-28 #754 entry. What the implementation decided, beyond that entry:
1. **The mark is published on the reading, not as a stable tag.** The entry and the RFC's freeze table said O6 marks the second unit's start "with a new stable tag" joining `STABLE_TAGS`. `Token` compares its tags, so a tag on a published token makes `Parser(policy=Policy(two_family_names=True)).parse("Gabriel García Márquez")` unequal to the same parse pickled by 2.8.0, which 4b and 4c each kept equal by dropping their kind tags at assemble. So O6 writes `vocab:second-surname`, private, and assemble drops it and sets `Reading.second_surname` on the word's reading, the field added with `ADDED_IN` 3.0.0 and `compare=False` as `Reading.languages` was; `tests/v2/test_types.py` holds the 2.8.0 pickle equal (`parsedname_v28_two_family_names.pickle`). Weighed against: a `ParsedName` field holding the unit's start, which `replace()` would have to keep in step with the tokens it rewrites, where a reading is simply absent from a token `replace()` or `Parser.revise()` writes; and a `ReadingKind` for a second surname's words, which would overwrite the reading a particle or a patronymic there took.
2. **Every word of the second unit is marked, not only its first.** The view is then a filter over the family's tokens, with no walk from a start and no rule about where the unit ends: the words O3 folds into the family (rendered first) and a particle P6 attached to the first unit are simply unmarked. Given-first, the unit is the family O6 found, before it took the middle word; family-first, it is the unit O6 took after the family. The mark rides the copy that moves the taken unit into the family, so family-first parses pay no frame for it.
3. **The word keeps the reading it took**, the mark set on it, so a patronymic run in the second unit reads in both views (`Ali Hassan bin Ahmad` gives surname2 and patronymic `bin Ahmad`), as does a word P6's attached patronymic particle makes the parent's name (`Mohamad Ali Mahathir bin` under FAMILY_FIRST). Assemble reads the word as it reads without the mark, so no reading's kind, source or files moves: over 1,296 parses of 36 names under each order, with and without `middle_as_family`, `two_family_names` and the RU and TR_AZ packs, tokens, tags and readings match the base apart from the new field.
4. **The view stays out of `as_dict()`, `--json`, `comparison_key()` and the repr**, as R8's and R9's do, and is a `render()` key and an assertable field of a rule's example lines, whose grammar now takes a digit in a field name. The snapshot records it where it is not empty (`tools/differential/snapshot.py`'s `SURNAME2_VIEW`), which no row is, since the snapshot parses the default policy; `tools/differential/cldr_agreement.py` scores it on its own beside the merged `family`, as its PENDING_ROLES were written to.
- Measured 2026-10-29 on the 4d branch over `8a1e6f5f`. The committed snapshot's 6,601 readings of the base: no row moves and none gains the key; the header names `surname2` among the recorded views, and rules/R10's examples add two names, 6,603 readings in all. The agreement records gain only `surname2` entries, each a second surname stated and not read, the default never reading one: CLDR 117 (`cldr_agreement.json`, most of them the `foreignFull` samples' two-word second surnames, `González Domingo`) and 22 more lines in the locale record, Wikidata 330 (its `surname2-es` 150, `surname2-mx` 100, `surname2-co` 80), conventions 2. With `two_family_names` on and each rendering's order kept, the view agrees on 263 of Wikidata's 330 (es 97, mx 91, co 75), on 7 of CLDR's 117 and on neither conventions row, and `family` on 318 of Wikidata's 330 against 55 under the default. The switch's cost on the renderings stating none: of each source's renderings whose expected `surname2` is empty, it gives one to CLDR 700 of 2,523 (`Prof. dr. Cornelia Margarita Sofia Aletta van den Berg jnr., MBChB PhD` reads 'van den Berg'), 102 of them the 252 renderings of a name CLDR states a second surname for that leave it out, Wikidata 160 of 338 (its patronymic strata, `Любовь Митрофановна Шуляк` reading 'Шуляк'), GND 107 of 801, the native-script source 39 of 1,027 and conventions 15 of 28. Recompute: for each row of a fields file, parse its text with `cldr_agreement._parser(None, shape)` and with the same parser's policy given `two_family_names=True`; where `expected_roles(...)` states a `surname2`, compare `.surname2` and `.family` with it by `same`, and where it states none, count the renderings whose `.surname2` is not empty. Frames per parse (`sys.setprofile` call events, a warm parse): the reference row `Dr. Juan0001 de la Vega III` 310.1 and `Gabriel García Márquez` 258 on both trees under the default policy; under `two_family_names`, `Gabriel García Márquez` 284 to 286 and `Juan Pérez de la Vega` 330 to 334, one copy per marked word and one for the marking call, and `García Márquez Gabriel` under FAMILY_FIRST 291 on both. A first draft that rebuilt each marked token at the end of assemble, and read the word with the mark among its tags, cost 13 frames for one word and 35 for three.
1 change: 1 addition & 0 deletions docs/design/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ Entry conventions:
- [R7 — no initials in a script that writes none](R7-no-initials-in-a-script-that-writes-none.md)
- [R8 — the generation and credentials views](R8-generation-and-credentials-views.md)
- [R9 — the patronymic view](R9-patronymic-view.md)
- [R10 — the second-surname view](R10-second-surname-view.md)

## Construction & configuration diagnostics (D)

Expand Down
2 changes: 2 additions & 0 deletions docs/design/decisions/vocabulary-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ The vocabulary was a set of base sets plus marker subsets (`particles` and `part
- A caller configures vocabulary by reading, `Lexicon` deriving the sets and gates the build derives, and the compiled sets become internal in 3.0 — 2026-10-28 #754, the `Lexicon` surface (Derek).
- A `Locale` may carry CLDR-shaped render patterns, display and sorting, a later addition whose shape is agreed; how a call reaches one is open — 2026-10-28 #754, render patterns.
- 3.0's output model keeps the seven roles and adds every new field as a view, read from `Token.reading`, the reading each token took, or for `surname2` from rules/O6's tag: `generation` and `credentials` partitioning `suffix`, `patronymic`, and `surname2` where rules/O6 read two family names; `middle` and `surnames` keep their meaning, a compound given name being a rule a caller or pack turns on; the default `str()` renders a name in the order it was read in with its written separators in a normalized form, through `render(spec: str | None = None)`, `None` being that default; a language file's `[rules]` compile into its `Locale`'s policy patch — 2026-10-28 #754 (Derek).
- `surname2` is read from O6's mark as `Token.reading` publishes it, `Reading.second_surname`, a field not compared, and not from a published tag, which would make an O6 parse unequal to its 2.x pickle (decisions/R10) — 2026-10-29 #754, sub-phase 4d.
- Gender is recorded only where a word gives a definitive clue: a feminine form or one half of a fixed pair, never an unmarked noun — 2026-10-15/16; 2026-10-19 #750, title kinds.
- Title-phrase modifiers count as titles only beside a head title, at the head of the name only and never in a comma part (`Wing Lau` → given 'Wing', `John Smith, Vice` → title 'Vice'), save a modifier that may be a given name, which the head's test reads in every part (`Lau, Wing Kai` → given 'Wing'), rules/H9 — 2026-10-15/16; 2026-10-19 #750, title kinds; 2026-10-23 #752; 2026-10-25 #752, phase 2i; 2026-10-26 #752, phase 2j, `wing`.
- Name readings are hand-declared, and only on a word some language file claims as vocabulary or refuses, so the vocabulary carries no surname list (the segmentation lists, Korean and the ZH pack's, are outside the per-word model) — 2026-10-15/16; 2026-10-19.
Expand Down Expand Up @@ -321,3 +322,4 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names)
- Not touched: no `Lexicon` or `Parser` field, no code. The vocabulary README's example of an exclusion is now `duke`, wrong everywhere, where it was `père`.
- **2026-10-29 #754, sub-phase 4b — `generation` AND `credentials` LAND AS VIEWS PARTITIONING `suffix`, AND A LISTED SUFFIX WORD'S KIND IS READ FROM THE READING INDEX; NO FIELD OF ANY PARSE MOVES.** Carries out item 4 of the 2026-10-28 #754 output-model entry and closes 4a's `Jr`/`Sr` seam: `_vocab.suffix_kind` asks the reading index for the kind of a word tagged `vocab:suffix-word`, so `Token.reading` and `Ambiguity.alternative` read `Jr` as a generation through the one predicate. What the implementation decided, the measurement and its recipe are in decisions/R8's 2026-10-29 entry, and the rule is rules/R8.
- **2026-10-29 #754, sub-phase 4c — `patronymic` LANDS AS A VIEW, READ FROM A PATRONYMIC PARTICLE'S RUN AND FROM THE RU AND TR_AZ RULES WHEREVER THEY ARE ON; NO FIELD OF ANY PARSE MOVES.** Carries out item 3 of the 2026-10-28 #754 output-model entry: the particle and the words after it in its field read `patronymic`, a particle's kind read from the reading index at classify as 4b reads a suffix word's, and `ReadingKind.PATRONYMIC`, which 4a added and no reading took (the 2026-10-29 sub-phase 4a entry, item 5), is now taken. What the implementation decided, the measurement and its recipe are in decisions/R9's 2026-10-29 entry, and the rule is rules/R9.
- **2026-10-29 #754, sub-phase 4d — `surname2` LANDS AS A VIEW, THE FAMILY'S SECOND UNIT WHERE rules/O6 READ TWO; O6'S MARK IS PUBLISHED ON THE READING, NOT AS A STABLE TAG; NO FIELD OF ANY PARSE MOVES.** Carries out item 5 of the 2026-10-28 #754 output-model entry, and amends it and the RFC's freeze table on one point: the "new stable tag" is a private tag assemble drops, the mark reaching the published token as `Reading.second_surname`, a field added with `ADDED_IN` and not compared, since a published tag would make an O6 parse unequal to its 2.x pickle. The last of the views lane's four (4a to 4d). What the implementation decided, the measurement and its recipe are in decisions/R10's 2026-10-29 entry, and the rule is rules/R10.
2 changes: 1 addition & 1 deletion docs/design/mechanisms.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ This catalog converts discovery into a one-time cost — it does not remove disc
**How it works.** One lookup site means one answer: a stage that re-derived vocabulary facts could disagree with the stage before it. Stable tags ("particle", "conjunction", "initial") are API;
"vocab:"-namespaced ones are not.

**Lives in.** nameparser/_pipeline/_classify.py (producer), the internal tag constants in nameparser/_pipeline/_state.py (among them the kind tags `vocab:generation-word`, `vocab:credential-word` and `vocab:patronymic`, which assemble reads for `Token.reading` and drops from the published token); consumers throughout _group/_assign/_post_rules, and nameparser/_render.py, which is not a stage but reads the same tags to decide what a view shows (#458 moved case repair's conjunction test onto the tag; initials() had read them since 2.0); the v1 facade's `_token_is_conjunction`, a consumer from #528, went with the facade in #357.
**Lives in.** nameparser/_pipeline/_classify.py (producer), the internal tag constants in nameparser/_pipeline/_state.py (among them the kind tags `vocab:generation-word`, `vocab:credential-word` and `vocab:patronymic`, which assemble reads for `Token.reading` and drops from the published token, as it drops the marks post_rules writes for it, P6's `vocab:attached` and O6's `vocab:second-surname`, the latter carried out as `Reading.second_surname`); consumers throughout _group/_assign/_post_rules, and nameparser/_render.py, which is not a stage but reads the same tags to decide what a view shows (#458 moved case repair's conjunction test onto the tag; initials() had read them since 2.0); the v1 facade's `_token_is_conjunction`, a consumer from #528, went with the facade in #357.

**Reach for it when.** A stage is about to import Lexicon to ask about a word classify already saw — or a view is, which is the harder one to notice, since a view legitimately holds a Lexicon for the questions classify never answered; the view side is RENDER-HONORS-THE-PARSE, which owns it and covers the decisions this entry does not produce.

Expand Down
5 changes: 5 additions & 0 deletions docs/design/rules/O6-two-family-names.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@
```
"Juan Pérez de la Vega" two_family_names → family="Pérez de la Vega"
```
- **The family keeps the two as units**, the one written second being the second-surname view (R10), whichever of the two the rule took.
```
"Gabriel García Márquez" two_family_names → surname2="Márquez"
"García Márquez Gabriel" family-first+two_family_names → surname2="Márquez"
```
- **Titles, suffixes and nicknames are no name words**, and a comma that sets off only suffixes leaves the name before it to this rule.
```
"Gabriel García Márquez Jr." two_family_names → family="García Márquez"
Expand Down
Loading
Loading