Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -681,7 +681,9 @@ listed below.
Slavic formal order (``EAST_SLAVIC``) or Turkic reversed order
(``TURKIC``) — but stands down under a declared
``FAMILY_FIRST`` or ``FAMILY_FIRST_GIVEN_LAST`` ``name_order``.
Defaults to empty.
Each rule also fills ``ParsedName.patronymic``, reading a middle
name with the East Slavic ending, and a Turkic marker with the
word before it. Defaults to empty.
* - ``middle_as_family``
- ``bool``
- Folds ``middle`` into ``family`` instead of splitting them —
Expand Down
25 changes: 25 additions & 0 deletions docs/design/decisions/R9-patronymic-view.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# R9 — the patronymic view

## Standing

- `ParsedName.patronymic` holds the words read as a patronymic, in whichever field the parse put them, which keep them; it is a view, not a role, so `family`, `middle`, `as_dict()`, `replace()`'s fields and the repr are unchanged, item 3 of decisions/vocabulary-model's 2026-10-28 #754 output-model entry — 2026-10-29 #754, sub-phase 4c.
- A patronymic particle and the words after it in its field all read `patronymic`, the particle by vocabulary and the rest by position, so the view is the tokens whose reading is `patronymic` — 2026-10-29 #754, sub-phase 4c.
- The East Slavic and Turkic rules write the reading wherever a policy turns them on, not only where their rotations fire: a middle word carrying the ending, and a marker with the name word before it — 2026-10-29 #754, sub-phase 4c.

Elsewhere: why the view leaves `family` and `middle` as they are, and the fielded-corpora measurements behind it, decisions/vocabulary-model's 2026-10-28 #754 entry, item 3; the rotations the two ending rules are named for, decisions/O1 and decisions/O2; the `clan`/`patronymic` line in the particle kinds, decisions/vocabulary-model's 2026-10-19 #749 entry.

Open: the Icelandic ending rule, which an `is` pack would turn on (the RFC's work order: 3b, 4c and 4f); whether the East Slavic endings should take the Ukrainian `-ivna`/`-ївна` and `-ovych`/`-ович` spellings, which would move O1's rotation too.

## Log

- 2026-10-29 #754, sub-phase 4c — IMPLEMENTED item 3 of decisions/vocabulary-model's 2026-10-28 #754 entry. What the implementation decided, beyond that entry:
1. **Every word of a particle's patronymic reads `patronymic`.** The entry left open whether the particle reads `patronymic`, or only the name after it; the view's text is `bin Mohamad`, so a view over one reading kind needs both, and `ReadingKind.PATRONYMIC`'s docstring already gave "bin Mohamad" as the words it names. The particle's reading is the vocabulary's (`vocabulary`, its files `ar ms` for `bin`), each word after it `position`: what makes it a patronymic is where it stands, after the particle, so it names no file, and its own reading (a name word, a bound given name, a chained particle) is not kept. Weighed against: the particle alone reading `patronymic` and the view adding the words after it, which would put a run-walk in the view and make `Token.reading` say `name` for a word the view counts as the parent's name.
2. **The run is the field, read at assemble.** The words after the particle in its field, a nickname skipped and a suffix ending it, measured over the final roles rather than group's pieces, since P1's fold and P6's attachment move words after group and a leading never-given particle stays a lone piece for the fold to find. So the run is what the family or middle holds after the particle: `ibn Ali Hasan` gives `ibn Ali Hasan` by default and `ibn Ali` under a family-first order, as the family does. Under `middle_as_family` the run follows the field the word stands as, so a folded middle's run stops at the family's own words.
3. **A particle names a parent only with a word after it**, or where P6 attached it after a family comma or a family-first order to the family written before it, whose words then join the run (`Mohamad, Mahathir bin` gives `bin Mohamad` and `Bakar, Ali bin Abu` `bin Abu Bakar`, written in the family's order since P6's words carry the folded mark the family writes first). P6 records the attachment on the words it attaches (`vocab:attached`, beside the folded mark, at both of its sites), which assemble reads and drops as it drops the kind tag, so it is read, not inferred: O3's middle fold writes the same folded mark, and two inferences from shape each lost a case (item 8). The first draft read every patronymic particle read as a particle, and the snapshot showed `Fritz Freiherr und von Bar` reading patronymic `Bar`: the German family name spelled as Hebrew `bar`, ending its field, which group's chain had left a particle.
4. **The kind is read at classify from the reading index**, as R8's suffix kind is: `particles` compiles every particle kind alike, so classify writes `vocab:patronymic` beside `particle` for a word the shipped files hold as a `patronymic` particle (`_vocab.is_patronymic_particle`, a memo over `_reading_index.holds_kind`), whatever the reading's default, and assemble drops the tag from the published token, a published tag making a name unequal to its 2.x pickle. Every patronymic particle has that kind only (`bar`, `bat`, `bin`, `bint`, `binte`, `binti`, `ibn` and the five native-script forms, 2026-10-29), so the lookup asks for the kind, not the first particle row. A word only a caller's `Lexicon` lists is a plain particle, `ben` included, which `ar.toml` excludes as a patronymic.
5. **The report's alternative moves with the reading.** A leading patronymic particle read as a given name names `patronymic as family` where it named `particle as family` (`bin Mohamad`), through `_vocab.particle_reading`, which both of assign's particle alternatives now ask, so `Token.reading` and `Ambiguity.alternative` agree, as R8's 4b made them for `Jr`.
6. **The ending rules read a word, not only a rotation.** O1 and O2 fire on exactly three and four name words; the patronymic is written in more shapes than that (`Ivanov, Ivan Ivanovich`, `Ivan Ivanovich Ivanov`, which the rotation leaves alone), so the East Slavic arm reads a middle word carrying O1's ending, and the Turkic arm a marker in a name field with the name word before it in its comma part, wherever the rule is on. The middle is the field a patronymic stands in after a given name, and a family word carrying the ending is a surname as often as not (`Roman Abramovich`). The patterns moved from `_post_rules` to `_vocab` as `has_patronymic_ending` and `is_patronymic_marker`, which the rotations and assemble both ask (mechanisms.md#ONE-PREDICATE-PER-QUESTION), the packs' hand copies pinned against them there.
7. **The view stays out of `as_dict()`, `--json`, `comparison_key()` and the repr**, as R8's do, and is a `render()` key and an assertable field of a rule's example lines. The snapshot records it where it is not empty (`tools/differential/snapshot.py`'s `PATRONYMIC_VIEW`, a `RECORDED_LATER` key), and `tools/differential/cldr_agreement.py` scores it on its own beside the merged `given_names`, as its PENDING_ROLES were written to once `ParsedName` had it.
8. **Review fixes (2026-10-29, the orchestrating review of #948).** (a) P6's attachment and O3's middle fold both write the folded mark, so under `middle_as_family` the first draft read the attached particle's field as the middle and dropped the exception (`Mohamad, Mahathir bin` and `Levi, Omer Bar` read no patronymic): the fix told the attachment by its shape, a folded family particle with nothing after it in its field behind a family word not folded. The second review found that shape wrong both ways: a particle P6 attached with another after it (`Bakar, Ali bin Abu`, a real naming pattern) opened a plain run and lost the family, reading 'bin Abu', under the default policy; and after a family comma, where the family is written first, a trailing particle P6 declines (no given word ahead of it) and O3 folds read as attached (`Rahman, Al bin` under `middle_as_family` gave 'bin Rahman'). P6 now marks what it attaches (item 3), and the field of a word P6 attached is the family under the fold, so a folded middle's run stops short of it (`Ali, Ahmad bin Ahmad Abu`). The third review found that the mark alone had dropped one condition of the shape test, a family word not folded for the attachment to name: behind an empty head (rules/C15) P6 attaches the particle as the whole family, which R2 makes name words without the fold and O3's fold gives a base, so `, Siti Aminah binti` under `middle_as_family` read patronymic 'binti'; an attached particle now names a parent only where the family holds a word P6 and O3 did not fold, and `test_middle_as_family_moves_no_patronymic_r2_does_not_move`, whose comparison skips a shape whose R2 marks differ, says that it cannot see these shapes and leaves them to its sibling's rows. (b) A Turkic marker a caller's `Lexicon` lists as a connective kept that reading while the word before it read `patronymic` (`Ali Ahmad Vali oglu Hasan` gave patronymic 'Vali'): the father is now read only once its marker reads as a name word, built once its marker is read, the token held until then. (c) The Turkic arm read a name field only, while the particle arm reads any: a marker in the maiden name now reads with the maiden word before it (`Leyla née Rasim qizi` gives 'Rasim qizi', `Ali née oglu` none). (d) The user docs said the ending rules fill the view under every order; the East Slavic arm reads only a middle, so it follows the order (`Ivan Ivanovich Ivanov` under FAMILY_FIRST reads given `Ivanovich` and none). (e) Each clause the review found no test for now has one that fails without it: the field test of the run's opener (`Ali bin, Ali`), the folded word left out of P6's family (`Ahmad Ahmad de bin` under FAMILY_FIRST), the suffix ending a run (`bin Ahmad Jr. Hasan, Ali`, whose family resumes behind the suffix) and the name-word test of the marker (the connective marker above); a clause requiring the attached particle to end the family was implied by the others and is gone. The first fix moved the run's field computation into a helper and cost a frame per token, seven on the reference row, caught by re-taking the frames below; it is inline again.
9. **Accepted after the second review.** R2 reads a name part made only of particles as name words, and O3's fold can give such a part a base, so its particle reads as one under the fold alone (`Ali, bin bin Abu` gives 'bin Abu' under `middle_as_family`, none without); `test_middle_as_family_moves_no_patronymic_r2_does_not_move` holds the two policies to the same patronymic words wherever R2's marks agree, 4,494 of 5,080 parses of its grid. Two Turkic patronymics, one in the name and one in the maiden name, are one view (`Leyla Vali qizi née Rasim qizi`), as two nasab particles are.
- Measured 2026-10-29 on the 4c branch over `a9884bfb`. The committed snapshot's 6,570 readings of the base: 23 gain a `patronymic` key, every one a patronymic particle's run (the snapshot parses no pack), and one row's alternative moves (`bin Stromae Elizabeth Windsor Maria Teresa MP` under FAMILY_FIRST, `particle as family` to `patronymic as family`); no field moves; rules/R9's examples add 31 names, 6,601 readings in all. Recompute: compare each row of `snapshot.jsonl` with its base row, `patronymic` and `alternatives` set apart. The agreement records gain only `patronymic` entries, each a patronymic stated and not read, none read where the source states none or another: Wikidata 334 (its `patronymic-ru` 126, `patronymic-ri` 41, `patronymic-su` 62, `patronymic-is` 100 and `patronymic-my` 5), the native-script source 17 (`uk-ua` 14, `kk-kz` 3), conventions 6. Under the RU pack, Wikidata's `patronymic-ru` agrees 126 of 126, `patronymic-ri` 41 of 41 and `patronymic-su` 61 of 62, the miss `Оксана Сергіївна Баюл`'s Ukrainian ending; `patronymic-my` agrees 4 of 9 in the default, the five misses written without `bin`. Recompute: parse each `wikidata_fields.jsonl` row stating a patronymic with `cldr_agreement.parse_rendering`, with and without `RU`, and compare `got_roles(...)["patronymic"]` with `expected_roles(...)["patronymic"]` by `same`. Frames per parse (`sys.setprofile` call events, a warm parse), re-taken after the second review's fixes: the reference row `Dr. Juan Q. Xavier de la Vega III` 369 and `John Smith` 153 on both trees under the default policy; a patronymic particle adds two (`Mahathir bin Mohamad` 217 to 219), P6's attached shape nine (`Mohamad, Mahathir bin` 263 to 272, `Bakar, Ali bin Abu` 289 to 298, its family's tokens rebuilt; the look for a family word not folded is a loop, a generator there costing two more and one per word behind an empty head). The opt-in rules pay per word, which is frame question (1) of root AGENTS.md, a cost that scales with the input, recorded here and guarded by the `patronymic_rules` row of `tests/v2/test_benchmark.py`'s `_POLICY_SHAPES`: RU one per middle word read as a name (`Ivan Ivanovich Ivanov` 211 to 213, `John Paul George Ringo Smith` 270 to 273, the reference row 369 to 370), TR_AZ one per parse (`_turkic_patronymics`), one per word in a name field or the maiden name (`is_patronymic_marker`, which O2's rotation also asks once of the last word), and two per marker found (`comma_bucket`), the father's token built once (`John Smith` 153 to 156, the reference row 369 to 376, `Ali Ahmad Vali oglu` 243 to 251: one, four in the arm and one in O2, two). The ending and marker tests are `_vocab`'s shared predicates rather than an inline pre-check, the rules being opt-in and their per-word cost one frame.
1 change: 1 addition & 0 deletions docs/design/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ Entry conventions:
- [R6 — the default rendering follows the written name](R6-default-rendering-follows-the-written-name.md)
- [R7 — no initials in a script that writes none](R7-no-initials-in-a-script-that-writes-none.md)
- [R8 — the generation and credentials views](R8-generation-and-credentials-views.md)
- [R9 — the patronymic view](R9-patronymic-view.md)

## Construction & configuration diagnostics (D)

Expand Down
3 changes: 2 additions & 1 deletion docs/design/decisions/vocabulary-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The vocabulary was a set of base sets plus marker subsets (`particles` and `part
- A language file holds the words the parser might meet in names written in that language, immigrant particles included, not the words of its etymology, and a word sits in the file of every language whose names it occurs in, a Latin transliteration in its source language's file; `mul.toml` holds the words shared on purpose instead (the generations `i` to `v` and `2`, `&`, and the European titles below) — 2026-10-15/16; 2026-10-18 #737; 2026-10-19 #749; 2026-10-19 #750, title kinds.
- The default loads every language and keeps working for names of any language. Choosing a language is to narrow it in phase 3 (#753): a pack is to settle only the clashes where its language has a vocabulary reading, a strict pack to drop other languages' name-level readings too, and a pack's preferred reading to be able to outrank one a shape rule derives — 2026-10-15/16.
- `e`, `y` and `i` stay in the default; choosing a language settles their one-case fork for that language's own reading, `may_be_initial` not holding on a chosen file's conjunction, which no shipped pack does yet — 2026-10-15/16; 2026-10-29 #753, phase 3b.
- A patronymic field is to come in 3.0, as a view that names its words without emptying `family`, and the particle kinds already draw the `clan`/`patronymic` line for it: a patronymic is followed by a parent's name, a clan prefix by a family name the lineage shares — 2026-10-15/16; 2026-10-19 #749; 2026-10-28 #754 (Derek).
- A patronymic field is to come in 3.0, as a view that names its words without emptying `family`, and the particle kinds already draw the `clan`/`patronymic` line for it: a patronymic is followed by a parent's name, a clan prefix by a family name the lineage shares — 2026-10-15/16; 2026-10-19 #749; 2026-10-28 #754 (Derek); landed as rules/R9, 2026-10-29 #754, sub-phase 4c.
- A language pack is a choice of languages on `Locale.languages` selecting which readings compile, a tag choosing its own file and its ancestors', and a reading index, loaded on its first lookup, carries the languages to run time; strict waits for 3i, and third-party language files are deferred — 2026-10-27 #753 (Derek); 2026-10-29 #753, phase 3a (Derek); 2026-10-29 #753, phase 3b.
- A choice is matched without regard to case, a tag whose file does not exist selects its ancestors' (`pt-PT` selects `pt`), `mul` and `und` are no language to choose, and a reading turned on also prefers, so `de`'s generation `i` drops `ca`'s and `pl`'s conjunction under a `de` choice; a choice is one call's: one chained through `base=` applies on top of the base's rather than unioning with it, and a choice drops the caller's own base entries it removes — 2026-10-29 #753, phase 3b; 2026-10-29 #753, phase 3b, chained choices (Derek).
- A reading right in its own language and wrong only as a default, because another language written in the same script collides with it, is written `default = false` in its file, never excluded, so that after the packs an exclusion means wrong everywhere: cs `st` since phase 2h, and cs `ml`, hr, sk and sl `ml` and `st` and fr `père` since 3c, the rest in their packs' PRs — 2026-10-27 #753 (Derek); 2026-10-29 #753, phase 3c.
Expand Down Expand Up @@ -320,3 +320,4 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names)
- **Open for 3f: a turned-on `ml` reads a lone comma part as the suffix, silently.** `ml` has no name reading, so under a choice it is an ungated suffix word: `Novak, ML` reads suffix `ML` with nothing reported, where the default reads given `ML`, and `Novak ml., Anna` family `Novak`, suffix `ml.`; the gated `st` reports `SUFFIX_OR_NAME`, and fr `Jean Père` reports `GIVEN_OR_FAMILY` (suffix `Père`). The initial pairs `Anna M. L. Novak` and `Anna ML Novak` do not move. Pinned in tests/v2/test_locales.py; whether `ml` takes a name reading or a gate is 3f's. So is the one-letter gap: es `h` and de `i` turned on stay inert written with a period (`Juan Pérez H.` keeps family `H.` under `es`, `Wilhelm Karl I.` family `I.` under `de`), a strict expected failure there.
- Not touched: no `Lexicon` or `Parser` field, no code. The vocabulary README's example of an exclusion is now `duke`, wrong everywhere, where it was `père`.
- **2026-10-29 #754, sub-phase 4b — `generation` AND `credentials` LAND AS VIEWS PARTITIONING `suffix`, AND A LISTED SUFFIX WORD'S KIND IS READ FROM THE READING INDEX; NO FIELD OF ANY PARSE MOVES.** Carries out item 4 of the 2026-10-28 #754 output-model entry and closes 4a's `Jr`/`Sr` seam: `_vocab.suffix_kind` asks the reading index for the kind of a word tagged `vocab:suffix-word`, so `Token.reading` and `Ambiguity.alternative` read `Jr` as a generation through the one predicate. What the implementation decided, the measurement and its recipe are in decisions/R8's 2026-10-29 entry, and the rule is rules/R8.
- **2026-10-29 #754, sub-phase 4c — `patronymic` LANDS AS A VIEW, READ FROM A PATRONYMIC PARTICLE'S RUN AND FROM THE RU AND TR_AZ RULES WHEREVER THEY ARE ON; NO FIELD OF ANY PARSE MOVES.** Carries out item 3 of the 2026-10-28 #754 output-model entry: the particle and the words after it in its field read `patronymic`, a particle's kind read from the reading index at classify as 4b reads a suffix word's, and `ReadingKind.PATRONYMIC`, which 4a added and no reading took (the 2026-10-29 sub-phase 4a entry, item 5), is now taken. What the implementation decided, the measurement and its recipe are in decisions/R9's 2026-10-29 entry, and the rule is rules/R9.
Loading
Loading