Add locales.NE, the first language pack: a choice of ne (3.0, 3d)

README.rst

@@ -82,7 +82,7 @@ Quick Start Example
     >>> russian.parse("Сидоров Иван Петрович").family
     'Сидоров'
     >>> locales.available()
-    ('ja', 'ru', 'tr_az', 'zh')
+    ('ja', 'ne', 'ru', 'tr_az', 'zh')
 
 Learn more
 ----------

docs/design/decisions/deviates-registry.md

@@ -2,11 +2,12 @@
 
 ## Standing
 
-- A locale pack declares its deviations from the defaults as a module-level `DEVIATES` callable, acceptance metadata kept off `Locale`, whose value equality and pickling a callable field would break; `test_registry_is_the_pack_contract` fails unless every registered pack ships `DEVIATES` and a rotator list, which is what lets mechanisms.md#LOCALE-PACKS-PURE-DATA promise packs no code paths of their own. A field on `Locale` is to be revisited only if a runtime consumer of the declaration, such as an `explain()`, appears, and none has — 2026-07-18. A vocabulary pack's deviations are derivable instead, from the reading index (`_reading_index.changed_words`), which tests/v2/test_locales.py's choice gate reads; the contract test still asks every registered pack for `DEVIATES` until the PR registering the first pack whose content is a choice of languages (3d) teaches it to accept the derived surface — 2026-10-29 #753, phase 3b.
+- A locale pack declares its deviations from the defaults as a module-level `DEVIATES` callable, acceptance metadata kept off `Locale`, whose value equality and pickling a callable field would break; `test_registry_is_the_pack_contract` fails unless every registered pack ships `DEVIATES` and a rotator list, which is what lets mechanisms.md#LOCALE-PACKS-PURE-DATA promise packs no code paths of their own. A field on `Locale` is to be revisited only if a runtime consumer of the declaration, such as an `explain()`, appears, and none has — 2026-07-18. A vocabulary pack's deviations are derivable instead, from the reading index (`_reading_index.changed_words`), which tests/v2/test_locales.py's choice gate reads; the gate ors the derived surface into every registered pack with `languages`, so a hand `DEVIATES` declares only what a fragment or a patch beside a choice changes; the contract test refuses a `DEVIATES` on a pack that is nothing but a choice of languages (`locales.NE`, the first) and requires one of every other pack — 2026-10-29 #753, phase 3b; 2026-10-30 #753, phase 3d.
 
 Open: none in this file.
 
 ## Log
 
 - 2026-07-18 (d4aaafa; the DEVIATES design note) — a `deviates` field ON Locale was rejected: callables break value equality and pickle-by-reference, and DEVIATES is acceptance metadata, not runtime behavior (revisit only if a runtime consumer like explain() appears). The REGISTRY became the contract instead: test_registry_is_the_pack_contract fails structurally unless every registered pack ships DEVIATES and its rotator list. This is why LOCALE-PACKS-PURE-DATA can promise "no code paths of their own" and mean it.
 - 2026-10-29 #753, phase 3b — A VOCABULARY PACK'S DEVIATIONS ARE DERIVED, NOT DECLARED. `Locale.languages` selects among the default's readings, so the words a choice can move are computable: `_reading_index.changed_words(chosen)` is every word joining or leaving a compiled constant under the choice, and tests/v2/test_locales.py's `test_non_interference_each_choice` requires the default's reading for every corpus name holding none of them, with a negative control (`test_the_choice_gate_fails_when_its_surface_misses_a_word`). No pack ships in 3b, so the gate runs over choices of the files' tags; the registry contract test still asks every REGISTERED pack for `DEVIATES` and a rotator list, and learns to read a derived surface for a pack with `languages` in the PR that registers the first one (3d). The policy and segmentation packs (JA, RU, TR_AZ, ZH) keep theirs, decisions/vocabulary-model's 2026-10-27 #753 entry, item 7.
+- 2026-10-30 #753, phase 3d — THE CONTRACT READS A DERIVED SURFACE. `locales.NE` registers the first pack whose content is a choice of languages and nothing else (`Locale.lexicon` empty, `Locale.policy` the empty patch). tests/v2/test_locales.py's `_deviates(code)` gives the gate, the rotator test and the combined gate every pack's predicate: the surface derived from its `languages` through `changed_words` and the 3b choice gate's `_surface_deviates`, or'd with its `DEVIATES`, so a pack with a fragment or a patch beside a choice needs to declare only that part (`test_a_pack_with_a_choice_and_a_patch_declares_only_the_patch`; nothing refuses a hand copy of the choice's words there); `test_registry_is_the_pack_contract` refuses a `DEVIATES` on a pack that is nothing but its choice, so a hand copy cannot drift beside the derived one, and still requires one of every other pack. Its rotator list stays required, as the gate's positive side. decisions/vocabulary-model's 2026-10-30 phase 3d entry, item 2.

docs/design/decisions/vocabulary-model.md

@@ -333,3 +333,12 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names)
   7. **ZH and JA keep their hand-written patches**, reversing in part the same clause of item 10 for ZH's surname list, which stays a Python list in its pack rather than becoming `[rules]` data. Both turn on `segment_scripts`, which the table could express as a set of script names, but neither turn-on stands alone: ZH's segmentation reads its surname list, a Python list outside the per-word model, and JA's reads the caller's segmenter, the one code hook, so a `[rules]` table would split each pack's behavior across a file and a module without making either pure data. ZH's moves when its surname list has a home in the vocabulary; JA's stays as it is, its turn-on serving the segmenter, the one code hook a pack keeps.
   - **Measured unchanged** 2026-10-29 on `23aacddc` plus this change, Python 3.11: `locales.RU` and `locales.TR_AZ` compare equal to the `Locale`s written by hand before, their rules `PatronymicRule` members (tests/v2/test_locales.py's `test_rules_packs_equal_their_hand_written_patches`); pickles of both, and of `parser_for(RU)`, `parser_for(TR_AZ)` and `parser_for(RU, TR_AZ)`, made under the 2.8.0 wheel and under `23aacddc`, load in this tree equal to its own and parse alike, and under `PYTHONHASHSEED=0` this tree's pickles of all five are byte-identical to `23aacddc`'s; the full suite, the snapshot and the locale agreement records (`cldr_agreement_locales.json`, `wikidata_native_agreement_locales.json`) do not move. Import, `nameparser` already imported in each case (medians of 25 fresh processes, each `python -c "import time, nameparser; t = time.perf_counter(); import nameparser.locales as L; L.RU; print(time.perf_counter() - t)"`, dropping `; L.RU` for the first figure): `import nameparser.locales` 0.23 ms before and 0.24 after; the first access of `locales.RU` or `locales.TR_AZ` 0.50 ms before and 0.69 after, which the review measured as 0.27 → 0.46 ms and 0.31 → 0.45 ms on other runs, about 0.2 ms each time, the import of `config/_rules.py`; packs still load lazily. A cold `import nameparser` is about 47 ms, unchanged. The default parse runs no new code, `rules_patch` being called once per pack at its import.
   - **Open:** #754, deriving `DEVIATES` from the rules (item 6), with 4g; #753, ZH's turn-on (item 7).
+- **2026-10-30 #753, phase 3d — THE FIRST LANGUAGE PACK, `locales.NE`: A CHOICE OF `ne` TURNS ON `given` READINGS OF `श्री`, `guru` AND `गुरु`, SO EACH IS A TITLE ONLY INSIDE A TITLE PHRASE AND ELSEWHERE A NAME WORD PLACED BY POSITION; THE DEFAULT DOES NOT MOVE.** Carries out 3d of item 11 of the 2026-10-27 #753 entry, the route the RFC's "Name readings in a pack" measured: a name reading turned on derives the marker any name reading derives (`TITLES_AMBIGUOUS`, rules/H9), and decision 15 holds inside the pack.
+  1. **The readings are in `ne.toml`, each `given = { default = false, note = … }`.** A chosen tag selects its own file and its ancestors' (the phase 3b entry), and `ne` has none, so `languages=("ne",)` reaches only `ne.toml`; a `given` reading in `hi.toml` or `bn.toml` would be turned on by a Hindi or Bengali choice instead, which is 3e's to weigh. The title readings stay where they were (`श्री` in `hi.toml` and `mr.toml`, `guru` in `bn.toml`, `hi.toml` and `pa.toml`, `गुरु` in `hi.toml`), and a name reading never prefers (`choose`'s "Name readings never leave and never settle anything"), so the choice keeps them and the pair derives the marker. `hi.toml` is untouched: its `श्री` note, which says a `given` reading there would make the title one only inside a title phrase, stays true of the default. A count on each reading is in its `note`, there being no count key for a corpus of pairs.
+  2. **`locales.NE` is nothing but its choice** (`Locale(code="ne", lexicon=Lexicon.empty(), languages=frozenset({"ne"}))`), registered in `locales/__init__.py`, and it **declares no `DEVIATES`**: mechanisms.md#LOCALE-PACKS-PURE-DATA says a vocabulary pack's deviations are derived, and a pack module may not import `_reading_index` (layering). tests/v2/test_locales.py's `_deviates` ors the surface derived from `changed_words` (`guru`, `गुरु`, `श्री` for NE, pinned by `test_ne_pack_contents`, through the 3b gate's `_surface_deviates`) into the predicate of EVERY pack with `languages`, so a hand `DEVIATES` needs to declare only what a fragment or a patch beside a choice changes, the choice's words no longer needing a hand copy (nothing refuses one that restates them) (`test_a_pack_with_a_choice_and_a_patch_declares_only_the_patch`); the registry contract refuses a `DEVIATES` on a pack that is nothing but its choice and requires one of every other pack. `snapshot.py --languages` and `alike --languages` read the choice alone, which is the whole of NE but not of a pack with a fragment or a patch beside its choice; both say so where they are described. NE's rotators cover each word in each script, at the head, behind a listed title and after a family comma. This is the pattern 3e to 3h copy where a pack is a choice alone.
+  3. **`local_corpora.py alike` takes `--languages`**, as `snapshot.py` has since 3b: the flags under the choice, against the default's flags (the tree's own, or `--base`'s), so the RFC's "Measuring a pack" pair measure is one command. A choice is read by `_parse(languages)`, which `surname` and `_read` now ask, and which joins `_ALIKE_CODE`, so the change re-keys every kept `alike` base once.
+  - **The default does not move.** `build.py` reports `_compiled.py` current, only `_index.py` gaining the three rows; tests/v2/test_snapshot.py passes with no re-record.
+  - **What `ne` moves,** measured 2026-10-30 on `40675146` plus this change, Python 3.11. Committed corpora (`snapshot.py --languages ne`): 6 entries, 4 of them held only by `corpus_cldr.jsonl`. Those four are CLDR's `hi`, `kok` and `mr` renderings opening with the salutation (`श्री ललित कुमार शर्मा एम॰डी॰`, CLDR title `श्री`), where `श्री` leaves the title for a name word placed by position: given in that one, family in the other three (`श्री शर्मा ललित कुमार, एम॰डी॰`, a family-first rendering, → family `श्री शर्मा ललित कुमार`), one of which, an `mr` rendering whose title was `श्री. एम.`, keeps `एम.`. The choice reads Hindi, Konkani and Marathi names as Nepali, the cost the user docs state. The other two are `corpus_issues.jsonl`'s `Guru Gobind Singh` and `Guru Nanak`, read given `Guru` where the Sikh Gurus' `Guru` is the title; decisions/indic-honorifics' accepted cost (`Guru Dutt` reads title `Guru`) is lifted under the choice and paid the other way. CLDR agreement per pack (`cldr_agreement.py`, `ne` mapped to NE in `LOCALE_FILES` and the `ne-np` stratum in `NATIVE_LABELS`): `ne` 15/24 with and without the pack, Wikidata's `ne-np` 10/10 both ways, neither file holding one of the words; the record gains NE's rows, unchanged. nepal-candidates (`snapshot.py --local nepal-candidates --base <this checkout> --languages ne`): 296 of its names move, all by the word leaving the title for the given name in the default order (a lone word after it going from given to family: `गुरु चौधरी`), 209 opening `श्री`, 42 `Guru`, 41 `गुरु`, 1 `GURU`, 1 `श्री.`, and 2 the glued `सु.श्री`; under either family-first order the word is placed as the family name instead (`श्री राम बहादुर थापा` → family `श्री`, given `राम` under `FAMILY_FIRST`, given `थापा` and middle `राम बहादुर` under `FAMILY_FIRST_GIVEN_LAST`). Pairs (`local_corpora.py alike nepal-candidates --languages ne`): 125,833 of 126,981 read alike under the default, 126,040 under `ne`, 211 now alike and 4 now apart. The 211 all hold `श्री` (4 `गुरु` too), the romanization writing it as a name word (Shree 191, Shri 6, Sri 2); the `Guru` and `गुरु` pairs move together and stay alike. Of the 4 apart, 3 romanize `श्री` as `Mr.` (`Mr. Bal Bahadur Singh`), which the romanization reads as the salutation and the pack does not, and 1 is the defect below. The RFC's 2026-10-27 figures (123,440 → 123,636, 198 fixed, 2 broken) were measured on `cfe8bf92` by roles per word; this measure is `alike`'s structure, on a later tree.
+  - **Not a rule change.** rules/H9's When already names "a title word that is also borne as a given name", and the choice adds three words to that class without moving its limits: `His Holiness Guru Prasad` keeps title `His Holiness Guru` (a style partner), `Swami Guru Prasad` reads title `Swami`, given `Guru` (H9's Accepted, a religious title behind any title but a style), and `Guru. Prasad Sharma` keeps title `Guru.` by rules/H2's shape, while `श्री. कृष्ण श्रेष्ठ` reads given `श्री.`, H2's shape never reading a Devanagari word (its Accepted clause on abugidas). Pinned in tests/v2/test_locales.py.
+  - **Open: a period-joined token whose only title chunk the choice makes possibly given reads as a name word, in both directions.** `_vocab.period_joined_vocab` makes a token a title where ANY chunk is in `titles`, and under `ne` `श्री` has moved to `titles_ambiguous`. That costs `सु.श्री.` (सुश्री, Ms., written with its periods), title under the default and given under `ne`, 2 nepal-candidates names; and it fixes a given name glued behind the word, `श्री.राम थापा`, title `श्री.राम` under the default and given `श्री.राम` under `ne`. Whether a possibly-given chunk counts there is open; a fix would move the default for every word of `TITLES_AMBIGUOUS`, and its grid holds both `सु.श्री.` and `श्री.X`.
+  - **Recompute:** the three commands above from a checkout of this change; the CLDR record with `cldr_agreement.py`; the word counts with `tools/name_usage.py guru गुरु श्री`.