Skip to content

Phase 4: order and shapes, the rules a pack can choose, and the output model #754

Open
0 of 6 issues completed

Description

@derek73

Part of #747. Mostly independent of phases 2 and 3.

Order versus shape (decided): a language supplies only a default order. The shapes each order accepts (where titles, suffixes, particles and commas sit) belong to that order, in the core.

The work:

Related questions are linked here as sub-issues.

Activity

  1. self-assigned this
    on Oct 19, 2026
  2. derek73 commented on Oct 23, 2026

    @derek73
    OwnerAuthor

    A ranking for the surname-span rule, from an unselected population

    The RFC's surname-span measurement (315 of 390 given-first double surnames disagreeing, GND and Wikidata) "was taken over corpora built to hold second surnames, so it shows that the gap exists, not how it ranks against the others". The local corpus psc-individuals (#849) can rank it. It holds Companies House's people with significant control, 1 in 8 of the snapshot dated 2026-10-22: 1,040,708 filed names, nobody chose them, and each filer typed the forename, middle name and surname into separate boxes.

    Size. 29,023 names (2.8%) have a two-word filed surname. 22,168 of them read with the surname's first word as a middle name (Ana García López → middle García, family López). It is the largest disagreement shape in the score.

    Who. Only 28% of the 22,168 (6,225) are Spanish, Portuguese or Latin American nationals. The rest are not:

    nationality records two-word surname read with a middle share of records
    British 680,873 10,236 7,688 1.1%
    Spanish 6,101 3,940 3,820 62.6%
    Indian 21,132 1,206 1,151 5.4%
    Portuguese 5,343 1,204 932 17.4%
    Italian 11,790 1,046 571 4.8%
    Moroccan 3,577 550 541 15.1%
    Pakistani 16,532 523 481 2.9%
    French 9,249 609 433 4.7%
    Brazilian 1,927 560 389 20.2%
    Sri Lankan 1,974 392 360 18.2%
    Mexican 405 271 262 64.7%
    Colombian 379 264 258 68.1%

    What it means for this phase.

    • Where an es pack would help: in Spanish, Mexican and Colombian names, nearly two in three names are misread. That is the strongest case in the data for an es pack's surname-span rule.
    • Where it would not: a rule chosen by a language pack reaches only callers who know the language, and most of these names come from nationalities no es/pt pack would be chosen for. Nationality is not a naming convention, so the British, Indian, Moroccan and Sri Lankan counts mix several conventions (South Asian two-word surnames are Tariq Abdul Aziz: should a two-word surname like Abdul Aziz stay whole after the given name? #694's question). Even so, they say the shape is not only Iberian.
    • What that leaves the default: the only general signal is the comma, and the RFC's own figure shows that index form disagrees on 0 of 60.

    Recompute. uv run python tools/differential/local_agreement.py psc-individuals reports the shape as middle 0->1, family 2->1 and middle 1->2, family 2->1. The per-nationality rows count the records in agreement.jsonl whose filed surname has two words and whose parsed family is its second word. The names stay in the cache; this comment prints counts only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

designvocabularyPer-word vocabulary model and language packs

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions