Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -877,6 +877,9 @@ Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py):
- 2026-09-28 (Derek), #544 — A PART READ WHOLLY AS SUFFIXES REPORTS NO NAME READING OF ITS WORDS. group's particle chain runs over every comma segment, and its two emitters — `particle-or-given` when a particle behind a word of both the title and the particle vocabulary chains (since 2.0.0, de264af1) and `suffix-or-name` when the chain takes an ambiguous acronym into the name (#289/#516, 59d8f38a, in no release) — reported inside a TAIL segment, which assign reads wholly as suffixes. Each such report named a reading the parse never made, against rules.md#A1's "A report names the reading the parse took". A family comma's tail was already silent, since group hands the chain no report list anywhere after a family comma; the suffix comma's tails were not. Measured on the released wheels: `John Smith, Jr., Freiherr von Richthofen` reports `particle-or-given` on 'von', a token in the suffix role, at 2.0.0, 2.1.0, 2.2.0 and 2.3.0; `John Smith, Jr., PhD van Ma` and `John Smith, Jr., PhD Do Ma` report it on 'van' and 'Do' at 2.0.0 and 2.1.0 only. The `suffix-or-name` half reached `John Smith, Jr., PhD Do Ma`, `John Smith, MA, PhD Do Ma` and `John Smith, Jr., PhD van Ma` on master (e10e83b4), and the run rule of the bullet above made it reachable behind ONE comma: `John Smith, PhD Do Ma` carried C1's flip and a second report on 'Ma'. FIXED by scope, not by a new test at the emitter: group passes the chain no report list in a tail segment either, so both emitters go quiet there together, and rules.md#C2 states the boundary for any part consumed wholly as suffixes. The maiden channel is a separate parameter and is untouched: a tail segment's reader is NONE, so the maiden walk reports nothing there to begin with. No mechanisms.md entry: this is AMBIGUITY-AT-THE-DECISION-SITE's own contract (a report fires only where the parse chose between live readings) applied to a stage whose reading a later stage overrides for the whole segment. MEASURED 2026-09-28, the tree against the same tree with `None if family_comma else ambiguities` restored in `group()` (the comparator), each parse recorded as its seven fields plus `(kind, [(token text, token role)])` per report, under all three name orders: 0 of the 1441 differential-corpus names move, so the gate has nothing to classify; over a comma grid — the prefixes `John Smith, `, `John Smith, Jr., ` and `Smith, John, ` times every run of one to three words drawn with repetition from {PhD, MA, Ma, Do, van, de, Jr, MEng, Ed, y, i}, each text as written, lowercased and uppercased, deduplicated to 11,049 texts — 360 parses (120 texts, every one of them mixed case) lose one `suffix-or-name` report apiece, every removed report on a token in the suffix role, 0 reports added, 0 field moves. 21 of the 120 texts carry one comma and are the run rule's reach; the other 99 carry two and moved the same way (297 parses) when the same one-line change was applied to master e10e83b4; the `Smith, John, ` prefix moves nothing, being a family comma. The grid holds no word of both the title and the particle vocabulary, so the `particle-or-given` half is witnessed by the case row alone. Pinned by the case rows `a_credential_run_after_the_comma_reports_no_chain_fork` and `a_part_past_the_second_reports_no_particle_fork`; the rows the chain still reports on outside a tail are `the_chain_reports_the_acronym_it_takes` and `titled_particle_chain_survives_a_title_that_is_also_a_particle`.
- 2026-10-01 (Derek), #562 — A PARTICLE CHAIN UNSETTLES A RUN THE CAPITALS SETTLED, AND THE COUNT READS IT. The 2026-09-27 bullet above left a run whose every member is listed and leans credential to the family-comma path, "which already reads it whole". That promise fails wherever two particles stand side by side in the part: group chains them into one particle run (P2), assign reads the part as name text, and P6 attaches the chain to the family — `John Smith, PhD DO DO` read given 'PhD', family 'DO DO John Smith', and `John Smith, DO DO DO` given 'DO', middle 'DO DO'. rules.md#S2 already said the capitals do not decide a member chained behind another particle ("the run attaches whatever the capitals say (P6)"), so C1's shortcut was resting on a premise S2 denies. Of the two fixes #562 weighed, the one taken narrows the shortcut and leaves S2 as written: a part holding two particles side by side is read by the count, which two name words before the comma flip to the credential run, reported (`suffix-or-name`). The other — letting C1's evidence or S2's credential-in-front company outrank the chain — would have contradicted S2's sentence and P6's `Doe, John van DO` example, so it needed S2 amended rather than a gap filled. The test is ANY two adjacent particles, not a member behind one: `vd` is a particle and an unambiguous suffix word, so `John Smith, PhD vd DO` and `John Smith, MA vd vd` chained and misread the same way, the second with no member behind a particle at all. Segment runs before classify, so it asks classify's own predicate (`_normalize(text) in lexicon.particles`) and only while the run is still settled. The test does not ask whether the family-comma path would actually have misread the part, which it could not without reading ahead to group: where that path did read the part whole — for example a pair opening the part with an unambiguous particle-and-suffix word (`John Smith, vd DO`, `John Smith, VD DO`), a credential that is also a title in front (`John Smith, MD DO DO`), or a credential closed by a period in front (`John Smith, Esq. DO DO`, `John Smith, Jr. DO DO`), the list being by example rather than a census — the count flips the part to the same fields and reports the call, as every flip at this comma does (rules.md#C1's "A decision either way at this comma is reported"). Those reports are ACCEPTED (Derek, 2026-10-01: none of these is a name anyone would write on purpose, so a report is the right signal): with the capitals no longer settling the run, the call is the count's, and the report says so; `tests/v2/cases.py` pins `John Smith, vd DO`. 1.4.0 read every one of these names as the fix does; 2.0.0 and 2.1.0 read `John Smith, PhD DO DO` as title 'PhD', given 'DO DO', and 2.2.0 and 2.3.0 as the issue describes. MEASURED 2026-10-01 against master 0eadedeb, py3.11, `nameparser.__file__` asserted on each side, each parse compared as its seven fields plus its sorted ambiguity kinds: 0 of the 1453 differential-corpus names move (the two names this change adds are the gate's only movers, under a new fix(#562) rule in the four 2.x ledgers); over tests/v2/test_properties.py's settled grid (5,580 texts) 15 move, 10 of them role moves, and the other 5 (`John Smith, MD DO DO`, `MS`, `Esq.`, `Sr`, `Ms` in front) keep their fields and gain the flip's report; over a wider grid — the prefixes `John Smith, `, `Smith, ` and `Doe, John ` times every run of one to three words drawn with repetition from {PhD, MA, Ma, DO, Do, do, vd, van, Jr, MD, Ms}, 4,389 texts — 30 move, 17 of them role moves, every mover a `John Smith, ` text now reading given 'John', family 'Smith' and the whole part as suffix, and every one reporting `suffix-or-name`. Recompute: check out the parent into a separate worktree, parse each grid in both trees under `PYTHONSAFEPATH=1` with the tree's root first on `sys.path`, and diff. The settled grid's own pin moves with it: tests/v2/test_properties.py's `_SETTLED_COUNT` reads 2,994 where it read 3,024, the ten `_SETTLED_EXCEPTIONS` that pinned #562 are gone, and its two recorded negative controls read 1,142 and 215 — the second had already moved from 751 to 215 with #563, before this change. LEFT OPEN, as #562 asked: `Smith, PhD DO DO` (ONE name word before the comma, so the count keeps the listing form, and P6 attaches the chain: given 'PhD', family 'DO DO Smith'), where no rule states S2's credential-in-front company against a chain; and `John Smith, PhD van der`, whose particles are no members and never reach the run test, reading given 'PhD', family 'van der John Smith' as before.
- 2026-10-01 (Derek) — A CAPITALS-SETTLED RUN OPENED BY A WORD OF THE CLASS REPORTS, AND RULES.MD#C1 NOW SAYS SO. C1 said a run whose every class word is written in capitals "reads whole in silence". That held only behind another credential: a class word OPENING the part is the first word after the comma, whose decision C1 reports either way, so `Doe, MA PhD`, `John Smith, MA MA` and `Smith, MA PhD` read wholly as suffixes and report `suffix-or-name` once, on that word, while `John Smith, PhD MA` and `Smith, PhD MA` are silent. The behavior predates #562 and is kept: nobody repeats `MA` at the end of their name on purpose, so the report is the right signal. Statement corrected, no parse moved. MEASURED 2026-10-01 on master b39c370c, the prefixes `John Smith, ` and `Smith, ` times every run of two or three words drawn with repetition from {MA, BA, ED, DO, JD, PhD, MD, Jr, Esq.} holding at least one class word: all 900 runs opening with a class word report, 895 of them read whole (14 of those hold a #562 particle pair, whose count flip reports over the whole part rather than on the first word) and the other five being `Smith, MA DO DO` and its like, the one-name-word chain #562 left open; of the 560 opening with another credential, 554 read whole in silence, and the other six hold a #562 particle pair — four reading whole and reporting (`John Smith, PhD DO DO`), two being `Smith, PhD DO DO` and `Smith, Jr DO DO`. Recompute: parse that grid and bucket by whether the first word is a class word, whether `ambiguities` is empty, and whether the suffix is the whole part. The same day (Derek) rules.md#S2's first-slot precedence sentence gained the pointer to that exception, so the next reader does not rediscover it: S2 says the count decides the first slot after a family comma before case, true for one word (`John Smith, MA` flips by the count) and false for a capitals-settled run, where C1's shortcut reads the capitals first and the comma keeps its family reading. And S2's list of reporting slots lost "the segments beyond it": no `suffix-or-name` report lands past the second comma (`Smith, John, MA` and `John Smith, Jr., MA` are silent). It once did: at #530's merge cc78c960 `John Smith, Jr., PhD Do Ma` reported `suffix-or-name` on 'Ma' beside its `comma-structure` flag, from group's chain emitter, which the 2026-09-28 #544 bullet above silenced in a tail. S2 now points at C2 for what such a part does report, which is not only `comma-structure` — an unclosed delimiter there adds `unbalanced-delimiter` (`John Smith, Jr., (Bob`), and a maiden clause standing in it is read as one (`Smith, John, Jr nee Jones MA`, maiden 'Jones MA'). Derek, 2026-10-01: the silence is right.
- 2026-10-01 (Derek), #554 — A WORD OF BOTH THE PARTICLE AND THE SUFFIX VOCABULARY COUNTS IN THE RUN, AND STILL SPEAKS FOR NOTHING. After #544 the run test (`_segment.py`) counts `vd` and `mc` as suffix words, while S2's company (`_pieces._anchors`) refuses to let them speak for the word behind them, an exclusion added within PR #552 after that PR's own anchor pass had brought back a split of `Smith vd Ma, John` into family 'Smith Ma', suffix 'vd'. That split is not only an intermediate state of #552: 2.2.0 and 2.3.0 shipped it (2.0.0 and 2.1.0 read family 'Smith vd Ma'), #530 removed it in this cycle, and #552 kept it removed. The issue framed that as two answers to one question and offered (b): end the run at such a particle, so that one "has a live name reading" predicate decides both. DECIDED (a), the shipped behavior, and no parse moves. They are two questions. Run membership asks what the word IS, and `vd` is unambiguous suffix vocabulary; company asks whether the word SPEAKS FOR the one behind it, and a particle cannot, P2 joining it forward to exactly that word. Both answers follow from the word's vocabulary, and they differ by design rather than by drift; S2's "end the run" is the company's run, not the part C1 counts. The clause reaches exactly `vd` and `mc`: `do`, also in both vocabularies, is in the ambiguous half and so is already a word of C1's class, reporting where these two do not (`John Smith, Jr do` against `John Smith, Jr vd`). Recompute the set: `L = Parser().lexicon; L.particles & (L.suffix_acronyms | L.suffix_words) - L.suffix_acronyms_ambiguous`, and drop the subtraction to see `do` with them.
The evidence is the writing that carries no other signal. Neither reading of `vd Ma` after a comma is realistic, a particle in front of a Chinese family name being as rare as `vd` or `mc` as a credential, so the mixed-case writing is an arbitrary edge case and decides nothing. Written in one case, where S2 has no capitals to read and the vocabulary is all there is, the part reads as the credential run, and that is the reading a reader would take: measured 2026-10-01 on master (fc682e36), `JOHN SMITH, VD MA` and `john smith, vd ma` read given 'JOHN'/'john', family 'SMITH'/'smith', suffix 'VD MA'/'vd ma' and report `suffix-or-name`, as `John Smith, vd Ma` does, and `John Smith, mc Ma` reads the same in every casing. It is also 1.4.0's reading, verified against the released wheel the same day: 1.4.0 read suffix 'vd Ma' and 'VD MA', and 2.0.0 through 2.3.0 read the whole name as the family. (b) would add a particle exception to C1, and to #563's paired-initials test besides, in order to give `JOHN SMITH, VD MA` family 'JOHN SMITH VD MA' with no given name: the worse reading, on exactly the input where nothing else speaks.
#563's site needs nothing of its own. `García Márquez, vd G.J.` reads as `García Márquez, PhD G.J.` does, in every casing: given 'García', family 'Márquez', suffix 'vd G.J.', and neither reports. If a surname split by a credential run in silence is a defect, it is the run's, not the particle's. Open: #573 — uniform-case `vd`/`mc` before a comma and after a family comma, and the silent mixed-case `Doe, Jane PhD vd Ma`. Recompute: `parse(s)` with `as_dict()` and `[a.kind.value for a in parse(s).ambiguities]` over each string above and its `.upper()` and `.lower()`.

### T1 — separators, not joiners

Expand Down
20 changes: 16 additions & 4 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -1703,10 +1703,17 @@ C1. Rationale: a credential run after the comma means the name is in
case: one letter is the shape a middle initial is written in,
and S3 retires single-character matches for the same reason,
while a multi-letter numeral or generational word stands in a
run like any suffix word ('John Smith, III Ma'). A word of both
the title and the suffix vocabulary opening such a part counts as
a suffix word there, the name before the comma being complete,
except as the titles of paired initials, above. Where every word
run like any suffix word ('John Smith, III Ma'). A word's
particle reading does not take it out of the run: a word of both
the particle and the unambiguous suffix vocabulary is a suffix
word there too ('John Smith, vd Ma'). Run membership asks what
the word is, and S2's company asks whether it speaks for the word
behind it, which a particle never does; the run S2 says such a
word ends is the company's, not this count's. A word of both the
title and the suffix vocabulary opening a part the count reads as
a run counts as a suffix word there, the name before the comma
being complete, except as the titles of paired initials, above.
Where every word
of this class in the part is a LISTED word written in capitals in
a mixed-case name, the writing has already made each of them the
credential (S2), and the part reads as the credential run on that
Expand Down Expand Up @@ -1812,6 +1819,11 @@ C1. Rationale: a credential run after the comma means the name is in
"John Smith, PhD DO DO" → suffix="PhD DO DO"
"John Smith, PhD DO DO" → ambiguities=("suffix-or-name",)
"John Smith, PhD vd DO" → suffix="PhD vd DO"
"John Smith, vd Ma" → suffix="vd Ma"
"JOHN SMITH, VD MA" → suffix="VD MA"
"JOHN SMITH, VD MA" → ambiguities=("suffix-or-name",)
"John Smith, Jr vd" → ambiguities=()
"John Smith, Jr do" → ambiguities=("suffix-or-name",)
"John Smith, X.Y.Z." → suffix="X.Y.Z."
"John Smith, X.Y.Z." → ambiguities=()
"John Smith, X.Y.Z." unlisted_dotted_suffixes-off → given="X.Y.Z."
Expand Down
Loading
Loading