A v1 Constants set entry carrying leading or trailing whitespace was inert on 1.4.0: lc() lowercases and strips periods but not whitespace, and no parsed piece ever carries whitespace, so the entry matched nothing. Through the 2.x shim the same entry does one of two things, and both break the shim's contract that it never raises on a config v1 accepted and never silently changes the parse (AGENTS.md, "The shim TRANSLATES").
It raises at the first parse where the entry lands in a set the shim's _snapshot() translates by set algebra. The algebra runs on the un-stripped strings, so 'ma ' is not subtracted from suffix_words and "'t " is not found in particles; Lexicon's _normalize then strips the whitespace and the invariant fires:
>>> c = Constants()
>>> c.suffix_not_acronyms.add("ma ")
>>> HumanName("John Smith", c)
ValueError: an ambiguous suffix acronym must not also be a suffix word; in both: ma. ...
>>> c = Constants()
>>> c.bound_first_names.add("'t ")
>>> HumanName("Gerard 't Hooft", c)
ValueError: bound_given_names entries that are also particles must be in particles_ambiguous; not in particles_ambiguous: 't. ...
Every release from 2.0.0 raises on both (do , ma and "'t " measured at 2.0.0, 2.3.0 and master; ba joins them from 2.3.0, when ba became ambiguous). 1.4.0 accepts all of them and parses as if the entry were absent.
It silently activates where the entry lands in a set the shim copies through: _normalize folds ' dean ' to dean, so the entry starts matching.
>>> c = Constants()
>>> c.titles.add(" dean ")
>>> HumanName("dean john smith", c).title
'dean' # 1.4.0: title '', first 'dean'
Reach. Only a caller who wrote whitespace into a config entry, which is a typo. But the raise surfaces at the first PARSE, in library code, with a message about a word the caller never typed, and the activation changes a parse with no diagnostic at all.
Precedent. _snapshot() already handles exactly this class for first_name_titles: an entry holding a whitespace run is dropped from given_name_titles with the comment "An entry holding a whitespace run was inert there (translating it would start matching)". The other eight SetManager fields have no such filter.
Options, in the order the shim doctrine ranks them:
- Reproduce v1: drop the entry from the translation wherever
_normalize(e) != lc(e) would change it, with the given_name_titles filter generalized to every set field. A construction-time UserWarning naming the entry is the diagnostic AGENTS.md reserves for a statically decidable, harmless-to-some-caller gap. No parse changes against 1.4.0.
- Strip whitespace in
SetManager itself at every entry point. Simpler, and it fixes both faces at once, but a v1-inert entry becomes active (' dean ' starts titling; "'t " starts binding, measured: 't Hooft Gerard moves from family-only to given 't Hooft), which is the silent-change half of the contract. Defensible as a typo fix; not what the doctrine says.
The _snapshot() comment roster (seven translations today) gains a bullet either way, and AGENTS.md's shim bullet with it.
Found in the #539 review while checking which Lexicon checks the shim's construction already satisfies.
A v1
Constantsset entry carrying leading or trailing whitespace was inert on 1.4.0:lc()lowercases and strips periods but not whitespace, and no parsed piece ever carries whitespace, so the entry matched nothing. Through the 2.x shim the same entry does one of two things, and both break the shim's contract that it never raises on a config v1 accepted and never silently changes the parse (AGENTS.md, "The shim TRANSLATES").It raises at the first parse where the entry lands in a set the shim's
_snapshot()translates by set algebra. The algebra runs on the un-stripped strings, so'ma 'is not subtracted fromsuffix_wordsand"'t "is not found inparticles;Lexicon's_normalizethen strips the whitespace and the invariant fires:Every release from 2.0.0 raises on both (
do,maand"'t "measured at 2.0.0, 2.3.0 and master;bajoins them from 2.3.0, whenbabecame ambiguous). 1.4.0 accepts all of them and parses as if the entry were absent.It silently activates where the entry lands in a set the shim copies through:
_normalizefolds' dean 'todean, so the entry starts matching.Reach. Only a caller who wrote whitespace into a config entry, which is a typo. But the raise surfaces at the first PARSE, in library code, with a message about a word the caller never typed, and the activation changes a parse with no diagnostic at all.
Precedent.
_snapshot()already handles exactly this class forfirst_name_titles: an entry holding a whitespace run is dropped fromgiven_name_titleswith the comment "An entry holding a whitespace run was inert there (translating it would start matching)". The other eightSetManagerfields have no such filter.Options, in the order the shim doctrine ranks them:
_normalize(e) != lc(e)would change it, with thegiven_name_titlesfilter generalized to every set field. A construction-timeUserWarningnaming the entry is the diagnostic AGENTS.md reserves for a statically decidable, harmless-to-some-caller gap. No parse changes against 1.4.0.SetManageritself at every entry point. Simpler, and it fixes both faces at once, but a v1-inert entry becomes active (' dean 'starts titling;"'t "starts binding, measured:'t Hooft Gerardmoves from family-only to given't Hooft), which is the silent-change half of the contract. Defensible as a typo fix; not what the doctrine says.The
_snapshot()comment roster (seven translations today) gains a bullet either way, and AGENTS.md's shim bullet with it.Found in the #539 review while checking which
Lexiconchecks the shim's construction already satisfies.