Skip to content

A Constants set entry with edge whitespace raises ValueError at the first parse: c.suffix_not_acronyms.add('ma ') then HumanName('John Smith', c) #541

Description

@derek73

A v1 Constants set entry carrying leading or trailing whitespace was inert on 1.4.0: lc() lowercases and strips periods but not whitespace, and no parsed piece ever carries whitespace, so the entry matched nothing. Through the 2.x shim the same entry does one of two things, and both break the shim's contract that it never raises on a config v1 accepted and never silently changes the parse (AGENTS.md, "The shim TRANSLATES").

It raises at the first parse where the entry lands in a set the shim's _snapshot() translates by set algebra. The algebra runs on the un-stripped strings, so 'ma ' is not subtracted from suffix_words and "'t " is not found in particles; Lexicon's _normalize then strips the whitespace and the invariant fires:

>>> c = Constants()
>>> c.suffix_not_acronyms.add("ma ")
>>> HumanName("John Smith", c)
ValueError: an ambiguous suffix acronym must not also be a suffix word; in both: ma. ...
>>> c = Constants()
>>> c.bound_first_names.add("'t ")
>>> HumanName("Gerard 't Hooft", c)
ValueError: bound_given_names entries that are also particles must be in particles_ambiguous; not in particles_ambiguous: 't. ...

Every release from 2.0.0 raises on both (do , ma and "'t " measured at 2.0.0, 2.3.0 and master; ba joins them from 2.3.0, when ba became ambiguous). 1.4.0 accepts all of them and parses as if the entry were absent.

It silently activates where the entry lands in a set the shim copies through: _normalize folds ' dean ' to dean, so the entry starts matching.

>>> c = Constants()
>>> c.titles.add(" dean ")
>>> HumanName("dean john smith", c).title
'dean'         # 1.4.0: title '', first 'dean'

Reach. Only a caller who wrote whitespace into a config entry, which is a typo. But the raise surfaces at the first PARSE, in library code, with a message about a word the caller never typed, and the activation changes a parse with no diagnostic at all.

Precedent. _snapshot() already handles exactly this class for first_name_titles: an entry holding a whitespace run is dropped from given_name_titles with the comment "An entry holding a whitespace run was inert there (translating it would start matching)". The other eight SetManager fields have no such filter.

Options, in the order the shim doctrine ranks them:

  1. Reproduce v1: drop the entry from the translation wherever _normalize(e) != lc(e) would change it, with the given_name_titles filter generalized to every set field. A construction-time UserWarning naming the entry is the diagnostic AGENTS.md reserves for a statically decidable, harmless-to-some-caller gap. No parse changes against 1.4.0.
  2. Strip whitespace in SetManager itself at every entry point. Simpler, and it fixes both faces at once, but a v1-inert entry becomes active (' dean ' starts titling; "'t " starts binding, measured: 't Hooft Gerard moves from family-only to given 't Hooft), which is the silent-change half of the contract. Defensible as a typo fix; not what the doctrine says.

The _snapshot() comment roster (seven translations today) gains a bullet either way, and AGENTS.md's shim bullet with it.

Found in the #539 review while checking which Lexicon checks the shim's construction already satisfies.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions