vaneseltine / nominally

A maximum-strength name parser for record linkage.

Hi there 👋

• 🔭 I’m currently working on ...
• 🌱 I’m currently learning ...
• 👯 I’m looking to collaborate on ...
• 🤔 I’m looking for help with ...
• 💬 Ask me about ...
• 📫 How to reach me: [email protected]
• 😄 Pronouns: he/him
• ⚡ Fun fact: ...

nominally's Issues

Do a light profiling to ensure there aren't crazy problems

python -m pip install snakeviz
python -m cProfile -o nominally cprof -m pytest
snakeviz nominally.crof

Test ticket from Code Climate

Make sure there are no hideous regressions

Run examples vs. name-parser to ensure nothing weird has slipped through the cracks.

Refactor using coverage

Validate final name parts

Including confirm r'[a-z]' on each part to make, e.g., Name("#") unparsable.

Core Infrastructure Initiative best practices

strip() hyphens from final results

Otherwise hyphens are left over in unusual cases, e.g., misplaced id number: J 2309-23492-3234 Smith ends up with a middle name of ---.

Contextualize and distribute examples

Just via github? In package at all? Links, perhaps?

The answer to this helps resolve #19

Add mc to prefixes like 'van'

Observing Gregor, Ewan Gordon Mc

Drop from OS to Python versions in CircleCI

Test in 3.6 and 3.8, lint and deploy from 3.7.

_extract_suffixes chokes on oddly spaced "andrews, dr "

junior -> jr

Hyphenated numbers are not fully ignored

ɪ (.venv) nominally › python -m nominally "14,A,A"
       raw: 14,A,A
   cleaned: {'14, a, a'}
    parsed: a, a
      list: ['', 'a', '', 'a', '', '']
     title:
     first: a
    middle:
      last: a
    suffix:
  nickname:
ɪ (.venv) nominally › python -m nominally "1-4,A,A"
       raw: 1-4,A,A
   cleaned: {'1-4, a, a'}
    parsed: a a
      list: ['', 'a', 'a', '', '', '']
     title:
     first: a
    middle: a
      last:
    suffix:
  nickname:

This was found via hypothesis generating "¼,A,A"

Retach missing last name prefixes

"Johann Gambolputty de von Ausfern" <-> "Ausfern, Johann Gambolputty de von"

Examine codeclimate plugins

https://codeclimate.com/repos/5d810ed5266c825c4200ed00/settings/plugins/edit

Support proper initializing (J.R.R. Tolkien)

Currently the periods are removed, leaving JRR||Tolkien

We want to recognize this use of periods are parse J|R R|Tolkien

Refactor Name._combine_rightmost_prefixes()

Flagged by codeclimate

Improve typing of list(name) and dict(name)

dict(Name())

nominally\__init__.py:15: error: No overload variant of "dict" matches argument type "Name"
nominally\__init__.py:15: note: Possible overload variants:
nominally\__init__.py:15: note:     def [_KT, _VT] __init__(self, map: Mapping[_KT, _VT], **kwargs: _VT) -> Dict[_KT, _VT]
nominally\__init__.py:15: note:     def [_KT, _VT] __init__(self, iterable: Iterable[Tuple[_KT, _VT]], **kwargs: _VT) -> Dict[_KT, _VT]
nominally\__init__.py:15: note:     <1 more non-matching overload not shown>
nominally\__init__.py:15: note:     def [_KT, _VT] __init__(self, map: Mapping[_KT, _VT], **kwargs: _VT) -> Dict[_KT, _VT]
nominally\__init__.py:15: note:     def [_KT, _VT] __init__(self, iterable: Iterable[Tuple[_KT, _VT]], **kwargs: _VT) -> Dict[_KT, _VT]

list(Name())

nominally\__init__.py:17: error: No overload variant of "list" matches argument type "Name"
nominally\__init__.py:17: note: Possible overload variant:
nominally\__init__.py:17: note:     def [_T] __init__(self, iterable: Iterable[_T]) -> List[_T]
nominally\__init__.py:17: note:     <1 more non-matching overload not shown> 

4th, 5th

Continue to support /[4-9]th/ suffixes; otherwise they turn into "th."

__slots__?

https://docs.python.org/3/reference/datamodel.html
https://stackoverflow.com/questions/472000/usage-of-slots

Name: blueberry, jr r

May be related to #7

Support multi-word prefixes?

E.g. 'von der X' and 'von dem Y' without necessarily picking up 'der' or 'dem' on their own. Might not be necessary.

Allow multiword single quote nicknames while still supporting apostrophes

Currently the apostrophes are conservatively considered part of the name; so Gwinifer 'Old Mother' Blackcap has 'Old Mother' as a middle name rather than Old Mother as a nickname.

Clean nicknames

Nicknames are not cleaned in the same way as post-nickname full names.

Add tests for api.py

from nominally.api import parse_name, report, prettier_print

def test_parse_name():
    pass

def test_report():
    pass

def test_prettier_print():
    pass

Empirically assess prefixes

Currently the list stands at:

PREFIXES = {
    "abu",
    "bin",
    "bon",
    "da",
    "dal",
    "de",
    "dei",
    "del",
    "dela",
    "della",
    "delle",
    "delli",
    "dello",
    "dem",
    "der",
    "di",
    "do",
    "dos",
    "du",
    "ibn",
    "la",
    "le",
    "mc",
    "mac",
    "san",
    "santa",
    "st",
    "ste",
    "van",
    "vel",
    "von",
}

Some of these are in fairly common use as given names:

Della   29,219
Van     22,943
Von      4,608
Del      4,454
Santa    2,852
San      2,308
La       1,663
Le       1,427
De       1,259

We certainly want to support Van, but what of Della and Del? San?

Improve parsing of names with nicknames

By treating the nickname as a field separator, a first/middle or middle/last split could potentially be maintained. Currently the nickname is invisible to the subsequent processing.

Resolve ambiguity in first-name prefixes

Currently "van meep de ook" -> first "van meep", last "de ook"
But "de ook, van meep" -> first "van", middle "meep", last "de ook"

So should van/de/etc operate on first names at all? Should we just identify the last name cluster and only run it on that?

_grab_junior() error

tried running the [...] code that used nominally, and we hit an error with _grab_junior()

Line 181, index function either returns value or ValueError

I'll run it again later to get the string.

Fix cleaned version of names

Refactor Name._lfm_from_list()

Lots of variations on a theme.

Reduce use of X as suffix

jones, xin x

Maybe related to #8

Leveraging last, first, middle comma split breaks idempotence

Name("von floogle, moogle mary, mcdoogle") loses the comma into str "von floogle, moogle mary mcdoogle" which is reparsed to move the mary into the middle name.

Fix "method_complexity" issue in nominally/parser.py _combine_rightmost_prefixes

Function _combine_rightmost_prefixes has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

https://codeclimate.com/github/vaneseltine/nominally/nominally/parser.py#issue_5d859e0248d9f0000100002d

Strip numbers following suffix extraction

zip safe?

Yea or nay?

Flag ? within names as possible encoding problem

Garc?a, Mart?n

Check fingerprint between cleaned and final

Continuing suffix issues

i should be initial; ii and iii and iv are sweepable

More thoroughly clean in preprocessing

", a, a is broken

Coverage on api.py

Refactor cleaning

Currently it's spread out across multiple stages, esp. for the swept strings.

Add general problem statement to README

only take one generational suffix

ii should probably be treated as a suffix more often, see #8

rogers, fred ii mcfeely
underhill, frodo v., ii

Refactor Name._sweep_suffixes()

Tagged by codeclimate.

Basic Name, .api documentation to readthedocs

Smart quotes

Fix raw = "Ramsay “R.J.” Jackson Canning"

Initialize Name() with a Name

Currently only string input is accepted.

Increase suffix expectations when further initials are unlikely

If two full names precede, it's unlikely in the target use cases that a last name will be abbreviated. E.g.

• William Henry Jameson V is the fifth of his name

• William H J V has a last name starting with a 'V'

Address Md <-> Muhammad?

Probably an issue to leave downstream -- but note in documentation.