smashinfries / pykatsuyou

Japanese verb/adjective inflections tool

Socials

Badges

My GitHub Stats

Top Repositories

pykatsuyou's Issues

Incorrect parsing by Igo causes an IndexError

When the Igo morphological analyzer fails to correctly parse the given verb (due to an outdated and/or incomplete bundled dictionary), we get this kind of stack trace:

  File "some_user_script.py", line 31, in generate_conjugated_entries
    for inflection in getInflections(base_verb)["list"]:
  File "pykatsuyou.py", line 127, in getInflections
    final = finalizeVerb(data, hira)
  File "pykatsuyou.py", line 89, in finalizeVerb
    data = ruDecider3000(hira)
  File "pykatsuyou.py", line 73, in ruDecider3000
    if (hira[-2] in PAIRS['あ']) or (hira[-2] in PAIRS['う']) or (hira[-2] in PAIRS['お'] or (text in EXCEPTIONS)):
IndexError: string index out of range

The IndexError happens because hira is only 1 character long. Why is hira 1 character long? Because hira is taken from the last morpheme found by Igo. In the case of unknown る verbs like 縮らせる, 荒げる, デコる, ファブる, デレる, 文字る, 火病る, 薫じる, or 倒る, Igo breaks down into multiple components where the features of the last morpheme are '助動詞,*,*,*,文語・ル,基本形,る,ル,ル' so we get "ル" → ”る”.

Naturally, correct parsing of the above examples is outside the scope of this project (that's a Igo issue) but PyKatsuyou could throw a more precise error than IndexError like UnknownVerbError or something.