turbopape / postagga

A Library to parse natural language in pure Clojure and ClojureScript

• 👋 Hi, I’m @turbopape
• 👀 I’m interested in software, music and literature
• 🌱 I’m currently learning to live on my own while launching automagic tools
• 📫 How to reach me: book 30 mins with me or send me an email

postagga's Issues

Add English language support

Use nlp-compromise as a tagger for browser bots !

We can use nlp-compromise as a JS postagger to showcase postagga running on the browser !

Enhance performance

On JS tagging is so slow sometimes eval times out.

Feature Request: OR logic for rule steps

Thanks for the great work so far on this, I'm having a blast trying it out. One thing I'd love to see is a way to express that one of a group of steps is required, rather than just a list of optional steps.

For instance, I have this working rule:

(def rules [{:id :statement
             :optional-steps []
             :rule [:actor
                    #{:get-value
                      #{"NN"} #{"NNP"} ;; singular, singular proper
                      #{"NNS"} #{"NNPS"}} ;; plural, plural proper
                    :verb
                    #{:get-value #{"VBD"}}
                    :object
                    #{:get-value
                      #{"NN"} #{"NNP"}
                      #{"NNS"} #{"NNPS"}}
                    #{#{"."}}
                    ]}])

"bob experienced enlightenment." => {:errors nil, :result {:rule :statement, :data {:actor ["bob"], :verb ["experienced"], :object ["enlightenment"]}}}

"people experienced enlightenment." => {:errors nil, :result {:rule :statement, :data {:actor ["people"], :verb ["experienced"], :object ["enlightenment"]}}}

note that some of the POS tags are different than the postagga defaults, I'm plugging in the Stanford NLP lib for tagging

This rule finds simple statements of actor - verb - object, where the actor and object can both be singular or plural. But it isn't possible to disambiguate between singular and plural in the result map, meaning that if I want to look specifically for singular or plural actors or objects, I need to either make both optional (which could result in a false hit with no actor/object), or branch and write more rules to cover all cases (singular actor - verb - singular object, plural actor - verb - singular object etc...) which would get pretty huge.

It would be great if I could specify something like an OR condition, either in the optional steps vector or the body of the rule, for example:

(def desired-rules
  [{:id :statement
    :optional-steps [;; grouping of two or more steps to indicate
                     ;; that one of them is required
                     [:actor-singular
                      :actor-plural]
                     ]
    :rule [:actor-singular
           #{:multi :get-value
             #{"NN"} #{"NNP"}} ;; singular, singular proper
           :actor-plural
           #{:multi :get-value
             #{"NNS"} #{"NNPS"}} ;; plural, plural proper
           :verb
           #{:get-value #{"VBD"}}
           :object
           #{:get-value
             #{"NN"} #{"NNP"}
             #{"NNS"} #{"NNPS"}}
           #{#{"."}}
           ]}])

"bob experienced enlightenment." => {:errors nil, :result {:rule :statement, :data {:actor-singular ["bob"], :verb ["experienced"], :object ["enlightenment"]}}}

"people experienced enlightenment." => {:errors nil, :result {:rule :statement, :data {:actor-plural ["people"], :verb ["experienced"], :object ["enlightenment"]}}}

Would something like this make sense to add? Is there some other property of the rule syntax that can achieve this already? Happy to contribute to make it happen.

Proof-Read the readme

Use external POS-Taggers into the parser

It might be interesting for people to add and document how they could add their POS-Taggers.
I red somewhere someone was able to use Standford's (see #13 ), it might be great if you can document this !

Shipped English model seems to behave incorrectly

I have many bizarre states plus inconsistencies when checking ... looks like I need to retrain the model.

Correct generated HMMS with Brill-based tagger

• I must enhance the output's precision with britt correction algo.

Implement a sample planning bot project

That would be easier for people to grasp what Postagga is for if they actually see it in action.
I would like to use the milestones project and postagga + nlp-compromise pos-tagger to develop a project planning assistant bot on slack.

Add Arabic language support

Write some more Tests

... And hopefully come up with some that fail so we detect maximum bugs :)

Create a "little rapper Logo" for postagga !

Postagga the name is a play on "POS-Tagger" mouthed as "gangsta"... etc.
As it is a lib about words and language, I thought it pertains the Rap culture, and so the name was picked.
To reflect that, postagga can use a logo showing a little comics-y rapper, that'd reflect its friendliness as welle as its toughness with words !

Design sentence detection

Design tokenization

Typo in website URL

First of all thanks for this library, it looks great!

Just a typo: http://fekr.tech seems wrong, it should be http://www.fekr.tech instead.

Include complete examples

It would be beneficial for beginner users to have a complete running example programs, preferably in English.

Common names not properly detected if they are not in the corpus

Maybe using an approach à la BRILL (see nlp_compromise) to solve the fact that unknown tokens in heads of sentences are arbitrarily computed.