oroszgy / hunlp

Hungarian NLP tools API

HuNLP: Hungarian language processing for Humans

The aim of this project to provide a unified access to Hungarian NLP tools. As such it provides wrapper classes and a REST API. Currently the project integrates:

• magyarlanc 3.0
• Szeged NER

Usage

Before running the application, make sure you have at least 3GB free memory.

Get the library

You can compile the library locally: $ make dependencies install (You would need Oracle Java 8 and Maven.)

Or find zipped jars here

Alternatively, you can directly use the prebuilt docker image: $ docker pull oroszgy/hunlp

Running the application

In case you directly compiled the sources: $ make serve

Having the binairies downloaded: $ hunlp.sh

Or running the docker container: $ docker run -it -p 9090:9090 oroszgy/hunlp

Using the application

Through the REST API

$ curl -X POST -H "Content-Type: application/json" -d '{"text": "Szia világ!"}' "http://localhost:9090/v1/annotate"

Java

import hu.nlp.api.HuNlp;

class MainApp {
    public static void main(String args[]) {
        HuNLlp nlp = new HuNlp();
        Document doc = nlp("Ez itt egy példa. Vajon működik?");
    }
}

Kotlin

import hu.nlp.api.HuNlp

fun main(args: Array<String>) {
    val nlp = HuNlp()
    val doc: Document = nlp("Ez itt egy példa. Vajon működik?")
}

Python

Make sure you installed the wrapper: pip install https://github.com/oroszgy/hunlp/releases/download/0.2/hunlp-0.2.0.tar.gz

from hunlp import HuNlp

nlp = HuNlp()
doc = nlp("Ez itt egy példa. Vajon működik?")

print(doc.ents)

for sent in doc:
    for tok in sent:
        print(tok.text, tok.lemma, tok.tag)

Changelog

0.2

Experimental Kotlinization

0.1

First experimental release containing magyarlanc (21-01-2016) and Szeged NER (20-06-2014).

License

HuNLP is licensed under LGPL3, however libraries integrated might use different (commercial) licenses:

• magyarlanc license
• Szeged NER builds upon Mallet

(c) Gyorgy Orosz, 2017

hunlp's Issues

NoSuchMethodError while running Docker images

Hello,
I experience an issue with Docker images. There seems to be an issue with the magyarlanc jar, because a NoSuchMethodError is raised when trying to start hunlp. Traceback:

Exception in thread "main" java.lang.NoSuchMethodError: hu.u_szeged.magyarlanc.Magyarlanc.fullInit()V
        at hu.nlp.api.HuNlp.<init>(HuNlp.kt:12)
        at hu.nlp.api.AppKt.main(App.kt:11)

Steps to reproduce

Just run docker run -it -p 9090:9090 oroszgy/hunlp. The exception above should appear.

Steps to fix

rm magyarlanc-3.0.jar
wget http://rgai.inf.u-szeged.hu/project/nlp/research/magyarlanc/magyarlanc-3.0.jar

The tool seems to be functional afterwards:

root@3e67897273ef:/app# ./hunlp.sh
29.51.633  is2.data.Cluster -1:<init> ->               Read cluster with 0 words
29.58.814  is2.parser.ParametersFloat -1:read ->       read parameters 134217727 not zero 18650065
29.58.819  is2.parser.Extractor -1:initStat ->         mult  (d4)
Used parser   class is2.parser.Parser
Creation date 2016.01.19 15:39:08
Training data szk.mate.train
Iterations    10 Used sentences 10000000
Cluster       null
29.58.821  hu.u_szeged.dep.parser.MyMateParser 108:readModel -> Reading data finnished
29.58.822  is2.parser.Extractor -1:initStat ->         mult  (d4)
[Thread-1] INFO org.eclipse.jetty.util.log - Logging initialized @45240ms
[Thread-1] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - == Spark has ignited ...
[Thread-1] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:9090
[Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.3.6.v20151106
[Thread-1] INFO org.eclipse.jetty.server.ServerConnector - Started ServerConnector@1989765a{HTTP/1.1,[http/1.1]}{0.0.0.0:9090}
[Thread-1] INFO org.eclipse.jetty.server.Server - Started @45373ms

Could you publish a new image? :)

Thank you for this cool tool! Happy new year!

Balint