winkNLP

Developer friendly Natural Language Processing ✨

winkNLP is a JavaScript library for Natural Language Processing (NLP). Designed specifically to make development of NLP solutions easier and faster, winkNLP is optimized for the right balance of performance and accuracy. The package can handle large amount of raw text at speeds over 525,000 tokens/second. And with a test coverage of ~100%, winkNLP is a tool for building production grade systems with confidence.

Features

WinkNLP has a comprehensive natural language processing (NLP) pipeline covering tokenization, sentence boundary detection (sbd), negation handling, sentiment analysis, part-of-speech (pos) tagging, named entity recognition (ner), custom entities recognition (cer):

At every stage a range of properties become accessible for tokens, sentences, and entities. Read more about the processing pipeline and how to configure it in the winkNLP documentation.

It packs a rich feature set into a small foot print codebase of under 1500 lines:

Fast, lossless & multilingual tokenizer
Developer friendly and intuitive API
Built-in API to aid text visualization
Extensive text processing features such as bag-of-words, frequency table, stop word removal, readability statistics computation and many more.
Pre-trained language models with sizes starting from <3MB onwards
BM25-based vectorizer
Multiple similarity methods
Word vector integration
No external dependencies
Runs on web browsers
Typescript support.

Installation

Use npm install:

npm install wink-nlp --save

In order to use winkNLP after its installation, you also need to install a language model. The following command installs the latest version of default language model — the light weight English language model called wink-eng-lite-model.

node -e "require( 'wink-nlp/models/install' )"

Any required model can be installed by specifying its name as the last parameter in the above command. For example:

node -e "require( 'wink-nlp/models/install' )" wink-eng-lite-model

How to install for Web Browser

If you’re using winkNLP in the browser use the wink-eng-lite-web-model instead. Learn about its installation and usage in our guide to using winkNLP in the browser.

Getting Started

The "Hello World!" in winkNLP is given below:

// Load wink-nlp package  & helpers.
const winkNLP = require( 'wink-nlp' );
// Load "its" helper to extract item properties.
const its = require( 'wink-nlp/src/its.js' );
// Load "as" reducer helper to reduce a collection.
const as = require( 'wink-nlp/src/as.js' );
// Load english language model — light version.
const model = require( 'wink-eng-lite-model' );
// Instantiate winkNLP.
const nlp = winkNLP( model );

// NLP Code.
const text = 'Hello   World🌎! How are you?';
const doc = nlp.readDoc( text );

console.log( doc.out() );
// -> Hello   World🌎! How are you?

console.log( doc.sentences().out() );
// -> [ 'Hello   World🌎!', 'How are you?' ]

console.log( doc.entities().out( its.detail ) );
// -> [ { value: '🌎', type: 'EMOJI' } ]

console.log( doc.tokens().out() );
// -> [ 'Hello', 'World', '🌎', '!', 'How', 'are', 'you', '?' ]

console.log( doc.tokens().out( its.type, as.freqTable ) );
// -> [ [ 'word', 5 ], [ 'punctuation', 2 ], [ 'emoji', 1 ] ]

Experiment with the above code on RunKit.

Explore Further

Dive into winkNLP's concepts or head to winkNLP recipes for common NLP tasks or just explore live showcases to learn:

Wikipedia Timeline ⏳

Reads any wikipedia article and generates a visual timeline of all its events.

NLP Wizard 🧙

Performs tokenization, sentence boundary detection, pos tagging, named entity detection and sentiment analysis of user input text in real time.

Naive Wikification Tool 🔗

Links entities such as famous persons, locations or objects to the relevant Wikipedia pages.

Speed & Accuracy

The winkNLP processes raw text at ~525,000 tokens per second with its default language model — wink-eng-lite-model, when benchmarked using "Ch 13 of Ulysses by James Joyce" on a 2.2 GHz Intel Core i7 machine with 16GB RAM. The processing included the entire NLP pipeline — tokenization, sentence boundary detection, negation handling, sentiment analysis, part-of-speech tagging, and named entity extraction. This speed is way ahead of the prevailing speed benchmarks.

The benchmark was conducted on Node.js versions 14.8.0, and 12.18.3.

The winkNLP delivers similar performance on browsers; its performance on a specific machine/browser combination can be measured using the Observable notebook — How to measure winkNLP's speed on browsers?.

It pos tags a subset of WSJ corpus with an accuracy of ~94.7% — this includes tokenization of raw text prior to pos tagging. The current state-of-the-art is at ~97% accuracy but at lower speeds and is generally computed using gold standard pre-tokenized corpus.

Its general purpose sentiment analysis delivers a f-score of ~84.5%, when validated using Amazon Product Review Sentiment Labelled Sentences Data Set at UCI Machine Learning Repository. The current benchmark accuracy for specifically trained models can range around 95%.

Memory Requirement

Wink NLP delivers this performance with the minimal load on RAM. For example, it processes the entire History of India Volume I with a total peak memory requirement of under 80MB. The book has around 350 pages which translates to over 125,000 tokens.

Documentation

Concepts — everything you need to know to get started.
API Reference — explains usage of APIs with examples.
Change log — version history along with the details of breaking changes, if any.
Showcases — live examples with code to give you a head start.

Need Help?

Usage query 👩🏽‍💻

Please ask at Stack Overflow or discuss at Wink JS GitHub Discussions or chat with us at Wink JS Gitter Lobby.

Bug report 🐛

If you spot a bug and the same has not yet been reported, raise a new issue or consider fixing it and sending a PR.

New feature ✨

Looking for a new feature, request it via the new features & ideas discussion forum or consider becoming a contributor.

About wink

Wink is a family of open source packages for Natural Language Processing, Machine Learning, and Statistical Analysis in NodeJS. The code is thoroughly documented for easy human comprehension and has a test coverage of ~100% for reliability to build production grade solutions.

Copyright & License

It is licensed under the terms of the MIT License.

e7dal / wink-nlp Goto Github PK

wink-nlp's Introduction