words / stemmer

Fast Porter stemmer implementation

stemmer

Porter stemming algorithm.

Contents

• What is this?
• When should I use this?
• Install
• Use
• API
  • stemmer(value)
• CLI
• Types
• Compatibility
• Related
• Contribute
• Security
• License

What is this?

This package exposes a stemming algorithm. That means it gets a certain string (typically an English word), and turns it into a shorter version (a stem), which can then be compared to other stems (of other words), to check if they are both (likely) the same term.

When should I use this?

You’re probably dealing with natural language, and know you need this, if you’re here!

Install

This package is ESM only. In Node.js (version 14.14+, 16.0+), install with npm:

npm install stemmer

In Deno with esm.sh:

import {stemmer} from 'https://esm.sh/stemmer@2'

In browsers with esm.sh:

<script type="module">
  import {stemmer} from 'https://esm.sh/stemmer@2?bundle'
</script>

Use

import {stemmer} from 'stemmer'

stemmer('considerations') // => 'consider'
stemmer('detestable') // => 'detest'
stemmer('vileness') // => 'vile'

API

This package exports the identifier stemmer. There is no default export.

stemmer(value)

Get the stem from a given value.

value

Value to stem (string, required).

Returns

Stem for value (string).

CLI

Usage: stemmer [options] <words...>

Porter Stemmer algorithm

Options:

  -h, --help           output usage information
  -v, --version        output version number

Usage:

# output stems
$ stemmer considerations
# consider

# output stems from stdin
$ echo "detestable vileness" | stemmer
# detest vile

Types

This package is fully typed with TypeScript. It exports no additional types.

Compatibility

This package is at least compatible with all maintained versions of Node.js. As of now, that is Node.js 14.14+ and 16.0+. It also works in Deno and modern browsers.

Related

• stmr.c — C API
• stmr — C CLI
• lancaster-stemmer — lancaster stemming algorithm
• double-metaphone — double metaphone algorithm
• soundex-code — soundex algorithm
• dice-coefficient — sørensen–dice coefficient
• levenshtein-edit-distance — levenshtein edit distance
• syllable — syllable count of English words

Contribute

Yes please! See How to Contribute to Open Source.

Security

This package is safe.

License

MIT © Titus Wormer

stemmer's Issues

Incorrect stemming of word "Day"

Guys: Can you kindly check on this. Not sure if this is affecting other words ending with "y" too or i'm doing something wrong.

Question - Stemmer API multiple words

Great library! Maybe a silly question but is there any way to stemmer a whole text, like the CLI does?

stemmer("considerations for all mankind")

Or should we just go with a legacy loop and replace?

input "status" and output "statu", is this a right result?

$ echo "report your status considerations" | stemmer
report your statu consider

I was wandering whether it is a right or expected result?

How do you 'require' stemmer if you absolutely have to?

I have an established project that is based on old-style node requires.

Is there any way to make stemmer play nice with common.js modules?

Support for languages other than English

Stemmer is great for English but is other languages support planned?

Missing cjs version of library

Issue

This library doesn't export a Common JS Package. This would be convenient for most users of this library in a backend context, since typescript typically compiles to CommonJS for node applications (see https://www.typescriptlang.org/tsconfig#module).

While researching how to use this library in a NestJS context, I came across this article: https://redfin.engineering/node-modules-at-war-why-commonjs-and-es-modules-cant-get-along-9617135eeca1. It has nice instructions for exporting both versions of a library without much work.

Workaround

For now, I am using a deep import to import the library (e.g. import { stemmer } from '../../../node_modules/stemmer/index'; instead of import { stemmer } from 'stemmer';.

Should have benchmarks

…including a comparison to the competition.