yomguithereal / clj-fuzzy

A handy collection of algorithms dealing with fuzzy strings and phonetics.

clj-fuzzy

clj-fuzzy is a native Clojure library providing a collection of famous algorithms dealing with fuzzy strings and phonetics.

It can be used in Clojure, ClojureScript, client-side JavaScript and Node.js.

Deprecation warning

Consider this library deprecated for JavaScript.

Indeed, the Talisman library can be seen as an improvement over clj-fuzzy and is, what's more, written directly in JavaScript.

Full documentation

The full documentation for this library is available there.

Available algorithms

Metrics

• Sorensen / Dice coefficient
• Levenshtein distance
• Hamming distance
• Jaccard / Tanimoto distance
• Jaro-Winkler distance
• MRA comparison
• Tversky index

Stemmers

• Lancaster stemmer (en)
• Lovins stemmer (en)
• Porter stemmer (en)
• Schinke stemmer (latin)

Phonetics

• Metaphone
• Double Metaphone
• Soundex
• NYSIIS (original & refined)
• Caverphone (original & revisited)
• Cologne Phonetic
• MRA codex

Contribution

Please feel free to contribute by forking this repo. Just be sure to add relevant unit tests and pass them all before submitting any code.

License

MIT

clj-fuzzy's Issues

Add doc about the algorithms to the code itself

Levenshtein Distance Error On Empty Sequence

If the first sequence passed to the levenshtein distance function is empty, an exception is thrown:

(fuzzy/levenshtein "" "abc")
ClassCastException clojure.lang.LazySeq cannot be cast to clojure.lang.IPersistentStack  clojure.lang.RT.peek (RT.java:710)

Whereas:

(fuzzy/levenshtein "abc" "")
3

An empty sequence for first position needs to be handled.

Spanish support?

There is no mention whatsoever about language support.
Schinke stemmer is supposed to be latin but it doesn't work as expected.
Thanks.

Add some tests to refined NYSIIS

Rework headers and descriptions

Optimize the Levensthein distance for JavaScript

Recursivity seems to make it slow in javascript.

Decide whether to reverse Jaccard or not?

Provide api for some helpers?

Plus some helpers for ngrams.

Refined caverphone

dice algorithm NaN

Just trying out this algorithm and seems the dice algorithm has some minor bugs (or I am not understanding it quite right):

These are the results I am getting with strings of length 0 and length 1, could this have anything to do with the input being characters rather than actual strings? Is that as intended?

Native

Switch to cljx or feature expressions

Cos similarity

Fix Levenshtein ortographe

Documentation website outdated

Hi,

I ran out on the issue of the dependency not found in Clojurescript until I found out I was using an outdated version of the lib 0.1.8, following install steps documented here http://yomguithereal.github.io/clj-fuzzy/clojure.html.

Looking more closely, website indicates Currently v0.3.2 in the sidebar and 0.1.8 in the Clojure install page, which are both wrong according to Clojars.

Is there anything I can do to help ?

Thanks!

Is it appropriate to add a non-formal algorithm?

Context:

I have ported the implementation of http://scottfrazersblog.blogspot.com/2009/12/emacs-better-ido-flex-matching.html?m=1 into Clojure.

Question:

Does this implementation belong to this library?

Cologne phonetic will drop the first place 0 erroneously

Utilities

closest to, best match etc.

Levenstein distance performance

The levenshtein/distance function has very poor performance for even short strings – is this a known issue? It takes 10 seconds on a MacBook Pro (3 GHz Intel Core i7) running Java 8.

user=> (time (clj-fuzzy.levenshtein/distance "feature" "get-project-features"))
"Elapsed time: 10438.251547 msecs"
13

Tanimoto & Sorensen Documentation

Add Normalization option to Hamming distance

Beider-Morse

Repo needs `index.js` so that it can be invoked with `require`

To reproduce, run var fuzzy = require('clj-fuzzy') in requirebin.com - throws error.

Adding node.js version

cmudict

Lovins stemmer

Metaphone3

Is this even authorized?

Big-O Performance

Great to see a library like this. I would love to see the Big-O performance of each fuzzy algorithm displayed so I know what size of data I can it for and maybe some advice about pros and cons.

I'm doing some fuzzy matching for accounting purposes ("McDonalds": "is this a business expense? probably not") and I wouldn't know which algo to pick to save the most time

Daitch Mokotoff

Switch to gulp

Fix the cljx pb

Miracode

Lancaster Stemmer

issue using this project as a dependency in clojurescript

Hello! Thanks for writing this cool library :).

I am using clojurescript version "0.0-3165" and clojure version "1.7.0-beta1" and I am unable to depend on clj-fuzzy from clojurescript. If it makes a difference, I am using the boot build tool.

Clojurescript should be a dev dependency

Onyx (https://github.com/onyx-platform/onyx) uses clj-fuzzy as a dependency, however we have to exclude clojurescript, as it is an unnecessary dependency for clojure users, and can cause conflicts in our user's projects. I think you would be best served by making it a dev dependency, as any of your cljs users will need clojurescript as a dependency anyway.

Thanks for a great project!

Add refined soundex

Add refined NYSIIS

Travis CI

interferes with other modules, do NOT use this module unless it's fixed

this module interferes with other node modules, like "later" (@breejs/later)
please don't do stuff like:
Date.prototype.s = function (a, b) { return b instanceof Date && this.valueOf() === b.valueOf()
this makes the whole module unusable

Sorensen alias

Add Tversky index

Jaro-Winkler returns unexpected values for two nil inputs

Is this intended behavior?

user=> (require '[clj-fuzzy.jaro-winkler])
nil
user=> (clj-fuzzy.jaro-winkler/jaro-winkler nil nil)
0.4

Something seems to be amiss with Jaro distance & long strings