Decide whether to reverse Jaccard or not? about clj-fuzzy

[PrayagAshwini]

Hi
I am trying to use jaccard matrix as an attribute to clusterfck -- here is my code :

var clusters = clusterfck.hcluster(dataarray, clj_fuzzy.metrics.levenshtein, clusterfck.COMPLETE_LINKAGE);

however, it always returns single node for each array item

[Yomguithereal]

Hello @PrayagAshwini. I am not very acquainted with how the clusterfck works so it is quite difficult to help you efficiently. However, your example uses the levenshtein function rather than the jaccard one.

[Yomguithereal]

Could you elaborate a bit more on what you are trying to do @PrayagAshwini, please?

[PrayagAshwini]

thanks for quick response Yomguithereal .
Actually I tried both Jaccard & levenshtein, however none of them working. same output with any matrix attribute.
I am using clj-fuzzy - v0.3.1 js file .
let me know if you get any other pointers.

[Yomguithereal]

Do you have a sample of the data you are trying to cluster somewhere so I can run tests on my side?

[PrayagAshwini]

here is sample data - ["economictimes.com","food.com","gaana.com","google.com","facebook.com","food.com","facebook.com","food.com"]

[Yomguithereal]

After running some tests on my own, I am not sure this library is really suited to analyze strings. It seem that it will only process numbers.

[Yomguithereal]

Moreover the library behaves quite strangely, sometimes it computes instantaneously but sometimes it just does not compute at all or very slowly. Besides the library is not maintained anymore.

[Yomguithereal]

Why don't you use something like hierarchical-clustering:

var cluster = require('hierarchical-clustering'),
    metrics = require('clj-fuzzy').metrics;

var data = [
  'economictimes.com',
  'food.com',
  'gaana.com',
  'google.com',
  'facebook.com',
  'food.com',
  'facebook.com',
  'food.com'
];

// Single-linkage clustering
function linkage(distances) {
  return Math.min.apply(null, distances);
}

var levels = cluster({
  input: data,
  distance: metrics.jaccard,
  linkage: linkage,
  minClusters: 2, // only want two clusters
});

var clusters = levels[levels.length - 1].clusters;

clusters = clusters.map(function (cluster) {
  return cluster.map(function (index) {
    return data[index];
  });
});

console.log(clusters);

[PrayagAshwini]

thanks Yomguithereal. The above code is working.. however metrics parameter is not working..
here is data -
[ 'chatting.com', 'chrysler.com', 'economictimes.com', 'food.com', 'toopgaana.com', 'health.nih.gov', 'ibncnn.com', 'nytimes.com', 'khaanakhajana.com','kidshealth.org','makaan.com', 'photography.com', 'pinrest.com', 'test.com', 'thetimes.co.uk', 'topgames.com', 'topnonprofits.com' ]

And output is -
[["kidshealth.org"],["topnonprofits.com","topgames.com","toopgaana.com","test.com","food.com","pinrest.com","nytimes.com","makaan.com","ibncnn.com","chrysler.com","chatting.com","thetimes.co.uk","economictimes.com","khaanakhajana.com","photography.com","health.nih.gov"]]

whether I use "metrics.jaccard," or " metrics.levenshtein" ..

Any pointers??

I would like to receive output something like below --

[["kidshealth.org"], ["topnonprofits.com", "topgames.com", "toopgaana.com"], ["test.com"],["food.com"], ["pinrest.com"], ["thetimes.co.uk", "economictimes.com", "nytimes.com"],["makaan.com"] ,["ibncnn.com"], ["chrysler.com", "chatting.com"], ["khaanakhajana.com"], ["photography.com"], ["health.nih.gov"]]

[Yomguithereal]

You can try to fiddle with the minClusters option to increase the number of possible clusters generated by the algorithm.

[PrayagAshwini]

I tried that.. however it just creates more clusters with single item, and remaining all in one..
If I provide manual distance formula, it clusters at least two values starting with "ch" and remaining items listed correctly as single item.

// Euclidean distance
function Edistance(a, b) {
var d = 0;
for (var i = 0; i < a.length; i++) {
d += Math.pow(a[i] - b[i], 2);
}
return Math.sqrt(d);
}

output
[["chrysler.com","chatting.com"],["economictimes.com"],["food.com"],["toopgaana.com"],["health.nih.gov"],["ibncnn.com"],["nytimes.com"],["khaanakhajana.com"],["kidshealth.org"],["makaan.com"],["photography.com"],["pinrest.com"],["test.com"],["thetimes.co.uk"],["topgames.com"],["topnonprofits.com"]]