The kvalid's discuss from theldus

kvalid's Issues

output of k-means and SSE does not match final number of clusters chosen

I am testing the KValid with the following dataset: https://www.kaggle.com/datasets/crawford/gene-expression

My settings:

the output:

KValid
======

=== Clustering validation, using: Elbow method (SSE) ===

For k = 3
SSE: 307911.9827729013

For k = 4
SSE: 284224.5253161817

For k = 5
SSE: 265002.0829083235

For k = 6
SSE: 271168.4306434722

For k = 7
SSE: 255451.7226422783

For k = 8
SSE: 248347.4649960015

For k = 9
SSE: 245387.35604437318

For k = 10
SSE: 240754.6914872431

so based on the k should be 5:
but what I see at the end, is 3 clsuters

=== Model and evaluation on training set ===

Clustered Instances

0      520 ( 65%)
1      145 ( 18%)
2      136 ( 17%)


Class attribute: Class
Classes to Clusters:

   0   1   2  <-- assigned to cluster
   0   0 136 | PRAD
 141   0   0 | LUAD
 300   0   0 | BRCA
   1 145   0 | KIRC
  78   0   0 | COAD

Cluster 0 <-- BRCA
Cluster 1 <-- KIRC
Cluster 2 <-- PRAD

Incorrectly clustered instances :	220.0	 27.4657 %

Elbow plot also shows optimal number of clusters to be 5.

Also,

When I set cascade=false: no graph shows up! but optimal number of clusters is determined approprietly:

=== Model and evaluation on training set ===

Clustered Instances

0      187 ( 23%)
1      145 ( 18%)
2      136 ( 17%)
3       74 (  9%)
4      259 ( 32%)


Class attribute: Class
Classes to Clusters:

   0   1   2   3   4  <-- assigned to cluster
   0   0 136   0   0 | PRAD
 141   0   0   0   0 | LUAD
  41   0   0   0 259 | BRCA
   1 145   0   0   0 | KIRC
   4   0   0  74   0 | COAD

Cluster 0 <-- LUAD
Cluster 1 <-- KIRC
Cluster 2 <-- PRAD
Cluster 3 <-- COAD
Cluster 4 <-- BRCA

Incorrectly clustered instances :	46.0	  5.7428 %