faryalamir / lab-clustering-songs

Lab | Clustering songs

## Introduction

Now it's time to cluster the songs of the hot_songs and not_hot_songs databases according to the song's audio features. For this purpose, you need to consider the following questions:

• Are you going to use all the audio features? I
  • If not, which ones do you think that makes more sense to be used?
  • It might make sense to use a dimensionality reduction technique to visualize the songs with only two features?
• What is the optimal number of clusters (for methods that need to know this beforehand)?
• What is the best distance to use?
• What clustering method provides better results?
• Does the clustering method need a transformer?

Considerations

Be aware that this process is extremely time-consuming!!! (it might take several hours on your laptop). Therefore, when testing different options, save the models into your disk in order to be able to use the best model later. You don't want to retrain the best model again when you know what are the optimal parameters for each.

To determine which clustering method performs best, you need to be practical and think about how many clusters you might want to have alongside with a clustering metric to evaluate how good or bad the songs were clustered. If the number of clusters is small, each cluster will be too big and generic. On the contrary, if the number of clusters is too big then each cluster will be too specific and it will be poorly populated (this also depends on how heterogeneous is your dataset).

On the other hand, when you train your clustering model make sure to concatenate both databases together (ie: hot_songs and not_hot_songs) before. If you don't combine both datasets, the clusters obtained with the hot_songs will be different than the ones obtained with the not_hot_songs database even though they might have the same label because they will contain different songs. However, after this, you will not know to which original dataframe belongs each song. To prevent this problem, before the concatenation you can add a new column named "dataset" with a "flag" to remind yourself in which dataset was included ("Hot", "Not hot") each song.

Finally, add a new column to the full dataset for each clustering method with the cluster membership of each song