newsclustering's Introduction

News Clustering via GibsLDA++

NewsClustering is a final project developed by a group of three students (Ilaria Ceppa, Marco Grandi and Marco Ponza) for the Information Retrieval course.

The goal of the project was to develop, experiment and analyze results of a clustering software which uses GibsLDA++ to generate clusters of italian news articles.

The final report is available in the current repository (italian only).

Setting up

The project can be compiled by typing:

make clean
make all

and the helper can be displayed with:

./clusteringLDA --help

Cluster Generation

To run the application on a news dataset type:

./clusteringLDA [-v] [-a alpha] [-b beta] [-n clusters] [-t terms] [-m size] [-i iter] [-s step] [-o file] [-c clust] [-d string] dataset_file

where:

-v shows the parameter values before running the application;
-a alpha set the alpha parameter of GibsLDA++;
-b beta set the beta parameter of GibsLDA++;
-n clusters set the number of clusters you want to generate;
-t terms set the number of terms that will be showed to the output file;
-m size minimum cluster size (clusters with a lower size will be removed);
-i iter set the number of iterations of GibsLDA++;
-s step set the number of iterations after which a temporary model will be generated;
-o file set the output file;
-c clust model name generated by GibsLDA++;
-d string set the preprocessing algorithms to NOT use:
. disables the punctuation filter;
s disables stopwords;
w disables shingling;
i disable the idf filter;
m disables cluster-size thresholding;
p disables document filter.

Recommend Projects