Goal: Classify Spam Messages. Compare Tfid with plain text results, Hash Vectorizer with n-Grams results where n=2, and Burrows Wheeler Transform Distance (BWTD) with plain text and n-gram results.
Approach:
- Supervised Learning task, because given labeled traning examples
- Binary Classification task
- Use plain text with Tfid
- Use N-Grams with Hash Vectorizer
- Use plain text with BWTD
- Use N-Grams with BWTD
- There is no continuous flow of data, no need to adjust to changing data, and the data is small enough to fit in memmory: Batch Learning
Data: SMS Spam Collection Dataset | Kaggle
Cover Picture: Mobile spam | Flickr
BWTD: Burrows Wheeler Transform Distance (BWTD) by Dr. Edward Raff
@inproceedings{Raff2020,
author = {Raff, Edward and Nicholas, Charles and McLean, Mark},
booktitle = {The Thirty-Fourth AAAI Conference on Artificial Intelligence},
title = {{A New Burrows Wheeler Transform Markov Distance}},
url = {http://arxiv.org/abs/1912.13046},
year = {2020}
}
Project Author: Maksim Ekin Eren