miguelfzafra / latest-news-classifier

Master in Data Science Final Project

Latest News Classifier

This is my Final Project from the MSc in Data Science at KSchool.

This project is intended to be a walkthrough on the development of a machine learning project from end to end. It covers the creation a real-time web application that gathers data from several newspapers and shows a summary of the different topics that are being treated in the news articles.

This is achieved with a supervised machine learning classification model that is able to predict the category of a given news article, a web scraping method that gets the latest news from the newspapers, and an interactive web application that shows the obtained results to the user.

The web application can be used at https://latestnewsclassifier.herokuapp.com. Please consider that a live web-scraping process is carried out every time you run the app, so there may be some crashes due to the failing status of some requests.

The full detailed process is explained in the Report. Following it and the different notebooks should be necessary for reaching a full understanding of the project.

The project consists of the following steps:

1. Dataset Creation
2. Exploratory Data Analysis
3. Feature Engineering
4. Model Training
5. News Scraping
6. App Creation
7. Annex - Installation
8. Annex - Deployment
9. Report
10. App Creation v2 (*I created a new version of the application after submitting the project.)

For any comments, suggestions or everything else, you can find me at my mail, linkedin or webpage!

latest-news-classifier's Issues

Error in confusion matrix axis description

I believe the correct labeling for y-axis is "Actual" label while the x-axis is for "Predicted". In your notebook for SVM training these are mixed up.

Aside from this, thanks a lot for these notebooks, they are great tutorials to get your hands on NLP end to end ! 👍

MT failed because of object type

Hello, I've been trying to run your code on Raspberry Pi but I'm having issue with the Model Training.

When I try :
`# First create the base model to tune
rfc = RandomForestClassifier(random_state=8)

Definition of the random search

random_search = RandomizedSearchCV(estimator=rfc,
param_distributions=random_grid,
n_iter=50,
scoring='accuracy',
cv=3,
verbose=1,
random_state=8)

Fit the random search model

random_search.fit(features_train, labels_train)`

I get the answer :

` File "/home/pi/.local/lib/python3.7/site-packages/sklearn/utils/multiclass.py", line 172, in check_classification_targets
raise ValueError("Unknown label type: %r" % y_type)
ValueError: Unknown label type: 'unknown'

FitFailedWarning)
/home/pi/.local/lib/python3.7/site-packages/sklearn/model_selection/_validation.py:552: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 531, in _fit_and_score`

I read online that the issue could be that when loading the data, the format isn't respected
see: https://stackoverflow.com/questions/45346550/valueerror-unknown-label-type-unknown

Any idea why this happens and what to do ?

Can't thank you enough for all the work you put in to this btw.

Thanks

Gettig always the same result

hello ;
when i test best_svc model with every sample i get the same result ;

The predicted category using the SVM model is sport.
The conditional probability is: 37.89371475423645 

Error : "Scrolled": true - name True undefined

Hello,

I'm quite new to python and I'm trying to make your program work on a Raspberry pi.

I'm having this error when I run 03. Features engineering:

"Scrolled": true - name True undefined

I checked that everything is up to date but I can't find a way around this issue.

Any help would be greatly appreciated .

Thanks for the great work anyway

10.App Creation v2 does not perform the web scrapping

Hi Miguel,

First of all thanks for this so well structured example. I really appreciate it.
Second is about the Folder 06 & 10. (related to App creation.
I realized that my dash version is new so I need to run the App in the folder 10. App Creation v2.
Everything works fine until the dashboard is displayed. And once I click in the webscrapping it does not perform any search neither any plot.

The error appears in the notebook:
File "C:\Users\ ... \anaconda3\lib\site-packages\requests\adapters.py", line 511, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='elpais.com', port=443): Max retries exceeded with url: /elpais/inenglish.html (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1045)')))
127.0.0.1 - - [25/Oct/2019 10:55:45] "POST /_dash-update-component HTTP/1.1" 500 -

Do you have any idea about how to fix it?

Many thanks again.