nicolai1233 / bigdatagruppe5

this git contians the code from the project

bigDataGruppe5

This git contians the code from the project

The folder Spark SQL contians the source code for the SparkSQL to be ran. The code runs spark SQL on the kaggle and EU Data from hdfs and cleans the output IF on the NODE the code could be ran if in the data folder by Running the command python mainTest.py

The folder Twiiter Streaming contains the source code for streaming data from twitter, and simple cleaning of the output. IF on the NODE the streaming code could be ran if in the folder data/denRigtig with the command python3 main.py. and to do simple cleaning and add to HDFS the command python addToHDFS.py could be called