''' This project is developed with python 3.6 in order to collect and process external data from different sources such as social media "Twitter" and other sites like "Google News". Those data will be integrated to a NoSQL database "MongoDB" to be consumed by another application. '''
''' To ensure that this project will work successfully you need to install first of all a virtualenv to prevent any conflict with other system libraries.Then, you should install libraries existant in text file "requirements.txt. '''
.
data-collection
├── common
├── api
├── config
├── constant
├── crawlers
│ ├── twitter_crawlers
│ │ └── init.py
│ ├── google_news_crawlers
│ │ ├── google_news_crawlers.py
│ │ └── init.py
│ └── wikipedia_crawlers
│ └── init.py
│
│
├──tests
├──config.json
├──requirements.txt
├── main.py
├── README.md
└── .gitignore
Enjoy !!!