First you need to clone this repo to your local computer.
$ git clone https://github.com/ShahinZeynalov/scrape_flta_with_selenium_celery_rabbitmq
$ cd scrape_flta_with_selenium_celery_rabbitmq/
After successfuly installed project you need to download requirements.
$ pip install -r requirements.txt
Or download via virtualenv
After successfuly download requirements you need to Build docker bellow command:
$ docker-compose up -d --build
For more info about docker visit to docker hub
After successfuly build docker you need to run celery with below command:
$ celery -A tasks worker --loglevel=INFO --autoscale=AUTOSCALE
For more info go to offical documentation.
$ python scrape.py
- Selenium: selenium offical
- Celery: celery
- BeautifulSoup: bs4