Use RabbitMQ to distribute tasks to multiple workers
One process will create task messages. Multiple worker processes will share the work.
- Fork this starter repo into your GitHub.
- Clone your repo down to your machine.
- View / Command Palette - then Python: Select Interpreter
- Select your conda environment.
- Read the RabbitMQ Tutorial - Work Queues
- Read the code and comments in this repo.
RabbitMQ comes with an admin panel. When you run the task emitter, reply y to open it.
(Python makes it easy to open a web page - see the code to learn how.)
- Run emitter_of_tasks.py (say y to monitor RabbitMQ queues)
Explore the RabbitMQ website.
- Run listening_worker.py
Will it terminate on its own? How do you know?
- Use your emitter_of_tasks to produce more task messages.
- Use your listening_worker.py script to launch a second worker.
Follow the tutorial. Add multiple tasks (e.g. First message, Second message, etc.) How are tasks distributed? Monitor the windows with at least two workers. Which worker gets which tasks?
Examine the excel file and put in the information you want to transmit Run the emitter and the listener to see results!
See a running example with at least 3 concurrent process windows here: