Find alternative ways to offload resource-heavy and time-consuming tasks about mlplayground

[nothingface0]

Asynchronous tasks that need to be offloaded to a separate process:

• Histogram File parsing (Very high cpu usage, takes a long time)
• RunRegistry/OMS API queries (takes a long time, may even timeout)

Celery seems to be an overkill. Perhaps simple Threads will do? (Histogram file parsing is already done this way)

---

Note: It must be taken into account that tasks may fail. Should there be a periodic task which cleans up/resumes those operations? Maybe the tasks are just put in a queue/table, and this periodic task checks what should be started/resumed/cleaned up.

---

Sources

1. https://stackoverflow.com/questions/21945052/simple-approach-to-launching-background-task-in-django
2. https://stackoverflow.com/questions/17601698/can-you-perform-multi-threaded-tasks-within-django