jonathanreadshaw / servingmlfastcelery

Working example for serving a ML model using FastAPI and Celery

ServingMLFastCelery

Working example for serving a ML model using FastAPI and Celery.

Usage

Install requirements:

pip install -r requirements.txt

Set environment variables:

• MODEL_PATH: Path to pickled machine learning model
• BROKER_URI: Message broker to be used by Celery e.g. RabbitMQ
• BACKEND_URI: Celery backend e.g. Redis

export MODEL_PATH=...
export BROKER_URI=...
export BACKEND_URI=...

Start API:

uvicorn app:app

Start worker node:

celery -A celery_task_app:worker worker -l info

servingmlfastcelery's Issues

dockerize the program

Thanks for sharing , really love the article on medium.
How does the cellery and worker being implemented ?on dedicated docker? separated ?
can u please consider share the dockrized version of it - it would be really helpful getting the whole idea of setup.

How to instantiate one single model instance shared by all threads

Thank you for this insightful example to host ML model via Celery and FastAPI. In your blog, you mentioned that "There will be a PredictTask object for each worker process. Therefore if a worker has four threads then the model will be loaded four times (each is then stored in memory as well". This is exactly my concern. I think each thread would try to instantiate its own model instance, but if the model is really expensive or some states need be shared by the calling threads, then we want only one single model instance shared by all threads, but how could we do it properly? Any suggestion?