Comments (5)
If anyone needs this, I did it as follows:
FROM nvidia/cuda:11.0.3-cudnn8-runtime-ubuntu18.04
RUN apt-get update
RUN apt-get install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa -y
RUN apt-get install python3.8 -y
RUN apt-get install python3.8-distutils -y
RUN apt-get install python-dev python3-dev python3.8-dev \
build-essential libssl-dev libffi-dev \
libxml2-dev libxslt1-dev zlib1g-dev -y
RUN apt-get update && \
apt-get install -y --no-install-recommends gcc
RUN apt-get install curl wget -y
RUN apt-get install linux-libc-dev -y
RUN apt-get install libc6-dev -y
RUN wget https://bootstrap.pypa.io/get-pip.py
RUN python3.8 get-pip.py
ENV APP_HOME=/app
WORKDIR $APP_HOME
COPY . /app
RUN pip3.8 install --no-cache-dir -r requirements.txt
EXPOSE 80
CMD ["gunicorn","main:app","-w","1","--threads", "1", "--timeout", "1500", \
"--worker-connections=300","-k","uvicorn.workers.UvicornWorker","--bind", "0.0.0.0:80"]
My requirements.txt:
tensorflow
tensorflow-hub
matplotlib
six
pillow
notebook
pydantic
fastapi
uvicorn[standard]
gunicorn
Probably don't need the notebook there. Also the big timeout time is due to me loading this in my laptop that doesn't have cuda installed so the model takes forever to load and I need to increase the timeout for testing. on a GPU you wouldn't need it.
from uvicorn-gunicorn-fastapi-docker.
Hmm, not really, everything runs on the CPU and only very specific custom and complex code would run on GPU.
To make something run on GPU you would use something like a deep learning framework like PyTorch or a tool to perform Numpy-like operations in GPU, like https://cupy.chainer.org/. But it would be only for Numpy-like operations. Linear algebra stuff in big matrices and vectors, that's what GPUs can do very well.
But the rest of the API code would still run in CPU.
Now, to use nvidia-docker
you might want to look at https://github.com/tiangolo/uvicorn-gunicorn-machine-learning-docker
I haven't been able to update it in a while but might still help you a bit.
from uvicorn-gunicorn-fastapi-docker.
When my number of workers is 2, there is a CUDA GPUS error while I want to use workers for 1 GPU. Can someone help me please
from uvicorn-gunicorn-fastapi-docker.
Thanks for the response. I eventually got a system working with nvidia-docker and flask. I wasn't aware of that ML repo you made. I'll check that out for my next project.
from uvicorn-gunicorn-fastapi-docker.
Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.
from uvicorn-gunicorn-fastapi-docker.
Related Issues (20)
- Application does not start up when combining this image and a database connection in Linux CentOS. HOT 4
- Upgrade from python3.8 to python3.11 fails HOT 1
- ModuleNotFoundError: No module named 'app' HOT 2
- APScheduler logs not showing in docker HOT 1
- websocket can not use
- Deploying HuggingFace model/pipeline using uvicorn-gunicorn-fastapi-docker on Google Cloud Run HOT 2
- Is --proxy-headers applied in the uvicorn CMD in the image? HOT 1
- getting timeout 504 in 1 minute
- Support for Apple sillicon
- New docker-image may be broken HOT 1
- apt-get update not working on specific version
- container build fails with pydantic-settings HOT 1
- Uvicorn -gunicorn-fastapi-docker
- Runtime Error: Can't start new thread HOT 1
- linux/arm64/v8 not supported HOT 2
- Unexpected Redirection of `/docs` Endpoint with Custom `APP_MODULE` Setup
- Current gunicorn version does not support Python 3.11
- Base Dockerfile HOT 3
- pydantic_settings package not supported
- Update fastapi to support Pydantic2 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uvicorn-gunicorn-fastapi-docker.