Possible to run on Jetson Nano? about whisper-asr-webservice

[hslr4]

For me it worked on a jetson orin using the following Dockerfile:

FROM swaggerapi/swagger-ui:v4.18.2 AS swagger-ui
FROM nvcr.io/nvidia/l4t-pytorch:r35.2.1-pth2.0-py3

RUN export DEBIAN_FRONTEND=noninteractive \
    && apt-get -qq update \
    && apt-get -qq install --no-install-recommends \
    python3-pip \
    ffmpeg \
    && rm -rf /var/lib/apt/lists/*

RUN pip3 install -U pip setuptools

WORKDIR /app

COPY requirements.txt ./

RUN pip3 install -r requirements.txt

COPY . .
COPY --from=swagger-ui /usr/share/nginx/html/swagger-ui.css swagger-ui-assets/swagger-ui.css
COPY --from=swagger-ui /usr/share/nginx/html/swagger-ui-bundle.js swagger-ui-assets/swagger-ui-bundle.js

CMD gunicorn --bind 0.0.0.0:9000 --workers 1 --timeout 0 app.webservice:app -k uvicorn.workers.UvicornWorker

It is based on nvidia's pytorch container so it can easily make use of the GPU. However this container is based on python 3.8 so additional adjustments are necessary. Since I haven't checked out poetry yet I replaced it with following requirements.txt:

unidecode >= 1.3.4, == 1.*
uvicorn [standard] >= 0.18.2, == 0.*
gunicorn >= 20.1.0, == 20.*
tqdm >= 4.64.1, == 4.*
transformers >= 4.22.1, == 4.*
python-multipart >= 0.0.5, == 0.*
ffmpeg-python >= 0.2.0, == 0.*
fastapi >= 0.95.1, == 0.*
llvmlite >= 0.39.1, == 0.*
numba >= 0.56.4, == 0.*
openai-whisper == 20230124
faster-whisper >= 0.4.1, == 0.*

Finally I removed the importlib dependency from webservice.py to fix some error.

I guess there are better ways to solve this problem and keep this repositories structure, so I am providing this just as a reference of one way to possibly achieve your goal.

[xiechengmude]

faster-whisper

Thanks for your script here.

How to set the model type =Largev3?