xcloudplatform / 1template_aiservicegpuhostedstabilityotterllama

FastAPI ⚡ 🤗 Huggingface Service Template

Question Answering API

🤗 Huggingface + ⚡ FastAPI = ❤️ Awesomeness. How to structure Deep Learning model serving REST API with FastAPI

How to server Hugging face models with FastAPI, the Python's fastest REST API framework.

The minimalistic project structure for development and production.

Installation and setup instructions to run the development mode model and serve a local RESTful API endpoint.

Project structure

Files related to application are in the huggingfastapi or tests directories. Application parts are:

huggingfastapi
├── api              - Main API.
│   └── routes       - Web routes.
├── core             - Application configuration, startup events, logging.
├── models           - Pydantic models for api.
├── services         - NLP logics.
└── main.py          - FastAPI application creation and configuration.
│
tests                - Codes without tests is an illusion 

Swagger Example

Requirements

Python 3.6+, [Make and Docker]

Installation

Install the required packages in your local environment (ideally virtualenv, conda, etc.).

python -m venv .venv
source .venv/bin/activate
make install

Running Localhost

make run

Running Via Docker

make deploy

Running Tests

make test

Setup

1. Duplicate the .env.example file and rename it to .env

2. In the .env file configure the API_KEY entry. The key is used for authenticating our API.
   Execute script to generate .env, and replace example_keywith the UUID generated:

make generate_dot_env
python -c "import uuid;print(str(uuid.uuid4()))"

Run without make for development

1. Start your app with:

PYTHONPATH=./huggingfastapi uvicorn main:app --reload

2. Go to http://localhost:8000/docs or http://localhost:8000/redoc for alternative swagger

3. Click Authorize and enter the API key as created in the Setup step.

Run Tests without using make

Intall testing libraries and run tox:

pip install tox pytest flake8 coverage bandit
tox

This runs tests and coverage for Python 3.8 and Flake8, Bandit.

TODO

• Change make to invoke: invoke is overrated
• Add endpoint for uploading text file and questions