wri / gfw-data-api

GFW Data API

GFW Data API

High-performance Async REST API, in Python. FastAPI + GINO + Uvicorn (powered by PostgreSQL).

Get Started

Run Locally with Docker

1. Clone this Repository. git clone https://github.com/wri/gfw-data-api.git
2. Run ./scripts/setup from the root directory. (Run pip install pipenv first, if necessary.)
3. Run locally using docker-compose. ./scripts/develop

Developing

• Generate a DB Migration: ./scripts/migrate (note app/settings/prestart.sh will run migrations automatically when running /scripts/develop)
• Run tests: ./scripts/test
  • --no_build - don't rebuild the containers
  • --moto-port=<port_number> - explicitly sets the motoserver port (default 50000)
• Run specific tests: ./scripts/test tasks/test_vector_source_assets.py::test_vector_source_asset
• Each development branch app instance gets its isolated database in AWS dev account that's cloned from geostore database. This database is named with the branch suffix (like geostore_<branch_name>). If a PR includes a database migration, once the change is merged to higher environments, the geostore database needs to also be updated with the migration. This can be done by manually replacing the existing database by a copy of a cleaned up version of the branch database (see ./prestart.sh script for cloning command).
• Debug memory usage of Batch jobs with memory_profiler:
  1. Install memory_profiler in the job's Dockerfile
  2. Modify the job's script to run with memory_profiler. Ex: pixetl "${ARG_ARRAY[@]}" -> mprof run -M -C -T 1 --python /usr/local/app/gfw_pixetl/pixetl.py "${ARG_ARRAY[@]}"
  3. scp memory_profiler's .dat files off of the Batch instance (found in /tmp by default) while the instance is still up

Features

Core Dependencies

• FastAPI: touts performance on-par with NodeJS & Go + automatic Swagger + ReDoc generation.
• GINO: built on SQLAlchemy core. Lightweight, simple, asynchronous ORM for PostgreSQL.
• Uvicorn: Lightning-fast, asynchronous ASGI server.
• Optimized Dockerfile: Optimized Dockerfile for ASGI applications, from https://github.com/tiangolo/uvicorn-gunicorn-docker.

Additional Dependencies

• Pydantic: Core to FastAPI. Define how data should be in pure, canonical python; validate it with pydantic.
• Alembic: Handles database migrations. Compatible with GINO.
• SQLAlchemy_Utils: Provides essential handles & datatypes. Compatible with GINO.
• PostgreSQL: Robust, fully-featured, scalable, open-source.

gfw-data-api's Issues

Bug: account creation confirmation email leads to an error page

Describe the bug
After creating an account and clicking the confirmation link on the email we get an error.

To Reproduce
Steps to reproduce the behavior:

1. Create a new account
2. Click the link on the confirmation email

Expected behavior
Probably should redirect somewhere else or display a success message.

Screenshots

Desktop (please complete the following information):

• OS: Debian 10.16 (Buster), Arch 6.3.8
• Browser: Firefox
• Version: 102.11.0esr (64-bit)

Bug: internal server error on the sign up

Describe the bug
An internal server error is happening on the sign-up, even tho the sign-up is working since I got the account creation confirmation email and I was able to verify it.

To Reproduce
Steps to reproduce the behavior:

curl "https://data-api.globalforestwatch.org/auth/sign-up" \
  -X POST \
  -d "{\n  \"name\": \"joaodemenech\",\n  \"email\": \"[email protected]\"\n}" \
  -H "content-type: application/json" 

Expected behavior
According to the docs, it should respond with 200.

Screenshots

Desktop (please complete the following information):

• OS: Debian 10.16 (Buster), Arch 6.3.8
• Browser: rest client/curl