n0nuser / fastapi-archetype

FastAPI Production-Ready Archetype based in DDD

FastAPI Production Archetype

Description

This is a FastAPI project archetype for production-ready applications. It provides a structure for building scalable and maintainable applications with FastAPI following Domain Driven Design (DDD) principles, best Docker practices, and a set of tools for development, testing, and deployment with the best code quality tools available.

Table of Contents

• FastAPI Production Archetype
  • Description
  • Table of Contents
  • Contributing
  • Getting Started
    • Built With
    • Prerequisites
    • Running the App
      • Terminal
      • VSCode
      • Docker
    • Development
  • Roadmap

Contributing

Check the contributing documentation for more information.

Getting Started

Built With

Development:

• Python 3: The programming language used.
• Poetry: A tool for dependency management and packaging in Python.
• FastAPI: A modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.
• Pydantic: Data validation and settings management using Python type annotations.
• HTTPX: A fully featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2.

Development Tools:

• Pre-Commit: A framework for managing and maintaining multi-language pre-commit hooks.
• Ruff: A tool for managing Python environments.
• Flake8: A tool that glues together pycodestyle, pyflakes, mccabe, and third-party plugins to check the style and quality of some Python code.
• Pylint: A tool that checks for errors in Python code, tries to enforce a coding standard, looks for code smells, and can offer simple refactoring suggestions.
• Bandit: A tool designed to find common security issues in Python code.

Databases:

• SQLAlchemy: The Python SQL Toolkit and Object-Relational Mapping (ORM) library.
• PostgreSQL: A powerful, open-source object-relational database system.

Deployment:

• Uvicorn: A lightning-fast ASGI server implementation, using uvloop and httptools.
• Docker: A set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers.
• Docker-Compose: A tool for defining and running multi-container Docker applications.

Testing:

• Pytest: A framework that makes it easy to write simple and scalable tests.
• Pytest-Mock: A thin-wrapper around the mock package for easier use with py.test.

Prerequisites

• Python 3.10 or higher
• Poetry: Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you. For documentation refer to Poetry or to a little guide we made here.
• Docker: Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. For documentation refer to Docker.

Running the App

You need to have the environment set up with Poetry by using poetry install and the dependencies will be installed. After that, you can start the application.

Terminal

To start the application, run the following command:

uvicorn src.app:app --reload --port 8000

This will start the application on port 8000. You can change the port by changing the --port argument.

VSCode

In VSCode you can use the Run button on the top right corner of the editor to start the application as the launch.json file is already set up.

Docker

Also, you can use the docker-compose to start the application with the following command:

cd docker
docker-compose up

If you don't know how to use Docker, you can check the Docker documentation and our deployment documentation.

Development

Important

Be sure to:

• Run pre-commit install to install the pre-commit hooks. This will run the linters and formatters before you commit your code. If you don't have pre-commit installed in your system, you can install it with pip install pre-commit.
• Check the project structure documentation for more information.
• Check the recommended extensions documentation for more information.
• Check the contributing documentation for more information.
• Check the deployment documentation for more information.

Roadmap

See the open issues for a full list of proposed features (and known issues).

fastapi-archetype's Issues

Add Celery support for heavy tasks

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository does not support handling heavy or long-running tasks efficiently. This can lead to performance issues and a poor user experience for applications that require background processing.

---

Describe the solution you'd like

• Integrate Celery to handle heavy and long-running tasks asynchronously. This includes:
  • Setting up Celery with the repository to offload tasks from the main application.
  • Configuring Celery to use a message broker like Redis or RabbitMQ.
• Integrate Flower to monitor and manage Celery tasks. This includes:
  • Setting up Flower to provide a web-based UI for monitoring task execution and status.
  • Ensuring Flower can be easily started alongside the main application and Celery worker.

---

Describe alternatives you've considered

• Implementing custom background task handling, which can be complex and less reliable compared to using Celery.
• Using other task queues or job processing systems, but Celery is widely adopted and has extensive community support and documentation.

---

Additional context

• Adding Celery support will significantly enhance the repository's ability to handle resource-intensive and time-consuming tasks without blocking the main application.
• Including Flower provides an intuitive way to monitor and manage tasks, improving the maintainability and reliability of applications built using this repository.
• Providing example configurations and usage documentation for Celery and Flower will help developers quickly set up and utilize these tools.

Add OpenTelemetry

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository lacks a unified way to collect, process, and visualize traces and metrics across distributed services, making it difficult to monitor and debug performance issues and service dependencies.

---

Describe the solution you'd like

• Integrate OpenTelemetry to provide comprehensive observability capabilities for tracing and metrics. This includes:
  • Setting up OpenTelemetry SDK to instrument the application for tracing and metrics collection.
  • Configuring exporters to send collected data to preferred backends (e.g., Jaeger, Prometheus, ELK Stack).
  • Providing example configurations and code snippets to demonstrate how to integrate OpenTelemetry with the application.

---

Describe alternatives you've considered

• Using other observability solutions such as Zipkin or New Relic, but OpenTelemetry offers a vendor-neutral and standardized approach to observability with broad language support and flexibility.
• Implementing custom tracing and metrics collection, which is complex and less feature-rich compared to using a comprehensive solution like OpenTelemetry.

---

Additional context

• Integrating OpenTelemetry will significantly enhance the observability of applications, making it easier to monitor performance, understand service dependencies, and troubleshoot issues.
• Providing detailed documentation and examples for setting up and using OpenTelemetry will help developers quickly adopt these practices and tools.
• This feature will improve the overall reliability and maintainability of applications built using this repository, ensuring that performance issues and bottlenecks can be identified and resolved efficiently.

Improve logging by adding compatibility with ELK stack

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository lacks integrated logging and monitoring capabilities, making it difficult to track, analyze, and visualize application performance and issues in real-time. This hampers the ability to quickly diagnose and resolve problems.

---

Describe the solution you'd like

• Integrate the ELK Stack (Elasticsearch, Logstash, Kibana) to provide comprehensive logging and monitoring capabilities. This includes:
  • Setting up Elasticsearch to store and index logs.
  • Configuring Logstash to collect, parse, and transform log data from the application.
  • Integrating Kibana to visualize and explore logs and metrics through an intuitive web interface.
• Provide example configurations and deployment scripts to easily set up and manage the ELK Stack alongside the application.

---

Describe alternatives you've considered

• Using other logging and monitoring solutions like the EFK Stack (Elasticsearch, Fluentd, Kibana) or Prometheus with Grafana, but the ELK Stack is well-established and offers a rich set of features for log management and analysis.
• Implementing custom logging solutions, which lack the robustness and visualization capabilities provided by the ELK Stack.

---

Additional context

• Integrating the ELK Stack will significantly enhance the repository's ability to monitor application health, debug issues, and gain insights into performance.
• Detailed documentation and examples for setting up and using the ELK Stack will help developers quickly adopt and leverage these powerful tools.
• This feature will improve overall observability and reliability, making it easier to maintain and optimize applications built using this repository.

Add Tests and improve Coverage

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository lacks sufficient tests and code coverage, which can lead to undetected bugs and regressions, reducing the reliability and maintainability of the codebase.

---

Describe the solution you'd like

• Implement comprehensive tests to cover existing code, ensuring functionality is verified and edge cases are handled. This includes:
  • Unit tests for individual functions and modules.
  • Integration tests to verify interactions between different parts of the system.
  • End-to-end tests to simulate user interactions and validate the entire workflow.
• Aim for a high code coverage percentage to ensure that most of the codebase is tested.
• Set up a Continuous Integration (CI) pipeline to run tests automatically on each commit and pull request, ensuring new changes do not introduce regressions.

---

Describe alternatives you've considered

• Manual testing, but this approach is time-consuming, less reliable, and not scalable for larger projects.
• Relying solely on developer discipline to write tests for new code, which can be inconsistent and lead to gaps in test coverage.

---

Additional context

• Implementing tests with good coverage will improve the reliability and maintainability of the repository, making it easier to identify and fix bugs early in the development process.
• Providing guidelines and examples for writing tests can help developers contribute high-quality code and maintain consistent testing practices.
• Integrating code coverage tools to monitor and report coverage metrics can help ensure that the codebase remains well-tested over time.

Add Redis Caching for DB

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• Our application experiences slow response times and high database load during peak traffic periods. This results in a poor user experience and increased costs for database scaling.

---

Describe the solution you'd like

• I would like to implement Redis caching to store frequently accessed data in memory. This will reduce the number of direct database queries, leading to faster response times and decreased load on the database.

---

Describe alternatives you've considered

• We've considered using other caching mechanisms such as Memcached, but Redis offers additional features like persistence, replication, and support for more complex data structures that are beneficial for our use case.
• Another alternative is optimizing database queries and indexing, but this alone does not sufficiently address the high load and response time issues during peak traffic.

---

Additional context

• Redis caching can also improve scalability and reliability of the application. Implementing Redis as a caching layer is a widely adopted best practice for improving application performance in high-traffic scenarios.
• Including benchmarks or case studies demonstrating the benefits of Redis caching in similar applications could support the request.

Add security and user management by default

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository lacks built-in security and user management features, which can lead to vulnerabilities and inefficiencies when setting up authentication and authorization for new applications.

---

Describe the solution you'd like

• Integrate the fastapi-users library to provide robust security and user management features by default. This will include:
  • User registration and authentication (with support for JWT and OAuth2).
  • Password reset functionality.
  • User profile management.
  • Role-based access control.

---

Describe alternatives you've considered

• Implementing custom user management and security features from scratch, but this approach is time-consuming and error-prone.
• Using other libraries for authentication and user management, but fastapi-users is well-integrated with FastAPI and offers comprehensive features out-of-the-box.

---

Additional context

• Adding fastapi-users will significantly reduce the setup time for new projects, providing a secure and reliable user management system.
• This feature ensures that security best practices are followed, enhancing the overall robustness of applications built using this repository.
• Including detailed documentation and examples for using fastapi-users within the repository can help developers quickly adopt and customize the provided features.

Add Payload Compression

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• There is no specific problem at the moment since the backend archetype does not have an active application yet. However, adding payload compression can proactively enhance data transfer efficiency and prepare the archetype for future high-traffic scenarios.

---

Describe the solution you'd like

• Integrate middleware that compresses HTTP responses using algorithms such as gzip or Brotli. This will reduce the size of the payloads sent from the server to the client, improving data transfer speeds and reducing bandwidth usage.

---

Describe alternatives you've considered

• One alternative is to leave payload compression to be handled by a reverse proxy or CDN. However, having built-in middleware for compression ensures that the feature is available regardless of the deployment environment and provides a consistent performance improvement.

---

Additional context

• Payload compression is a common optimization technique that can significantly improve web application performance, especially for users with slower internet connections.
• Including payload compression in the backend archetype sets a standard for efficient data handling and can serve as a valuable feature for developers who use this archetype for their projects.

Exclusive Filters

❓Exclusive Filters

Describe your question or ask for support.

• When adding multiple filter to the get list function from the CRUD base, these filters work exclusively, i.e. these filters work as an "and" condition. Before an "or" option was commented when adding the filter but now it was deleted. Should we recover this option or is it implemented somewhere else?

Improve versioning of repository

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository lacks a systematic approach to versioning, leading to potential issues with tracking changes, managing releases, and maintaining backward compatibility.

---

Describe the solution you'd like

• Implement a versioning strategy using Semantic Versioning (SemVer) to clearly communicate changes and their impact.
• Integrate automated tools for version management such as:
  • git tags to mark release points in the repository.
  • CHANGELOG.md to document changes in each version.
  • Continuous Integration (CI) pipeline updates to automatically increment versions based on commit messages (using tools like semantic-release).

---

Describe alternatives you've considered

• Manually managing versions and changelogs, which is prone to errors and inconsistencies.
• Using a date-based versioning system, but this does not convey the nature and impact of changes as effectively as Semantic Versioning.

---

Additional context

• Implementing a standardized versioning system helps maintain clear communication with users and contributors about the state of the repository.
• Automating versioning and changelog updates can save time and reduce the risk of human error, ensuring that the repository is always in a state ready for release.
• This feature can enhance the repository’s professionalism and reliability, making it easier to manage and contribute to.

Add Bulk Insert and Update to CRUD

🚀 Add Bulk Insert and Update to CRUD

Is your feature request related to a problem? Please describe.

• As of now, the CRUD only allows for creating elements one by one. Bulk inserting and updating is necessary for more efficient, performant operations.

---

Describe the solution you'd like

• Implement Bulk Inserting and Updating into the CRUD.

Improve Documentation

📚 Documentation Issue Report

Describe the bug

• Documentation might be a little too hard for newbies. Might not have enough code examples or not have explicit best practices written for intermediate users. And may be some files could use some examples of their modification for expert users, like Dockerfiles.

---

Describe the solution you'd like

For Beginners:

• Simplify Technical Jargon: The documentation contains technical terms and jargon that might be overwhelming for beginners. A glossary of terms or simpler explanations could make the material more accessible.
• Step-by-Step Guide: A more detailed step-by-step guide for setting up the development environment, including installing Python, Poetry, and other dependencies, could help beginners who are unfamiliar with these tools.
• Troubleshooting Section: Adding a troubleshooting section for common setup and runtime errors could help beginners navigate issues more independently.

For Intermediate Users:

• Code Examples: More code examples, especially in the context of FastAPI and how it integrates with other components of the project (like SQLAlchemy for database interactions), could provide clearer insights into practical usage.
• Best Practices: While the repository follows Domain Driven Design principles, elaborating on these principles and their implementation in the project could offer valuable learning opportunities for intermediate users.

For Expert Users:

• Advanced Features and Customization: Documentation on advanced features, customization options, and extending the project could engage expert users. This might include customizing the Docker setup, extending the database models, or integrating additional services.
• Performance Optimization: Insights into performance considerations and optimization strategies used in the project could be valuable for expert users looking to adapt the archetype for high-performance applications.
• Security Best Practices: While the repository likely considers security practices, explicitly documenting these practices, recommendations for secure deployment, and how to handle sensitive data would benefit users aiming to use this in production environments.

General Improvements:

• Interactive Documentation: Incorporating interactive API documentation using tools like Swagger UI (already part of FastAPI but should be highlighted in the documentation) can help users explore the API endpoints more intuitively.
• Community and Contribution: Encouraging community involvement through a clearer contribution guide, how to submit issues, and request features could foster a more active community. The existing CONTRIBUTING.md file is a good start but highlighting how users can contribute to the documentation itself could also be beneficial.
• Accessibility: Ensuring the documentation is accessible, including considerations for screen readers and other assistive technologies, can make the project more inclusive.

---

Additional context

Add Alembic and documentation for it

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository does not include a database migration tool, making it difficult to manage schema changes over time and ensuring that different environments remain in sync.

---

Describe the solution you'd like

• Integrate Alembic to manage database migrations. This includes:
  • Setting up Alembic with the repository to handle schema migrations for supported databases.
  • Configuring Alembic to work seamlessly with the existing database models.
• Provide comprehensive documentation on how to use Alembic for database migrations. This should include:
  • Instructions for setting up Alembic in a new project.
  • Examples of creating, applying, and rolling back migrations.
  • Best practices for managing database migrations in development and production environments.

---

Describe alternatives you've considered

• Using other migration tools such as Flyway or Liquibase, but Alembic is specifically designed to work with SQLAlchemy and is widely adopted in the Python ecosystem.
• Writing custom scripts for database schema changes, which is error-prone and lacks the robustness and features of a dedicated migration tool.

---

Additional context

• Adding Alembic will streamline the process of managing database schema changes, reducing the risk of inconsistencies and deployment issues.
• Comprehensive documentation will help developers quickly get up to speed with using Alembic, ensuring a smooth workflow for handling database migrations.
• This feature will enhance the repository's overall functionality and usability, making it more robust and developer-friendly.

Spectral Validation with Automatic Export of OpenAPI specification

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• There is no specific problem at the moment since the backend archetype does not have an active application yet. However, ensuring that the OpenAPI documentation is up-to-date and validated can streamline the development process and improve API reliability.

---

Describe the solution you'd like

• Implement automation for extracting the OpenAPI.json from the codebase. This process should be triggered during the build or deployment pipeline to ensure the API documentation is always current.
• Integrate a pre-commit hook using Spectral to validate the OpenAPI specification. This will enforce consistency and correctness in the API documentation before changes are committed to the codebase.

---

Describe alternatives you've considered

• Manually extracting and validating the OpenAPI.json, but this approach is error-prone and can lead to outdated documentation.
• Using other tools for OpenAPI validation, but Spectral is specifically designed for this purpose and offers a wide range of rules and customization options.

---

Additional context

• Automating the extraction of OpenAPI.json ensures that the documentation reflects the latest state of the API, reducing the risk of discrepancies and improving developer experience.
• Implementing a pre-commit hook with Spectral helps maintain high standards for API design and documentation quality, catching issues early in the development process.
• This feature can be particularly beneficial for teams working in collaborative environments, ensuring consistent and accurate API documentation across the board.

Add Prometheus compatible metrics possibility

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository lacks integrated metrics collection and monitoring, making it difficult to track application performance and resource usage over time. This limits the ability to proactively identify and address potential issues.

---

Describe the solution you'd like

• Integrate Prometheus-compatible metrics collection to enable robust monitoring and alerting. This includes:
  • Setting up instrumentation in the application to expose metrics in a Prometheus-compatible format.
  • Providing example configurations and code snippets to demonstrate how to collect and expose metrics (e.g., using prometheus_client for Python).
  • Configuring Prometheus to scrape the metrics endpoint and store collected metrics.
  • Optionally, integrating Grafana to visualize and create dashboards for the collected metrics.

---

Describe alternatives you've considered

• Using other monitoring solutions such as Datadog or New Relic, but Prometheus offers a widely-adopted, open-source solution with extensive community support.
• Implementing custom metrics collection and storage, which is more complex and less standardized compared to using Prometheus.

---

Additional context

• Integrating Prometheus-compatible metrics will enhance the observability of applications, allowing developers to track performance and resource usage in real-time.
• Providing detailed documentation and examples for setting up and using Prometheus with the application will help developers quickly adopt these monitoring practices.
• This feature will improve the overall reliability and maintainability of applications built using this repository, enabling proactive performance monitoring and issue resolution.

Add Load Balancing with Traefik

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository does not include support for load balancing, which can lead to performance bottlenecks and reduced reliability in high-traffic scenarios. Efficiently distributing traffic across multiple instances is essential for scaling applications.

---

Describe the solution you'd like

• Integrate Traefik as a load balancer to distribute traffic across multiple application instances. This includes:
  • Configuring Traefik to automatically discover and route traffic to healthy instances.
  • Setting up Traefik to handle dynamic configuration changes based on the application's needs.
• Provide automatic HTTPS support via Traefik using Let's Encrypt, ensuring secure communication for all routed traffic.

---

Describe alternatives you've considered

• Using other load balancing solutions such as Nginx or HAProxy, but Traefik offers modern features like automatic service discovery and built-in Let's Encrypt support with simpler configuration.
• Implementing a custom load balancing solution, which is more complex and less feature-rich compared to using Traefik.

---

Additional context

• Traefik's seamless integration with containerized environments (e.g., Docker, Kubernetes) makes it an ideal choice for modern application architectures.
• Including detailed setup guides and example configurations for Traefik will help developers quickly deploy and manage load-balanced applications.
• This feature will enhance the repository's scalability and reliability, providing a robust solution for handling high-traffic scenarios.

Add Connection Pooling

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• I'm always frustrated when database connections are not efficiently managed, leading to increased latency and potential bottlenecks in our application. Currently, each database request opens a new connection, which increases overhead and decreases performance, especially under high load conditions.

---

Describe the solution you'd like

• I would like to have a connection pooling feature implemented. This feature should allow the reuse of existing database connections rather than opening a new connection for each request. Connection pooling would manage a pool of database connections, improving performance by reducing the overhead associated with establishing new connections and allowing for more efficient resource usage.

---

Describe alternatives you've considered

• Alternatives considered include manually managing database connections within the application code, but this approach is error-prone and does not scale well. Another alternative is using third-party libraries for connection pooling, but native support would be more integrated and efficient.

---

Additional context

• Connection pooling is a common feature in many database systems and frameworks, and its absence is a significant drawback for applications that require high performance and scalability. Implementing this feature would greatly enhance the efficiency and reliability of the database interactions in our application.

Add Nginx and possibility of automatic HTTPS generation

🚀 Feature Request

Is your feature request related to a problem? Please describe.

• The current repository does not include support for setting up Nginx for routing and serving as a reverse proxy, nor does it provide a streamlined method for automatically generating HTTPS certificates. This can make it challenging to deploy applications securely and efficiently.

---

Describe the solution you'd like

• Integrate Nginx configuration within the repository to handle routing and serve as a reverse proxy for the application. This includes:
  • Example Nginx configuration files tailored for common use cases.
  • Guidelines for customizing Nginx settings to suit specific needs.
• Add support for automatic HTTPS certificate generation using Let's Encrypt. This involves:
  • Setting up Certbot or a similar tool to automate the generation and renewal of SSL certificates.
  • Configuring Nginx to use these certificates for secure HTTPS connections.

---

Describe alternatives you've considered

• Manually configuring Nginx and setting up HTTPS certificates, which can be error-prone and time-consuming.
• Using other reverse proxy solutions like Traefik or Caddy, which offer automatic HTTPS but might require additional setup and learning.

---

Additional context

• Including Nginx with automatic HTTPS generation will simplify the deployment process, making it easier to set up secure and scalable applications.
• Providing detailed documentation and examples for setting up Nginx and Let's Encrypt can help developers quickly configure their environments and ensure best practices are followed.
• This feature enhances the repository's functionality by offering a complete solution for secure and efficient web application deployment.