Coder Social home page Coder Social logo

docker-postgresql-patroni's Introduction

Build Status Coverage Status

Patroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul

There are many ways to run high availability with PostgreSQL; for a list, see the PostgreSQL Documentation.

Patroni is a template for you to create your own customized, high-availability solution using Python and — for maximum accessibility — a distributed configuration store like ZooKeeper, etcd or Consul. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in the datacenter—or anywhere else—will hopefully find it useful.

We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.

How Patroni Works

Patroni originated as a fork of Governor, the project from Compose. It includes plenty of new features.

For an example of a Docker-based deployment with Patroni, see Spilo, currently in use at Zalando.

For additional background info, see:

Running the service

The docker container has a number of environment variables that are available for use: Most can be seen in the Dockerfile, the most common are described below - SYNCHRONOUS - set to on to make replication synchronous - ADMINUSER - the main admin user for the database - ADMINPASS - the password for the admin user - ETCD_TTL - the period before a postgres master election occurs after the current master dies - ETCD_TIMEOUT - the period before the call to etcd times out, this is described as a string eg. "2s - 2 seconds, 1m - 1 minute"

Example usage: ` docker run -t -i -e ADMINUSER="postgres" -e ADMINPASS="password" patroni-dev --etcd=192.168.99.100 `

Development Status

Patroni is in active development and accepts contributions. See our Contributing section below for more details.

Technical Requirements/Installation

For Mac

To install requirements on a Mac, run the following:

brew install postgresql etcd haproxy libyaml python
pip install psycopg2 pyyaml

Running and Configuring

To get started, do the following from different terminals:

> etcd --data-dir=data/etcd
> ./patroni.py postgres0.yml
> ./patroni.py postgres1.yml

You will then see a high-availability cluster start up. Test different settings in the YAML files to see how the cluster’s behavior changes. Kill some of the components to see how the system behaves.

Add more postgres*.yml files to create an even larger cluster.

Patroni provides an HAProxy configuration, which will give your application a single endpoint for connecting to the cluster's leader. To configure, run:

> haproxy -f haproxy.cfg
> psql --host 127.0.0.1 --port 5000 postgres

YAML Configuration

Go here for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see postgres0.yml.

Replication Choices

Patroni uses Postgres' streaming replication, which is asynchronous by default. For more information, see the Postgres documentation on streaming replication.

Patroni's asynchronous replication configuration allows for maximum_lag_on_failover settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the follower. This setting should be increased or decreased based on business requirements.

When asynchronous replication is not optimal for your use case, investigate Postgres's synchronous replication. Synchronous replication ensures consistency across a cluster by confirming that writes are written to a secondary before returning to the connecting client with a success. The cost of synchronous replication: reduced throughput on writes. This throughput will be entirely based on network performance.

In hosted datacenter environments (like AWS, Rackspace, or any network you do not control), synchronous replication significantly increases the variability of write performance. If followers become inaccessible from the leader, the leader effectively becomes read-only.

To enable a simple synchronous replication test, add the follow lines to the parameters section of your YAML configuration files:

synchronous_commit: "on"
synchronous_standby_names: "*"

When using synchronous replication, use at least three Postgres data nodes to ensure write availability if one host fails.

Choosing your replication schema is dependent on your business considerations. Investigate both async and sync replication, as well as other HA solutions, to determine which solution is best for you.

Applications Should Not Use Superusers

When connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from an application, you can potentially use the entire connection pool, including the connections reserved for superusers, with the superuser_reserved_connections setting. If Patroni cannot access the Primary because the connection pool is full, behavior will be undesirable.

Contributing

Patroni accepts contributions from the open-source community; see the Issues Tracker for current needs.

Before making a contribution, please let us know by posting a comment to the relevant issue. If you would like to propose a new feature, please first file a new issue explaining the feature you’d like to create.

docker-postgresql-patroni's People

Contributors

osterzel avatar pixie79 avatar lewismarshall avatar

Stargazers

Mehboob Alam avatar  avatar

Watchers

Marcin Ciszak avatar Jits avatar Rustem Suniev avatar Luke Ashe-Browne avatar Douglas Gardner avatar James Cloos avatar Leigh Eyles avatar Colin Gallagher avatar Samuel.Hughes avatar  avatar Mohammud Yassine Jaffoo avatar Leon de Jager avatar Tim Gent avatar Martin Devlin avatar Iqbal Shaikh avatar Geoffrey Martin avatar  avatar Vijay Jadhav avatar Syed Rafiq avatar  avatar Ben Eustace avatar  avatar

Forkers

uk-gov-mirror

docker-postgresql-patroni's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.