Coder Social home page Coder Social logo

cnpg-i's Introduction

CNCF Landscape Latest Release GitHub License Documentation Stack Overflow

Welcome to the CloudNativePG project!

CloudNativePG is a comprehensive open source platform designed to seamlessly manage PostgreSQL databases within Kubernetes environments, covering the entire operational lifecycle from initial deployment to ongoing maintenance. The main component is the CloudNativePG operator.

CloudNativePG was originally built and sponsored by EDB.

Table of content

Getting Started

The best way to get started is with the "Quickstart" section in the documentation.

Scope

The goal of CloudNativePG is to increase the adoption of PostgreSQL, one of the most loved DBMS in traditional VM and bare metal environments, inside Kubernetes, thus making the database an integral part of the development process and GitOps CI/CD automated pipelines.

In scope

CloudNativePG has been designed by Postgres experts with Kubernetes administrators in mind. Put simply, it leverages Kubernetes by extending its controller and by defining, in a programmatic way, all the actions that a good DBA would normally do when managing a highly available PostgreSQL database cluster.

Since the inception, our philosophy has been to adopt a Kubernetes native approach to PostgreSQL cluster management, making incremental decisions that would answer the fundamental question: "What would a Kubernetes user expect from a Postgres operator?".

The most important decision we made is to have the status of a PostgreSQL cluster directly available in the Cluster resource, so to inspect it through the Kubernetes API. We've fully embraced the operator pattern and eventual consistency, two of the core principles upon which Kubernetes is built for managing complex applications.

As a result, the operator is responsible for managing the status of the Cluster resource, keeping it up to date with the information that each PostgreSQL instance manager regularly reports back through the API server. Changes to the cluster status might trigger, for example, actions like:

  • a PostgreSQL failover where, after an unexpected failure of a cluster's primary instance, the operator itself elects the new primary, updates the status, and directly coordinates the operation through the reconciliation loop, by relying on the instance managers

  • scaling up or down the number of read-only replicas, based on a positive or negative variation in the number of desired instances in the cluster, so that the operator creates or removes the required resources to run PostgreSQL, such as persistent volumes, persistent volume claims, pods, secrets, config maps, and then coordinates cloning and streaming replication tasks

  • updates of the endpoints of the PostgreSQL services that applications rely on to interact with the database, as Kubernetes represents the single source of truth and authority

  • updates of container images in a rolling fashion, following a change in the image name, by first updating the pods where replicas are running, and then the primary, issuing a switchover first

The latter example is based on another pillar of CloudNativePG: immutable application containers - as explained in the blog article "Why EDB Chose Immutable Application Containers".

The above list can be extended. However, the gist is that CloudNativePG exclusively relies on the Kubernetes API server and the instance manager to coordinate the complex operations that need to take place in a business continuity PostgreSQL cluster, without requiring any assistance from an intermediate management tool responsible for high availability and failover management like similar open source operators.

CloudNativePG also manages additional resources to help the Cluster resource manage PostgreSQL - currently Backup, ClusterImageCatalog, ImageCatalog, Pooler, and ScheduledBackup.

Fully embracing Kubernetes means adopting a hands-off approach during temporary failures of the Kubernetes API server. In such instances, the operator refrains from taking action, deferring decisions until the API server is operational again. Meanwhile, Postgres instances persist, maintaining operations based on the latest known state of the cluster.

Out of scope

CloudNativePG is exclusively focused on the PostgreSQL database management system maintained by the PostgreSQL Global Development Group (PGDG). We are not currently considering adding to CloudNativePG extensions or capabilities that are included in forks of the PostgreSQL database management system, unless in the form of extensible or pluggable frameworks.

CloudNativePG doesn't intend to pursue database independence (e.g. control a MariaDB cluster).

Communications

Resources

Adopters

A list of publicly known users of the CloudNativePG operator is in ADOPTERS.md. Help us grow our community and CloudNativePG by adding yourself and your organization to this list!

CloudNativePG at KubeCon

Useful links

Star History

Star History Chart

Trademarks

Postgres, PostgreSQL and the Slonik Logo are trademarks or registered trademarks of the PostgreSQL Community Association of Canada, and used with their permission.

cnpg-i's People

Contributors

armru avatar fcanovai avatar gbartolini avatar leonardoce avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

cnpg-i's Issues

[Docs]: Automated documentation

Is there an existing issue already for your request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new documentation enhancement to be evaluated.

What problem in the existing documentation this issue aims to solve?

We need an automated way to generate the documentation of the API.

Describe what additions need to be done to the documentation

Use https://github.com/pseudomuto/protoc-gen-doc

Describe what pages need to change in the documentation, if any

No response

Describe what pages need to be removed from the documentation, if any

No response

Additional context

No response

Backport?

N/A

Code of Conduct

  • I agree to follow this project's Code of Conduct

[Feature]: Metadata for a plugin

Is there an existing issue already for this feature request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

We are lacking a way for each plugin to document itself.

Describe the solution you'd like

We should add some ways to set metadata in each plugin. Look at this for ideas: https://github.com/cloudnative-pg/cloudnative-pg/blob/main/config/olm-manifests/bases/cloudnative-pg.clusterserviceversion.yaml

Describe alternatives you've considered

N/A

Additional context

N/A

Are you willing to actively contribute to this feature?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

[Feature]: CNPG-I service for management of WAL files

Is there an existing issue already for this feature request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

We need a generic interface for managing WAL files

Describe the solution you'd like

Write a generic interface for managing WAL files, primarily archiving and restore. It would be useful to also think about diagnostic information/status (e.g. first available WAL, last available WAL, history files, ...) and introspection.

Describe alternatives you've considered

N/A

Additional context

Prototype. Related to #3699

Are you willing to actively contribute to this feature?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/continuous-delivery.yml
  • actions/checkout v4
  • arduino/setup-task v2
.github/workflows/release-please.yml
  • google-github-actions/release-please-action v3
gomod
go.mod
  • go 1.21
  • google.golang.org/grpc v1.63.2
  • google.golang.org/protobuf v1.33.0
regex
.github/workflows/continuous-delivery.yml
  • dagger/dagger 0.10.3
Taskfile.yml
  • protolint main@f3ddf3912214a4f992e58ff43b6bc92be4b757d2
  • protoc-gen-go-grpc main@f3ddf3912214a4f992e58ff43b6bc92be4b757d2
  • golangci-lint main@9a3074edb9cb21746ea2f9dcdd018d520b47f2e6
  • protoc-gen-go-grpc main@f3ddf3912214a4f992e58ff43b6bc92be4b757d2
  • golang main@57352e06a1cfbcb5307c009d37c0201b2719b935
  • commitlint main@f3ddf3912214a4f992e58ff43b6bc92be4b757d2
  • uncommitted main@f3ddf3912214a4f992e58ff43b6bc92be4b757d2

  • Check this box to trigger a request for Renovate to run again on this repository

[Feature]: move dagger modules to their own repo

Is there an existing issue already for this feature request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

The modules used in the cnpg-i CI are generic, and could be reused in different projects. They should live in their own repository.

Describe the solution you'd like

Have a cloudnative-pg/daggerverse repo where we keep our modules

Describe alternatives you've considered

N/A

Additional context

N/A

Are you willing to actively contribute to this feature?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

[Feature]: Export metadata in JSON/YAML output

Is there an existing issue already for this feature request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

Metadata is currently in the source code.

Describe the solution you'd like

Provide a default method that every plugin inherits to convert the metadata (#4) in JSON and/or YAML format.

Describe alternatives you've considered

N/A

Additional context

N/A

Are you willing to actively contribute to this feature?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

[Feature]: Pipeline MVP

Is there an existing issue already for this feature request/idea?

  • I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

CNPG-I should have a pipeline to validate the source, build documentation, linting, check build is coherent with the protocol, spellcheck ...

Describe the solution you'd like

Create a GitHub action to fulfil the above. Feel free to extend the scope during investigation.

Describe alternatives you've considered

N/A

Additional context

No response

Are you willing to actively contribute to this feature?

Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.