arnavforks / pipebird

Pipebird is open source infrastructure for securely sharing data with customers.

Welcome to Pipebird

Pipebird is the open source platform used for syncing data to customers' data warehouses. SaaS companies deploy Pipebird to boost revenue by offering customers secure data syncing, without the headache of building and maintaining sharing infrastructure.

With Pipebird you can immediately:

• select sources to push data from (such as PostgreSQL).
• let customers configure pipelines and apply transformations (such as type casting).
• sync data directly to customers' warehouses (such as Snowflake).

Pipebird is for companies that want to offer enterprise-grade data sharing.

• Minimize security and compliance risks created by involving third-party ETL providers. Pipebird enables direct data sharing from your source to a customer's data warehouse. Your data never hits our servers.
• Eliminate pipeline complexity for customers and partners. Customers can trust verified pipelines offered directly from your product. It takes minutes to activate customer-defined pipelines using a declarative configuration language.
• Internalize revenue previously captured by third-party ETL providers. Rather than contracting a third-party, customers pay you for higher-quality data, ease of use, and security enhancements.

Get started for free

Deploy on your own infrastructure and keep control of your data.

Click here to view our deployment guide.

Join the Pipebird Slack Community or email [email protected] if you'd like help with your deployment.

Platform features

Sources

Data originates from one of your company's sources, which can be any of:

• Postgres
• Redshift
• CockroachDB
• MySQL
• MariaDB
• etc.

Destinations

Your customers can define their own destinations, which your team can configure through our Destinations API.

• Snowflake
• Amazon S3
• Amazon Redshift
• BigQuery [in progress]
• Databricks [in progress]
• CSV Export

Data transformers

Customers can choose to define some set of transformations to be applied on data by uploading Configurations which define mutations on the source data. For example, a consumer may want the Date column updated_at to be casted into a DateTime object in the destination.

We currently support renaming columns between sources and destinations and will be expanding destinations and working on transformations like:

• casting data types
• sums
• averages
• sorts
• groupby
• etc.

Our goal for Pipebird

We believe that SaaS companies sharing data directly with their customers is the future of ELT/ELT. In this world, creating a data pipeline is as simple as pressing a button from a SaaS vendor's dashboard.

Companies like Stripe and Customer.io have already invested in building out native data sharing features for their customers. Pipebird makes it easy for any company to offer the same powerful data sharing features.

We'd love to work with you to grow Pipebird. Feel free to message us in the Pipebird Slack Community.

If you like Pipebird, please show support by starring this repo 🌟

Open source vs. paid versions

This repo is entirely MIT licensed, with the exception of the ee directory (if applicable).

Premium features (contained in the ee directory) require a Pipebird license. Contact us at [email protected] for more information, or see our pricing page.

Pipebird is entirely free for developers. We'll make money by charging larger companies that have more specific needs for additional features around security and scale.

Want to book a meeting with someone on our team? Choose a time here!