Topic: data-engineering Goto Github
Some thing interesting about data-engineering
Some thing interesting about data-engineering
data-engineering,A list of useful resources to learn Data Engineering from scratch
User: adilkhash
data-engineering,The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Organization: airbytehq
Home Page: https://airbyte.com
data-engineering,Implementing best practices for PySpark ETL jobs and applications.
User: alexioannides
data-engineering,The Data Engineering Cookbook
User: andkret
Home Page: https://learndataengineering.com/
data-engineering,Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Organization: apache
Home Page: https://airflow.apache.org/
data-engineering,Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
Organization: apache
Home Page: https://devlake.apache.org/
data-engineering,Apache Superset is a Data Visualization and Data Exploration Platform
Organization: apache
Home Page: https://superset.apache.org/
data-engineering,Workflow Engine for Kubernetes
Organization: argoproj
Home Page: https://argo-workflows.readthedocs.io/
data-engineering,Turns Data and AI algorithms into production-ready web applications in no time.
Organization: avaiga
Home Page: https://www.taipy.io
data-engineering,pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Organization: aws
Home Page: https://aws-sdk-pandas.readthedocs.io
data-engineering,Fancy stream processing made operationally mundane
Organization: benthosdev
Home Page: https://www.benthos.dev
data-engineering,The open source high performance ELT framework powered by Apache Arrow
Organization: cloudquery
Home Page: https://cloudquery.io
data-engineering,An orchestration platform for the development, production, and observation of data assets.
Organization: dagster-io
Home Page: https://dagster.io
data-engineering,Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
Organization: dagworks-inc
Home Page: https://hamilton.dagworks.io/en/latest/
data-engineering,Compare tables within or across databases
Organization: datafold
Home Page: https://docs.datafold.com
data-engineering,Roadmap to becoming a data engineer in 2021
Organization: datastacktv
Home Page: https://datastack.tv
data-engineering,Free Data Engineering course!
Organization: datatalksclub
data-engineering,data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
Organization: dlt-hub
Home Page: https://dlthub.com/docs
data-engineering,📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
User: eugeneyan
data-engineering,Distributed DataFrame for Python designed for the cloud, powered by Rust
Organization: eventual-inc
Home Page: https://getdaft.io
data-engineering,Business intelligence as code: build fast, interactive data visualizations in pure SQL and markdown
Organization: evidence-dev
Home Page: https://evidence.dev
data-engineering,Feature Store for Machine Learning
Organization: feast-dev
Home Page: https://feast.dev
data-engineering,Feathr – A scalable, unified data and AI engineering platform for enterprise
Organization: feathr-ai
Home Page: https://join.slack.com/t/feathrai/shared_invite/zt-1ffva5u6v-voq0Us7bbKAw873cEzHOSg
data-engineering,Learn how to design, develop, deploy and iterate on production-grade ML applications.
User: gokumohandas
Home Page: https://madewithml.com
data-engineering,Learn how to design, develop, deploy and iterate on production-grade ML applications.
User: gokumohandas
Home Page: https://madewithml.com
data-engineering,Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017
Organization: googlecloudplatform
data-engineering,Always know what to expect from your data.
Organization: great-expectations
Home Page: https://docs.greatexpectations.io/
data-engineering,Open Source Feature Flagging and A/B Testing Platform
Organization: growthbook
Home Page: https://www.growthbook.io
data-engineering,An Awesome List of Open-Source Data Engineering Projects
User: gunnarmorling
data-engineering,A collection of scientific methods, processes, algorithms, and systems to build stories & models. Whether you are a fresher in the field or an experienced professional who wants to transition into Data Science & AI
User: hemansnation
Home Page: https://www.himanshuramchandani.co/
data-engineering,CSVs sliced, diced & analyzed.
User: jqnatividad
data-engineering,:bar_chart: :clipboard: Dashboards using YAML or JSON files
User: kantord
Home Page: https://kantord.github.io/just-dashboard/
data-engineering,Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
Organization: kestra-io
Home Page: https://kestra.io
data-engineering,🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
Organization: mage-ai
Home Page: https://www.mage.ai/
data-engineering,Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Organization: meltano
Home Page: https://meltano.com/
data-engineering,Memphis.dev is a highly scalable and effortless data streaming platform
Organization: memphisdev
Home Page: https://memphis.dev
data-engineering,A low code Machine Learning personalized ranking service for articles, listings, search results, recommendations that boosts user engagement. A friendly Learn-to-Rank engine
Organization: metarank
Home Page: https://metarank.ai
data-engineering,Data Science Roadmap from A to Z
User: moataz-elmesmary
data-engineering,Build AI Assistants with function calling and connect LLMs to external tools.
Organization: phidatahq
Home Page: https://docs.phidata.com
data-engineering,The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
Organization: ploomber
Home Page: https://docs.ploomber.io
data-engineering,Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines
Organization: prefecthq
Home Page: https://prefect.io
data-engineering,Quadratic | Data Science Spreadsheet with Python & SQL
Organization: quadratichq
Home Page: https://QuadraticHQ.com
data-engineering,Quilt is a data mesh for connecting people with actionable data
Organization: quiltdata
Home Page: https://quiltdata.com
data-engineering,Scalable Postgres for stream processing, analytics, and management. KsqlDB and Apache Flink alternative. 🚀 10x more productive. 🚀 10x more cost-efficient.
Organization: risingwavelabs
Home Page: https://www.risingwave.com/slack
data-engineering,Datart is a next generation Data Visualization Open Platform
Organization: running-elephant
Home Page: https://running-elephant.github.io/datart-docs/
data-engineering,Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
User: san089
data-engineering,:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io
Organization: sodadata
Home Page: https://go.soda.io/core-docs
data-engineering,lakeFS - Data version control for your data lake | Git for data
Organization: treeverse
Home Page: https://docs.lakefs.io
data-engineering,SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.
User: whoiskatrin
Home Page: https://www.sqltranslate.app/
data-engineering,:shell: Python-powered, cross-platform, Unix-gazing shell.
Organization: xonsh
Home Page: http://xon.sh
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.