Topic: databricks Goto Github

Some thing interesting about databricks

👇 Here are 635 public repositories matching this topic...

adidas / lakehouse-engine

databricks,The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

Organization: adidas

Home Page: https://adidas.github.io/lakehouse-engine-docs/

big-data configuration-driven data-engineering data-quality databricks delta-lake framework great-expectations lakehouse spark

aehrc / variantspark

databricks,machine learning for genomic variants

Organization: aehrc

Home Page: http://bioinformatics.csiro.au/variantspark

variant-spark gwas random-forest genome association-studies vcf variantspark notebook databricks bioinformatics

aloneguid / stowage

databricks,Bloat-free, no BS cloud storage SDK.

User: aloneguid

databricks aws-s3 azure-storage gcp-storage

amrit-hub / databricks-certified-data-engineer-associate-questions

databricks,This repo contains "Databricks Certified Data Engineer Associate" Questions and related docs.

User: amrit-hub

Home Page: https://amrit-hub.github.io/Databricks-Certified-Data-Engineer-Associate-Questions/

sql certification data-engineering databricks

azure-samples / azure-databricks-mlops-mlflow

databricks,Azure Databricks MLOps sample for Python based source code using MLflow without using MLflow Project.

Organization: azure-samples

azure databricks databricks-notebooks mlflow mlflow-projects mlops ml ml-production ml-monitoring ml-ops

azure-samples / modern-data-warehouse-dataops

databricks,DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.

Organization: azure-samples

devops dataops data mdw azure databricks datafactory cicd automatedtesting

azure / azure-cosmosdb-spark

databricks,Apache Spark Connector for Azure Cosmos DB

Organization: azure

azure-cosmos-db cosmos-db spark connector jupyter-notebook databricks azure-databricks lambda-architecture changefeed databricks-notebooks

azure / azure-databricks-client

databricks,Client library for Azure Databricks

Organization: azure

databricks azure

azure / azure-event-hubs-spark

databricks,Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Organization: azure

spark spark-streaming azure scala eventhubs real-time streaming continuous apache apache-spark microsoft event-hubs connector databricks stream structured-streaming bigdata ingestion kafka

benchaliah / timeseriesgan

databricks,GANs for Time series analysis (Synthetic data generation, anomaly detection and interpolation), Hypertuning using Optuna, MLFlow and Databricks

User: benchaliah

causal-convolution databricks dilated-convolution generative-adversarial-network hyperparameter-optimization hyperparameter-tuning mlops spark stock-market

cartodb / analytics-toolbox-core

databricks,A set of UDFs and Procedures to extend BigQuery, Snowflake, Redshift, Postgres and Databricks with Spatial Analytics capabilities

Organization: cartodb

bigquery snowflake carto gis redshift geospatial databricks postgres sql analytics-toolbox

crflynn / databricks-api

databricks,A simplified, autogenerated API client interface using the databricks-cli package

User: crflynn

databricks api-client

databricks / cli

databricks,Databricks CLI

Organization: databricks

command-line-interface databricks

databricks / databricks-sdk-py

databricks,Databricks SDK for Python (Beta)

Organization: databricks

Home Page: https://databricks-sdk-py.readthedocs.io/

databricks python databricks-sdk

databricks / databricks-sql-python

databricks,Databricks SQL Connector for Python

Organization: databricks

databricks dwh python3 sql

databricks / dbt-databricks

databricks,A dbt adapter for Databricks.

Organization: databricks

Home Page: https://databricks.com

dbt sql etl databricks

databricks / terraform-databricks-examples

databricks,Examples of using Terraform to deploy Databricks resources

Organization: databricks

Home Page: https://registry.terraform.io/modules/databricks/examples/databricks/latest

aws azure databricks gcp lakehouse terraform terraform-module databricks-module

databricks / terraform-provider-databricks

databricks,Databricks Terraform Provider

Organization: databricks

Home Page: https://registry.terraform.io/providers/databricks/databricks/latest

databricks terraform-provider aws azure terraform gcp databricks-automation

databrickslabs / cicd-templates

databricks,Manage your Databricks deployments and CI with code.

Organization: databrickslabs

cd-pipeline databricks azure-devops mlops ci gitlab github-actions aws azure

databrickslabs / dbldatagen

databricks,Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POCs, and other uses in Databricks environments including in Delta Live Tables pipelines

Organization: databrickslabs

Home Page: https://databrickslabs.github.io/dbldatagen

datagen pyspark python data-generation faker spark spark-streaming delta-live-tables deltalake databricks synthetic-data datageneration datagenerator

databrickslabs / dbx

databricks,🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.

Organization: databrickslabs

Home Page: https://dbx.readthedocs.io

databricks databricks-api databricks-cli ci cicd mlops

databrickslabs / dolly

databricks,Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

Organization: databrickslabs

Home Page: https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html

databricks gpt chatbot dolly

databrickslabs / jupyterlab-integration

databricks,DEPRECATED: Integrating Jupyter with Databricks via SSH

Organization: databrickslabs

jupyter jupyter-notebook databricks databricks-api databricks-deploy

databrickslabs / overwatch

databricks,Capture deep metrics on one or all assets within a Databricks workspace

Organization: databrickslabs

databricks monitoring

databrickslabs / ucx

databricks,Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.

Organization: databrickslabs

databricks unity-catalog databricks-cli-installable

dataflint / spark

databricks,Performance Observability for Apache Spark

Organization: dataflint

apache-spark big-data observability optimization data-pipeline data-pipelines databricks dataproc emr etl spark-operator

datathirstltd / azure.databricks.cicd.tools

databricks,Tools for Deploying Databricks Solutions in Azure

Organization: datathirstltd

powershell databricks devops azure

delta-io / delta-rs

databricks,A native Rust library for Delta Lake, with bindings into Python

Organization: delta-io

Home Page: https://delta-io.github.io/delta-rs/

delta rust delta-lake databricks python pandas pandas-dataframe

djouallah / testing_bi_engine

databricks,TPC-H_SF10

User: djouallah

tpc-h snowflake synapse-analytics bigquery duckdb-database databricks olap sqlserver alloydb

dotlas / databricks_helpers

databricks,🧱 A collection of supplementary utilities and helper notebooks to perform admin tasks on Databricks

Organization: dotlas

admin administration databricks migration notebook workflows pandas pydantic sqlalchemy

dotnet / spark

databricks,.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Organization: dotnet

Home Page: https://dot.net/spark

spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core

flmu / mlflow-tracking-server

databricks,MLFLow Tracking Server based on Docker and AWS S3

User: flmu

Home Page: https://www.mlflow.org/docs/latest/tracking.html

docker docker-image mlflow mlflow-tracking-server docker-file databricks machine-learning ui aws aws-s3

getredash / redash

databricks,Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Organization: getredash

Home Page: http://redash.io/

redash python visualization analytics bi redshift bigquery athena mysql postgresql

hystax / optscale

databricks,FinOps and MLOps platform to run ML/AI and regular cloud workloads with optimal performance and cost.

Organization: hystax

Home Page: https://hystax.com

aws azure devops gcp kubernetes cost-optimization finops mlops ml cloud-cost cloud-cost-intelligence cloud databricks mlflow s3-optimization data-science experiment-tracking paas-instrumentation paas-profiling

inspera / blackbricks

databricks,Black for Databricks notebooks

Organization: inspera

black formatter sql python databricks databricks-notebooks pre-commit pre-commit-hook

lamastex / scalable-data-science

databricks,Scalable Data Science, course sets in big data Using Apache Spark over databricks and their mathematical, statistical and computational foundations using SageMath.

User: lamastex

Home Page: https://lamastex.github.io/scalable-data-science/

apache-spark databricks data-science scala

lhbench / lhbench

databricks,Lakehouse storage system benchmark

Organization: lhbench

Home Page: https://lhbench.cs.berkeley.edu

apache-hudi apache-iceberg lakehouse benchmark cidr database databricks delta-lake

martandsingh / apachespark

databricks,This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

User: martandsingh

apachespark data-analysis data-engineering database databricks datalake deltalake etl-pipeline hadoop hive

microsoft / dstoolkit-mlops-databricks

databricks,ML Ops Accelerator: Databricks & Azure Machine Learning Unification

Organization: microsoft

databricks azure mlops mlflow databricks-feature-store machine-learning cicd azure-machine-learning

microsoft / nutter

databricks,Testing framework for Databricks notebooks

Organization: microsoft

databricks-notebooks databricks azuredevops

microsoft / synapseml

databricks,Simple and Distributed Machine Learning

Organization: microsoft

Home Page: http://aka.ms/spark

spark pyspark azure scala microsoft ml machine-learning databricks cognitive-services lightgbm

mlcraft-io / mlcraft

databricks,Synmetrix – open source semantic layer / Boost your LLM precision

Organization: mlcraft-io

Home Page: https://synmetrix.org

business-intelligence clickhouse redshift big-data cube cubejs databricks prestodb druid firebolt

osin-vladimir / architect_big_data_solutions_with_spark

databricks,code, labs and lectures for the course

User: osin-vladimir

databricks databricks-notebooks aws-machine-learning spark-streaming etl data-analysis spark machine-learning python data-science

prefecthq / prefect-databricks

databricks,Prefect integrations for interacting with Databricks.

Organization: prefecthq

Home Page: https://prefecthq.github.io/prefect-databricks/

prefect databricks python

tencent / apijson

databricks,🏆 零代码、全功能、强安全 ORM 库 🚀 后端接口和文档零代码，前端(客户端) 定制返回 JSON 的数据和结构。 🏆 A JSON Transmission Protocol and an ORM Library 🚀 provides APIs and Docs without writing any code.

Organization: tencent

Home Page: http://apijson.cn

postgresql-database tidb sqlserver oracle clickhouse postgresql hive hadoop tdengine low-code