venosov / kubedbt

:calendar: Run, schedule, and manage your dbt jobs using Kubernetes.

Kubedbt

Run, schedule, and manage your dbt (data build tool) jobs using Kubernetes!

Requirements

Since this projects uses different tools, you'll need to have access to a Kubernetes cluster and to a dbt suported database.

Usage

First, replace the sample project called project-test with yours or use this one as a test. The tool dbt also expect a profiles.yaml file with the connection details. The best option is to let dbt generate it from the environmental variables you can specify in the default profile.

If that doesn't work for you, you can also create a profile following the official instructions and mount it using Docker volumes (example).

At this point, if the dbt run command is working locally, tweak the schedule and other parameters in dbt-conjob.yaml, create the proper secrets in Kubernetes and deploy with make deploy. This should start running your dbt model with your custom schedule!

License

Copyright (c) David Gasquez Licensed under the MIT license.