tonanhngo / kfp-tekton

Kubeflow Pipeline compiler to compile KFP DSL to Tekton YAML.

Kubeflow Pipelines DSL Compiler for Tekton

Project to bring Kubeflow Pipelines DSL and Tekton together. The work is being driven in accordance with this evolving design doc specifications. The work is split in phases. While the details of the phases are listed in the design doc, the current effort in this repository is focused on creating a Kubeflow Pipeline compiler for Tekton, which can take KFP DSL, and compile it to Tekton YAML. You can then run those Pipelines on Tekton directly.

If you would like to take the compiled output to Kubeflow Pipelines (KFP) engine and run it through KFP SDK and UI, please follow the evolving code in the backend repository

Tekton

The Tekton Pipelines project provides Kubernetes-style resources for declaring CI/CD-style pipelines. Tekton introduces several new CRDs including Task, Pipeline, TaskRun, and PipelineRun. A PipelineRun represents a single running instance of a Pipeline and is responsible for creating a Pod for each of its Tasks and as many containers within each Pod as it has Steps. Please look for more details in Tekton repo.

Kubeflow Pipeline DSL to Tekton Compiler

We are currently using Kubeflow Pipelines 0.5.0 and Tekton 0.13.0 for this project.

Getting Started

Getting started with KFP Tekton Compiler SDK

Developer Guide

Developer Guide

Available Features and Implementation Details

Available Features and Implementation Details

Compiler Status Report

Compilation Tests Status Report

Samples

Samples being run end to end for verification

KFP, Argo and Tekton Features Comparison

KFP, Argo and Tekton Features Comparison

Design Doc

Design Doc

CD Foundation

• CD Foundation MLOps Sig.
• Instructions to join

Additional Reference Materials: KFP and TFX

• Kubeflow and TFX Pipelines
• Kubeflow and TFX Pipelines talk at Tensorflow World