Overview

The Kubeflow Pipelines Operator (KFP Operator) offers a declarative API designed to streamline the management and execution of ML pipelines across various providers using resource definitions. A “provider” refers to a runtime environment that handles the orchestration and execution of these pipelines and associated resources.

Why KFP Operator

This project was initiated with the goal of promoting best practices in Machine Learning engineering while minimizing the operational complexities involved in deploying, executing, and maintaining training pipelines. This project seeks to move away from manual, error-prone, copy-and-paste deployments, and towards a declarative, transparent, and self-service model.

By configuring simple Kubernetes resources, machine learning practitioners can run their desired training pipelines in each environment on the path to production in a repeatable, testable and scalable way. When linked with serving components, this provides a fully testable path to production for machine learning systems.

The KFP Operator bridges the gap between Continuous Delivery (CD) and Continuous Training (CT), enabling Level 2 of the MLOps Maturity model.