Kubeflow Pipelines SDK (KFP SDK)

To create a KFP SDK pipeline:

  • Ensure your Provider supports KFP SDK by specifying the KFP SDK image in spec.frameworks[].
  • Create a Pipeline resource, specifying:
    • the kfpsdk framework in spec.framework.name. This needs to match the name specified in the Provider.
    • the fully qualified name of the Python function that creates a KFP SDK pipeline under spec.framework.parameters[].pipeline.

KFP SDK Parameters

NameDescription
pipelineFully qualified name of the Python function creating a KFP SDK pipeline. This function should be wrapped using the kfp.dsl.Pipeline decorator.

Environment Variables

The KFP SDK compiler automatically injects the following environment variables during compilation:

VariableDescription
KFP_PIPELINE_IMAGESet to the value of spec.image from the Pipeline resource. Use this in your pipeline code to dynamically set component base images, e.g. @dsl.component(base_image=os.environ.get("KFP_PIPELINE_IMAGE")).

KFP SDK Pipeline resource example

apiVersion: pipelines.kubeflow.org/v1beta1
kind: Pipeline
metadata:
  name: kfpsdk-quickstart
spec:
  provider: provider-namespace/kfp
  image: kfp-operator-kfpsdk-quickstart:v1
  framework:
    name: kfpsdk
    parameters:
      pipeline: getting_started.pipeline.add_pipeline