Installation

We recommend the installation using Helm as it allows a declarative approach to managing Kubernetes resources.

This guide assumes you are familiar with Helm.

Prerequisites

Build and Install

Create basic values.yaml with the following content:

fullnameOverride: kfp-operator
manager:
  argo:
    serviceAccount: pipeline-runner
  configuration:
    defaultExperiment: Default
    pipelineStorage: {STORAGE_LOCATION}
  provider:
    type: kfp
    configuration:
      kfpNamespace: kubeflow
      restKfpApiUrl: http://ml-pipeline.kubeflow:8888
      grpcKfpApiAddress: ml-pipeline.kubeflow:8887
      grpcMetadataStoreAddress: metadata-grpc-service.kubeflow:8080

Install the latest version of the operator

helm install oci://ghcr.io/kfp-operator/kfp-operator -f values.yaml

Configuration Values

Valid configuration options to override the Default values.yaml are:

Parameter nameDescription
containerRegistryContainer Registry base path for all container images
namespace.createCreate the namespace for the operator
namespace.nameOperator namespace name
manager.argo.containerDefaultsContainer Spec defaults to be used for Argo workflow pods created by the operator
manager.argo.metadataContainer Metadata defaults to be used for Argo workflow pods created by the operator
manager.argo.ttlStrategyTTL Strategy used for all Argo Workflows
manager.argo.stepTimeoutSeconds.compileTimeout in seconds for compiler steps - defaults to 1800 (30m)
manager.argo.stepTimeoutSeconds.defaultDefault timeout in seconds for workflow steps - defaults to 300 (5m)
manager.argo.serviceAccount.nameThe k8s service account used to run Argo workflows
manager.argo.serviceAccount.createCreate the Argo Workflows service account (or assume it has been created externally)
manager.argo.serviceAccount.metadataOptional Argo Workflows service account default metadata
manager.metadataObject Metadata for the manager’s pods
manager.rbac.createCreate roles and rolebindings for the operator
manager.serviceAccount.nameManager service account’s name
manager.serviceAccount.createCreate the manager’s service account or expect it to be created externally
manager.replicasNumber of replicas for the manager deployment
manager.resourcesManager resources as per k8s documentation
manager.configurationManager configuration as defined in Configuration (note that you can omit compilerImage and kfpSdkImage when specifying containerRegistry as default values will be applied)
manager.monitoring.createCreate the manager’s monitoring resources
manager.monitoring.rbacSecuredEnable addtional RBAC-based security
manager.monitoring.serviceMonitor.createCreate a ServiceMonitor for the Prometheus Operator
manager.monitoring.serviceMonitor.endpointConfigurationAdditional configuration to be used in the service monitor endpoint (path, port and scheme are provided)
manager.multiversion.enabledEnable multiversion API. Should be used in production to allow version migration, disable for simplified installation
manager.webhookCertificates.providerK8s conversion webhook TLS certificate provider - choose cert-manager for Helm to deploy certificates if cert-manager is available or custom otherwise (see below)
manager.webhookCertificates.secretNameName of a K8s secret deployed into the operator namespace to secure the webhook endpoint with, required if the custom provider is chosen
manager.webhookCertificates.caBundleCA bundle of the certificate authority that has signed the webhook’s certificate, required if the custom provider is chosen
manager.provider.typeProvider type (kfp for Kubeflow Pipelines or vai for Vertex AI Pipelines)
manager.provider.configurationConfiguration block for the specific provider (see Provider Configuration), automatically mounted as a file
logging.verbosityLogging verbosity for all components - see the logging documentation for valid values
eventsourceServer.metadataObject Metadata for the eventsource server’s pods
eventsourceServer.rbac.createCreate roles and rolebindings for the eventsource server
eventsourceServer.serviceAccount.nameEventsource server’s service account
eventsourceServer.serviceAccount.createCreate the eventsource server’s service account or expect it to be created externally
eventsourceServer.resourcesEventsource server resources as per k8s documentation
providersDictionary of providers (see below)

Examples for these values can be found in the test configuration

Providers

The providers block contains a dictionary of provider names to provider configurations:

Parameter nameDescription
typeProvider type (kfp or vai)
serviceAccount.nameName of the service account to run provider-specific operations
serviceAccount.createCreate the service account (or assume it has been created externally)
serviceAccount.metadataOptional service account default metadata
configurationSee Provider Configuration for all available providers and their respective configuration options

Example:

providers:
  kfp:
    type: kfp
    serviceAccount:
      name: kfp-operator-kfp
      create: false
    configuration:
      ...
  vai:
    type: vai
    serviceAccount: 
      name: kfp-operator-kfp
      create: true
      metadata:
        annotations:
          iam.gke.io/gcp-service-account: kfp-operator-vai@my-project.iam.gserviceaccount.com
    configuration:
      ...