TensorFlow Extended (TFX)

To create a TFX pipeline:

  1. Ensure your Provider supports TFX by specifying the TFX image in spec.frameworks[].
  2. Create a Pipeline resource, specifying:
  • the tfx framework in spec.framework.name. This needs to match the name specified in the Provider.
  • the fully qualified name of the Python function creating TFX pipeline components under spec.framework.parameters[].components.
  • any required beam arguments under spec.framework.parameters[].beamArgs.

TFX Parameters

NameDescription
componentsFully qualified name of the Python function creating TFX pipeline components.
beamArgs[]List of named objects. These will be provided as beam_pipeline_args when compiling the pipeline.

TFX Pipeline resource example

apiVersion: pipelines.kubeflow.org/v1beta1
kind: Pipeline
metadata:
  name: penguin-pipeline
spec:
  provider: provider-namespace/provider-name
  image: kfp-quickstart:v1
  framework:
    name: tfx
    parameters:
      pipeline: penguin_pipeline.pipeline.create_components
      beamArgs:
      - name: anArg
        value: aValue