Samples

ML Pipeline Services - Authoring Guideline

Setup

• Create a python3 envionronment.

Python 3.5 or above is required and if you don't have Python3 set up, we suggest the following steps to install Miniconda.

In Debian/Ubuntu/Cloud shell environment:

apt-get update; apt-get install -y wget bzip2
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

In Windows environment, download the installer and remember to select "Add Miniconda to my PATH environment variable" option during the installation.

In Mac environment, download the installer and run the following command:

bash Miniconda3-latest-MacOSX-x86_64.sh

Then, create a clean python3 envionrment

conda create --name mlpipeline python=3.6
source activate mlpipeline

If conda command is not found, be sure to add the Miniconda path:

export PATH=MINICONDA_PATH/bin:$PATH

• Clone the repo.

• Install DSL library and DSL compiler

pip install https://storage.googleapis.com/ml-pipeline/release/0.0.26/kfp-0.0.26.tar.gz --upgrade

After successful installation the command "dsl-compile" should be added to your PATH.

Compile the samples

The sample pipelines are represented as Python code. To run these samples, you need to compile them, and then upload the output to the Pipeline system from the web UI.

dsl-compile --py [path/to/py/file] --output [path/to/output/tar.gz]

For example:

dsl-compile --py [ML_REPO_DIRECTORY]/samples/basic/sequential.py --output [ML_REPO_DIRECTORY]/samples/basic/sequential.tar.gz

Deploy the samples

Upload the generated .tar.gz file through the ML pipeline UI.

Advanced: Build your own components

See the guide to building your own pipeline components.