CodeFlare Pipelines

CodeFlare Pipelines reimagined pipelines to provide a more intuitive API for the data scientist to create AI/ML pipelines, data workflows, pre-processing, post-processing tasks, and many more which can scale from a laptop to a cluster seamlessly.

See the API documentation here, and reference use case documentation in the Examples section.

Examples are provided as executable notebooks.

To run examples, if you haven’t done so yet, clone the CodeFlare project with:

git clone https://github.com/project-codeflare/codeflare.git

Example notebooks require JupyterLab, which can be installed with:

pip3 install --upgrade jupyterlab

Use the command below to run locally:

jupyter-lab codeflare/notebooks/<example_notebook>

The step above should automatically open a browser window and connect to a running Jupyter server.

If you are using any one of the recommended cloud based deployments, examples are found in the codeflare/notebooks directory in the container image. The examples can be executed directly from the Jupyter environment.

As a first example, we recommend the sample pipeline.