Skip to content

Commit

Permalink
Removed docker build docs + fixed CLI command for zenml pipeline buil…
Browse files Browse the repository at this point in the history
…d list (#2938)

* Hydrated pipeline build

* removed docs for reuse docker build
  • Loading branch information
htahir1 authored Aug 21, 2024
1 parent 0baa9d6 commit ed62d10
Show file tree
Hide file tree
Showing 6 changed files with 10 additions and 83 deletions.
2 changes: 0 additions & 2 deletions docs/book/how-to/configure-python-environments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@ When running locally, there is no real concept of an `execution` environment as

ZenML handles the Docker image configuration, creation, and pushing, starting with a [base image](https://hub.docker.com/r/zenmldocker/zenml) containing ZenML and Python, then adding pipeline dependencies. To manage the Docker image configuration, follow the steps in the [containerize your pipeline](../../how-to/customize-docker-builds/README.md) guide, including specifying additional pip dependencies, using a custom parent image, and customizing the build process.

The execution environments do not need to be built each time a pipeline is run - you can [reuse builds from previous runs to save time](../../how-to/customize-docker-builds/reuse-docker-builds.md).

## Image Builder Environment

By default, execution environments are created locally in the [client environment](#client-environment-or-the-runner-environment) using the local Docker client. However, this requires Docker installation and permissions. ZenML offers [image builders](../../component-guide/image-builders/image-builders.md), a special [stack component](../../component-guide/README.md), allowing users to build and push Docker images in a different specialized _image builder environment_.
Expand Down

This file was deleted.

30 changes: 0 additions & 30 deletions docs/book/how-to/customize-docker-builds/reuse-docker-builds.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Use code repositories to speed up Docker build times

While [reusing Docker builds](reuse-docker-builds.md) is useful, it can be limited. This is because specifying a custom build when running a pipeline will **not run the code on your client machine** but will use the code **included in the Docker images of the build**. As a consequence, even if you make local code changes, reusing a build will _always_ execute the code bundled in the Docker image, rather than the local code. Therefore, if you would like to reuse a Docker build AND make sure your local code changes are also downloaded into the image, you need to disconnect your code from the build.
While reusing Docker builds is useful, it can be limited. This is because specifying a custom build when running a pipeline will **not run the code on your client machine** but will use the code **included in the Docker images of the build**. As a consequence, even if you make local code changes, reusing a build will _always_ execute the code bundled in the Docker image, rather than the local code. Therefore, if you would like to reuse a Docker build AND make sure your local code changes are also downloaded into the image, you need to disconnect your code from the build.

You can do so by connecting a git repository. Registering a code repository lets you avoid building images each time you run a pipeline **and** quickly iterate on your code. When running a pipeline that is part of a local code repository checkout, ZenML can instead build the Docker images without including any of your source files, and download the files inside the container before running your code. This greatly speeds up the building process and also allows you to reuse images that one of your colleagues might have built for the same stack.

Expand Down
2 changes: 0 additions & 2 deletions docs/book/toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,7 @@
* [Specify pip dependencies and apt packages](how-to/customize-docker-builds/specify-pip-dependencies-and-apt-packages.md)
* [Use your own Dockerfiles](how-to/customize-docker-builds/use-your-own-docker-files.md)
* [Which files are built into the image](how-to/customize-docker-builds/which-files-are-built-into-the-image.md)
* [Reuse Docker builds to speed up Docker build times](how-to/customize-docker-builds/reuse-docker-builds.md)
* [Use code repositories to automate Docker build reuse](how-to/customize-docker-builds/use-code-repositories-to-speed-up-docker-build-times.md)
* [Build the pipeline without running](how-to/customize-docker-builds/build-the-pipeline-without-running.md)
* [Define where an image is built](how-to/customize-docker-builds/define-where-an-image-is-built.md)
* [⚒️ Manage stacks & components](how-to/stack-deployment/README.md)
* [Deploy a cloud stack with ZenML](how-to/stack-deployment/deploy-a-cloud-stack.md)
Expand Down
11 changes: 9 additions & 2 deletions src/zenml/cli/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -516,7 +516,7 @@ def list_pipeline_builds(**kwargs: Any) -> None:
client = Client()
try:
with console.status("Listing pipeline builds...\n"):
pipeline_builds = client.list_builds(**kwargs)
pipeline_builds = client.list_builds(hydrate=True, **kwargs)
except KeyError as err:
cli_utils.error(str(err))
else:
Expand All @@ -526,7 +526,14 @@ def list_pipeline_builds(**kwargs: Any) -> None:

cli_utils.print_pydantic_models(
pipeline_builds,
exclude_columns=["created", "updated", "user", "workspace"],
exclude_columns=[
"created",
"updated",
"user",
"workspace",
"images",
"stack_checksum",
],
)


Expand Down

0 comments on commit ed62d10

Please sign in to comment.