From d3d529cf7e0e52b7c84d5b36cdac82c1b20ed054 Mon Sep 17 00:00:00 2001 From: Guillaume Smet Date: Fri, 27 Dec 2024 12:50:05 +0100 Subject: [PATCH] Fix implementation and documentation of init tasks The documentation was not in line with the implementation so I fixed things in both (and also fixed some typos in passing). Fixes #39230 --- docs/src/main/asciidoc/init-tasks.adoc | 50 +++++++++----- .../spi/KubernetesInitContainerBuildItem.java | 67 +++++++++++++------ .../deployment/ContainerConfig.java | 4 +- .../kubernetes/deployment/InitTaskConfig.java | 13 +++- .../deployment/InitTaskProcessor.java | 2 + .../deployment/KubernetesCommonHelper.java | 2 + .../deployment/KubernetesConfig.java | 6 +- .../deployment/OpenShiftConfig.java | 8 ++- 8 files changed, 106 insertions(+), 46 deletions(-) diff --git a/docs/src/main/asciidoc/init-tasks.adoc b/docs/src/main/asciidoc/init-tasks.adoc index 6c750cc087955..8a2818e72523f 100644 --- a/docs/src/main/asciidoc/init-tasks.adoc +++ b/docs/src/main/asciidoc/init-tasks.adoc @@ -12,7 +12,7 @@ https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc There are often initialization tasks performed by Quarkus extensions that are meant to be run once. For example, Flyway or Liquibase initialization falls into that category. But what happens when the scaling needs of an application requires more instances of the application to run? Or what happens when the application -restarts ? +restarts? A common environment where both of these cases are pretty common is Kubernetes. To address these challenges, Quarkus allows externalization of such tasks as Kubernetes https://kubernetes.io/docs/concepts/workloads/controllers/job/[Jobs] and uses https://kubernetes.io/docs/concepts/workloads/pods/init-containers/[init containers] to ensure that an @@ -23,7 +23,7 @@ This approach is reflected in the manifests generated by xref:deploying-to-kuber == Disabling the feature -The feature can be explictily disabled per task (enabled by default). +The feature can be explicitly disabled per task (enabled by default). The default behavior can change by setting the following property to `false`: [source,properties] @@ -89,59 +89,77 @@ Any customization to the original deployment resource (via configuration or exte == Controlling the generated init container The name of the generated init container is `wait-for-${task name}` by default. -Given that the init container is part of the same pod as the actual application it will get the same service account (and therefore permissions) and volumes as the application. -Further customization to the container can be done using using the configuration options for init containers (see `quarkus.kubernetes.init-containers` or `quarkus.openshift.init-containers`). +Given that the init container is part of the same pod as the actual application, it will get the same service account (and therefore permissions) and volumes as the application. +Further customization to the container can be done using the configuration options for init containers (see `quarkus.kubernetes.init-containers` or `quarkus.openshift.init-containers`). Examples: -To set the imagePullPolicy to `IfNotPresent` on the init container that waits for the `flyway` job: +To set the `imagePullPolicy` to `IfNotPresent` on the init container that waits for the `flyway` job: [source,properties] ---- -quarkus.kubernetes.init-containers.wait-for-flyway.image-pull-policy=IfNotPresent +quarkus.kubernetes.init-containers.flyway.image-pull-policy=if-not-present ---- To set custom command (say `custom-wait-for`) on the init container that waits for the `flyway` job: [source,properties] ---- -quarkus.kubernetes.init-containers.wait-for-flyway.command=custom-wait-for +quarkus.kubernetes.init-containers.flyway.command=custom-wait-for ---- - == Orchestration of the initialization tasks The deployment resource should not start until the job has been completed. The typical pattern that is used among Kubernetes users is the -use of init containers to achieve this. An init container that `wait for` the job to complete is enough to enforce that requirement. +use of init containers to achieve this. An init container that `waits for` the job to complete is enough to enforce that requirement. === Using a custom wait-for container image -To change the `wait-for` image which by default is `groundnuty/k8s-wait-for:no-root-v1.7` you can use: +By default, the `wait-for` image is `groundnuty/k8s-wait-for:no-root-v1.7`. +You can define another image: [source,properties] ---- quarkus.kubernetes.init-task-defaults.wait-for-container.image=my/wait-for-image:1.0 ---- -To change the `wait-for` image for a particular init container (e.g. `wait-for-flway`) you can use: +The `imagePullPolicy` can also be configured: + +[source,properties] +---- +quarkus.kubernetes.init-task-defaults.wait-for-container.image-pull-policy=if-not-present +---- + +To change the `wait-for` image for a particular init container (e.g. one that waits for the `flyway` job), you can use: [source,properties] ---- -quarkus.kubernetes.init-containers.wait-for-flyway=my/wait-for-image:1.0 +quarkus.kubernetes.init-tasks.flyway.wait-for-container.image=my/wait-for-image:1.0 +---- + +You can define the `imagePullPolicy` for this particular init container with: + +[source,properties] +---- +quarkus.kubernetes.init-tasks.flyway.wait-for-container.image-pull-policy=if-not-present ---- === Configuring permissions -For an init container to be able to perform the `wait for job` it needs to be able to perform `get` operations on the job resource. +For an init container to be able to perform the `wait for job`, it needs to be able to perform `get` operations on the job resource. This is done automatically and the generated manifests include the required `Role` and `RoleBinding` resources. -If for any reason additional permissions are required either by the init container or the job, they can be configured with through the xref:deploying-to-kubernetes.adoc#generating-rbac-resources[Kubernetes RBAC configuration]. +If, for any reason, additional permissions are required either by the init container or the job, they can be configured with the xref:deploying-to-kubernetes.adoc#generating-rbac-resources[Kubernetes RBAC configuration]. -**Note**: The application, the init container and the job use the same `ServiceAccount` and therefore, share the same permissions. +[NOTE] +==== +The application, the init container and the job use the same `ServiceAccount` and therefore, share the same permissions. +==== -== Extension providing Initialization Tasks +== Extensions providing Initialization Tasks Currently, this feature is used by the following extensions: + - xref:flyway.adoc[Flyway] - xref:liquibase.adoc[Liquibase] - xref:liquibase-mongodb.adoc[Liquibase MongoDB] diff --git a/extensions/kubernetes/spi/src/main/java/io/quarkus/kubernetes/spi/KubernetesInitContainerBuildItem.java b/extensions/kubernetes/spi/src/main/java/io/quarkus/kubernetes/spi/KubernetesInitContainerBuildItem.java index 469facd99e490..b2287f7744a2e 100644 --- a/extensions/kubernetes/spi/src/main/java/io/quarkus/kubernetes/spi/KubernetesInitContainerBuildItem.java +++ b/extensions/kubernetes/spi/src/main/java/io/quarkus/kubernetes/spi/KubernetesInitContainerBuildItem.java @@ -7,7 +7,7 @@ import io.quarkus.builder.item.MultiBuildItem; /** - * A Built item for generating init containers. + * A Build item for generating init containers. * The generated container will have the specified fields * and may optionally inherit env vars and volumes from the app container. *

@@ -15,9 +15,12 @@ */ public final class KubernetesInitContainerBuildItem extends MultiBuildItem implements Targetable { + private static final String DEFAULT_IMAGE_PULL_POLICY = "Always"; + private final String name; private final String target; private final String image; + private final String imagePullPolicy; private final List command; private final List arguments; private final Map envVars; @@ -25,16 +28,31 @@ public final class KubernetesInitContainerBuildItem extends MultiBuildItem imple private final boolean sharedFilesystem; public static KubernetesInitContainerBuildItem create(String name, String image) { - return new KubernetesInitContainerBuildItem(name, null, image, Collections.emptyList(), Collections.emptyList(), - Collections.emptyMap(), false, false); + return create(name, image, DEFAULT_IMAGE_PULL_POLICY); + } + + public static KubernetesInitContainerBuildItem create(String name, String image, String imagePullPolicy) { + return new KubernetesInitContainerBuildItem(name, null, image, DEFAULT_IMAGE_PULL_POLICY, Collections.emptyList(), + Collections.emptyList(), Collections.emptyMap(), false, false); } - public KubernetesInitContainerBuildItem(String name, String target, String image, List command, + @Deprecated(forRemoval = true, since = "3.18") + public KubernetesInitContainerBuildItem(String name, String target, String image, + List command, + List arguments, + Map envVars, boolean sharedEnvironment, boolean sharedFilesystem) { + this(name, target, image, DEFAULT_IMAGE_PULL_POLICY, command, arguments, envVars, sharedEnvironment, sharedFilesystem); + } + + private KubernetesInitContainerBuildItem(String name, String target, String image, + String imagePullPolicy, // using a string here as we don't have the enum in the classpath + List command, List arguments, Map envVars, boolean sharedEnvironment, boolean sharedFilesystem) { this.name = name; this.target = target; this.image = image; + this.imagePullPolicy = imagePullPolicy; this.command = command; this.arguments = arguments; this.envVars = envVars; @@ -47,8 +65,8 @@ public String getName() { } public KubernetesInitContainerBuildItem withName(String name) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } public String getTarget() { @@ -56,8 +74,8 @@ public String getTarget() { } public KubernetesInitContainerBuildItem withTarget(String target) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } public String getImage() { @@ -66,8 +84,17 @@ public String getImage() { @SuppressWarnings("unused") public KubernetesInitContainerBuildItem withImage(String image) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); + } + + public String getImagePullPolicy() { + return imagePullPolicy; + } + + public KubernetesInitContainerBuildItem withImagePullPolicy(String imagePullPolicy) { + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } public List getCommand() { @@ -75,8 +102,8 @@ public List getCommand() { } public KubernetesInitContainerBuildItem withCommand(List command) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } public List getArguments() { @@ -84,8 +111,8 @@ public List getArguments() { } public KubernetesInitContainerBuildItem withArguments(List arguments) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } public Map getEnvVars() { @@ -94,8 +121,8 @@ public Map getEnvVars() { @SuppressWarnings("unused") public KubernetesInitContainerBuildItem withEnvVars(Map envVars) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } /** @@ -111,8 +138,8 @@ public boolean isSharedEnvironment() { @SuppressWarnings("unused") public KubernetesInitContainerBuildItem withSharedEnvironment(boolean sharedEnvironment) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } /** @@ -130,7 +157,7 @@ public boolean isSharedFilesystem() { @SuppressWarnings("unused") public KubernetesInitContainerBuildItem withSharedFilesystem(boolean sharedFilesystem) { - return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment, - sharedFilesystem); + return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars, + sharedEnvironment, sharedFilesystem); } } diff --git a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/ContainerConfig.java b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/ContainerConfig.java index 0b20a782b14f3..db75e97c9e167 100644 --- a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/ContainerConfig.java +++ b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/ContainerConfig.java @@ -52,11 +52,11 @@ public interface ContainerConfig extends EnvVarHolder { /** * Image pull policy. */ - @WithDefault("Always") + @WithDefault("always") ImagePullPolicy imagePullPolicy(); /** - * The image pull secret + * The image pull secrets. */ Optional> imagePullSecrets(); diff --git a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskConfig.java b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskConfig.java index d5b8c147a0c58..954ab5f7cd872 100644 --- a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskConfig.java +++ b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskConfig.java @@ -2,6 +2,7 @@ import java.util.Optional; +import io.dekorate.kubernetes.annotation.ImagePullPolicy; import io.smallrye.config.WithDefault; public interface InitTaskConfig { @@ -12,11 +13,11 @@ public interface InitTaskConfig { boolean enabled(); /** - * The init task image to use by the init-container. + * The init task image to use by the init container. * * @deprecated use waitForContainer.image instead. */ - @Deprecated + @Deprecated(forRemoval = true, since = "3.5") Optional image(); /** @@ -26,9 +27,15 @@ public interface InitTaskConfig { interface InitTaskContainerConfig { /** - * The init task image to use by the init-container. + * The init task image to use by the init container. */ @WithDefault("groundnuty/k8s-wait-for:no-root-v1.7") String image(); + + /** + * Image pull policy. + */ + @WithDefault("always") + ImagePullPolicy imagePullPolicy(); } } diff --git a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskProcessor.java b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskProcessor.java index c67776b419f8b..1b96e87b73492 100644 --- a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskProcessor.java +++ b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/InitTaskProcessor.java @@ -46,6 +46,7 @@ public static void process( .replaceAll("^" + Pattern.quote(name + "-"), "") .replaceAll(Pattern.quote("-init") + "$", ""); String jobName = name + "-" + taskName + "-init"; + InitTaskConfig config = initTasksConfig.getOrDefault(taskName, initTaskDefaults); if (config == null || config.enabled()) { generateRoleForJobs = true; @@ -69,6 +70,7 @@ public static void process( String waitForImage = config.image().orElse(config.waitForContainer().image()); initContainers .produce(KubernetesInitContainerBuildItem.create(INIT_CONTAINER_WAITER_NAME + taskName, waitForImage) + .withImagePullPolicy(config.waitForContainer().imagePullPolicy().name()) .withTarget(target) .withArguments(List.of("job", jobName))); } diff --git a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesCommonHelper.java b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesCommonHelper.java index 6c1f43f2e3d21..e939c0b5f2d22 100644 --- a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesCommonHelper.java +++ b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesCommonHelper.java @@ -22,6 +22,7 @@ import org.jboss.logging.Logger; +import io.dekorate.kubernetes.annotation.ImagePullPolicy; import io.dekorate.kubernetes.config.Annotation; import io.dekorate.kubernetes.config.ConfigMapVolumeBuilder; import io.dekorate.kubernetes.config.EnvBuilder; @@ -636,6 +637,7 @@ public static List createInitContainerDecorators(String targ io.dekorate.kubernetes.config.ContainerBuilder containerBuilder = new io.dekorate.kubernetes.config.ContainerBuilder() .withName(item.getName()) .withImage(item.getImage()) + .withImagePullPolicy(ImagePullPolicy.valueOf(item.getImagePullPolicy())) .withCommand(item.getCommand().toArray(new String[0])) .withArguments(item.getArguments().toArray(new String[0])); diff --git a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesConfig.java b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesConfig.java index 11dbb0744fc15..a00a94e3760c7 100644 --- a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesConfig.java +++ b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/KubernetesConfig.java @@ -11,6 +11,7 @@ import io.quarkus.deployment.Capabilities; import io.quarkus.deployment.Capability; import io.quarkus.kubernetes.spi.DeployStrategy; +import io.quarkus.runtime.annotations.ConfigDocMapKey; import io.quarkus.runtime.annotations.ConfigPhase; import io.quarkus.runtime.annotations.ConfigRoot; import io.smallrye.config.ConfigMapping; @@ -96,11 +97,12 @@ default String targetPlatformName() { /** * Init tasks configuration. *

- * The init tasks are automatically generated by extensions like Flyway to perform the database migration before staring + * The init tasks are automatically generated by extensions like Flyway to perform the database migration before starting * up the application. *

* This property is only taken into account if `quarkus.kubernetes.externalize-init` is true. */ + @ConfigDocMapKey("task-name") Map initTasks(); /** @@ -112,7 +114,7 @@ default String targetPlatformName() { InitTaskConfig initTaskDefaults(); /** - * Optionally set directory generated kubernetes resources will be written to. Default is `target/kubernetes`. + * Optionally set directory generated Kubernetes resources will be written to. Default is `target/kubernetes`. */ Optional outputDirectory(); diff --git a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/OpenShiftConfig.java b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/OpenShiftConfig.java index e472eead4b82a..533389dc0b9b4 100644 --- a/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/OpenShiftConfig.java +++ b/extensions/kubernetes/vanilla/deployment/src/main/java/io/quarkus/kubernetes/deployment/OpenShiftConfig.java @@ -12,6 +12,7 @@ import io.quarkus.deployment.Capabilities; import io.quarkus.deployment.Capability; import io.quarkus.kubernetes.spi.DeployStrategy; +import io.quarkus.runtime.annotations.ConfigDocMapKey; import io.quarkus.runtime.annotations.ConfigPhase; import io.quarkus.runtime.annotations.ConfigRoot; import io.smallrye.config.ConfigMapping; @@ -70,7 +71,7 @@ enum OpenshiftFlavor { Map containers(); /** - * Openshift route configuration + * OpenShift route configuration */ RouteConfig route(); @@ -104,15 +105,16 @@ enum OpenshiftFlavor { /** * Init tasks configuration. *

- * The init tasks are automatically generated by extensions like Flyway to perform the database migration before staring + * The init tasks are automatically generated by extensions like Flyway to perform the database migration before starting * up the application. *

* This property is only taken into account if `quarkus.openshift.externalize-init` is true. */ + @ConfigDocMapKey("task-name") Map initTasks(); /** - * Default Init tasks configuration. + * Default init tasks configuration. *

* The init tasks are automatically generated by extensions like Flyway to perform the database migration before staring * up the application.