Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix implementation and documentation of init tasks #45296

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 34 additions & 16 deletions docs/src/main/asciidoc/init-tasks.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
There are often initialization tasks performed by Quarkus extensions that are meant to be run once.
For example, Flyway or Liquibase initialization falls into that category. But what happens when the scaling
needs of an application requires more instances of the application to run? Or what happens when the application
restarts ?
restarts?

A common environment where both of these cases are pretty common is Kubernetes. To address these challenges,
Quarkus allows externalization of such tasks as Kubernetes https://kubernetes.io/docs/concepts/workloads/controllers/job/[Jobs] and uses https://kubernetes.io/docs/concepts/workloads/pods/init-containers/[init containers] to ensure that an
Expand All @@ -23,7 +23,7 @@ This approach is reflected in the manifests generated by xref:deploying-to-kuber

== Disabling the feature

The feature can be explictily disabled per task (enabled by default).
The feature can be explicitly disabled per task (enabled by default).
The default behavior can change by setting the following property to `false`:

[source,properties]
Expand Down Expand Up @@ -89,59 +89,77 @@ Any customization to the original deployment resource (via configuration or exte
== Controlling the generated init container

The name of the generated init container is `wait-for-${task name}` by default.
Given that the init container is part of the same pod as the actual application it will get the same service account (and therefore permissions) and volumes as the application.
Further customization to the container can be done using using the configuration options for init containers (see `quarkus.kubernetes.init-containers` or `quarkus.openshift.init-containers`).
Given that the init container is part of the same pod as the actual application, it will get the same service account (and therefore permissions) and volumes as the application.
Further customization to the container can be done using the configuration options for init containers (see `quarkus.kubernetes.init-containers` or `quarkus.openshift.init-containers`).

Examples:

To set the imagePullPolicy to `IfNotPresent` on the init container that waits for the `flyway` job:
To set the `imagePullPolicy` to `IfNotPresent` on the init container that waits for the `flyway` job:

[source,properties]
----
quarkus.kubernetes.init-containers.wait-for-flyway.image-pull-policy=IfNotPresent
quarkus.kubernetes.init-containers.flyway.image-pull-policy=if-not-present
----

To set custom command (say `custom-wait-for`) on the init container that waits for the `flyway` job:

[source,properties]
----
quarkus.kubernetes.init-containers.wait-for-flyway.command=custom-wait-for
quarkus.kubernetes.init-containers.flyway.command=custom-wait-for
----


== Orchestration of the initialization tasks

The deployment resource should not start until the job has been completed. The typical pattern that is used among Kubernetes users is the
use of init containers to achieve this. An init container that `wait for` the job to complete is enough to enforce that requirement.
use of init containers to achieve this. An init container that `waits for` the job to complete is enough to enforce that requirement.

=== Using a custom wait-for container image

To change the `wait-for` image which by default is `groundnuty/k8s-wait-for:no-root-v1.7` you can use:
By default, the `wait-for` image is `groundnuty/k8s-wait-for:no-root-v1.7`.
You can define another image:

[source,properties]
----
quarkus.kubernetes.init-task-defaults.wait-for-container.image=my/wait-for-image:1.0
----

To change the `wait-for` image for a particular init container (e.g. `wait-for-flway`) you can use:
The `imagePullPolicy` can also be configured:

[source,properties]
----
quarkus.kubernetes.init-task-defaults.wait-for-container.image-pull-policy=if-not-present
----

To change the `wait-for` image for a particular init container (e.g. one that waits for the `flyway` job), you can use:

[source,properties]
----
quarkus.kubernetes.init-containers.wait-for-flyway=my/wait-for-image:1.0
quarkus.kubernetes.init-tasks.flyway.wait-for-container.image=my/wait-for-image:1.0
----

You can define the `imagePullPolicy` for this particular init container with:

[source,properties]
----
quarkus.kubernetes.init-tasks.flyway.wait-for-container.image-pull-policy=if-not-present
----

=== Configuring permissions

For an init container to be able to perform the `wait for job` it needs to be able to perform `get` operations on the job resource.
For an init container to be able to perform the `wait for job`, it needs to be able to perform `get` operations on the job resource.
This is done automatically and the generated manifests include the required `Role` and `RoleBinding` resources.

If for any reason additional permissions are required either by the init container or the job, they can be configured with through the xref:deploying-to-kubernetes.adoc#generating-rbac-resources[Kubernetes RBAC configuration].
If, for any reason, additional permissions are required either by the init container or the job, they can be configured with the xref:deploying-to-kubernetes.adoc#generating-rbac-resources[Kubernetes RBAC configuration].

**Note**: The application, the init container and the job use the same `ServiceAccount` and therefore, share the same permissions.
[NOTE]
====
The application, the init container and the job use the same `ServiceAccount` and therefore, share the same permissions.
====

== Extension providing Initialization Tasks
== Extensions providing Initialization Tasks

Currently, this feature is used by the following extensions:

- xref:flyway.adoc[Flyway]
- xref:liquibase.adoc[Liquibase]
- xref:liquibase-mongodb.adoc[Liquibase MongoDB]
Original file line number Diff line number Diff line change
Expand Up @@ -7,34 +7,52 @@
import io.quarkus.builder.item.MultiBuildItem;

/**
* A Built item for generating init containers.
* A Build item for generating init containers.
* The generated container will have the specified fields
* and may optionally inherit env vars and volumes from the app container.
* <p>
* Env vars specified through this build item, will take precedence over inherited ones.
*/
public final class KubernetesInitContainerBuildItem extends MultiBuildItem implements Targetable {

private static final String DEFAULT_IMAGE_PULL_POLICY = "Always";

private final String name;
private final String target;
private final String image;
private final String imagePullPolicy;
private final List<String> command;
private final List<String> arguments;
private final Map<String, String> envVars;
private final boolean sharedEnvironment;
private final boolean sharedFilesystem;

public static KubernetesInitContainerBuildItem create(String name, String image) {
return new KubernetesInitContainerBuildItem(name, null, image, Collections.emptyList(), Collections.emptyList(),
Collections.emptyMap(), false, false);
return create(name, image, DEFAULT_IMAGE_PULL_POLICY);
}

public static KubernetesInitContainerBuildItem create(String name, String image, String imagePullPolicy) {
return new KubernetesInitContainerBuildItem(name, null, image, DEFAULT_IMAGE_PULL_POLICY, Collections.emptyList(),
Collections.emptyList(), Collections.emptyMap(), false, false);
}

public KubernetesInitContainerBuildItem(String name, String target, String image, List<String> command,
@Deprecated(forRemoval = true, since = "3.18")
public KubernetesInitContainerBuildItem(String name, String target, String image,
List<String> command,
List<String> arguments,
Map<String, String> envVars, boolean sharedEnvironment, boolean sharedFilesystem) {
this(name, target, image, DEFAULT_IMAGE_PULL_POLICY, command, arguments, envVars, sharedEnvironment, sharedFilesystem);
}

private KubernetesInitContainerBuildItem(String name, String target, String image,
String imagePullPolicy, // using a string here as we don't have the enum in the classpath
List<String> command,
List<String> arguments,
Map<String, String> envVars, boolean sharedEnvironment, boolean sharedFilesystem) {
this.name = name;
this.target = target;
this.image = image;
this.imagePullPolicy = imagePullPolicy;
this.command = command;
this.arguments = arguments;
this.envVars = envVars;
Expand All @@ -47,17 +65,17 @@ public String getName() {
}

public KubernetesInitContainerBuildItem withName(String name) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

public String getTarget() {
return target;
}

public KubernetesInitContainerBuildItem withTarget(String target) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

public String getImage() {
Expand All @@ -66,26 +84,35 @@ public String getImage() {

@SuppressWarnings("unused")
public KubernetesInitContainerBuildItem withImage(String image) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

public String getImagePullPolicy() {
return imagePullPolicy;
}

public KubernetesInitContainerBuildItem withImagePullPolicy(String imagePullPolicy) {
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

public List<String> getCommand() {
return command;
}

public KubernetesInitContainerBuildItem withCommand(List<String> command) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

public List<String> getArguments() {
return arguments;
}

public KubernetesInitContainerBuildItem withArguments(List<String> arguments) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

public Map<String, String> getEnvVars() {
Expand All @@ -94,8 +121,8 @@ public Map<String, String> getEnvVars() {

@SuppressWarnings("unused")
public KubernetesInitContainerBuildItem withEnvVars(Map<String, String> envVars) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

/**
Expand All @@ -111,8 +138,8 @@ public boolean isSharedEnvironment() {

@SuppressWarnings("unused")
public KubernetesInitContainerBuildItem withSharedEnvironment(boolean sharedEnvironment) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}

/**
Expand All @@ -130,7 +157,7 @@ public boolean isSharedFilesystem() {

@SuppressWarnings("unused")
public KubernetesInitContainerBuildItem withSharedFilesystem(boolean sharedFilesystem) {
return new KubernetesInitContainerBuildItem(name, target, image, command, arguments, envVars, sharedEnvironment,
sharedFilesystem);
return new KubernetesInitContainerBuildItem(name, target, image, imagePullPolicy, command, arguments, envVars,
sharedEnvironment, sharedFilesystem);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@ public interface ContainerConfig extends EnvVarHolder {
/**
* Image pull policy.
*/
@WithDefault("Always")
@WithDefault("always")
ImagePullPolicy imagePullPolicy();

/**
* The image pull secret
* The image pull secrets.
*/
Optional<List<String>> imagePullSecrets();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.util.Optional;

import io.dekorate.kubernetes.annotation.ImagePullPolicy;
import io.smallrye.config.WithDefault;

public interface InitTaskConfig {
Expand All @@ -12,11 +13,11 @@ public interface InitTaskConfig {
boolean enabled();

/**
* The init task image to use by the init-container.
* The init task image to use by the init container.
*
* @deprecated use waitForContainer.image instead.
*/
@Deprecated
@Deprecated(forRemoval = true, since = "3.5")
Optional<String> image();

/**
Expand All @@ -26,9 +27,15 @@ public interface InitTaskConfig {

interface InitTaskContainerConfig {
/**
* The init task image to use by the init-container.
* The init task image to use by the init container.
*/
@WithDefault("groundnuty/k8s-wait-for:no-root-v1.7")
String image();

/**
* Image pull policy.
*/
@WithDefault("always")
ImagePullPolicy imagePullPolicy();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ public static void process(
.replaceAll("^" + Pattern.quote(name + "-"), "")
.replaceAll(Pattern.quote("-init") + "$", "");
String jobName = name + "-" + taskName + "-init";

InitTaskConfig config = initTasksConfig.getOrDefault(taskName, initTaskDefaults);
if (config == null || config.enabled()) {
generateRoleForJobs = true;
Expand All @@ -69,6 +70,7 @@ public static void process(
String waitForImage = config.image().orElse(config.waitForContainer().image());
initContainers
.produce(KubernetesInitContainerBuildItem.create(INIT_CONTAINER_WAITER_NAME + taskName, waitForImage)
.withImagePullPolicy(config.waitForContainer().imagePullPolicy().name())
.withTarget(target)
.withArguments(List.of("job", jobName)));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@

import org.jboss.logging.Logger;

import io.dekorate.kubernetes.annotation.ImagePullPolicy;
import io.dekorate.kubernetes.config.Annotation;
import io.dekorate.kubernetes.config.ConfigMapVolumeBuilder;
import io.dekorate.kubernetes.config.EnvBuilder;
Expand Down Expand Up @@ -636,6 +637,7 @@ public static List<DecoratorBuildItem> createInitContainerDecorators(String targ
io.dekorate.kubernetes.config.ContainerBuilder containerBuilder = new io.dekorate.kubernetes.config.ContainerBuilder()
.withName(item.getName())
.withImage(item.getImage())
.withImagePullPolicy(ImagePullPolicy.valueOf(item.getImagePullPolicy()))
.withCommand(item.getCommand().toArray(new String[0]))
.withArguments(item.getArguments().toArray(new String[0]));

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import io.quarkus.deployment.Capabilities;
import io.quarkus.deployment.Capability;
import io.quarkus.kubernetes.spi.DeployStrategy;
import io.quarkus.runtime.annotations.ConfigDocMapKey;
import io.quarkus.runtime.annotations.ConfigPhase;
import io.quarkus.runtime.annotations.ConfigRoot;
import io.smallrye.config.ConfigMapping;
Expand Down Expand Up @@ -96,11 +97,12 @@ default String targetPlatformName() {
/**
* Init tasks configuration.
* <p>
* The init tasks are automatically generated by extensions like Flyway to perform the database migration before staring
* The init tasks are automatically generated by extensions like Flyway to perform the database migration before starting
* up the application.
* <p>
* This property is only taken into account if `quarkus.kubernetes.externalize-init` is true.
*/
@ConfigDocMapKey("task-name")
Map<String, InitTaskConfig> initTasks();

/**
Expand All @@ -112,7 +114,7 @@ default String targetPlatformName() {
InitTaskConfig initTaskDefaults();

/**
* Optionally set directory generated kubernetes resources will be written to. Default is `target/kubernetes`.
* Optionally set directory generated Kubernetes resources will be written to. Default is `target/kubernetes`.
*/
Optional<String> outputDirectory();

Expand Down
Loading
Loading