Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Buildpack environment related content #754

Closed
wants to merge 14 commits into from
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ title="Add labels to the application image"
weight=99
+++

<!--more-->

Labels are key-value pairs, stored as strings, that are attached to an image (i.e., arbitrary metadata). Labels are used to add helpful descriptions or attributes to an application image, which are meaningful to users.

<!--more-->

Labels are usually added at the time an image is created. Images can have multiple labels; however, each key must be unique.

## Key Points
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,20 @@ title="Clear the buildpack environment"
weight=99
+++

"Clearing" the buildpack environment with `clear-env` is the process of preventing end-users from customizing a buildpack's behavior through environment variables.

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
Buildpack authors may elect to clear user-defined environment variables when `bin/detect` and `bin/build` are executed. This is achieved by setting `clear-env` to `true` in [buildpack.toml](https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml); by default `clear-env` is set to `false`.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

* When `clear-env` is set to `true` for a given buildpack, the `lifecycle` will not set user-provided environment variables when running `/bin/detect` or `/bin/build` for the given buildpack.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
* If a buildpack does allow customization by the end-user through the environment (`clear-env` is `false`), there is a special convention for naming the available environment variables, shown in the following table:
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

| Env Variable | Description | Detect | Build | Launch |
|------------------------|---------------------------------------------------|--------|-------|--------|
| `BP_*` | User-provided variable for buildpack | [x] | [x] | |
| `BPL_*` | User-provided variable for exec.d | | | [x] |

### Further Reading

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
For more about how environment variables are specified by end-users, see the page for how to [customize buildpack behavior with build-time environment variables](https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/).
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@ A buildpack can control how a layer will be used by creating a `<layer>.toml` wi

A buildpack might create a `$CNB_LAYERS_DIR/python` directory and a `$CNB_LAYERS_DIR/python.toml` with the following contents:

```
```toml
launch = true
cache = true
build = true
```

In this example:

* the final app image will contain a layer with `python`, as this is needed to run the app
* the `$CNB_LAYERS_DIR/python` directory will be pre-created for future builds, avoiding the need to re-download this large dependency
* buildpacks that follow in the build will be able to use `python`
Expand All @@ -36,7 +37,7 @@ In this example:

This is a simple `./bin/build` script for a buildpack that runs Python's `pip` package manager to resolve dependencies:

```
```bash
#!/bin/sh

PIP_LAYER="$CNB_LAYERS_DIR/pip"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ A buildpack must contain a `buildpack.toml` file in its root directory.

### Example

```
```toml
api = "0.10"

[buildpack]
Expand All @@ -46,7 +46,7 @@ For more information, see [buildpack config](/docs/reference/config/buildpack-co

### Usage

```
```txt
bin/detect
```

Expand All @@ -72,7 +72,7 @@ Other exit codes indicate an error during detection.
This is a simple example of a buildpack that detects a Python application
by checking for the presence of a `requirements.txt` file:

```
```bash
#!/bin/sh

if [ -f requirements.txt ]; then
Expand All @@ -87,7 +87,7 @@ fi

### Usage

```
```txt
bin/build
```

Expand All @@ -108,4 +108,4 @@ It is important to note that multiple buildpacks may work together to create the
each contributing a subset of the dependencies or configuration needed to run the application.
In this way, buildpacks are modular and composable.

[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan
[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,143 @@ title="Specify the environment"
weight=99
+++

Environment variables are a common way to configure buildpacks at build-time and the application at runtime.

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
### Preparing the environment at build time

When the `lifecycle` runs each buildpack, it first tears down any environment variables defined on the `build-time` base image of the environment. It only allows a [specific set](https://github.com/buildpacks/lifecycle/blob/a43d5993a4f2cc23c44b6480ba2ab09fe81d57ed/env/build.go#L9-L19) of pre-configured environment variables through.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

For the detect phase, the lifecycle then applies user-provided environment variables. For more information, see [clearing the buildpack environment](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/clear-env/)
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

For the `build` phase, the process is more complex. Before applying user-provided environment variables, the `lifecycle` applies buildpack-provided environment variables, which is anything that a previous buildpack (a buildpack that ran earlier in the `build` phase) might have configured in its `layers` directory.

>Note that buildpacks cannot set environment variables for other buildpacks during the `detect` phase.

#### Example

Let's look at the following directory tree to see how layers created by a previous buildpack (with id `some-buildpack-id`) would affect the environment for the current buildpack.

```text

layers/
└── some-buildpack-id
├── some-build-layer
│   ├── bin
│   │   └── some-binary
│   ├── env
│   │   └── SOME_VAR # contents foo
│   └── lib
│   └── some-static-library
└── some-build-layer.toml # has build = true in the [types] table

hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
```

With this tree:

* The current buildpack will see `SOME_VAR=foo` in its environment
* The current buildpack will find `some-binary` in `PATH`
* The current buildpack will find `some-static-library` in `LIBRARY_PATH`

Thus, any `<layers>/<layer>/<env>` directory is for setting environment variables directly, and `<layers>/<layer>/<bin>`, `<layers>/<layer>/<lib>`, etc. offer a convenient way to modify `POSIX` path variables.

The full list of convenience directories is summarized in the table below:

| Env Variable | Layer Path | Contents | Build | Launch |
|--------------------------------------------|--------------|------------------|-------|--------|
| `PATH` | `/bin` | binaries | [x] | [x] |
| `LD_LIBRARY_PATH` | `/lib` | shared libraries | [x] | [x] |
| `LIBRARY_PATH` | `/lib` | static libraries | [x] | |
| `CPATH` | `/include` | header files | [x] | |
| `PKG_CONFIG_PATH` | `/pkgconfig` | pc files | [x] | |

* If `env/SOME_VAR` and `env.build/SOME_VAR` have a conflict, a given procedure applies to figure out who "wins" when applying the environment [modification rules](https://github.com/buildpacks/spec/blob/main/buildpack.md#environment-variable-modification-rules).
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
* `User-defined` variables are then applied; meaning it's possible to override `buildpack-defined` variables.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
* Finally, `platform-defined` variables are applied, which eventually override any previous values.
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

### Preparing the environment at runtime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can flesh out this section a little more

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you elaborate on this? thanks

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to land this change now, and flesh out this discussion in subsequent changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AidanDelaney

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natalieparellano still waiting for you


For `runtime` and `process` environment variables, the [tree above](#example) is still applicable except that it doesn't include a `build` layer. Instead it has a`launch` layer, i.e., `some-launch-layer.toml` that has
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

```yaml

[types]
launch = true

```

>Note that the `launcher` binary sets up the environment at `runtime`. The `launcher` is found inside the app image at `/cnb/lifecycle/launcher` and is the entrypoint for any `CNB-built` image.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

### When multiple buildpacks define the same variable

When multiple buildpacks define the same variable, the ["environment modification rules"](https://github.com/buildpacks/spec/blob/main/buildpack.md#environment-variable-modification-rules) come into play.

The lifecycle MUST consider the name of the environment variable to be the name of the file up to the first period (`.`) or to the end of the name if no periods are present. In all cases, file contents MUST NOT be evaluated by a shell or otherwise modified before inclusion in environment variable values.
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

For each environment variable file the period-delimited suffix SHALL determine the modification behavior as follows.

| Suffix | Modification Behavior |
|------------|-------------------------------------------|
| none | [Override](#override) |
| `.append` | [Append](#append) |
| `.default` | [Default](#default) |
| `.delim` | [Delimiter](#delimiter) |
| `.override`| [Override](#override) |
| `.prepend` | [Prepend](#prepend) |

#### Append

The value of the environment variable MUST be a concatenation of the file contents and the contents of other files representing that environment variable.

#### Default

The value of the environment variable MUST only be the file contents if the environment variable is empty.

#### Delimiter

The file contents MUST be used to delimit any concatenation within the same layer involving that environment variable.

#### Override

The value of the environment variable MUST be the file contents.

#### Prepend

The value of the environment variable MUST be a concatenation of the file contents and the contents of other files representing that environment variable.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

To better understand the above modification rules, let's take a look at the tree below,
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

```text

layers/
├── some-buildpack-id
│   ├── some-layer
│   │   └── env
│   │   ├── SOME_VAR # contents foo
│   │   └── SOME_VAR.append # contents :
│   └── some-layer.toml
└── some-other-buildpack-id
├── some-layer
│   └── env
│   ├── SOME_VAR # contents bar
│   └── SOME_VAR.append # contents :
└── some-layer.toml

```

Assuming that `some-buildpack-id` comes before `some-other-buildpack-id` in the buildpack group, the final value of `SOME_VAR` shown above would be `foo:bar`
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

>For more information on modifying environment variables, see [Environment Variable Modification Rules](https://github.com/buildpacks/spec/blob/main/buildpack.md#environment-variable-modification-rules) specification.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

Please note that when `clear-env` is not set to `true`, the `lifecycle` MUST set user-provided environment variables in the environment of `/bin/detect` or `/bin/build` such that:

* For layer path environment variables, user-provided values are prepended before any existing values and are delimited by the OS path list separator.
* For all other environment variables, user-provided values override any existing values.
* The environment variable prefix `CNB_` is reserved. It MUST NOT be used for environment variables that are not defined in this specification or approved extensions.

>For more information on clearing user-defined environment variables, see [Clear the buildpack environment](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/clear-env/) documentation.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
### Further reading
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved

To cover:
* POSIX paths
* Build env
* Process env
* Runtime env
* Modifiers
For more about environment variables, see the [customize buildpack behavior with build-time environment variables](https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/) documentation and the [Environment](https://github.com/buildpacks/spec/blob/main/buildpack.md#environment) specification.
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ title="Use exec.d binaries to configure the application at runtime"
weight=99
+++

The [buildpacks `exec.d` interface](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd) allows buildpack authors to execute custom scripts or binaries when the application image is started.

<!--more-->

The [buildpacks `exec.d` interface](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd) allows buildpack authors to execute custom scripts or binaries when the application image is started. This interface can be particularly useful for injecting dynamic behavior or environment variables into the runtime environment of an application.
This interface can be particularly useful for injecting dynamic behavior or environment variables into the runtime environment of an application.

## Key Points

Expand Down Expand Up @@ -51,23 +53,23 @@ And a `Go` example is:
package main

import (
"fmt"
"os"
"fmt"
"os"
)

func main() {
// Open file descriptor 3 for writing
fd3 := os.NewFile(3, "fd3")
if fd3 == nil {
fmt.Println("Failed to open file descriptor 3")
return
}

// Write the environment variable to file descriptor 3
_, err := fd3.WriteString(`EXAMPLE="test"\n`)
if err != nil {
fmt.Println("Error writing to file descriptor 3:", err)
}
// Open file descriptor 3 for writing
fd3 := os.NewFile(3, "fd3")
if fd3 == nil {
fmt.Println("Failed to open file descriptor 3")
return
}

// Write the environment variable to file descriptor 3
_, err := fd3.WriteString(`EXAMPLE="test"\n`)
if err != nil {
fmt.Println("Error writing to file descriptor 3:", err)
}
}
```

Expand Down
Loading