Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

[orabos] A/B root partitions, Third for persistence #14

Draft
wants to merge 140 commits into
base: orabos
Choose a base branch
from

Conversation

fwiesel
Copy link
Member

@fwiesel fwiesel commented Jun 14, 2024

We could either have two completely separate paritions, and copy things manually around, or this.
Btrfs offers the option create subvolumes for partitioning the data further up.
Otherwise we would have to statically determine the size of the home, var, and var/tmp partitions (or at least: home and var).
Home should be fairly limited, so that would not be an insurmountable problem.

As the VMs will (eventually) only run with non-local storage, the partition will only be used for our data, which should be practically ephemeral, and is only persisted for a faster life-cycle.

Brtfs also would offer us the option to use snapshots for rollback, partition the paths further, or and makes it easy to add more paths, in case of need.

dependabot bot and others added 30 commits March 11, 2024 20:07
Bumps [azure/login](https://github.com/azure/login) from 1.6.1 to 2.0.0.
- [Release notes](https://github.com/azure/login/releases)
- [Commits](Azure/login@cb79c77...8c334a1)

---
updated-dependencies:
- dependency-name: azure/login
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Add a feature to make the Garden Linux image as compliant as possible to the [Canonical Ubuntu 20.04 LTS Security Technical Implementation Guide](https://www.stigviewer.com/stig/canonical_ubuntu_20.04_lts/2023-09-08/)

Note that some items in that STIG simply don't apply to Garden Linux, for example it does not contain a GUI or AppArmor.

STIG-specific configuration is encapsulated in the 'stig' feature. To opt in into it, build your image with that feature enabled. Note that this might break other features.

The 'stigDev' feature provides a known user/password and ssh key for the purpose of testing. This feature is only intended for development/testing purposes, not for production.
So far the SapMachine version we include needs to be updated manually.

This PR adds a script to help with that, but the update is still a manual process.

The script finds the latest minor/patch version of a given major version along with the checksums of the tarballs.

Running `update-sapmachine.py` will print out the variables to be used in `exec.config`.
A potential improvement is to edit the `exec.config` script automatically via `update-sapmachine.py`.

We still want the checksum and the exact SapMachine version to be stored in the git repo, so we don't want to hide this by automatically running `update-sapmachine.py` during the build for reproducibility reasons.

Co-authored-by: Vincent Riesop <[email protected]>
…x#2046)

Bumps [boto3](https://github.com/boto/boto3) from 1.34.55 to 1.34.65.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.34.55...1.34.65)

---
updated-dependencies:
- dependency-name: boto3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Remove docker from gardener feature

* Remove docker-in-gardener-specific test
…rs_tests

feat: add tests for bare flavours
…x#2060)

* security update annotations are not tracked in package repository,
but in security tracker and other internal tooling.
* new package pipeline does not contain annotations for CVEs, thus
we need to remove the packages updates section in the release page automation
=> need to include it manually
Bumps [actions/cache](https://github.com/actions/cache) from 4.0.1 to 4.0.2.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@ab5e6d0...0c45773)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
…b_actions/azure/login-2.0.0

build(deps): bump azure/login from 1.6.1 to 2.0.0
…b_actions/actions/cache-4.0.2

build(deps): bump actions/cache from 4.0.1 to 4.0.2
* Stop building openstack and vmware for arm64

Those builds fail because of missing open-vm-tools for arm64.

Disable builds as they have no consumers anyways.

* fix s3 upload
* update LICENSE date
* improve readme
* badge showing latest LTS version of Garden Linux
* simple quick start sections: build, test, download
* remove unecessary license badge. we have LICENSE.md file which is commin practise and Github also displays it in UI
* remove toc. README is simple enough.
* add reference to bare container
* add reference to base container
* add reference to nvidia installer
* remove details about integration tests. Details are explained in developer docs. User readind README only needs to know how to test locally and that we test automatically

Co-authored-by: Florian Wilhelm <[email protected]>
This PR introduces a workflow for building disk images and yaml manifest for use with lima-vm. Uploading those build artefacts will be done in a separate PR.

Co-authored-by: Vincent Riesop <[email protected]>
Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](kjd/idna@v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…nux#2070)

[no ci] Bumps [azure-cli](https://github.com/Azure/azure-cli) from 2.57.0 to 2.59.0.
- [Release notes](https://github.com/Azure/azure-cli/releases)
- [Changelog](https://github.com/Azure/azure-cli/blob/dev/doc/try_new_features_before_release.md)
- [Commits](Azure/azure-cli@azure-cli-2.57.0...azure-cli-2.59.0)

---
updated-dependencies:
- dependency-name: azure-cli
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…linux#2063)

[no ci] Bumps [google-auth](https://github.com/googleapis/google-auth-library-python) from 2.28.1 to 2.29.0.
- [Release notes](https://github.com/googleapis/google-auth-library-python/releases)
- [Changelog](https://github.com/googleapis/google-auth-library-python/blob/main/CHANGELOG.md)
- [Commits](googleapis/google-auth-library-python@v2.28.1...v2.29.0)

---
updated-dependencies:
- dependency-name: google-auth
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
gardenlinux#2062)

[no ci] Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 2.15.0 to 2.16.0.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](googleapis/python-storage@v2.15.0...v2.16.0)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ardenlinux#2029)

[no ci] Bumps [azure-identity](https://github.com/Azure/azure-sdk-for-python) from 1.16.0b1 to 1.16.0b2.
- [Release notes](https://github.com/Azure/azure-sdk-for-python/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md)
- [Commits](Azure/azure-sdk-for-python@azure-identity_1.16.0b1...azure-identity_1.16.0b2)

---
updated-dependencies:
- dependency-name: azure-identity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* gh: create config for auto changelog generation

* GitHub supports the generation of automatic release notes based on PRs. 
* PRs can be filtered and sorted into categories via PR labels
* more details: https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes

* gh: remove unused labels in changelog config
defo89 and others added 20 commits June 10, 2024 10:12
It breaks the build, and that is easier for now.
That is leaving systemd-resolved in hanging state even if systemd-networkd.service is up
is also disabled in gardener feature
conflicts with nodeport services
* for NFS PVs (needed for prometheus)
The prior one is if you deploy in an external network,
the new one also works in private networks with dhcp.
Configuring them late in the system setup takes a significant
time, doing so very early via a kexec-reboot should be faster.
Potentially, changing them on-the-fly early enough might
have been enough, but I haven't tested that.
We expect a bond, but when testing with qemu, there will be only
a single interface. Cover that as well.
This gives us a place to collect all changes relevant to metal3
provisioning (as opposed to adding it to orabos).
Presumably that is the feature we want to get closest to
@fwiesel fwiesel marked this pull request as draft June 14, 2024 12:23
@fwiesel
Copy link
Member Author

fwiesel commented Jun 18, 2024

Okay, I'll switch away from btrfs... the main reason for that is, that it is delays the bootup.
Switching to two partitions for home and var.

Doing it in the same unit has a race-condition,
as the osdb-server forks of and will create
eventually the socket, and at the same time
the ExecStartPost script is running.
By moving it to its own unit, the script can
wait for the socket and adjust the permissions
without delaying other units.
We could either have two completely separate paritions,
and copy things manually around, or this.
Btrfs offers the option create subvolumes for partitioning
the data further up.
Otherwise we would have to statically determine the size
of the home, var, and var/tmp partitions (or at least:
home and var).
Home should be fairly limited, so that would not be an insurmountable
problem.

As the VMs will (eventually) only run with non-local storage, the
partition will only be used for our data, which should be practically
ephemeral, and is only persisted for a faster life-cycle.

Brtfs also would offer us the option to use snapshots for rollback,
partition the paths further, or and makes it easy to add more paths,
in case of need.

This includes the read-only /usr directory, now disabled.
Adding a normal a/b partition doubles the risk of having
chosen the wrong size and having to redeploy from scratch to
repartition it.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants