Integrates upgrades tests into Cloud Build #4037

igooch · 2024-11-18T17:24:14Z

What type of PR is this?

/kind feature

What this PR does / Why we need it:

This PR integrates the in place upgrades tests into the Cloud Build CI / CD pipeline. Once these tests are in place they will run in parallel with the e2e tests for each build.

Builds, pushes, and applies the necessary .yaml files to the upgrade test clusters for running the in place upgrades tests as part of the CI / CD testing suite.
Includes evictable "balloon" pods in the autopilot clusters that will maintain a buffer of available node space at all times. This will prevent the tests from failing on autopilot clusters due to scaling up and the quick churn of the test pods.
Updates the test container to watch game servers and pod events rather than pods. This covers more edge failure cases (such as ImagePullBackOff) than previously.
Adds a few more permissions that were causing logged errors with using Helm to delete Agones resources as part of the test cleanup.
Makes a few updates to versions to remove old cluster versions that are no longer in use from the version map, and adds new cluster versions.

Which issue(s) this PR fixes:

Working on #3795

Special notes for your reviewer:

While writing this PR I discovered that when the test container exists correctly (sdk.shutdown is called and the container exits with status 0), the pod attached to the game server exits with status 2. This will require more investigation as to why the pod reports a non-zero exit code when the side car and main container were shut down correctly.

agones-bot · 2024-11-18T17:36:12Z

Build Failed 😭

Build Id: a87125df-ffe9-4923-b8a1-64d777648220

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T05:18:53Z

Build Succeeded 🥳

Build Id: 78bead89-d9f0-49ee-8584-48d67165b2f9

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.45.0-dev-864e332
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.45.0-dev-864e332
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.45.0-dev-864e332-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.45.0-dev-864e332
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.45.0-dev-864e332
Linux C++ SDK (build): agonessdk-1.45.0-dev-864e332-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.45.0-dev-864e332.zip

A preview of the website (the last 30 builds are retained):

https://864e332-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4037/head:pr_4037 && git checkout pr_4037
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.45.0-dev-864e332

agones-bot · 2024-11-19T19:46:22Z

Build Failed 😭

Build Id: 824ef53b-7110-4070-9c13-f011d59add5c

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T19:56:03Z

Build Failed 😭

Build Id: d9982ce5-87ee-4bf0-8329-3b39eeb74ef7

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T20:08:48Z

Build Failed 😭

Build Id: 93b2a488-08c0-492a-8c0b-3ed45fae46b5

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T20:33:53Z

Build Failed 😭

Build Id: 33282874-f60f-4520-ac96-33ae555bfa21

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T20:55:41Z

Build Failed 😭

Build Id: a3514a3a-08e8-4940-bafe-1795133abc3b

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T21:14:13Z

Build Failed 😭

Build Id: 70c6a664-728c-482c-b9dc-a3bbc46dc52c

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T21:47:52Z

Build Failed 😭

Build Id: 46ebd711-db56-4f46-bdf0-46fa65c16381

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T22:08:24Z

Build Failed 😭

Build Id: 979dd3fe-2774-4b89-b7ab-3c550b933070

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-19T22:38:05Z

Build Failed 😭

Build Id: 90781c0c-1ede-4cc0-85e2-24b09dc2acc0

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-20T01:30:47Z

Build Succeeded 🥳

Build Id: 88d19424-f9de-4619-92cd-589ef8097f94

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.45.0-dev-2e4f2ec
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.45.0-dev-2e4f2ec
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.45.0-dev-2e4f2ec-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.45.0-dev-2e4f2ec
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.45.0-dev-2e4f2ec
Linux C++ SDK (build): agonessdk-1.45.0-dev-2e4f2ec-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.45.0-dev-2e4f2ec.zip

A preview of the website (the last 30 builds are retained):

https://2e4f2ec-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4037/head:pr_4037 && git checkout pr_4037
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.45.0-dev-2e4f2ec

agones-bot · 2024-11-20T19:03:15Z

Build Failed 😭

Build Id: c9ca05de-9173-4084-81a3-6b96f3fc671a

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-20T19:38:57Z

Build Failed 😭

Build Id: 0604a19d-e93c-4aeb-bff4-fac3fc6d4992

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-20T22:21:30Z

Build Succeeded 🥳

Build Id: c3fa65df-9c7b-442b-a673-9fa49a52540b

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.46.0-dev-7168dd3
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.46.0-dev-7168dd3
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.46.0-dev-7168dd3-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.46.0-dev-7168dd3
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.46.0-dev-7168dd3
Linux C++ SDK (build): agonessdk-1.46.0-dev-7168dd3-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.46.0-dev-7168dd3.zip

A preview of the website (the last 30 builds are retained):

https://7168dd3-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4037/head:pr_4037 && git checkout pr_4037
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.46.0-dev-7168dd3

agones-bot · 2024-11-27T00:07:07Z

Build Failed 😭

Build Id: a500691b-e080-48e2-a16c-5c11019bb209

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-11-27T01:47:26Z

Build Succeeded 🥳

Build Id: 00a863ef-291d-49b4-a63a-7ede70293bb4

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.46.0-dev-280ccb9
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.46.0-dev-280ccb9
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.46.0-dev-280ccb9-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.46.0-dev-280ccb9
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.46.0-dev-280ccb9
Linux C++ SDK (build): agonessdk-1.46.0-dev-280ccb9-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.46.0-dev-280ccb9.zip

A preview of the website (the last 30 builds are retained):

https://280ccb9-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4037/head:pr_4037 && git checkout pr_4037
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.46.0-dev-280ccb9

agones-bot · 2024-12-02T23:31:18Z

Build Failed 😭

Build Id: 98183a9b-72f0-4e43-bdee-1c4c6c53f79e

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

cloudbuild.yaml

test/upgrade/main.go

agones-bot · 2024-12-03T18:35:48Z

Build Failed 😭

Build Id: 2658950a-9bf4-45ef-9f43-e8a59a3fb47c

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-12-03T18:54:08Z

Build Failed 😭

Build Id: a49616a6-2f23-417f-87f9-ae843bb04f44

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-12-03T19:29:38Z

Build Failed 😭

Build Id: bb899c2d-fc81-4408-92bd-d806ca9b08b9

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2024-12-04T02:46:03Z

Build Succeeded 🥳

Build Id: 4f0207dd-dae7-4724-9635-2a511f8fcf09

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.46.0-dev-fa639b0
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.46.0-dev-fa639b0
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.46.0-dev-fa639b0-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.46.0-dev-fa639b0
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.46.0-dev-fa639b0
Linux C++ SDK (build): agonessdk-1.46.0-dev-fa639b0-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.46.0-dev-fa639b0.zip

A preview of the website (the last 30 builds are retained):

https://fa639b0-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4037/head:pr_4037 && git checkout pr_4037
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.46.0-dev-fa639b0

…g the upgrade test

…t game servers

The same events are covered by AddFunc as by UpdateFunc with less overhead. Any container crashes are caught by the Game Server Unhealthy watch, and are not needed in this watch.

agones-bot · 2024-12-06T21:21:34Z

Build Succeeded 🥳

Build Id: e34da770-2893-49f8-adcf-192030919507

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.46.0-dev-5a56169
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.46.0-dev-5a56169
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.46.0-dev-5a56169-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.46.0-dev-5a56169
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.46.0-dev-5a56169
Linux C++ SDK (build): agonessdk-1.46.0-dev-5a56169-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.46.0-dev-5a56169.zip

A preview of the website (the last 30 builds are retained):

https://5a56169-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4037/head:pr_4037 && git checkout pr_4037
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.46.0-dev-5a56169

github-actions bot added kind/feature New features for Agones size/L labels Nov 18, 2024

igooch force-pushed the testInPlaceUpgrades branch from 18dd6f8 to f264223 Compare November 19, 2024 19:55

igooch force-pushed the testInPlaceUpgrades branch from f264223 to 1df7ed0 Compare November 19, 2024 20:25

igooch force-pushed the testInPlaceUpgrades branch from 1df7ed0 to 44756f4 Compare November 19, 2024 20:39

igooch force-pushed the testInPlaceUpgrades branch from 44756f4 to 26010de Compare November 19, 2024 21:01

igooch force-pushed the testInPlaceUpgrades branch from 26010de to f56d250 Compare November 19, 2024 21:32

igooch force-pushed the testInPlaceUpgrades branch from f56d250 to 2371cb0 Compare November 19, 2024 21:50

igooch force-pushed the testInPlaceUpgrades branch from 2371cb0 to d95f1df Compare November 19, 2024 22:11

igooch force-pushed the testInPlaceUpgrades branch from 554a867 to 2e4f2ec Compare November 20, 2024 00:19

igooch force-pushed the testInPlaceUpgrades branch 2 times, most recently from f834aa9 to 9f64d99 Compare November 20, 2024 18:45

igooch force-pushed the testInPlaceUpgrades branch from 9f64d99 to 5e62c26 Compare November 20, 2024 19:19

igooch force-pushed the testInPlaceUpgrades branch from 5e62c26 to 7168dd3 Compare November 20, 2024 20:54

igooch marked this pull request as ready for review November 20, 2024 22:22

igooch force-pushed the testInPlaceUpgrades branch from 75a42c5 to 280ccb9 Compare November 27, 2024 00:40

gongmax reviewed Dec 3, 2024

View reviewed changes

cloudbuild.yaml Outdated Show resolved Hide resolved

test/upgrade/main.go Outdated Show resolved Hide resolved

igooch force-pushed the testInPlaceUpgrades branch from cc41308 to 296ef94 Compare December 3, 2024 18:42

igooch force-pushed the testInPlaceUpgrades branch from 296ef94 to 13b81c9 Compare December 3, 2024 19:04

igooch force-pushed the testInPlaceUpgrades branch 2 times, most recently from 083bf3c to fa639b0 Compare December 4, 2024 00:51

gongmax approved these changes Dec 4, 2024

View reviewed changes

igooch added 12 commits December 6, 2024 11:26

Integrates upgrades tests into Cloud Build

30974ac

Templates current build version to pass through to test containers

e4afddf

Use sed for variable replacement as envsubst not found in cloud build

d6cab18

Upgrade tests wait to be oldest builds before running

61d1fc8

Bumps versions

e1f1bce

Allows the Helm install of the current Agones development build durin…

73c1524

…g the upgrade test

Changes per linter

2645629

Bumps retry timeout by 10s

f1fca3f

Deletes all dangling agones resources

455bc3f

Updates evictable pods to have the same pod affinity rules as the tes…

9843c02

…t game servers

Removes unnecessary check from the cloudbuild file

f5bac5b

Changes watch events to AddFunc rather than UpdateFunc

5a56169

The same events are covered by AddFunc as by UpdateFunc with less overhead. Any container crashes are caught by the Game Server Unhealthy watch, and are not needed in this watch.

igooch force-pushed the testInPlaceUpgrades branch from fa639b0 to 5a56169 Compare December 6, 2024 19:26

igooch merged commit 171def9 into googleforgames:main Dec 6, 2024
4 checks passed

igooch deleted the testInPlaceUpgrades branch December 6, 2024 21:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrates upgrades tests into Cloud Build #4037

Integrates upgrades tests into Cloud Build #4037

igooch commented Nov 18, 2024 •

edited

Loading

agones-bot commented Nov 18, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 27, 2024

agones-bot commented Nov 27, 2024

agones-bot commented Dec 2, 2024

agones-bot commented Dec 3, 2024

agones-bot commented Dec 3, 2024

agones-bot commented Dec 3, 2024

agones-bot commented Dec 4, 2024

agones-bot commented Dec 6, 2024

Integrates upgrades tests into Cloud Build #4037

Integrates upgrades tests into Cloud Build #4037

Conversation

igooch commented Nov 18, 2024 • edited Loading

agones-bot commented Nov 18, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 19, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 20, 2024

agones-bot commented Nov 27, 2024

agones-bot commented Nov 27, 2024

agones-bot commented Dec 2, 2024

agones-bot commented Dec 3, 2024

agones-bot commented Dec 3, 2024

agones-bot commented Dec 3, 2024

agones-bot commented Dec 4, 2024

agones-bot commented Dec 6, 2024

igooch commented Nov 18, 2024 •

edited

Loading