⚠️ Treating VSphereVM wasNotFoundByBIOSUUID as transient error

[zhanggbj]

What this PR does / why we need it:
We used to treate WasNotFoundByUUID as terminal error, and set failureMessage and failureReason, so VSphereVM won't be reconciled anymore. However, in some situation, this error is recoverable, for instance, if the storage was temporarily disconnected but later recovered, this error will be transient.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2102

[zhanggbj]

Also CC @sbueringer for a code review, thx!

[chrischdi]

Thanks for the PR!

One question but as of that this looks great.

/hold

To wait for other opinions 👍

[chrischdi]

To summarise what happens on current code state when "not found by biosUUID" gets hit:

• All 3 objects (Machine, VSphereMachine, VSphereVM) will get set additional conditions:
  • Machine:
    • type: Ready
    • type: InfrastructureReady
  • VSphereMachine & VSphereVM:
    • type: Ready
    • type: VMProvisioned

For all of these six conditions, message, reason and severity is the same:

   message: vm with bios uuid 420a150b-bb27-8db3-5a61-6e3f472a4137aaa not found
   reason: NotFoundByBIOSUUID
   severity: Error

Additional for VSphereVM only: (to be dropped according above comments)

• .status.ready gets removed (because it gets set to the default false and by that omitted)

Note: tested via kubectl edit vspherevm <name> + changing the .spec.biosUUID` field

Note2: Fixing the issue gets everything back to a clean healthy state.

[zhanggbj]

@chrischdi @sbueringer Thanks again for your valuable feedback and quick action:) All comments are addressed and PR is ready for a second review.

[chrischdi]

/lgtm

sbueringer for approval :-)

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: fb9d01b7b4457f36cff01e44ef77d4131acd6850

[sbueringer]

/retitle ⚠️ Treating VSphereVM wasNotFoundByBIOSUUID as transient error
(for the behavioral change)

[sbueringer]

Thank you very much!!

/lgtm
/approve

(let's not backport this one given the change in behavior)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sbueringer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sbueringer]

Ups forgot

/hold cancel

[sbueringer]

Unrelated flake

/retest