Added an additional check to make sure etcdadm and capi clusters are available

[panktishah26]

Description of changes:
We are having an issue in creating workload cluster using the controller if external Etcd is configured on the workload cluster. It used to fail with failed calling webhook "mutation.vspheremachineconfig.anywhere.amazonaws.com error. This issue happens because reconciler tries to get the CAPI cluster before the CAPI cluster becomes available for the workload cluster.
Added additional checks to make sure CAPI cluster are Etcdadm cluster are available before moving to check if the etcd machines are available or not.

Testing (if applicable):

Documentation added/planned (if applicable):

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[panktishah26]

/test eks-anywhere-presubmit

[codecov]

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (3c6cece) 71.84% compared to head (e9905c9) 71.99%.
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6806      +/-   ##
==========================================
+ Coverage   71.84%   71.99%   +0.14%     
==========================================
  Files         531      532       +1     
  Lines       41365    41597     +232     
==========================================
+ Hits        29718    29947     +229     
+ Misses       9981     9978       -3     
- Partials     1666     1672       +6     

Files	Coverage Δ
pkg/controller/clusters/status.go	94.54% <100.00%> (+2.14%)	⬆️

... and 23 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[chrisdoherty4]

I don't understand how this is possible. What are the circumstances for this happening? It feels peculiar to conditionally update conditions based on the existence of objects we're responsible for creating all in the same algorithm.

[tatlat]

I don't understand how this is possible. What are the circumstances for this happening? It feels peculiar to conditionally update conditions based on the existence of objects we're responsible for creating all in the same algorithm.

When we apply the workload cluster config in the API tests, the controller would attempt to update the status of the workload cluster before the CAPI cluster could be created. In UpdateClusterStatusForControlPlane, controller.GetCAPICluster would return nil and the controller would get stuck in a a cycle of panicking and restarting. Since the controller was always restarting, the webhooks were unreachable.

[chrisdoherty4]

the controller would attempt to update the status of the workload cluster before the CAPI cluster could be created

Do you mean for the cluster to actually be created or for the Cluster API Cluster object to be submitted to the cluster?

[eks-distro-bot]

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment