Add e2e test for installer flow

[shubham14bajpai]

Signed-off-by: Shubham [email protected]

What this PR does / why we need it:
This PR adds e2e test for the new installer flow of provisioning workload cluster. This flow will replace the current way of provisioning so keeping that in mind few choices were made while implementing the test and to avoid writing complex tests which might be removed soon:

• another cluster template was added with installer changes
• the current test was duplicated with 1 key difference of --use-installer-controller enabled via host agent
• another step was added to github action to run installer test separately as the both cannot be run together due to difference in cluster template
• to reuse the same provider.yaml file sed the template path before running installer test.

The above choices also makes it easier to remove the old test when we decide to deprecate the old flow, which I assume is pretty soon. We can just remove the old test & template and rename/restructure some new test & templates.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #468

Additional information
I was facing the below error while testing the installer flow. Reversing the order of kubelet and kubeadm in the install script fixed it

+ dpkg --install /var/lib/byoh/bundles/projects.registry.vmware.com/cluster_api_provider_bringyourownhost/byoh-bundle-ubuntu_20.04.1_x86-64_k8s:v1.23.5/kubeadm.deb
�1Selecting previously unselected package kubeadm.
�G(Reading database ... 8761 files and directories currently installed.)
�(Preparing to unpack .../kubeadm.deb ...
�"Unpacking kubeadm (1.23.5-00) ...
��dpkg: dependency problems prevent configuration of kubeadm:
 kubeadm depends on kubelet (>= 1.19.0); however:
  Package kubelet is not installed.
dpkg: error processing package kubeadm (--install):
 dependency problems - leaving unconfigured

Special notes for your reviewer

[dharmjit]

LGTM on a first pass. minor nit only. Do we have a successful run to link in the PR?

[shubham14bajpai]

LGTM on a first pass. minor nit only. Do we have a successful run to link in the PR?

The e2e check in this PR runs both the old and flow test.

[sachinkumarsingh092]

We should also close #255 after merging this PR.

[anusha94]

lgtm