Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installer e2e test #255

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions agent/installer/e2e/Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
# The most common configuration options are documented and commented below.
# For a complete reference, please see the online documentation at
# https://docs.vagrantup.com.

# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
config.vm.box = "bento/ubuntu-20.04"
config.vm.define "byoh-installer-e2e-test-vm"

# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
# config.vm.box_check_update = false

# Create a forwarded port mapping which allows access to a specific port
# within the machine from a port on the host machine. In the example below,
# accessing "localhost:8080" will access port 80 on the guest machine.
# NOTE: This will enable public access to the opened port
# config.vm.network "forwarded_port", guest: 80, host: 8080

# Create a forwarded port mapping which allows access to a specific port
# within the machine from a port on the host machine and only allow access
# via 127.0.0.1 to disable public access
# config.vm.network "forwarded_port", guest: 80, host: 8080, host_ip: "127.0.0.1"

# Create a private network, which allows host-only access to the machine
# using a specific IP.
# config.vm.network "private_network", ip: "192.168.33.10"

# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
# config.vm.network "public_network"

# Share an additional folder to the guest VM. The first argument is
# the path on the host to the actual folder. The second argument is
# the path on the guest to mount the folder. And the optional third
# argument is a set of non-required options.
# config.vm.synced_folder "../data", "/vagrant_data"

# Provider-specific configuration so you can fine-tune various
# backing providers for Vagrant. These expose provider-specific options.
# Example for VirtualBox:
#
# config.vm.provider "virtualbox" do |vb|
# # Display the VirtualBox GUI when booting the machine
# vb.gui = true
#
# # Customize the amount of memory on the VM:
# vb.memory = "1024"
# end
#
# View the documentation for the provider you are using for more
# information on available options.

# Enable provisioning with a shell script. Additional provisioners such as
# Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
# documentation for more information about their specific syntax and use.
# config.vm.provision "shell", inline: <<-SHELL
# apt-get update
# apt-get install -y apache2
# SHELL
end
10 changes: 10 additions & 0 deletions agent/installer/e2e/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
services:
ingredients-deb:
build:
context: ../bundle_builder/ingredients/deb/
dockerfile: Dockerfile

bundle-builder:
build:
context: ../bundle_builder/
dockerfile: Dockerfile
31 changes: 31 additions & 0 deletions agent/installer/e2e/core.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
services:
ingredients-deb:
image: ingredients-deb
volumes:
- bundle-vol:/ingredients

bundle-builder:
image: bundle-builder
command: bundle-repo.local:5000/byoh/byoh-bundle-ubuntu_20.04.1_x86-64_k8s_${K8SVER:-v1.22.3}
depends_on:
- ingredients-deb
- bundle-repo.local
environment:
- BUILD_ONLY=0
networks:
- app-net
volumes:
- bundle-vol:/ingredients

bundle-repo.local:
image: registry
ports:
- ${REPOPORT:-5005}:5000
networks:
- app-net

networks:
app-net:

volumes:
bundle-vol: {}
21 changes: 21 additions & 0 deletions agent/installer/e2e/test-in-vm.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#~/bin/bash

# host machine is accessible as gateway by guest
REPO_IP=$(/sbin/ip route | awk '/default/ { print $3 }')
REPO_PORT=$1
K8SVER=$2

echo "===Install Pre-requisites"
apt-get install socat ebtables ethtool conntrack -y

set -e

echo "===Run Install"
./cli --install --bundle-repo $REPO_IP:$REPO_PORT/byoh --k8s $K8SVER

echo "===Run kubeadm preflight"
kubeadm init phase preflight --ignore-preflight-errors=Cpu,Mem

echo "===Run Uninstall"
./cli --uninstall --k8s $K8SVER

45 changes: 45 additions & 0 deletions agent/installer/e2e/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#~/bin/bash

set -e

TAG=byoh-installer-e2e-test
export REPOPORT=5005
export K8SVER="v1.22.3"

function cleanup()
{
ARG=$?

set +e

echo "==Clean up"

docker-compose -p $TAG -f build.yml -f core.yml down --rmi all --volumes

vagrant destroy -f

rm cli
rm -rf $K8SVER*

exit $ARG
}

trap cleanup EXIT

echo "===Download bundle ingredients"
docker-compose -p $TAG -f core.yml -f build.yml up --build ingredients-deb

echo "===Starting bundle repository"
docker-compose -p $TAG -f core.yml -f build.yml up --build bundle-repo.local &

echo "===Build and publish bundle"
docker-compose -p $TAG -f core.yml -f build.yml up --build bundle-builder

echo "===Build cli"
go build ../cli
echo "===Spin up test vm"
vagrant up
echo "===Install bundle inside vm"
vagrant ssh -c "cd /vagrant && sudo ./test-in-vm.sh $REPOPORT $K8SVER"

echo "===PASS"
17 changes: 17 additions & 0 deletions docs/local_dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -291,3 +291,20 @@ sudo ./cli --install --os Ubuntu_20.04.1_x86-64 --bundle-repo 10.26.226.219:5000
# Will override the OS detection, use the specified repo and uninstall
sudo ./cli --uninstall --os Ubuntu_20.04.1_x86-64 --bundle-repo 10.26.226.219:5000/repo --k8s v1.22.3
```

## Run end to end test
Vagrant is required.

```shell
(cd agent/installer/bundle_builder/e2e && ./test.sh)
```
This test executes the following workflow:
- download upstream bundle contents
- start local repo
- build a BYOH bundle and upload it to the local repo
- spin up a Ubuntu 20.04 VM and inside it run
- cli install with local repo
- kubeadm init phase preflight
- cli uninstall

Upon success return 0.