Skip to content

Commit

Permalink
Merge pull request #64 from sp-yduck/develop/v0.3.0
Browse files Browse the repository at this point in the history
release v0.3.0
  • Loading branch information
sp-yduck authored Aug 10, 2023
2 parents 9aa4667 + 398a609 commit eed957f
Show file tree
Hide file tree
Showing 34 changed files with 1,121 additions and 386 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/golangci-lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: golangci-lint

on:
pull_request:
types: [opened, edited, synchronize, reopened]
types: [opened, synchronize, reopened]

# Remove all permissions from GITHUB_TOKEN except metadata.
permissions: {}
Expand All @@ -15,10 +15,10 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: 1.19
go-version: 1.20.5
check-latest: true
- name: golangci-lint
uses: golangci/[email protected]
with:
version: v1.52.2
version: v1.53.3
args: --timeout 5m
12 changes: 11 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ fmt: ## Run go fmt against code.
vet: ## Run go vet against code.
go vet ./...

.PHONY: lint
lint: ## Run golangci-lint
$(GOLANGCI_LINT) run

CLUSTER_NAME := cappx-test

.PHONY: create-workload-cluster
Expand Down Expand Up @@ -180,6 +184,7 @@ CONTROLLER_GEN ?= $(LOCALBIN)/controller-gen
ENVTEST ?= $(LOCALBIN)/setup-envtest
ENVSUBST ?= $(LOCALBIN)/envsubst
KUBECTL ?= $(LOCALBIN)/kubectl
GOLANGCI_LINT ?= $(LOCALBIN)/golangci-lint

## Tool Versions
KUSTOMIZE_VERSION ?= v5.0.0
Expand Down Expand Up @@ -222,4 +227,9 @@ $(KUBECTL): $(LOCALBIN)
.PHONY: setup-envtest
setup-envtest: $(SETUP_ENVTEST)
$(SETUP_ENVTEST): go.mod # Build setup-envtest from tools folder.
GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@$(SETUP_ENVTEST_VER)
GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@$(SETUP_ENVTEST_VER)

.PHONY: golangci-lint
golangci-lint: $(GOLANGCI_LINT)
$(GOLANGCI_LINT): $(LOCALBIN)
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(LOCALBIN) v1.54.0
27 changes: 12 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,25 +17,20 @@ for more information : https://cluster-api.sigs.k8s.io/user/quick-start.html#ini
```sh
# install cluster-api components
export EXP_CLUSTER_RESOURCE_SET=true
clusterctl init --infrastructure=proxmox:v0.2.3 --config https://raw.githubusercontent.com/sp-yduck/cluster-api-provider-proxmox/main/clusterctl.yaml
clusterctl init --infrastructure=proxmox:v0.3.0 --config https://raw.githubusercontent.com/sp-yduck/cluster-api-provider-proxmox/main/clusterctl.yaml
```
**Note:** container images are available at [ghcr.io/sp-yduck/cluster-api-provider-proxmox:\<tag\>](https://github.com/sp-yduck/cluster-api-provider-proxmox/pkgs/container/cluster-api-provider-proxmox)

2. Create your first workload cluster
```sh
# export env variables
export CONTROLPLANE_HOST=X.X.X.X # control-plane vip
export CONTROLPLANE_HOST=X.X.X.X # control-plane vip
export PROXMOX_URL=https://X.X.X.X:8006/api2/json
# export PROXMOX_PASSWORD=password # (optional)
# export PROXMOX_USER=user@pam # (optional)
export PROXMOX_TOKENID='root@pam!api-token-id' # (optional)
export PROXMOX_SECRET=aaaaaaaa-bbbb-cccc-dddd-ee12345678 # (optional)
export NODE_URL=node.ssh.url:22
export NODE_USER=node-ssh-user
export NODE_PASSWORD=node-ssh-password
export PROXMOX_PASSWORD=password
export PROXMOX_USER=user@pam

# generate manifests (available flags: --target-namespace, --kubernetes-version, --control-plane-machine-count, --worker-machine-count)
clusterctl generate cluster cappx-test --control-plane-machine-count=3 --infrastructure=proxmox:v0.2.3 --config https://raw.githubusercontent.com/sp-yduck/cluster-api-provider-proxmox/main/clusterctl.yaml > cappx-test.yaml
clusterctl generate cluster cappx-test --control-plane-machine-count=3 --infrastructure=proxmox:v0.3.0 --config https://raw.githubusercontent.com/sp-yduck/cluster-api-provider-proxmox/main/clusterctl.yaml > cappx-test.yaml

# inspect and edit
vi cappx-test.yaml
Expand All @@ -62,21 +57,23 @@ kubectl delete cluster cappx-test

## Fetures

- No need to prepare vm templates. You can specify any vm image in `ProxmoxMachine.Spec.Image`.
- No need to prepare vm templates. You can specify any vm image in `ProxmoxMachine.Spec.Image`. CAPPX bootstrap your vm from scratch.

- Supports custom cloud-config (user data). CAPPX uses ssh for bootstrapping nodes so it can applies custom cloud-config that can not be achieved by only Proxmox API.
- Supports mutiple image format. CAPPX uses VNC websocket for downloading/installing node images so it can support multiple image format not only ISO (Proxmox API can only support ISO)

- Supports custom cloud-config (user data). CAPPX uses VNC websockert for bootstrapping nodes so it can applies custom cloud-config that can not be achieved by only Proxmox API.

### Node Images

CAPPX is compatible with `qcow2` image. You can build your own node image and use it for `ProxmoxMachine`.
CAPPX is compatible with `iso`, `qcow2`, `qed`, `raw`, `vdi`, `vpc`, `vmdk` format of image. You can build your own node image and use it for `ProxmoxMachine`.

CAPPX relies on a few prerequisites which have to be already installed in the used operating system images, e.g. a container runtime, kubelet, kubeadm,.. .

To build your custom node image, you can use [kubernetes-sigs/image-builder](https://github.com/kubernetes-sigs/image-builder) project.

Also there are some available out-of-box images published other communities such as [Metal3](https://github.com/metal3-io). For example https://artifactory.nordix.org/ui/native/metal3/images/. Example MD can be found [metal3-ubuntu2204-k8s127.yaml](examples/machine_deployment/metal3-ubuntu2204-k8s127.yaml).

If it isn't possible to pre-install those prerequisites in the image, you can always deploy and execute some custom scripts through the `ProxmoxMachine.spec.cloudInit` or `KubeadmConfig` . Example MD can be found [ubuntu2204.yaml](examples/machine_deployment/ubuntu2204.yaml).
If it isn't possible to pre-install those prerequisites in the image, you can always deploy and execute some custom scripts through the `ProxmoxMachine.spec.cloudInit` or `KubeadmConfig`. Example MD can be found [ubuntu2204.yaml](examples/machine_deployment/ubuntu2204.yaml).

## Compatibility

Expand All @@ -99,7 +96,7 @@ This project aims to follow the Cluster API [Provider contract](https://cluster-

### ProxmoxCluster

Because Proxmox-VE does not provide LBaaS solution, CAPPX does not follow the [typical infra-cluster logic](https://cluster-api.sigs.k8s.io/developer/providers/cluster-infrastructure.html#behavior). ProxmoxCluster controller reconciles only Proxmox storages used for instances. You need to prepare control plane load balancer by yourself if you creates HA control plane workload cluster.
Because Proxmox-VE does not provide LBaaS solution, CAPPX does not follow the [typical infra-cluster logic](https://cluster-api.sigs.k8s.io/developer/providers/cluster-infrastructure.html#behavior). ProxmoxCluster controller reconciles only Proxmox storages used for instances. You need to prepare control plane load balancer by yourself if you creates HA control plane workload cluster. In the [cluster-template.yaml](./templates/cluster-template.yaml), you can find HA control plane example with [kube-vip](https://github.com/kube-vip/kube-vip).

### ProxmoxMachine

Expand Down
165 changes: 165 additions & 0 deletions api/v1beta1/options_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
package v1beta1

import "strconv"

// import "encoding/json"

// +kubebuilder:validation:Enum:=x86_64;aarch64
type Arch string

// +kubebuilder:validation:Enum:=seabios;ovmf
type BIOS string

// +kubebuilder:validation:Enum:=0;2;1024
type HugePages int

// +kubebuilder:validation:Enum:=backup;clone;create;migrate;rollback;snapshot;snapshot-delete;suspending;suspended
type Lock string

// +kubebuilder:validation:Enum:=other;wxp;w2k;w2k3;w2k8;wvista;win7;win8;win10;win11;l24;l26;solaris
type OSType string

// +kubebuilder:validation:Pattern:="[a-zA-Z0-9-_.;]+"
type Tag string

type Tags []Tag

func (h *HugePages) String() string {
if h == nil {
return ""
} else if *h == 0 {
return "any"
}
return strconv.Itoa(int(*h))
}

func (t *Tags) String() string {
var tags string
for _, tag := range *t {
tags += string(tag) + ";"
}
return tags
}

// Options
type Options struct {
// Enable/Disable ACPI. Defaults to true.
ACPI bool `json:"acpi,omitempty"`

// Virtual processor architecture. Defaults to the host. x86_64 or aarch64.
Arch Arch `json:"arch,omitempty"`

// +kubebuilder:validation:Minimum:=0
// Amount of target RAM for the VM in MiB. Using zero disables the ballon driver.
Balloon int `json:"balloon,omitempty"`

// Description for the VM. Shown in the web-interface VM's summary.
// This is saved as comment inside the configuration file.
Description string `json:"description,omitempty"`

// Script that will be executed during various steps in the vms lifetime.
// HookScripts []Hookscript `json:"hookScripts,omitempty"`

// enable hotplug feature. list og devices.
// network, disk, cpu, memory, usb. Defaults to [network, disk, usb].
// HotPlug []HotPlugDevice `json:"hotPlug,omitempty"`

// enable/disable hugepages memory. 0 or 2 or 1024. 0 indicated 'any'
HugePages *HugePages `json:"hugePages,omitempty"`

// Use together with hugepages. If enabled, hugepages will not not be deleted
// after VM shutdown and can be used for subsequent starts. Defaults to false.
KeepHugePages bool `json:"keepHugePages,omitempty"`

// Enable/disable KVM hardware virtualization. Defaults to true.
KVM bool `json:"kvm,omitempty"`

// Set the real time clock (RTC) to local time.
// This is enabled by default if the `ostype` indicates a Microsoft Windows OS.
LocalTime bool `json:"localTime,omitempty"`

// Lock/unlock the VM.
Lock Lock `json:"lock,omitempty"`

// Set maximum tolerated downtime (in seconds) for migrations.
// MigrateDowntime json.Number `json:"migrateDowntime,omitempty"`

// Set maximum speed (in MB/s) for migrations. Value 0 is no limit.
// MigrateSpeed `json:"migrateSpeed,omitempty"`

// Enable/disable NUMA.
NUMA bool `json:"numa,omitempty"`

// Specifies whether a VM will be started during system bootup.
OnBoot bool `json:"onBoot,omitempty"`

// Specify guest operating system. This is used to enable special
// optimization/features for specific operating systems.
OSType OSType `json:"osType,omitempty"`

// Sets the protection flag of the VM.
// This will disable the remove VM and remove disk operations.
// Defaults to false.
Protection bool `json:"protection,omitempty"`

// Allow reboot. If set to 'false' the VM exit on reboot.
// Defaults to true.
Reboot bool `json:"reboot,omitempty"`

// +kubebuilder:validation:Minimum:=0
// +kubebuilder:validation:Maximum:=5000
// Amount of memory shares for auto-ballooning. The larger the number is, the more memory this VM gets.
// Number is relative to weights of all other running VMs. Using zero disables auto-ballooning.
// Auto-ballooning is done by pvestatd. 0 ~ 5000. Defaults to 1000.
Shares int `json:"shares,omitempty"`

// Set the initial date of the real time clock.
// Valid format for date are:'now' or '2006-06-17T16:01:21' or '2006-06-17'.
// Defaults to 'now'.
// StartDate string `json:"startDate,omitempty"`

// StartUp string `json:"startUp,omitempty`

// Enable/disable the USB tablet device. This device is usually needed to allow
// absolute mouse positioning with VNC. Else the mouse runs out of sync with normal VNC clients.
// If you're running lots of console-only guests on one host,
// you may consider disabling this to save some context switches.
// This is turned off by default if you use spice (`qm set <vmid> --vga qxl`).
// Defaults to true.
Tablet bool `json:"tablet,omitempty"`

// Tags of the VM. This is only meta information.
Tags Tags `json:"tags,omitempty"`

// Enable/disable time drift fix. Defaults to false.
TimeDriftFix bool `json:"timeDriftFix,omitempty"`

// Enable/disable Template. Defaults to false.
Template bool `json:"template,omitempty"`

// TPMState string `json:"tpmState,omitempty"`

// +kubebuilder:validation:Minimum:=0
// Number of hotplugged vcpus. Defaults to 0.
VCPUs int `json:"vcpus,omitempty"`

// VGA string `json:"vga,omitempty"`

// +kubebuilder:validation:Pattern:="(?:[a-fA-F0-9]{8}(?:-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}|[01])"
// The VM generation ID (vmgenid) device exposes a 128-bit integer value identifier to the guest OS.
// This allows to notify the guest operating system when the virtual machine is executed with a different configuration
// (e.g. snapshot execution or creation from a template).
// The guest operating system notices the change, and is then able to react as appropriate by marking its copies of distributed databases as dirty,
// re-initializing its random number generator, etc.
// Note that auto-creation only works when done through API/CLI create or update methods, but not when manually editing the config file.
// regex: (?:[a-fA-F0-9]{8}(?:-[a-fA-F0-9]{4}){3}-[a-fA-F0-9]{12}|[01]). Defaults to 1 (autogenerated)
VMGenerationID string `json:"vmGenerationID,omitempty"`

// Default storage for VM state volumes/files.
// VMStateStorage string `json:"vmStateStorage,omitempty"`

// Create a virtual hardware watchdog device. Once enabled (by a guest action),
// the watchdog must be periodically polled by an agent inside the guest or else
// the watchdog will reset the guest (or execute the respective action specified)
// WatchDog string `json:"watchDog,omitempty"`
}
6 changes: 0 additions & 6 deletions api/v1beta1/proxmoxcluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ import (
clusterv1 "sigs.k8s.io/cluster-api/api/v1beta1"
)

// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

const (
// ClusterFinalizer
ClusterFinalizer = "proxmoxcluster.infrastructure.cluster.x-k8s.io"
Expand All @@ -38,9 +35,6 @@ type ProxmoxClusterSpec struct {
// ServerRef is used for configuring Proxmox client
ServerRef ServerRef `json:"serverRef"`

// NodesRef contains reference of nodes used for ProxmoxCluster
NodeRefs []NodeRef `json:"nodeRefs,omitempty"`

// storage is for proxmox storage used by vm instances
// +optional
Storage Storage `json:"storage"`
Expand Down
19 changes: 10 additions & 9 deletions api/v1beta1/proxmoxmachine_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,31 +17,27 @@ limitations under the License.
package v1beta1

import (
"github.com/sp-yduck/proxmox-go/api"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
clusterv1 "sigs.k8s.io/cluster-api/api/v1beta1"
"sigs.k8s.io/cluster-api/errors"
)

// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

const (
// MachineFinalizer
MachineFinalizer = "proxmoxmachine.infrastructure.cluster.x-k8s.io"
)

// ProxmoxMachineSpec defines the desired state of ProxmoxMachine
type ProxmoxMachineSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file

// ProviderID
ProviderID *string `json:"providerID,omitempty"`

// Node is proxmox node hosting vm instance which used for ProxmoxMachine
// +optional
Node string `json:"node,omitempty"`

// +kubebuilder:validation:Minimum:=0
// VMID is proxmox qemu's id
// +optional
VMID *int `json:"vmID,omitempty"`
Expand All @@ -60,9 +56,11 @@ type ProxmoxMachineSpec struct {
// Network
Network Network `json:"network,omitempty"`

// Options
// +optional
Options Options `json:"options,omitempty"`

// FailureDomain is the failure domain unique identifier this Machine should be attached to, as defined in Cluster API.
// For this infrastructure provider, the ID is equivalent to an AWS Availability Zone.
// If multiple subnets are matched for the availability zone, the first one returned is picked.
FailureDomain *string `json:"failureDomain,omitempty"`
}

Expand All @@ -84,7 +82,10 @@ type ProxmoxMachineStatus struct {
// Conditions
Conditions clusterv1.Conditions `json:"conditions,omitempty"`

// InstanceStatus is the status of the GCP instance for this machine.
// Configuration
Config api.VirtualMachineConfig `json:"config,omitempty"`

// InstanceStatus is the status of the proxmox instance for this machine.
// +optional
InstanceStatus *InstanceStatus `json:"instanceStatus,omitempty"` // InstanceStatus
}
Expand Down
Loading

0 comments on commit eed957f

Please sign in to comment.