HashiCorp Packer and VMware vSphere to Build Private Cloud Machine Images

Table of Contents

1. Introduction
2. Requirements
3. Configuration
4. Build
5. Troubleshoot
6. Credits

Introduction

This repository provides examples to automate the creation of virtual machine images and their guest operating systems on VMware vSphere using HashiCorp Packer and the Packer Plugin for VMware vSphere (vsphere-iso). All examples are authored in the HashiCorp Configuration Language ("HCL2").

By default, the machine image artifacts are transferred to a vSphere Content Library as an OVF template and the temporary machine image is destroyed. If an item of the same name exists in the target content library, Packer will update the existing item with the new OVF template. This method is extremely useful for vRealize Automation as image mappings do not need to be updated when a virtual machine image update is executed and finalized.

The following builds are available:

Linux Distributions

• VMware Photon OS 4
• Ubuntu Server 20.04 LTS
• Ubuntu Server 18.04 LTS
• Red Hat Enterprise Linux 8 Server
• AlmaLinux 8
• Rocky Linux 8
• CentOS Stream 8
• CentOS Linux 8

Microsoft Windows - Core and Desktop Experience

• Microsoft Windows Server 2022 - Standard and Datacenter
• Microsoft Windows Server 2019 - Standard and Datacenter
• Microsoft Windows Server 2016 - Standard and Datacenter

NOTE: Guest customization is not supported for AlmaLinux and Rocky Linux in vCenter Server 7.0 Update 2.

Requirements

Packer:

• HashiCorp Packer 1.7.4 or higher.

• HashiCorp Packer Plugin for VMware vSphere (vsphere-iso) 1.0.1 or higher.

• Packer Plugin for Windows Updates 0.14.0 or higher - a community plugin for HashiCorp Packer.

  Required plugins are automatically downloaded and initialized when using ./build.sh. For dark sites, you may download the plugins and place these same directory as your Packer executable /usr/local/bin or $HOME/.packer.d/plugins.

Operating Systems:

• Ubuntu Server 20.04 LTS

• macOS Big Sur (Intel)

• Microsoft Windows Server 2019

  Operating systems and versions tested with the repository examples.

Additional Software Packages:

• Git command line tools.
• A command-line .iso creator. Packer will use one of the following:
  • xorriso (Ubuntu: apt-get install xorriso)
  • mkisofs (Ubuntu: apt-get install mkisofs)
  • hdiutil (macOS)
  • oscdimg (Windows: requires Windows ADK)

Platform:

• VMware Cloud Foundation 4.2 or higher, or
• VMware vSphere 7.0 Update 2 or higher

Configuration

Step 1 - Clone the Repository

Clone the GitHub repository using Git.

Example:

git clone https://github.com/rainpole/packer-vsphere.git

The directory structure of the repository.

├── build.sh
├── LICENSE
├── NOTICE
├── README.md
├── builds
│   ├── ansible.pkvars.hcl
│   ├── build.pkvars.hcl
│   ├── common.pkvars.hcl
│   ├── rhsm.pkvars.hcl
│   ├── vsphere.pkvars.hcl
│   ├── linux
│   │   └── distribution-version
│   │       ├── *.pkr.hcl
│   │       ├── *.auto.pkrvars.hcl
│   │       └── http
│   │           └── ks.pkrtpl.hcl
│   └── windows
│       └── version
│           ├── *.pkr.hcl
│           ├── *.auto.pkrvars.hcl
│           └── cd
│               └── autounattend.pkrtpl.hcl
├── certificates
│   ├── root-ca.crt
│   └── root-ca.p7b
├── manifests
└── scripts
    ├── linux
    │   └── *.sh
    └── windows
        └── *.ps1

The files are distributed in the following directories.

• builds - contains the build templates, variables, and configuration files.
• scripts - contains scripts that are used to initialize and prepare the machine image builds.
• certificates - contains the Trusted Root Authority certificates.
• manifests - manifests created after the completion of each build.

Step 2 - Prepare the Guest Operating Systems ISOs

1. Download the x64 guest operating system .iso images.

   Linux Distributions

   • VMware Photon OS 4 Server
     • Download the latest release of the FULL .iso image.
   • Ubuntu Server 20.04 LTS
     • Download the latest LIVE release .iso image.
   • Ubuntu Server 18.04 LTS
     • Download the latest legacy NON-LIVE release .iso image.
   • Red Hat Enterprise Linux 8 Server
     • Download the latest release of the FULL (e.g. RHEL-8-x86_64-dvd1.iso) .iso image.
   • AlmaLinux 8 Server
     • Download the latest release of the FULL (e.g. AlmaLinux-8-x86_64-dvd1.iso) .iso image.
   • Rocky Linux 8 Server
     • Download the latest release of the FULL (e.g. Rocky-8-x86_64-dvd1.iso) .iso image.
   • CentOS Stream 8 Server
     • Download the latest release of the FULL (e.g. CentOS-Stream-8-x86_64-dvd1.iso) .iso image.
   • CentOS Linux 8 Server
     • Download the latest release of the FULL (e.g. CentOS-8-x86_64-dvd1.iso) .iso image.

   Microsoft Windows

   • Microsoft Windows Server 2022
   • Microsoft Windows Server 2019
   • Microsoft Windows Server 2016

2. Rename your guest operating system .iso images. The examples in this repository generally use the format of iso-family-vendor-type-version.iso.

   Example: iso-linux-ubuntu-server-20-04-lts.iso

3. Obtain the SHA-512 checksum for each guest operating system .iso image. This will be use in the build input variables.

   Example:

   • macOS terminal: shasum -a 512 [filename.iso]
   • Linux shell: sha512sum [filename.iso]
   • Windows command: certutil -hashfile [filename.iso] sha512

4. Upload your guest operating system .iso images to the datastore and path defined in your common variables.

   Example: [sfo-w01-ds-nfs01] /iso.

Step 3 - Configure the Variables

The variables are defined in .pkvars.hcl files.

Build Variables

Edit the /builds/build.pkvars.hcl file to configure the following:

• Credentials for the default account on machine images.

Example: /builds/build.pkvars.hcl

build_username           = "rainpole"
build_password           = "<plaintext_password>"
build_password_encrypted = "<sha512_encrypted_password >"
build_key                = "<public_key>"

Generate a SHA-512 encrypted password for the build_password_encrypted using various other tools like OpenSSL, mkpasswd, etc.

Example: OpenSSL on macOS:

rainpole@macos>  openssl passwd -6 
Password: ***************
Verifying - Password: ***************
[password hash]

Example: mkpasswd on Linux:

rainpole@linux>  mkpasswd --method=SHA-512 --rounds=4096
Password: ***************
[password hash]

Generate a public key for the build_password_encrypted for public key authentication.

Example: macOS and Linux.

rainpole@macos> cd .ssh/
rainpole@macos ~/.ssh> ssh-keygen -t ecdsa -b 521 -C "[email protected]"
Generating public/private ecdsa key pair.
Enter file in which to save the key (/Users/rainpole/.ssh/id_ecdsa): 
Enter passphrase (empty for no passphrase): **************
Enter same passphrase again: **************
Your identification has been saved in /Users/rainpole/.ssh/id_ecdsa.
Your public key has been saved in /Users/rainpole/.ssh/id_ecdsa.pub.

The content of the public key, build_key, is added the key to the .ssh/authorized_keys file of the build_username on the guest operating system.

WARNING: Replace the default public keys and passwords. By default, both Public Key Authentication and Password Authentication are enabled for Linux distributions. If you wish to disable Password Authentication and only use Public Key Authentication, comment or remove the portion of the associated script in the /scripts directory.

Ansible Variables

Edit the /builds/ansible.pkvars.hcl file to configure the following:

• Credentials for the Ansible account on Linux machine images.

Example: /builds/ansible.pkvars.hcl

ansible_username = "ansible"
ansible_key      = "<public_key>"

NOTE: A random password is generated for the Ansible user.

Common Variables

Edit the /builds/common.pkvars.hcl file to configure the following:

• Common Virtual Machine Settings
• Common Template and Content Library Settings
• Common Removable Media Settings
• Common Boot and Provisioning Settings

Example: /builds/common.pkvars.hcl

common_template_conversion     = false
common_content_library_name    = "sfo-w01-lib01"
common_content_library_ovf     = true
common_content_library_destroy = true

vSphere Variables

Edit the /buils/vsphere.pkvars.hcl file to configure the following:

• vSphere Endpoint and Credentials
• vSphere Settings

Example: /builds/vsphere.pkvars.hcl

vsphere_endpoint             = "sfo-w01-vc01.sfo.rainpole.io"
vsphere_username             = "[email protected]"
vsphere_password             = "<plaintext_password>"
vsphere_insecure_connection  = true
vsphere_datacenter           = "sfo-w01-dc01"
vsphere_cluster              = "sfo-w01-cl01"
vsphere_datastore            = "sfo-w01-cl01-ds-vsan01"
vsphere_network              = "sfo-w01-seg-dhcp"
vsphere_folder               = "sfo-w01-fd-templates"

Red Hat Subscription Manager Variables

Edit the /builds/redhat.pkvars.hcl file to configure the following:

• Credentials for your Red Hat Subscription Manager account.

Example: /builds/redhat.pkvars.hcl

rhsm_username = "rainpole"
rhsm_password = "<plaintext_password>"

These variables are only used if you are performing a Red Hat Enterprise Linux Server build to register the image with Red Hat Subscription Manager and run a sudo yum update -y within the shell provisioner. Before the build completes, the machine image is unregistered from Red Hat Subscription Manager.

Machine Image Variables

Edit the *.auto.pkvars.hcl file in each builds/<type>/<build> folder to configure the following virtual machine hardware settings, as required:

• CPU Sockets (init)

• CPU Cores (init)

• Memory in MB (init)

• Primary Disk in MB (init)

• .iso Image File (string)

• .iso Image SHA-512 Checksum (string)

  Note: All variables.auto.pkvars.hcl default to using the the recommended firmware for the guest operating system, the VMware Paravirtual SCSI controller and the VMXNET 3 network card device types.

Using Environmental Variables

Some of the variables may include sensitive information and environmental data that you would prefer not to save to clear text files. You can add there to environmental variables using the example below:

export PKR_VAR_vsphere_endpoint="<vsphere_endpoint_fqdn>"
export PKR_VAR_vsphere_username="<vsphere_username>"
export PKR_VAR_vsphere_password="<vsphere_password>"
export PKR_VAR_vsphere_datacenter="<vsphere_datacenter>>"
export PKR_VAR_vsphere_cluster="<vsphere_cluster>"
export PKR_VAR_vsphere_datastore="<vsphere_datastore>>"
export PKR_VAR_vsphere_network="<vsphere_network>"
export PKR_VAR_vsphere_folder="<vsphere_folder>"
export PKR_VAR_build_username="<build_password>"
export PKR_VAR_build_password="<build_password>"
export PKR_VAR_build_password="<build_password_encrypted>"
export PKR_VAR_build_key="<build_key>"
export PKR_VAR_ansible_username="<ansible_password>"
export PKR_VAR_ansible_key="<ansible_key>"
export PKR_VAR_rhsm_username="<rhsm_password>"
export PKR_VAR_rhsm_password="<rhsm_password>"

Step 4 - Modify the Configurations and Scripts

If required, modify the configuration and scripts files, for the Linux distributions and Microsoft Windows.

Linux Distribution Kickstart and Scripts

Username and password variables are passed into the kickstart or cloud-init files for each Linux distribution as Packer template files (.pkrtpl.hcl) to generate these on-demand.

A SHA-512 encrypted password for the root account and the build_username (e.g. rainpole). It also adds the build_username to the sudoers.

Microsoft Windows Unattended amd Scripts

Variables are passed into the Microsoft Windows unattend files (autounattend.xml) as Packer template files (autounattend.pkrtpl.hcl) to generate these on-demand.

By default, each unattended file set the Product Key to use the KMS client setup keys.

Need help customizing the configuration files?

• VMware Photon OS - Read the Photon OS Kickstart Documentation.

• Ubuntu Server - Install and run system-config-kickstart on a Ubuntu desktop.

  sudo apt-get install system-config-kickstart
  ssh -X rainpole@ubuntu-desktop
  sudo system-config-kickstart
  

• Red Hat Enterprise Linux (as well as CentOS Linux/Stream, AlmaLinux, and Rocky Linux) - Use the Red Hat Kickstart Generator.

• Microsoft Windows - Use the Microsoft Windows Answer File Generator if you need to customize the provided examples further.

Step 5 - Configure Certificates

Save a copy of your Root Certificate Authority certificate to /certificates in .crt and .p7b formats.

These files are copied to the guest operating systems with a Packer file provisioner; after which, the a shell provisioner adds the certificate to the Trusted Certificate Authority of the guest operating system.

NOTE: If you do not wish to install the certificates on the guest operating systems, comment or remove the portion of the associated script in the /scripts directory and the file provisioner from the prk.hcl file for each build. If you need to add an intermediate certificate, add the certificate to /certificates and update the shell provisioner scripts in the scripts directory with your requirements.

Build

Start a pre-defined build by running the build script (./build.sh). The script presents a menu the which simply calls Packer and the respective build(s).

Example: Menu for ./build.sh.

    ____             __                ____        _ __    __     
   / __ \____ ______/ /_____  _____   / __ )__  __(_) /___/ /____ 
  / /_/ / __  / ___/ //_/ _ \/ ___/  / __  / / / / / / __  / ___/ 
 / ____/ /_/ / /__/ ,< /  __/ /     / /_/ / /_/ / / / /_/ (__  )  
/_/    \__,_/\___/_/|_|\___/_/     /_____/\__,_/_/_/\__,_/____/   

  Select a HashiCorp Packer build for VMware vSphere:

      Linux Distribution:

         1  -  VMware Photon OS 4
         2  -  Ubuntu Server 20.04 LTS
         3  -  Ubuntu Server 18.04 LTS
         4  -  Red Hat Enterprise Linux 8 Server
         5  -  AlmaLinux 8 Server
         6  -  Rocky Linux 8 Server
         7  -  CentOS Stream 8 Server
         8  -  CentOS Linux 8 Server

      Microsoft Windows:

         9  -  Windows Server 2022 - All
        10  -  Windows Server 2022 - Standard Only
        11  -  Windows Server 2022 - Datacenter Only
        12  -  Windows Server 2019 - All
        13  -  Windows Server 2019 - Standard Only
        14  -  Windows Server 2019 - Datacenter Only
        15  -  Windows Server 2016 - All
        16  -  Windows Server 2016 - Standard Only
        17  -  Windows Server 2016 - Datacenter Only

      Other:
      
        I   -  Information
        Q   -  Quit

You can also start a build based on a specific source for some of the virtual machine images.

For example, if you simply want to build a Microsoft Windows Server 2022 Standard Core, run the following:

Initialize the plugins:

rainpole@macos packer-examples> cd builds/windows/windows-server-2022/
rainpole@macos packer-examples> packer init windows-server-2022.pkr.hcl

Build a specific machine image:

rainpole@macos windows-server-2022> packer build -force \
      --only vsphere-iso.windows-server-standard-core \
      -var-file="../../vsphere.pkrvars.hcl" \
      -var-file="../../build.pkrvars.hcl" \
      -var-file="../../common.pkrvars.hcl" .

Build a specific machine image using environmental variables:

rainpole@macos windows-server-2022> packer build -force \
      --only vsphere-iso.windows-server-standard-core \
      -var-file="../../common.pkrvars.hcl" .

Happy building!!!

-- Your friends at rainpole.io.

Troubleshoot

• Read Debugging Packer Builds.

Credits

• Maher AlAsfar @vmwarelab

  Linux Bash scripting hints.

• Owen Reynolds @OVDamn

  VMware Tools for Windows installation PowerShell script.