Skip to content

Service deployment

Zarquan edited this page Jul 17, 2023 · 16 revisions

The configuration for these machines comes from the heliodines private git repository, a copy of which is installed at /var/local/projects/heliodines/git.

Physical machines

The physical hyper-visor machine trop01 has two physical network interfaces named br0 and br1.

#[user@trop02]

    /sbin/ifconfig

        br0     Link encap:Ethernet  HWaddr 0c:c4:7a:35:12:06  
                inet addr:129.215.175.97  Bcast:129.215.175.255  Mask:255.255.255.0
                inet6 addr: fe80::ec4:7aff:fe35:1206/64 Scope:Link
                ....

        br1     Link encap:Ethernet  HWaddr 0c:c4:7a:35:12:07  
                inet addr:192.168.137.233  Bcast:192.168.137.255  Mask:255.255.255.0
                inet6 addr: fe80::ec4:7aff:fe35:1207/64 Scope:Link
                ....

Interface br0 is connected to the local VLAN created for the rack of machines in the ROE machine room, and interface br1 is connected to the internal VLAN for the SQLServer databases.

The source configuration for these interfaces comes from the heliodines private git repository at /var/local/projects/heliodines/git/src/cfg/tropo/trop02/etc/network/interfaces.

#[user@trop02]

    less /var/local/projects/heliodines/git/src/cfg/tropo/trop02/etc/network/interfaces 

        # This file describes the network interfaces available on your system
        # and how to activate them. For more information, see interfaces(5).

        # The loopback network interface
        auto lo
        iface lo inet loopback

        # Public interface
        auto br0
        iface br0 inet static
            address   129.215.175.97
            netmask   255.255.255.0
            network   129.215.175.0
            gateway   129.215.175.126
            broadcast 129.215.175.255
            # dns-* options are implemented by the resolvconf package, if installed
            dns-nameservers 195.194.120.1 195.194.120.2
            dns-search roe.ac.uk
            # Configure bridge port and STP.
            bridge_ports eth0
            bridge_fd 0
            bridge_stp off
            bridge_maxwait 0

        # Private interface
        auto br1
        iface br1 inet static
            address   192.168.137.233
            netmask   255.255.255.0
            network   192.168.137.0
            broadcast 192.168.137.255
            # Configure bridge port and STP.
            bridge_ports eth1
            bridge_fd 0
            bridge_stp off
            bridge_maxwait 0

Both of these interfaces are configured as bridges with static IP addresses, allowing them to be used as routes to access other hosts on the other networks.

Interface br0 is allocated an external public IP address, 129.215.175.97, on the 129.215.175.0 network, providing access to/from the public internet.

Interface br1 is allocated an internal IP address, 192.168.137.233, on the 192.168.137.0 network, providing access to the SQLServer databases.

Each of these interfaces also has a corresponding internal port that inherits the MAC address and IP address of the bridge, creating an internal interface that the OS on the physical hyper-visor machine itself can use.

#[user@trop02]

    /sbin/ifconfig

        eth0    Link encap:Ethernet  HWaddr 0c:c4:7a:35:12:06  
                ....

        eth1    Link encap:Ethernet  HWaddr 0c:c4:7a:35:12:07  
                ....

Virtual networks

The physical host contains two virtual networks provided by the libvirt system.

#[user@trop02]

    virsh \
        --connection 'qemu:///system' \
        net-list

         Name                 State      Autostart     Persistent
        ----------------------------------------------------------
         bridged              active     yes           yes
         default              active     yes           yes

The source configuration for these virtual networks comes from the heliodines private git repository at /var/local/projects/heliodines/git/src/cfg/tropo/trop02/etc/libvirt/qemu/networks/.

#[user@trop02]

    ls /var/local/projects/heliodines/git/src/cfg/tropo/trop02/etc/libvirt/qemu/networks/

        bridged.xml
        default.xml

The bridged network is configured as a forwarding bridge connected to the br0 external interface.

#[user@trop02]

    less /var/local/projects/heliodines/git/src/cfg/tropo/trop02/etc/libvirt/qemu/networks/bridged.xml

        <network ipv6='yes'>
          <name>bridged</name>
          <uuid/>
          <forward mode='bridge'/>
          <bridge name='br0'/>
        </network>

This configuration means any virtual machines connected to this network is effectively connected to the external br0 bridge interface on the ROE machine room VLAN. This means that the virtual machine can use this interface for outbound access the external public internet, and, if the virtual machine is given a public IP address, it can be reached by inbound traffic from the public internet.

The default network is configured as a standard libvirt NAT network with settings for 8 virtual machines.

#[user@trop02]

    less /var/local/projects/heliodines/git/src/cfg/tropo/trop02/etc/libvirt/qemu/networks/default.xml

        <network ipv6='yes'>
          <name>default</name>
          <uuid/>
          <forward mode='nat'>
            <nat>
              <port start='1024' end='65535'/>
            </nat>
          </forward>
          <bridge name='virbr0' stp='off' delay='0'/>
          <mac address='52:54:00:02:02:01'/>
          <ip family='ipv4' address='192.168.202.1' netmask='255.255.255.0'>
            <dhcp>
              <range start='192.168.202.8' end='192.168.202.15'/>
              <host mac='52:54:00:02:02:08' ip='192.168.202.8'  name='Araybwyn'/>
              <host mac='52:54:00:02:02:09' ip='192.168.202.9'  name='Lothigometh'/>
              <host mac='52:54:00:02:02:0A' ip='192.168.202.10' name='Ulov'/>
              <host mac='52:54:00:02:02:0B' ip='192.168.202.11' name='Dwiema'/>
              <host mac='52:54:00:02:02:0C' ip='192.168.202.12' name='Ibalehar'/>
              <host mac='52:54:00:02:02:0D' ip='192.168.202.13' name='Eterathiel'/>
              <host mac='52:54:00:02:02:0E' ip='192.168.202.14' name='Siamond'/>
              <host mac='52:54:00:02:02:0F' ip='192.168.202.15' name='Acilamwen'/>
            </dhcp>
          </ip>
        </network>

The MAC addresses and IP addresses for the NAT network on each of the hyper-visor machines, trop01 to trop04`, are allocated within specific ranges.

+-------------+---------------------+--------------------+
| hyper-visor |  MAC address range  |  IP address range  |
+-------------+---------------------+--------------------+
|    trop01   |  52:54:00:02:01:xx  |   192.168.201.xxx  |
|    trop02   |  52:54:00:02:02:xx  |   192.168.202.xxx  |
|    trop03   |  52:54:00:02:03:xx  |   192.168.203.xxx  |
|    trop04   |  52:54:00:02:04:xx  |   192.168.204.xxx  |
+-------------+---------------------+--------------------+

This pattern makes it easier to identify which physical hyper-visor a virtual machine belongs to given an IP or MAC address from a network snoop trace.

Virtual machines

TAP services

Clone this wiki locally