-
Notifications
You must be signed in to change notification settings - Fork 20
README
Terraform
Installation Instructions
Need: Java 6 (or higher)
vmrun (only for VMware environments)
Maven (only if you are building from source code)
To use the VMware portion of Terraform, you will need to have access to a VCenter server.
To use the Amazon Web Services portion of Terraform, you will need to have an AWS Account with the EC2 services. This service is free, but see Amazon for limitations. ( to get an AWS account: https://console.aws.amazon.com/console/home )
We include dependencies for you in the terraform/lib directory for the distributions in the Downloads tab.
Set your current working directory to the root terraform directory.
To build terraform, run:
$ mvn package
You will find a shell script in terraform/bin. This is used to run Terraform.
How to run terraform:
terraform [create/destory] [input-xml-file] [input-credentials-file] [prop1=val1 prop2=val2 ...]
You will need to create a credentials file. Instructions for this are found in the provider-specific sections (VMware or Amazon Web Services).
The Amazon Web Services Templates do not require any properties. For more information on the AWS portion of Terraform, see the AWS section below.
The VMware Templates require specific properties. See below for more details. For more information on the VMware portion of Terraform, see the VMware section below.
Environment Template examples can be found in example-config/xml-templates
A template is the structure of an environment, the blueprints. They are formatted in xml and have a required structure to them. The root element of every template must be and must have an xmlns attribute. Underneath the must be an element. What sits under the depends on the type of environment you want to create. See each individual provider section below.
Properties can be used in the XML template with ${property.name} and. The properties are
passed in as arguments on the command line (anything after the 3rd argument will be interpreted
as a property).
e.g. if you set uDeploy.host=12.34.56.78 and uDeploy.port=7918, you can then use these with <param value="${uDeploy.host}"/> <param value="${uDeploy.port}"/>
Some properties are defined by the provider. See each provider's section below (VMware / AWS).
Example command line call to Terraform with a VMware template:
terraform create vmware-template.xml vmware-1.key datacenter=my-datacenter host.name=my-host.mysite.com datastore=my-datastore destination=my-datacenter/path/to/my-destination/folder
VMware vSphere instances require four properties to start, which are detailed later in the VMware section.
To destroy an instance:
Call terraform.sh with the following arguments: First argument: destroy Second argument: the xml instance file generated when you created your instance (NOT the original template) Third argument: the credentials file
No properties are required to destroy an instance. Example command line call to Terraform:
terraform destroy terra-env-1234.xml vmware-1.key
This will power down the VMs, delete them, delete the folder that Terraform created, and delete any virtual switches and port groups it created.
Below is a hierarchy of VMware elements in their templates. The number in parentheses represents how many of those elements are allowed.
context(1)
└── environment(1)
├── clone(n)
│ ├── network-ref(n)
│ ├── security-group-ref(n)
│ ├── post-create-task(n)
│ ├── event-listener(n)
│ └── command(n)
├── network(n)
└── security-group(n)
└──port-range(n)
All vmware templates must include one context element and one environment element under the context:
<context
xmlns="org.urbancode.terraform.tasks.vmware">
<environment
name="terra-env"
folder-name="terra-folder">
</environment>
</context>
To add clones, insert "clone" elements in to the environment element. Clone elements look like this:
<clone instance-name="instance-1"
order="1"
image-path="/mydatacenter/myimagefolder/linux-clone-image"
snapshot-name="my-snapshot"
count="1">
</clone>
Required attributes on a clone element: instance-name(string), order(positive int), image-path(string)
Recommended attributes on a clone element: user(string), password(string). These specify the VM user name and password (which may need to be root, depending on what configurations you are doing).
The user and password attributes are required on any router clone, any clone with a command element, and any clone with a uDeploy post-create-task element.
User and password should be a valid login on the VM.
The order attribute distinguishes the order VMs will be created - lowest to highest. If clones have the same number they are created in parallel. The order in which parallel clones complete can be unpredictable.
Recommended attribute: snapshot(string). This should be a valid snapshot on the VM you are cloning (cannot be a template if you include a snapshot). When this attribute is specified, a linked clone will be created instead of a standard clone. Linked clones finish very quickly (5 seconds to clone as opposed to 1-10 minutes). You might have a performance degradation if you make a long chain of linked clones (10+ in chain length, ex. B is a linked clone of A, C is a linked clone of B, D is a linked clone of C, etc.).
If no snapshot is specified, a normal thin provisioned clone will be created. If no snapshot is specified, it is recommended that the original VM be marked as a template.
Terraform also supports multiple different clone elements - useful if you want them to have different name prefixes or orders in which they are created.
You can add Unix shell commands to be executed once the VM is created using the "command" element. The command element has one attribute called "cmd" which takes a shell command. You can also put a series of commands to be executed separated by semicolons. Note that you should not use any "shortcuts" in your commands; use fully qualified paths to programs. Example - use /home/myusername instead of ~, use /bin/sh instead of ./ or bash
Examples:
<command cmd="/bin/sleep 3" />
<command cmd="/bin/sh /home/me/scripts/my-shell-script" />
<command cmd="/bin/sh /home/me/scripts/my-shell-script;/bin/sh /home/me/scripts/second-script" />
<command cmd="/bin/mkdir /mnt/test;/bin/mount -t nfs nas.myhost.net:/volumes/public /mnt/test"/>
For creating a private network, include a network element in the environment for each subnet.
<network network-name="my-net-web" port-count="32"/>
A network element must have a network-name(string) and a port-count(int). This creates a virtual switch and port group.
There is an optional parameter vlan-id for a network to specify the VLAN ID. By default it is 0, which represents no VLAN.
Example: <network network-name="my-net-web" port-count="32" vlan-id="2096" />
All clones on the private network must have a network-ref element pertaining to the proper network, referenced by name.
<network-ref network-name="my-net-web" nic-index="1"/>
The name should be the same as the network name they are on. The nic-index corresponds to the network card which will be attached to the network.
A general note about networking - there must be some way for your VMs to have access to a physical NIC in order for them to get an IP. There are two ways to do this. One is for your image to have a NIC attached that is on an internet-connected virtual switch (that is, a vSwitch with a physical NIC attached). The second way is for the VM to be connected to a private network serviced by a router. The router must have one NIC attached to a vSwitch with a physical NIC and a NIC attached to the vSwitch that your VMs are on.
If you are creating a router to use for the private network, you must use a Linux image that can act as a router. We recommend Debian or Ubuntu.
You must also include these elements on the clone:
<post-create-task/>
<event-listener/>
These elements will configure the networking files on the router. It is highly recommended that the router be created BEFORE any clones that are on its network (specify using the order attribute). In addition, for any non-router clone on the router, add this attribute: assign-host-ip="true" . This will tell the clone to get its IP from the router instead of from your DHCP service.
By default, on a private network, all incoming traffic to ports on the network are blocked. You can open ports by adding a security group element to the environment.
<security-group name="default-group">
<port-range first-port="22" last-port="22"/>
<port-range first-port="80" last-port="80"/>
<port-range first-port="3306" last-port="3306"/>
<port-range first-port="8080" last-port="9090"/>
</security-group>
This security group will open ports 22, 80, 3306, and 8080-9090 on any instance that references this security group. To reference a security group on a clone, add this element to the clone:
<security-group-ref name="default-group"/>
The name should be equal to the security group name. This feature only works if you have a router in your template.
Finally, an example template with a private network, an application server, and a database server:
<context
xmlns="org.urbancode.terraform.tasks.vmware">
<environment
name="terra-env"
folder-name="terra-folder">
<network network-name="my-net-web" port-count="32"/>
<security-group name="default-group">
<port-range first-port="22" last-port="22"/>
<port-range first-port="80" last-port="80"/>
<port-range first-port="3306" last-port="3306"/>
<port-range first-port="8080" last-port="9090"/>
</security-group>
<clone instance-name="router-1"
order="1"
image-path="/mydatacenter/myimagefolder/router-image"
snapshot-name="my-snapshot"
user="root"
password="mypassword">
<network-ref network-name="my-net-web" nic-index="1"/>
<post-create-task/>
<event-listener/>
</clone>
<clone instance-name="app-instance-1"
order="2"
image-path="/mydatacenter/myimagefolder/linux-clone-image"
snapshot-name="my-snapshot"
assign-host-ip="true"
count="1">
<security-group-ref name="default-group"/>
<network-ref network-name="my-net-web" nic-index="0"/>
</clone>
<clone instance-name="db-instance-1"
order="2"
image-path="/mydatacenter/myimagefolder/linux-clone-image"
snapshot-name="my-snapshot"
assign-host-ip="true"
count="1">
<security-group-ref name="default-group"/>
<network-ref network-name="my-net-web" nic-index="0"/>
</clone>
</environment>
</context>
A credentials file is a text file ending in a .key suffix. It contains name=value pairs separated by lines.
Example file - creds-example.key
Contents:
type=org.urbancode.terraform.credentials.vmware.CredentialsVmware name=my.vmware.creds username=my-vcenter-user password=my-vcenter-password url=https://vcenter.mysite.com/sdk
For vmware, the "type", "name", "username", "password", and "url" properties are all mandatory. The "name" property whatever you want to name your credentials. The name should not contain spaces. Username is your vCenter login name. Password is your vCenter password. Url is the url pointing to your vCenter server, followed by a /sdk suffix.
Vmware environments have 4 required properties when creating an environment. EC2 environments have none. Here are the four required properties for vmware that must be passed in the command line arguments:
datacenter=my-datacenter host.name=my-host.mysite.com destination=path/to/my-destination/folder datastore=my-datastore
These correspond to the vSphere datacenter name, the name of the host the VMs will be created on, the destination folder for the environment folder, and the datastore for the VMs.
You will need to create a credentials file with information from your Amazon Web Services account. There is a template file in example-config/credentials-templates/aws.creds.template.key
You will need to replace the access.key and secret.key with the access key and secret key associated with your AWS account. You can find this information at : https://portal.aws.amazon.com/gp/aws/securityCredentials access.key = Access Key ID secret.key = Secret Access Key
The general layout of an AWS Environment Template is the following: The number in the parenthesis is the number of those elements allowed.
context(1)
└── environment(1)
├── instance(n)
│ ├── boot-actions(1)
│ │ └── script(n)
│ │ └── param(n)
│ ├── post-create-actions(1)
│ │ └── ssh(n)
│ └── security-group-ref(n)
├── load-balancer(n)
│ ├── health-check(1)
│ └── listener(n)
└── vpc(1)
├── inet-gwy(1)
├── route-table(n)
│ └── route(n)
├── subnet(n)
└── vpc-security-group(n)
└── rule(n)
The following is a list of elements available in the xml: [ The hierarchy represents which elements go under which in the xml ]
o context: The context holds the whole environment. It has one or more xmlns attributes, which specify map an xml namespace to a package. The packages it points to should contain a properties file named: terralib.classes for VMware use: xmlns="com.urbancode.uprovision.tasks.vmware" for AWS use: xmlns="com.urbancode.uprovision.tasks.aws"
o environment: The environment contains the Virtual Private Cloud (only 1 VPC per environment). The name of the environment is used as a prefix on the name tags in AWS.
o vpc: This is the core network. It is made with a CIDR (x.x.x.x/y). For more information on CIDR notation, see: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing
o subnet: Subnets are smaller networks inside of the VPC. They each have their own CIDR which is a sub-set of the VPC's CDIR. Each subnet has a zone property which specifies a region that the subnet will be located in. If any instances in this subnet need to be behind an Elastic Load Balancer, the zone of this subnet must match that of the load balancer.
o inet-gwy: The Internet Gateway attaches to a VPC and allows internet access from inside the VPC.
o route-table: Each VPC has a "Main" route table that cannot be deleted. Route Tables can be associated with a single subnet subnet. There can only be one "default" Route Table, which all new subnets will use if no Route Table is specified. The default Route Table will attempt to make a Route to allow instances to reach the internet if an Internet Gateway exists.
o route: Routes are elements of a Route Table. They contain a target, which canbe an Internet Gateway, a NAT (a specially configured instance), or an ENI (Elastic Network Interface). They also contain a destination, which is described by a CIDR block.
- For more information on Route Tables and Routes, see:
http://docs.amazonwebservices.com/AmazonVPC/latest/UserGuide/VPC_Routing.html
o vpc-security-group: A Security Group essentially acts as a firewall. They can be applied to multiple instances (and/or load balancers) and allow or disallow incoming or outgoing traffic on specified ports. A Security Group contains a collection of Rules, which contain the details on the traffic rules. They also have a name and description.
o rule: A Rule contains a source, which will apply to rule to just traffic from the specified CIDR. A Rule also contains a protocol, which the rule will apply only to traffic using that protocol, a port range, which is the range of ports the rule will apply to, and an inbound flag, which if set to false, will apply the rule to outbound traffic.
o instance: The representation of the EC2 instance that will be launched.
-name: the name of the instance. The number of the instance will be automatically
appended to the end of the instance name once it has been launched.
e.g. an instance with the name "web" and count "2" will launch instances:
web00 and web01
-ami-id: the id of of the Amazon Machine Image to use for the instance
-subnet-name: the name of the subnet to launch the instance into
-count: the number of instances with these same details you would like to launch
-elastic-ip: a flag that if set to true, will assign an Elastic Ip Address to this instance. This allows you to reach the instance from outside of the VPC.
* AWS has a default of 5 EIPs per account. For more info, see:
http://aws.amazon.com/contact-us/eip_limit_request/
-private-key-ref: The Key Pair (SSH key) that the instance will be accessable by.
You can find these in your AWS account. This is also the key that will be used
to run the SSH Post Create Actions.
-image-size: The size, or type, of the instance you want to create. VPCs currently
cannot support instances of size t1.micro.
valid values: m1.small, c1.medium, m1.medium, m1.large, m1.xlarge, m2.xlarge,
m2.2xlarge, m2.4xlarge, c1.xlarge
For more info, see: http://aws.amazon.com/ec2/instance-types/
-kernel-id: This is the id of the kernel you would like to use. It is recommended that you leave this empty, as it will use the default kernel for the AMI.
-ramdisk-id: This is the id of the RAM disk you would like to use. It is recommended that you leave this empty, as it will use the default RAM disk for the AMI.
-load-balancer: the name of the load balancer the instance will be registered with
-priority: This is the order in which the instance will be launched in. The instances with the lowest priority will be completely launched first. This means that the instance will hit the 'running' state, the 'ok' status, all boot-actions will be ran, then the post-create-actions will be ran. You can have multiple instances with the same priority and they will all be launched at once.
o security-group-ref: This is just a reference to the Security Group you would like to apply to the instance. You can have multiple of these.
o boot-actions: This is a list of actions that will be run at FIRST BOOT of the instance. These actions are run as root and are referered to as User-Data by Amazon. This will automatically convert the script to Base64 as required by Amazon. The whole user-data script (but not including the contents of external scripts) will be resolved for any properties that may show up. Script can be 16KB max.
-shell: This is the shell that the user-data script will be run with.
o script: This is a action that will grab a script from the given URL and run it with the specified shell. This grabs the script using 'wget'. You can have multiple script elements.
o param: These are parameters for a given script. They will be ran in order from first to last defined.
o post-create-actions: This contains all the actions to be ran after the instance has fully started.
o ssh: Connects to an instance via ssh. The instance should have an Elastic Ip, allowing outside traffic in, along with allowing inbound traffic on port 22. * include any used key-pair files obtained from Amazon (.pem) in ~/.terraform/
o load-balancer: An Elastic Load Balancer by Amazon for load balancing instances.
-name: the name of the load balancer. MUST BE UNIQUE ON YOUR AWS ACCOUNT!
-subnet-name: the subnet to associate with the load balancer
-app-cookie-name: the name of the cookie your application uses. You can leave this blank if you want the load balancer to handle the cookies.
o listener: This specified how the load balancer should balance the loads
-protocol: Valid values: http, https, tcp, ssh
-instance-port: The port on the instance to send traffic to
-load-balancer-port: The port on the load balancer to listen for traffic
-cert-id: Use only if you are using a secure protocol
o health-check: This is how the load balancer determines whether or not an instance is health and can accept traffic or not. The instance must return a status 200 OK in order to be considered a successful check.
-interval: the frequency of checks on instances
-timeout: how long before a check on the instance will timeout
-unhealthy-count: How many consecutive checks before an instance is determined to be unhealthy
-healthy-count: How many consecuctive checks before an instance is determined to be healthy
-protocol: the protocol on which to check the instance
-port: the port to check the instance on
-path: the path on the instance to check
-
For more information on any of these elements, see the Amazon Web Services documentation at: http://aws.amazon.com/documentation/ec2/
Other Notes
When an instance is launched, two properties are set. One for the public ip address, and one for the private ip address. These can be used with ${INSTANCENAME.public.ip} ${INSTANCENAME.private.ip}
e.g. If you launch an instance with name "web" and count "2", you can access the ips of the two instances with these properties:
${web00.private.ip} ${web01.public.ip}Note: a public ip address (aka Elastic IP) will only be assigned if the elastic-ip flag on an instance is set to true.
Check the example AWS template in the example-config/xml-templatesfor more information on creating templates.
In order for any Post Create Actions to run on an Amazon EC2 instance, the instance must have an Elastic Ip Address assigned to it. You must also have the key-pair (ssh key) that the instance was created with in your ~/.terraform directory.
The Elastic Ip Address is neccessary since the instances are launched inside of a Virtual Private Cloud. The EIP allows you to communicate with those instances from outside of the VPC.
File Information
The key-pair file is needed for connecting to the server. You must also know what user that key-pair was associated with. This is determined when the AMI (Amazon Machine Image) is created.
IMPORTANT: Launching public images without a key pair ID will leave them inaccessible. The public key material is made available to the instance at boot time by placing it in the openssh_id.pub file on a logical device that is exposed to the instance as /dev/sda2 (the ephemeral store). The format of this file is suitable for use as an entry within ~/.ssh/authorized_keys (the OpenSSH format). This can be done at boot (e.g., as part of rc.local) allowing for secure access without passwords. Optional user data can be provided in the launch request. All instances that collectively comprise the launch request have access to this data For more information, see Instance Metadata.
NOTE: If any of the AMIs have a product code attached for which the user has not subscribed, the instance launch will fail.
[ Global ]
- log4j.properties Located in the $TERRAFORM_HOME/conf folder. This file contains the properties used to configure logging.
AWS-specific
(none)
VMware-specific
- ippool.conf - Located in the $TERRAFORM_HOME/conf folder. Contains the start and end points for the IP pool used by vCenter. The start is the begining IP address to start allocating for VMs. It should be in the following format: start=192.168.2.1 end=192.168.2.250
The files below are created in the $TERRAFORM_HOME/temp folder and deleted when the environment is deleted.
.temp files - these are the beginning forms of the conf files listed below. Content is added to them and placed in the appropriate conf file.
- dhcpd.conf - This is the standard DHCPD configuration file for dhcpd, the Internet Systems Consortium DHCP Server.
See http://linux.die.net/man/5/dhcpd.conf
- interfaces - This is a debian/ubuntu networking configuration file for the interfaces on the router machine.
See http://support.arpnetworks.com/kb/vps/example-etcnetworkinterfaces-for-debian-and-ubuntu
- iptables.conf -
This is the standard iptables configuration file for configuring a firewall and allowable network traffic.
See http://linux.die.net/man/8/iptables
- isc-dhcp-server -
This file indicates which network interfaces will be serving up DHCP addresses. It is typically located in /etc/default and referenced by /etc/init.d/isc-dhcp-server . The only required content of this file is one line, as below
INTERFACES="eth1 eth2"
Quotation marks must be included. All network interfaces to be served DHCP addresses should be listed, separated by one space. The above line represents an example where the router will be serving DHCP addresses to interfaces eth1 and eth2 (so 2 subnets). In our router configuration, is one interface per subnet, starting at eth1 for the first.
======================================