Added cleanup meachnism for tendrl setup

The cleanup playbook performs the below steps
- Stops collectd, tendrl-node-agent and tendrl-gluster-integration
on storage nodes
- Stops tendrl-node-agent, tendrl-monitoring-integration, tendrl-notifier,
tendrl-api on tendrl server node
- Stops grafana-server, carbon-cache and etcd services on storage nodes
- Cleans up etcd, grafana and carbon data files

tendrl-bug-id: Tendrl/issues/63
Signed-off-by: Shubhendu <[email protected]>

README.md

@@ -15,6 +15,12 @@ there](https://github.com/Tendrl/documentation/wiki/Tendrl-release-latest) to
 have basic understanding of various machine roles in Tendrl cluster before
 using tendrl-ansible.
 
+tendrl-ansible also provides a playbook to cleanup the existing tendrl setup. It
+performs the below steps
+* Stops all the tendrl specific services on storage nodes
+* Stops all the tendrl specific services on tendrl server node
+* Cleans etcd, grafana and carbon data files
+
 
 ## How to get tendrl-ansible?
 
@@ -61,6 +67,9 @@ Ansible roles for Tendrl:
   (where Tendrl api, web and etcd are running)
 * `tendrl-ansible.tendrl-storage-node`: installation of **Tendrl Storage Node**
   machines (Gluster servers, which you would like to monitor by Tendrl)
+* `tendrl-ansible.tendrl-server-cleanup`: cleanup tendrl setup from server node
+* `tendrl-ansible.tendrl-storage-node-cleanup`: cleanup tendrl setup from
+storage nodes
 
 Roles installing yum repositories of Tendrl dependencies:
 
@@ -79,6 +88,8 @@ Playbook files:
   (see comments inside the playbook file for references)
 * `site.yml`: main playbook of tendrl-ansible, which one will use to install
   Tendrl
+* `cleanup.yml`: playbook for cleaning up the tendrl setup from storage nodes
+and tendrl server node
 
 
 ## Where are the roles and playbooks if I use rpm package?
@@ -317,6 +328,63 @@ tendrl-ansible:
     on [TEN-257](https://tendrl.atlassian.net/browse/TEN-257)).
 
 
+## How do I cleanup Tendrl setup using tendrl-ansible?
+
+1)  If you use tendrl-ansible from rpm package, copy `cleanup.yml` playbook into
+    working directory (where you already store the inventory file):
+
+    ```
+    $ cp /usr/share/doc/tendrl-ansible-VERSION/cleanup.yml .
+    ```
+
+2)  Check that ssh can connect to all machines from the inventory file without
+    asking for password or validation of public key by running:
+
+    ```
+    $ ansible -i inventory_file -m ping all
+    ```
+
+    You should see ansible to show `"pong"` message for all machines.
+    In case of any problems, you need to fix it before going on. If you are not
+    sure what's wrong, consult documentation of ansible and/or ssh.
+
+    The following example shows how to use [ansible become
+    feature](https://docs.ansible.com/ansible/latest/become.html) **when direct
+    ssh login of root user is not allowed** and you are connecting via non-root
+    `cloud-user` account, which can leverage `sudo` to run any command as root
+    without any password:
+
+    ```
+    $ ansible --become -u cloud-user -i inventory_file -m ping all
+    ```
+
+    If this is your case, you may consider converting command line arguments
+    related to *Ansbile become feature* into [behavioral inventory
+    parameters](https://docs.ansible.com/ansible/latest/intro_inventory.html#list-of-behavioral-inventory-parameters)
+    and adding them into the inventory file. This way, you don't need to
+    specify these arguments again for every ansible command. Example of this
+    update which matches previous command line example follows (it should be
+    appended to the `[all:vars]` section):
+
+    ```
+    ansible_become=yes
+    ansible_user=cloud-user
+    ```
+
+    After this edit, you can re run the ping example without become command
+    line arguments:
+
+    ```
+    $ ansible -i inventory_file -m ping all
+    ```
+
+3)  Then we are ready to run ansible to cleanup Tendrl setup:
+
+    ```
+    $ ansible-playbook -i inventory_file cleanup.yml
+    ```
+
+
 ## How do I expand cluster with tendrl-ansible?
 
 See [Tendrl wiki](https://github.com/Tendrl/documentation/wiki) for full

cleanup.yml

@@ -0,0 +1,29 @@
+---
+# This Ansible playbook is to cleanup Tendrl setup
+# It just stops all the running services related to Tendrl
+# on storage nodes as well as server. Also it cleans the
+# existing etcd, grafana and carbon data (if any) for existing
+# setup
+
+- hosts: gluster_servers
+  user: root
+
+  roles:
+    - tendrl-storage-node-cleanup
+
+- hosts: tendrl_server
+  user: root
+  vars:
+    # etcd db file to be deleted
+    etcd_db_path: "/var/lib/etcd/default.etcd"
+
+    # grafana db file to be deleted
+    grafana_db_file: "/var/lib/grafana/grafana.db"
+
+    # carbon data to be deleted
+    carbon_data_path: "/var/lib/carbon/whisper/tendrl/"
+
+  roles:
+    - tendrl-server-cleanup
+
+

roles/tendrl-ansible.tendrl-server-cleanup/main.yml

@@ -0,0 +1,42 @@
+---
+# Task file for storage nodes
+
+- name: Stop tendrl services
+  service:
+    name: tendrl-node-agent
+    state: stopped
+
+- name: Stop etcd service
+  service:
+    name: etcd
+    state: stopped
+
+- name: Stop grafana-server service
+  service:
+    name: grafana-server
+    state: stopped
+
+- name: Stop carbon-cache service
+  service:
+    name: carbon-cache
+    state: stopped
+
+- name: Stop httpd service
+  service:
+    name: httpd
+    state: stopped
+
+- name: Cleanup etcd data
+  file:
+    path: "{{etcd_db_path}}/"
+    state: absent
+
+- name: Cleanup grafana data
+  file:
+    path: "{{grafana_db_file}}"
+    state: absent
+
+- name: Cleanup carbon data
+  file:
+    path: "{{carbon_data_path}}/"
+    state: absent

roles/tendrl-ansible.tendrl-storage-node-cleanup/main.yml

@@ -0,0 +1,18 @@
+
+---
+# Task file for storage nodes
+
+- name: Stop tendrl services
+  service:
+    name: tendrl-gluster-integration
+    state: stopped
+
+- name: Stop tendrl services
+  service:
+    name: tendrl-node-agent
+    state: stopped
+
+- name: Stop collectd
+  service:
+    name: collectd
+    state: stopped