Setting up a test Grafana

This wiki describes how to set up Grafana and InfluxDB, feed DSC XML data to it and get your own site like this test site up and running.

If you installed it before any of the following updates you should update the cloned repository.

cd $HOME/dsc-datatool
git pull

InfluxDB, Downsampling and Data Retention

This subject will not be covered here, please refer to InfluxDB documentation in order to set it up in the way you want it.

Installation

For the instructions of this wiki the base Linux distributions used will be Debian Jessie, other distributions should work also as long as all dependencies are installed.

When you have a Debian Jessie up and running do the following to setup all repositories needed:

sudo apt-get install -y apt-transport-https

wget -O - https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian jessie stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

wget -O - https://packagecloud.io/gpg.key | sudo apt-key add -
echo "deb https://packagecloud.io/grafana/stable/debian/ jessie main" | sudo tee /etc/apt/sources.list.d/grafana.list

wget -O - https://jelu.github.io/package.gpg | sudo apt-key add -
echo "deb http://jelu.github.io/zonemaster/debian jessie main" | sudo tee /etc/apt/sources.list.d/jelu.github.io.zonemaster.list

Now we can update API and install both InfluxDB and Grafana:

sudo apt-get update
sudo apt-get install -y influxdb grafana

For some reason InfluxDB does not automatically start on installation and Grafana is disabled, so we fix that to make sure it automatically starts on next boot:

sudo service influxdb start
sudo systemctl daemon-reload
sudo systemctl enable grafana-server
sudo systemctl start grafana-server

Dependencies for dsc-datatool needs to be installed also:

sudo apt-get install -y libcommon-sense-perl libyaml-tiny-perl libxml-libxml-simple-perl libmodule-find-perl libnetaddr-ip-perl libip-country-perl

Grafana Pie Chart Panel

The DSC graphs uses an additional panel that is not included in the standard installation so we need to install it:

sudo grafana-cli plugins install grafana-piechart-panel
sudo service grafana-server stop
sudo service grafana-server start

InfluxDB

We now need to setup the database used in InfluxDB:

influx -execute "create database dsc"

Grafana Data Source

Log in to Grafana at http://$IP:3000/, the default administrator and password is admin/admin.

Now create a Data Source with the following settings:

• Name: influxdb
• Default: yes
• Type: InfluxDB
• Url: http://localhost:8086
• Access: proxy
• Basic Auth: no
• With Credentials: no
• Database: dsc
• User:
• Password:
• Default group by time: >60s

Save & Test and check that it is working.

Grafana Dashboard

As a starting dashboard you can use the one that is in the contrib/test-site, to import it into Grafana do the following:

• Download the dashboard JSON file here
• Press Home at the top left next to the Grafana logo
• In the dialog that opens, press Import button at the bottom
• Browse and select the JSON file you downloaded
• Press the save icon top left or press CTRL+S so actually save the dashboard (may not be needed with latest version of Grafana)

DSC XML data

The instructions below will need access to the XML data from DSC, it is up to you how to make that available.

dsc-datatool

Time to clone to dsc-datatool:

git clone https://github.com/DNS-OARC/dsc-datatool.git

Fetch Labler YAML (optional)

The IANA DNS parameters can be used with the Labler transformer to rewrite RCODE, QTYPE and OPCODE numbers into their names.

For it to work the Labler needs a YAML file with the label names and you can run the following contributed script in order to download them from IANA:

$HOME/dsc-datatool/contrib/iana-dns-parameters/fetch-yaml-for-labler.pl > $HOME/labler.yaml

Import!

We can now run dsc-datatool and import some DATA!

Following environment variables are used in the shell code below:

• $SERVER: Used to set the server name (--server)
• $NODE: Used to set the node name (--node)
• $XML: Used as the input XML file (--xml)

perl -I$HOME/dsc-datatool/lib $HOME/dsc-datatool/bin/dsc-datatool \
  --server $SERVER \
  --node $NODE \
  --output ";InfluxDB;file=influx.txt;dml=1;database=dsc" \
  --transform ";Labler;*;yaml=$HOME/labler.yaml" \
  --transform ";ReRanger;rcode_vs_replylen;range=/64;pad_to=5" \
  --transform ";ReRanger;qtype_vs_qnamelen;range=/16;pad_to=3" \
  --transform ";ReRanger;client_port_range;key=low;range=/2048;pad_to=5" \
  --transform ";ReRanger;edns_bufsiz,priming_queries;key=low;range=/512;pad_to=5;allow_invalid_keys=1" \
  --transform ";ReRanger;priming_responses;key=low;range=/128;pad_to=4" \
  --transform ";NetRemap;client_subnet,client_subnet2,client_addr_vs_rcode,ipv6_rsn_abusers;net=8" \
  --generator client_subnet_country,client_subnet_authority \
  --xml $XML &&
influx -import -path=influx.txt

View the Dashboard

You should now have data in your dashboard at http://$IP:3000/dashboard/db/dsc.

Enjoy!