Setting up a test Grafana

This wiki describes how to set up Grafana and InfluxDB, feed DSC XML data to it and get your own site up and running. Previous test site is being reworked!

Please Note!

The setup and graphs describe in this Wiki are meant as a example of what can be done. It's important to understand and learn how DSC's datasets work, how they are exported to InfluxDB and how graphs in Grafana works.

Grafana is an amazing tool, and you will greatly benefit in customizing your own graphs for your own purpose and needs.

InfluxDB's Downsampling and Data Retention

This subject will not be covered here, please refer to InfluxDB documentation in order to set it up in the way you want it.

Installation

For the instructions of this wiki the base Linux distributions used will be Debian 10, other distributions will work also see:

• dsc-datatool: https://dev.dns-oarc.net/packages/
• InfluxDB: https://docs.influxdata.com/influxdb/latest/introduction/install/
• Grafana: https://grafana.com/docs/grafana/latest/installation/

When you have a Debian up and running do the following to setup all repositories needed:

sudo apt-get install -y wget gnupg apt-transport-https

wget -O - https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian `lsb_release -c -s` stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list

wget -O - https://pkg.dns-oarc.net/dns-oarc.distribution.key.gpg | sudo apt-key add -
echo "deb http://pkg.dns-oarc.net/debian `lsb_release -c -s` main" | sudo tee /etc/apt/sources.list.d/dns-oarc.list

Now we can install all software needed:

sudo apt-get update
sudo apt-get install -y influxdb grafana dsc-datatool

For some reason InfluxDB does not automatically start on installation and Grafana is disabled, so we fix that to make sure it automatically starts on next boot:

sudo systemctl daemon-reload
sudo systemctl enable grafana-server
sudo systemctl start influxdb grafana-server

Grafana Pie Chart Panel

The DSC graphs uses an additional panel that is not included in the standard installation so we need to install it:

sudo grafana-cli plugins install grafana-piechart-panel
sudo systemctl restart grafana-server

InfluxDB

We now need to setup the database used in InfluxDB:

influx -execute "create database dsc"

Grafana Data Source

Log in to Grafana at http://$IP:3000/, the default administrator and password is admin/admin.

Now create a InfluxDB Data Source keeping all default setting except these:

• URL: http://localhost:8086
• Database: dsc
• Min time interval: >60s

Save & Test and check that it is working.

Grafana Dashboard

As a starting dashboard you can use any or all that is in the contrib/test-site, to import it into Grafana do the following:

• Download the dashboard JSON files here
• Press + on the left panel and select Import
• Use Upload JSON file to add the JSON file(s) and import it
• Repeat from step 2 until all JSON files have been uploaded

DSC XML data

The instructions below will need access to the XML data from DSC, it is up to you how to make that available.

Fetch Labler YAML (optional)

The IANA DNS parameters can be used with the Labler transformer to rewrite RCODE, QTYPE and OPCODE numbers into their names.

For it to work the Labler needs a YAML file with the label names and you can run the following contributed script in order to download them from IANA:

wget https://github.com/DNS-OARC/dsc-datatool/raw/develop/contrib/iana-dns-params-toyaml.py
python3 iana-dns-params-toyaml.py > $HOME/labler.yaml

MaxMind Database Setup

To use the client_subnet_country generator it will need the databases from MaxMind. These databases are no longer distributed because of privacy regulations but are still available for free, see https://dev.maxmind.com/geoip/geoip2/geolite2/.

Once you have your access and geoip.cfg use geoipupdate to fetch the databases, see https://github.com/maxmind/geoipupdate .

wget https://launchpad.net/~maxmind/+archive/ubuntu/ppa/+files/geoipupdate_4.3.0-0+maxmind1~xenial_amd64.deb
sudo dpkg -i geoipupdate_4.3.0-0+maxmind1~xenial_amd64.deb
sudo geoipupdate -f geoip.cfg

Import!

We can now run dsc-datatool and import some DATA!

Following environment variables are used in the shell code below:

• $SERVER: Used to set the server name (--server)
• $NODE: Used to set the node name (--node)
• $XML: Used as the input XML file/directory (--xml)

dsc-datatool \
  --server "$SERVER" \
  --node "$NODE" \
  --output ";InfluxDB;file=influx.txt;dml=1;database=dsc" \
  --transform ";Labler;*;yaml=$HOME/labler.yaml" \
  --transform ";ReRanger;rcode_vs_replylen;range=/64;pad_to=5" \
  --transform ";ReRanger;qtype_vs_qnamelen;range=/16;pad_to=3" \
  --transform ";ReRanger;client_port_range;key=low;range=/2048;pad_to=5" \
  --transform ";ReRanger;edns_bufsiz,priming_queries;key=low;range=/512;pad_to=5;allow_invalid_keys=1" \
  --transform ";ReRanger;priming_responses;key=low;range=/128;pad_to=4" \
  --transform ";NetRemap;client_subnet,client_subnet2,client_addr_vs_rcode,ipv6_rsn_abusers;net=8" \
  --generator client_subnet_country \
  --generator ";client_subnet_authority;fetch=yes" \
  --xml "$XML" &&
influx -import -path=influx.txt

View the Dashboard

You should now have data in your dashboard at http://$IP:3000/dashboard/db/dsc.

Enjoy!