docker

Quick project reference

• Source code: github.com/SK-CERT/Taranis-NG
• Docker images: hub.docker.com/u/skcert
• Maintained by: SK-CERT
• Project web page: taranis.ng
• Where to file issues (no vulnerability reports please): GitHub issues page
• Where to send security issues and vulnerability reports: [email protected]

About Taranis NG

Taranis NG is an OSINT gathering and analysis tool for CSIRT teams and organisations. It allows osint gathering, analysis and reporting; team-to-team collaboration; and contains a user portal for simple self asset management.

Taranis crawls various data sources such as web sites or tweets to gather unstructured news items. These are processed by analysts to create structured report items, which are used to create products such as PDF files, which are finally published.

Taranis supports team-to-team collaboration, and includes a light weight self service asset management which automatically links to the advisories that mention vulnerabilities in the software.

Deploying Taranis NG with docker-compose

Taranis NG supports deployment in Docker containers. The docker/ folder on GitHub repository contains a sample docker-compose.yml file which runs the whole application in one stack.

The same folder also contains additional support files for the creation of the Docker containers. These include start and pre-start scripts, the application entrypoint, and the gunicorn configuration file.

Prerequisites

• Docker
• docker-compose >= 1.27.0 (In July 2023, Compose V1 has been deprecated)

or

• Compose V2, which is part of standard Docker Engine installation
• (Optional) Vim or other text editor - for configuration and development

Please note it is important to use the abovementioned version of docker-compose or newer, otherwise the build and deploy will fail.

Quickly build and run Taranis NG using docker-compose or docker compose

First, you need to clone the source code repository:

git clone https://github.com/SK-CERT/Taranis-NG.git
cd Taranis-NG

Then, remove .example extension from file docker/.env.example and files in docker/secrets. Use your favorite text editor and change default passwords. Taranis NG uses Docker secrets to store sensitive data. (Saving passwords in variables defined in docker/.env is not advised and you will need to modify Docker compose YAML files to make it work correctly.

vim docker/.env

Optionally: you may modify other settings in the docker/.env and docker/docker-compose.yml files to your liking. More information on container configuration can be found here.

Finally, either deploy the ready-made images from Docker hub with:

docker-compose -f docker/docker-compose.yml pull
docker-compose -f docker/docker-compose.yml up --no-build

or

docker compose -f docker/docker-compose.yml pull
docker compose -f docker/docker-compose.yml up --no-build

or, alternatively, build and run the containers with:

TARANIS_NG_TAG=build docker-compose -f docker/docker-compose.yml build --pull
TARANIS_NG_TAG=build docker-compose -f docker/docker-compose.yml up

or

TARANIS_NG_TAG=build docker compose -f docker/docker-compose.yml build --pull
TARANIS_NG_TAG=build docker compose -f docker/docker-compose.yml up

(--pull updates the base images)

Voila, Taranis NG is up and running. Visit your instance by navigating to https://localhost:4443/ using your web browser.

Your Taranis NG instance now needs to be configured. Continue here.

The default credentials are user / user and admin / admin.

Advanced build methods

Individually build the containers

To build the Docker images individually, you need to clone the source code repository.

git clone https://github.com/SK-CERT/Taranis-NG.git

Afterwards go to the cloned repository and launch the docker build command for the specific container image, like so:

cd Taranis-NG
docker build -t taranis-ng-bots . -f ./docker/Dockerfile.bots
docker build -t taranis-ng-collectors . -f ./docker/Dockerfile.collectors
docker build -t taranis-ng-core . -f ./docker/Dockerfile.core
docker build -t taranis-ng-gui . -f ./docker/Dockerfile.gui
docker build -t taranis-ng-presenters . -f ./docker/Dockerfile.presenters
docker build -t taranis-ng-publishers . -f ./docker/Dockerfile.publishers

There are several Dockerfiles and each of them builds a different component of the system. These Dockerfiles exist:

• Dockerfile.bots
• Dockerfile.collectors
• Dockerfile.core
• Dockerfile.gui
• Dockerfile.presenters
• Dockerfile.publishers

Configuration

Container variables

redis

Any configuration options are available at https://hub.docker.com/_/redis.

database

Any configuration options are available at https://hub.docker.com/_/postgres.

core

Environment variable	Description	Example
REDIS_URL	Redis database URL. Used for SSE events.	redis://redis
DB_URL	PostgreSQL database URL.	127.0.0.1
DB_DATABASE	PostgreSQL database name.	taranis-ng
DB_USER	PostgreSQL database user.	taranis-ng
DB_POOL_SIZE	SQLAlchemy QueuePool number of active connections to the database.	100
DB_POOL_RECYCLE	SQLAlchemy QueuePool maximum connection age.	300
DB_POOL_TIMEOUT	SQLAlchemy QueuePool connection timeout.	5
OPENID_LOGOUT_URL	Keycloak logout URL.	https://example.com/realms/master/protocol/openid-connect/logout
WORKERS_PER_CORE	Number of gunicorn worker threads to spawn per CPU core.	4
SKIP_DEFAULT_COLLECTOR	Set to true to prevent initialization of a default docker collector at first run	``

Secrets file	Description	Example
postgres_password	PostgreSQL database password.	supersecret
jwt_secret_key	JWT token secret key.	supersecret

Taranis NG can use connection pooling to maintain multiple active connections to the database server. Connection pooling is required when your deployment serves hundreds of customers from one instance. To enable connection pooling, set the DB_POOL_SIZE, DB_POOL_RECYCLE, and DB_POOL_TIMEOUT environment variables.

bots, collectors, presenters, publishers

Environment variable	Description	Example
TARANIS_NG_CORE_URL	URL of the Taranis NG core API.	http://127.0.0.1:8080/api/v1
WORKERS_PER_CORE	Number of gunicorn worker threads to spawn per CPU core.	4

Secrets file	Description	Example
api_key	Shared API key.	cuBG/4H9lGTeo47F9X6DUg

gui

Environment variable	Description	Example
VUE_APP_TARANIS_NG_CORE_API	URL of the Taranis NG core API.	http://127.0.0.1:8080/api/v1
VUE_APP_TARANIS_NG_CORE_SSE	URL of the Taranis NG SSE endpoint.	http://127.0.0.1:8080/sse
VUE_APP_TARANIS_NG_URL	URL of the Taranis NG frontend.	http://127.0.0.1
VUE_APP_TARANIS_NG_LOCALE	Application locale.	en
NGINX_WORKERS	Number of NginX worker threads to spawn.	4
NGINX_CONNECTIONS	Maximum number of allowed connections per one worker thread.	16

Management script how-to

Taranis NG core container comes with a simple management script that may be used to set up and configure the instance without manual interaction with the database.

To run the management script, launch a shell inside of the docker container for the core component with this command:

docker exec -it [CONTAINER] python manage.py [COMMAND] [PARAMETERS]

Currently, you may manage the following:

Command	Description	Parameters
account	(WIP) List, create, edit and delete user accounts.	--list, -l : list all user accounts --create, -c : create a new user account --edit, -e : edit an existing user account --delete, -d : delete a user account --username : specify the username --name : specify the user's name --password : specify the user's password --roles : specify a list of roles, divided by a comma (,), that the user belongs to
role	(WIP) List, create, edit and delete user roles.	--list, -l : list all roles --filter, -f : filter roles by their name or description --create, -c : create a new role --edit, -e : edit an existing role --delete, -d : delete a role --id : specify the role id (in combination with --edit or --delete) --name : specify the role name --description : specify the role description (default is "") --permissions : specify a list of permissions, divided with a comma (,), that the role would allow
collector	(WIP) List, create, edit, delete and update collector nodes.	--list, -l : list all collector nodes --create, -c : create a new node --edit, -e : edit an existing node --delete, -d : delete a node --update, -u : re-initialize collector node --all, -a : update all collector nodes (in combination with --update) --show-api-key : show API key in plaintext (in combination with --list) --id : specify the node id (in combination with --edit, --delete or --update) --name : specify the node name --description : specify the collector description (default is "") --api-url : specify the collector node API url --api-key : specify the collector node API key
dictionary	Update CPE, CWE and CVE dictionaries.	--upload-cpe : upload the CPE dictionary (expected on STDIN in XML format) to the path indicated by CPE_UPDATE_FILE environment variable, and update the database from that file. --upload-cve : upload the CVE dictionary (expected on STDIN in XML format) to the path indicated by CVE_UPDATE_FILE environment variable, and update the database from that file. --upload-cwe : upload the CWE dictionary (expected on STDIN in XML format) to the path indicated by CWE_UPDATE_FILE environment variable, and update the database from that file.
apikey	List, create and delete apikeys.	--list, -l : list all apikeys --create, -c : create a new apikey --delete, -d : delete a apikey --name : specify the apikey name --user : specify the user's name --expires : specify the apikey expiration datetime

Example usage

Create a new role with a set of permissions

manage.py role \
    --create \
    --name "Custom role 1" \
    --description "Custom role with analysis and assessment access" \
    --permissions "ANALYZE_ACCESS, ANALYZE_CREATE, ANALYZE_UPDATE, \
    ANALYZE_DELETE, ASSESS_ACCESS, ASSESS_CREATE, ASSESS_UPDATE, \
    ASSESS_DELETE, MY_ASSETS_ACCESS, MY_ASSETS_CREATE, MY_ASSETS_CONFIG"

Command output:

Role 'Custom role 1' with id 3 created.

Role filter

manage.py role \
    --list \
    --filter "Custom role 1"

Command output:

Id: 3
	Name: Custom role 1
	Description: Custom role with analysis and assessment access
	Permissions: ['ANALYZE_ACCESS', 'ANALYZE_CREATE', 'ANALYZE_UPDATE', 'ANALYZE_DELETE', 'ASSESS_ACCESS', 'ASSESS_CREATE', 'ASSESS_UPDATE', 'ASSESS_DELETE', 'MY_ASSETS_ACCESS', 'MY_ASSETS_CREATE', 'MY_ASSETS_CONFIG']

Create a new collector node

manage.py collector \
    --create \
    --name "Docker collector" \
    --description "A simple collector hosted in a Docker container" \
    --api-url "http://collector.example.com" \
    --api-key "supersecret"

Command output:

Collector node 'Docker collector' with id 1 created.

Re-initialize a collector node

manage.py collector \
    --update \
    --name "Docker"

Command output:

Collector node 1 updated.
Collector node 2 updated.
Unable to update collector node 3.
    Response: [401] ""

Create a new user account

manage.py account \
    --create \
    --name "John Doe" \
    --username "test_user" \
    --password "supersecret" \
    --roles 3

Command output:

User 'test_user' created.

Upload a CPE dictionary

zcat official-cpe-dictionary_v2.3.xml.gz | manage.py dictionary --upload-cpe

Command output:

Processed CPE items: 1000
Processed CPE items: 2000
...
...
Processed CPE items: 789000
Processed CPE items: 789704
Dictionary was uploaded.

Create new ApiKey

manage.py apikey \
    --create \
    --name "My ApiKey"

Create a new API key for a user with an expiration date

manage.py apikey \
    --create \
    --name "My ApiKey" \
    --user "test_user" \
    --expire "2022-12-31 16:55"

Command output:

ApiKey 'My ApiKey' with id 3 created.