Streaming Logs

This is only available in version v9.7.0 or higher.

Overview

The log_stream is a Redis stream that stores logs from the game server in sequential order on a transient basis (they are not persisted to the database and are cleared on service startup) to support pushing new logs to external tools through a websocket endpoint.

Configuration

{
  "enabled": false,
  "stream_size": 1000,
  "startup_since_mins": 2,
  "refresh_frequency_sec": 1,
  "refresh_since_mins": 2
}

stream_size: The number of logs the stream will retain before discarding the oldest logs.

startup_since_mins: The number of minutes of logs to request from the game service when the service starts up

refresh_frequency_sec: The poll rate for asking for new logs from the game server

refresh_since_mins The number of minutes of logs to request from the game service each loop

Permissions

For new installs these are added to the owner and admin group.

Existing installs can add these permissions to users or groups:

1. api.can_view_log_stream_config
2. api.can_change_log_stream_config

Service

The log_stream service will start by default but will only actually do anything if enabled is set to true in its config.

Each start up the redis stream is purged and logs (if any are available) are pulled from the game server.

Stream IDs

In a Redis stream the IDs must be unique and sequentially increasing.

We use the timestamp of a log entry and a 0 indexed incrementing suffix because more than one log can occur at the same timestamp, e.g. 1711657986-0 and 1711657986-1.

Connecting / Authentication

To connect you must make a web socket connection to your CRCON at your normal RCONWEB_PORT or RCONWEB_PORT_HTTPS with the /ws/logs endpoint.

For example: ws://localhost:8010/ws/logs

You must include a valid API key for an account with the appropriate permisisons, there is no username/password based authentication.

Request Formats

After successfully connecting you must send a JSON message with your criteria before you will receive any logs.

You can specify a past stream ID (last_seen_id) and/or filter by action type (actions) (any valid type as defined in rcon.types.AllLogTypes).

If you don't care about past logs, simply pass null for last_seen_id, or if you don't care about either you can pass {}.

No filter and only new logs:

{}

Older logs:

{ "last_seen_id": null }

Filtered logs:

{ "actions": ["KILL", "TEAM KILL"] }

Response Formats

Data is return as JSON and is in the following formats:

Service disabled:

{
    "error": "Log stream is not enabled in your config",
    "last_seen_id": null,
    "logs": []
}

If more than 25 logs would be returned, they're batched 25 logs per response and multiple responses are sent to avoid any potential issues with payload size.

Logs:

{
    "last_seen_id": "1711657986-1",
    "logs": [
        {
            "id": "1711657986-0",
            "log": {
                "version": 1,
                "timestamp_ms": 1711657986000,
                "relative_time_ms": -188.916,
                "raw": "[449 ms (1711657986)] KILL: maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
                "line_without_time": "KILL: maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
                "action": "KILL",
                "player": "maxintexas13",
                "steam_id_64_1": "76561199195150101",
                "player2": "Twisted",
                "steam_id_64_2": "76561199096984835",
                "weapon": "KARABINER 98K",
                "message": "maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
                "sub_content": null
            }
        },
        {
            "id": "1711657986-1",
            "log": {
                "version": 1,
                "timestamp_ms": 1711657986000,
                "relative_time_ms": -188.916,
                "raw": "[340 ms (1711657986)] TEAM KILL: Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
                "line_without_time": "TEAM KILL: Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
                "action": "TEAM KILL",
                "player": "Untreated HSV",
                "steam_id_64_1": "76561198039482575",
                "player2": "zeto61",
                "steam_id_64_2": "76561198178304393",
                "weapon": "MK2 GRENADE",
                "message": "Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
                "sub_content": null
            }
        }
    ],
    "error": null
}

Sample Python Code

Courtesy of @excuseme on the Discord, this is an example of using the service

import asyncio
import json
from datetime import datetime

import logging

import websockets
from websockets.exceptions import ConnectionClosed

ALL_ACTIONS_TO_MONITOR = ['KILL', 'TEAM KILL', 'MATCH ENDED']


class CustomDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(object_hook=self.try_datetime, *args, **kwargs)

    @staticmethod
    def try_datetime(d):
        ret = {}
        for key, value in d.items():
            try:
                ret[key] = datetime.fromisoformat(value)
            except (ValueError, TypeError):
                ret[key] = value
        return ret


class CRCONWebSocketClient:

    def __init__(self, server, ):
        self.server = server
        self.stop_event = None

    async def start_socket(self, stop_event):
        self.stop_event = stop_event

        headers = {"Authorization": f"Bearer {self.server.rcon_api_key}"}
        if self.server.rcon_login_headers:
            headers.update(self.server.rcon_login_headers)
        websocket_url = self.server.rcon_web_socket + "/ws/logs"
        while not self.stop_event.is_set():
            try:
                async with websockets.connect(websocket_url, extra_headers=headers,
                                              max_size=1_000_000_000) as websocket:
                    logging.info(f"Connecting to {websocket_url}")
                    try:
                        await websocket.send(json.dumps({"last_seen_id": None, "actions": ALL_ACTIONS_TO_MONITOR}))
                    except ConnectionClosed:
                        logging.warning(
                            f"ConnectionClosed exception",
                            exc_info=cc)
                        break
                    logging.info(f"Connected to CRCON websocket {websocket_url}")

                    while not self.stop_event.is_set():
                        try:
                            message = await asyncio.wait_for(websocket.recv(), timeout=1.0)
                            await self.handle_incoming_message(websocket, message)

                        except asyncio.TimeoutError:
                            logging.debug("timeout error")
                        except asyncio.exceptions.CancelledError:
                            logging.debug("cancelled error")
                        except ConnectionClosed as cc:
                            logging.warning(
                                f"ConnectionClosed exception",
                                exc_info=cc)
                            break

                        except Exception as e:
                            logging.error(f"Error during handling of message by {message}", exc_info=e)

            except Exception as e:
                logging.error(f"Error connecting to {websocket_url}", exc_info=e)
            await asyncio.sleep(5)

    async def handle_incoming_message(self, websocket, message):
        json_object = json.loads(message, cls=CustomDecoder)
        if json_object:
            logs_bundle = json_object.get('logs', [])
            # do stuff with the logs