JSON Querylog Answer is obfuscated?

[pyrodex]

I've been using filebeat to consume the dumped querylog json and parse the data in elastic into pretty graphs. I did notice when looking at the data set from adguardhome the querylog always returns the ANSWER in an obfuscated format regardless of the upstream server type (e.g. DoT, DoH, Standard).

Below is an example:

{"T":"2022-02-03T10:28:20.577969524-05:00","QH":"api.amazonalexa.com","QT":"A","QC":"IN","CP":"","Answer":"+M2BgAABAAMAAAAAA2FwaQthbWF6b25hbGV4YQNjb20AAAEAAcAMAAUAAQAAADwAGAJ0cBJiMTYwNjYzO TAtZnJvbnRpZXLAEMAxAAUAAQAAADwAHw5kMWdzZzA1cnExdmpkdwpjbG91ZGZyb250A25ldADAVQABAAEAAAA8AAQ0VVOK","Result":{},"Upstream":"https://dns.quad9.net:443/dns-query","IP":"192.168.xx.xx","Elapsed" :496594,"Cached":true}

How does on decode the answer or is there a configuration setting I am missing? It would be nice to see what the client is getting back from the DNS server outside of the GUI for long term storage.

Thanks!

[ainar-g]

Hello. The Answer field isn't obfuscated, it's just the binary, wire format body of a DNS message.

[pyrodex]

Any way to decode this response? Looking to do this in Elastic.

[ainar-g]

@pyrodex, I don't think there is one. At least, not one suitable for Elastic. That log has an unstable format that is basically for use by AdGuard Home itself only, and it will probably be replaced completely in v0.108.0. You're probably better off waiting for it to release, or at least for it to gain the new query log HTTP API, and use that API to get the answer information for your analyses.

[MichalMarchewka]

Hello @ainar-g
Any news on log format?

[romanlex]

Any news about HTTP API ?

[tuyafeng]

In querylog, the answer can be parsed, and I've found the method to do so. I'll paste the code here to help anyone else facing the same issue.

Python code:

import struct
import base64

def parse_dns_response(data):
    # Parse DNS data header
    transaction_id = data[:2]
    flags = data[2:4]
    qdcount = struct.unpack('!H', data[4:6])[0]
    ancount = struct.unpack('!H', data[6:8])[0]
    nscount = struct.unpack('!H', data[8:10])[0]
    arcount = struct.unpack('!H', data[10:12])[0]

    print("Transaction ID:", transaction_id.hex())
    print("Flags:", flags.hex())
    print("Questions:", qdcount)
    print("Answer RRs:", ancount)
    print("Authority RRs:", nscount)
    print("Additional RRs:", arcount)

    # Skip header length
    offset = 12

    # Parse query part
    for _ in range(qdcount):
        offset, qname = parse_name(data, offset)
        qtype, qclass = struct.unpack('!HH', data[offset:offset+4])
        offset += 4
        print("Query Name:", qname)
        print("Query Type:", qtype)
        print("Query Class:", qclass)

    # Parse answer part
    for _ in range(ancount):
        offset, name = parse_name(data, offset)
        atype, aclass, ttl, rdlength = struct.unpack('!HHIH', data[offset:offset+10])
        offset += 10
        rdata = data[offset:offset+rdlength]
        offset += rdlength
        print("Answer Name:", name)
        print("Answer Type:", atype)
        print("Answer Class:", aclass)
        print("Answer TTL:", ttl)
        print("Answer Data Length:", rdlength)
        if atype == 1:  # If A record
            ip = struct.unpack('!BBBB', rdata)
            print("Answer Address:", ".".join(map(str, ip)))
        else:
            print("Answer Data:", rdata.hex())

def parse_name(data, offset):
    labels = []
    while True:
        length = data[offset]
        if length & 0xc0 == 0xc0:  # If pointer
            pointer = struct.unpack('!H', data[offset:offset+2])[0]
            offset += 2
            return offset, parse_name(data, pointer & 0x3fff)[1]
        if length == 0:  # Domain end if length == 0
            offset += 1
            break
        offset += 1
        labels.append(data[offset:offset+length].decode('utf-8'))
        offset += length
    return offset, ".".join(labels)

encoded_string = '+M2BgAABAAMAAAAAA2FwaQthbWF6b25hbGV4YQNjb20AAAEAAcAMAAUAAQAAADwAGAJ0cBJiMTYwNjYzO TAtZnJvbnRpZXLAEMAxAAUAAQAAADwAHw5kMWdzZzA1cnExdmpkdwpjbG91ZGZyb250A25ldADAVQABAAEAAAA8AAQ0VVOK'

decoded_bytes = base64.b64decode(encoded_string)

print(f'Raw: {encoded_string}\n')
parse_dns_response(decoded_bytes)

Output:

Raw: +M2BgAABAAMAAAAAA2FwaQthbWF6b25hbGV4YQNjb20AAAEAAcAMAAUAAQAAADwAGAJ0cBJiMTYwNjYzO TAtZnJvbnRpZXLAEMAxAAUAAQAAADwAHw5kMWdzZzA1cnExdmpkdwpjbG91ZGZyb250A25ldADAVQABAAEAAAA8AAQ0VVOK

Transaction ID: f8cd
Flags: 8180
Questions: 1
Answer RRs: 3
Authority RRs: 0
Additional RRs: 0
Query Name: api.amazonalexa.com
Query Type: 1
Query Class: 1
Answer Name: api.amazonalexa.com
Answer Type: 5
Answer Class: 1
Answer TTL: 60
Answer Data Length: 24
Answer Data: 027470126231363036363339302d66726f6e74696572c010
Answer Name: amazonalexa.com
Answer Type: 5
Answer Class: 1
Answer TTL: 60
Answer Data Length: 31
Answer Data: 0e64316773673035727131766a64770a636c6f756466726f6e74036e657400
Answer Name: d1gsg05rq1vjdw.cloudfront.net
Answer Type: 1
Answer Class: 1
Answer TTL: 60
Answer Data Length: 4
Answer Address: 52.85.83.138

[Somalia0932]

@tuyafeng 非常非常感谢您共享的代码，使得我对于querylog的利用又更近了一步！

[MichalMarchewka]

Alternatively, I created my own solution
https://github.com/MichalMarchewka/adguard-to-elasticsearch