-
Notifications
You must be signed in to change notification settings - Fork 0
koolakoff/log_parse
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
A demo project of logs parser tool contain - parse_lib.py - engine for parsing specified events - parse_usb.py - an example of usage - a use case for tracking USB stick event detection from dmesg logs - log_example_1_usb - the piece of dmesg report used as a log example Example usage: - insert a USB stick to Linux PC (tested only with Transcend mass storage USB stick) - get the kernel logs to separate file and apply parser over the file: $ dmesg > log $ ./parse_usb.py log log_example_1_usb +17 [ 199.309030] USB storage detected: sda1 The parse engine supports combination of easy-to-use API and flexibility. Parser engine supports - single regexp - external functions for parsing specific use cases - events represented with multiple logs lines Usage: to use the engine, just need register set of Parser objects and then feed the logs lines to each of these objects parser = Parser(key, [report_line]) - where 'key' defines what would be parsed (see details below) - report_line - the actual text to display if event found in the log the 'key' supports the following use cases: - track one line log. In scope of this use case it is possible to register 'key' as a -- string regexp - that would be chedked by logs parser engine automatically Example: parser_usb = Parser(r"usb.*Product:\ Mass\ Storage\ Device", report_line="USB detected") Note, when passing a line as a regexp use letter r in front of string like this r"some\ regexp\ line" the 'r' tells python to store all '\' as a symbol of '\' but not as a hash-combo Note, it is used re.VERBOSE regexp style -- external function handler - the parser engine will call this handler for log analyze this allow parse more detailed info (i.e. when needs parse some info from log line) Example: def parse_hello(line): if "hello" in line: return "a Hello event" parser_usb = Parser(parse_hello) - track multiple lines log. some events may be represented in logs with multiple lines. For example the USB storage detection in dmesg are represented with a set of lines that include report of USB mass storage detection, and the corresponding device name is reported on a separate line. in this case it is possible to register the list of handlers (regex or external functions) for each of line. The parser will print result only when each of lines from the set is catched Example: def parse_world(line): if "world" in line: return "a Hello world event" parser_usb = Parser([r"hello",parse_world])
About
a Python demo tool of logs parser
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published