-
Notifications
You must be signed in to change notification settings - Fork 110
SC4S Log Path Development workflow
Using either a network capture tool such as wireshark OR SC4S directly capture a full example of relative events. Its important to capture the preamble to the header typically in the form of <n> where n is a number 1-3 digits long.
Using SC4S configure the device to send events to the default port (514) then review events in Splunk using the search index=main sourcetype=sc4s:fallback. The event in Splunk should include the full original message in a field called $RAWMSG.
Develop at least one test message harness using pytest. Additional tests may be required if conditional logic such as sub source typing or alternate time stamps are required.
- Create a new test case in the tests folder named in the format "test_vendor_product.py" see existing test cases as examples
- Copy the header (imports section) from an existing test case to the new test case module
- Identify a similar existing test case and copy the test case into the new test module and rename the function appropriately.
- Using a comment line place the reference message as captured above the new test case function
- Using JINJA2 syntax replace the existing test message "mt = ...." with a new message based on the collected sample. Note the "host" value must be the randomly generated value provided by "{{ host }}" syntax. Note the data format string may be required in more than one location if the actual event time stamp is not sourced from the syslog header.
- run all tests, All tests except for the new test case should pass and the new test cases must fail before continuing.
Note 1: Log paths are located relative to the project root in package/etc/conf.d/log_paths Note 2: Filters are located relative to the project root in a vendor specific sub folder package/etc/conf.d/filters
- Review the event indexed in the test instance of Splunk look for the indexed field sc4s_syslog_format, indexed fields are available in the event field list but are not in the raw event displayed
- Identify a similar log path as required for the new format and copy the file. Name the log path using the syntax "p_format-vendor_product.conf.tmpl" Note SC4S uses golang templating to render the conf file for syslog-ng use.
- Standard compliant RFC 5424 events and some legacy RFC 3164 events can be identified using the PROGRAM or message contents fields. If that is possible create a new filter in the filters path above.
- Update the new log path to utilize a unique vendor_product notation for this technology
- Update package/etc/context-local/splunk_index.conf to place any new events in an appropriate default index other than main
- Update package/etc/context-local/vendor_product_by_source.conf and package/etc/context-local/vendor_product_by_source.csv to allow ip/host pattern identification by example
All tests should now pass, verify using the output of pytest new tests did run.