-
Notifications
You must be signed in to change notification settings - Fork 14
How to use splparser.cmdrules.common
Although the syntax of SPL commands can be quite idiosyncratic, there is overlap in their rules. In order to avoid redundancy (each parser containing copies of the same rules), parsers can import rules from splparser.cmdparsers.common. Doing this is highly recommended. If you wish to use these rules (and you should), you need to be careful to define all the requisite tokens, which are noted below.
At a high-level, there are two types of common rules: those which can be used with search-style lexers, and those which can be used with eval-style lexers. The main difference between these two types of lexers is that the former is able to tokenize commonly occurring complex data types, such as email addresses, IP addresses, hostnames, etc., whereas the latter can't do that because characters which commonly occur in these data types (such as '.' and '-') are used as operators for arithmetic, string manipulation, and so on. Thus, these two types of lexers necessarily tokenize the same strings differently. For example, consider the string day-average. search-style lexers can and should tokenize this as:
ID
whereas eval-style lexers can and should tokenize this as:
WORD MINUS WORD
because subtraction is a valid eval operator.
If your command uses eval functions, then you should use eval-style lexers. Otherwise, please use search-style lexers. You should define a new lexer for each command, but almost all commands should be able to use either splparser.cmdparsers.searchregexes or splparser.cmdparsers.evalregexes, so think very hard if you are not doing this.
The modules within this package are as follows:
These are very simple rules for as, which is used by many SPL commands. These rules match both the uppercase and lowercase version of the word "as".
Required tokens
'ASLC''ASUC'
Lexer style: any
Similar to the previous module (splparser.cmdparsers.common.asrules).
Required tokens:
'BYLC''BYUC'
Lexer style: any
These rules are all those necessary to parse invocations of eval functions, which are functions that can be used with the eval command, and others. These are defined here.
Required tokens:
'EVAL_FN''COMMON_FN''LPAREN''RPAREN''COMMA''PERIOD''PLUS''MINUS''TIMES''DIVIDES''MODULUS''EQ''LT''LE''GE''GT''NE''DEQ''AND''OR''NOT''XOR''LIKE'- See:
splparser.cmdparsers.common.simplevaluerules
Lexer style: eval
These rules parse lists of fields, delimited by space or by comma.
Required tokens:
- See:
splparser.cmdparsers.common.fieldrules
Lexer style: search
These rules parse fields, which have been discovered to be, via trial-and-error, of any of the token types listed below.
Required tokens:
'WORD''ID''LITERAL''HOSTNAME''NBSTR''WILDCARD''BIN''OCT''HEX''INT''FLOAT'
Lexer style: search
Like splparser.cmdparsers.common.fieldlistrules, but for simplefield rules instead of fieldrules.
Required tokens:
- See:
splparser.cmdparsers.common.simplefieldrules
Lexer style: eval
Like splparser.cmdparsers.common.fieldrules, but uses a smaller, simpler set of tokens that don't attempt to identify field type.
Required tokens:
'WORD''ID''LITERAL''NBSTR''BIN''OCT''HEX''INT''FLOAT'
Lexer style: eval
This module contains rules for simplevalues, which is a stand-in for almost any argument.
Required tokens:
'IPV4ADDR''IPV6ADDR'- See:
splparser.cmdparsers.common.simplefieldrules
Lexer style: eval
These rules are all those necessary to parse invocations of stats functions, which are functions that can be used with the stats command, and others. These functions can be found here. Since eval and its respective functions are also valid arguments to stats and its respective functions, there is a dependency on splparser.cmdparsers.common.evalfnexprrules (i.e., some rules in this module use evalfnexpr rules) and the tokens required by that module.
Required tokens:
'STATS_FN''COMMON_FN''EVAL''SPARKLINE'- See:
splparser.cmdparsers.common.evalfnexprrules - See:
splparser.cmdparsers.common.simplevaluerules
Lexer style: eval
This rule is necessary for commands which both need to parse 'MINUS' as a token, because it's used as an operator, and parse negative integers. This rule ensures that the 'MINUS' token in the case of negative integers has the proper associativity. See the ply documentation for more detail.
Required tokens:
'INT'-
'UMINUS'(precedence token placeholder, not an actual token)
Lexer style: eval
These rules are like simplevaluerules, but include a broader set of token types.
Required tokens:
'IPV4ADDR''IPV6ADDR''EMAIL''HOSTNAME''URL''PATH''US_PHONE'- See:
splparser.cmdparsers.common.simplefieldrules
Lexer style: search