Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transform: luaxform transform script #11938

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions doc/userguide/configuration/suricata-yaml.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2829,8 +2829,9 @@ Lua
~~~

Suricata 8.0 sandboxes Lua rules by default. The restrictions on the sandbox for Lua rules can be
modified in the ``security.lua`` section of the configuration file. Additionally, Lua rules
can be completely disabled the same as the Suricata 7.0 default:
modified in the ``security.lua`` section of the configuration file. This section also applies to
Lua transforms. Additionally, Lua rules can be completely disabled in the same way as for as the
Suricata 7.0 default:

::

Expand Down
15 changes: 8 additions & 7 deletions doc/userguide/lua/lua-functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ Lua functions
Differences between `output` and `detect`:
------------------------------------------

Currently, the ``needs`` key initialization varies, depending on what is the goal of the script: output or detection.
Currently, the ``needs`` key initialization varies, depending on what is the goal of the script: output or detection. The
Lua script for the ``luaxform`` transform **does not use ``needs``**.

If the script is for detection, the ``needs`` initialization should be as seen in the example below (see :ref:`lua-detection` for a complete example of a detection script):

Expand Down Expand Up @@ -812,7 +813,7 @@ Example:
return 0
end
end

HasshServerGet
~~~~~~~~~~~~~~

Expand All @@ -828,7 +829,7 @@ Example:
return 0
end
end

HasshServerGetString
~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -998,7 +999,7 @@ index so in our case we need to use 0.
SCFlowintSet(0, a + 1)
else
SCFlowintSet(0, 1)
end
end

SCFlowintGet
~~~~~~~~~~~~
Expand Down Expand Up @@ -1031,7 +1032,7 @@ SCFlowvarSet
Set a Flowvar. First parameter is the index, second is the data
and third is the length of data.

You can use it to set string
You can use it to set string

::

Expand All @@ -1041,7 +1042,7 @@ You can use it to set string
needs["flowvar"] = {"cnt"}
return needs
end

function match(args)
a = SCFlowvarGet(0);
if a then
Expand All @@ -1050,7 +1051,7 @@ You can use it to set string
else
a = tostring(1)
SCFlowvarSet(0, a, #a)
end
end

Misc
----
Expand Down
19 changes: 14 additions & 5 deletions doc/userguide/lua/lua-usage.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,29 @@
Lua usage in Suricata
=====================

Lua scripting can be used in two components of Suricata. The first is in
output and the second one in rules in the detection engine.
Lua scripting can be used in two components of Suricata:

* Output
* Detection: ``lua`` keyword and ``luaxform`` transform

Both features are using a list of functions to access the data extracted by
Suricata. You can get the list of functions in the :ref:`lua-functions` page.

.. note:: Currently, there is a difference in the ``needs`` key in the ``init`` function, depending on what is the usage: ``output`` or ``detection``. The list of available functions may also differ.
.. note:: Currently, there is a difference in the ``needs`` key in the ``init`` function,
depending on what is the usage: ``output`` or ``detection``. The list of available
functions may also differ. The ``luaxform`` doesn't use the ``needs`` key.

Lua output
----------

Lua can be used to write arbitrary output. See :ref:`lua-output` for more information.
Lua scripts can be used to write arbitrary output. See :ref:`lua-output` for more information.

Lua detection
-------------

Lua script can be used as a filter condition in signatures. See :ref:`lua-detection` for more information.
Lua scripts can be used as a filter condition in signatures. See :ref:`lua-detection` for more information.

Lua transform
-------------

The ``luaxform`` transform can be used in signatures. See :ref:`lua-transform` for more information.
19 changes: 16 additions & 3 deletions doc/userguide/rules/lua-detection.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,18 @@
Lua Scripting for Detection
===========================

There are 2 ways that Lua can be used with detection. These are

* ``lua`` rule keyword.
* ``luaxform`` transform.

.. note:: Lua is disabled by default for use in rules, it must be
enabled in the configuration file. See the ``security.lua``
section of ``suricata.yaml`` and enable ``allow-rules``.

Lua Rule Keyword
^^^^^^^^^^^^^^^^

Syntax:

::
Expand Down Expand Up @@ -103,8 +111,13 @@ Entire script:

return 0

Sandbox and Available functions
-------------------------------
Lua Transform: ``luaxform``
^^^^^^^^^^^^^^^^^^^^^^^^^^^

More details in :ref:`lua-transform`

Lua Sandbox and Available functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Lua rule scripts are run in a sandbox environment the applies the
following restrictions:
Expand Down Expand Up @@ -140,7 +153,7 @@ Of note, the following standard libraries are not available:

This behavior can be modified via the ``security.lua`` section of :ref:`suricata-yaml-lua-config`

.. note:: Suricata 8.0 has moved to Lua 5.4 and has builtin support for bitwise and utf8 operations now.
.. note:: Suricata 8.0 has moved to Lua 5.4 and now has builtin support for bitwise and utf8 operations.

A comprehensive list of existing lua functions - with examples - can
be found at :ref:`lua-functions` (some of them, however, work only for
Expand Down
107 changes: 107 additions & 0 deletions doc/userguide/rules/transforms.rst
Original file line number Diff line number Diff line change
Expand Up @@ -243,3 +243,110 @@ This example transforms `"Zm 9v Ym Fy"` to `"foobar"`::

content:"/?arg=Zm 9v Ym Fy"; from_base64: offset 6, mode rfc2045; \
content:"foobar";

.. _lua-transform:

luaxform
--------

This transform allows a Lua script to apply a transformation
to a buffer.

Lua scripts that are used for transformations *must* contain a function
named ``transform``.

Lua transforms can be passed optional arguments -- see the examples below -- but they
are not required to do so. Optional arguments are included with the transform (see example
below).

Note that the arguments and values are passed without validation
nor interpretation. There is a maximum of 10 arguments.

The Lua ``transform`` function receive parameters:

* `input-length` The number of bytes in the buffer provided to the transform
* `input` The buffer provided to the transform
* `argument` The number of arguments provided in the following parameters. If there are
no arguments to the Lua transform, this value will be `0`.
* `arguments` The list of arguments.

The Lua ``transform`` must return two values

* `transformed-buffer` The buffer with the transformed bytes.
* `transformed-buffer-byte-count` The number of bytes in the transformed buffer.

This example supplies the HTTP data to a Lua transform with the transform
results being checked with `content`.

Example::

alert http any any -> any any (msg:"Lua Xform example"; flow:established; \
file.data; luaxform:./lua/lua-transform.lua; content: "abc"; sid: 2;)


The following example supplies the HTTP data to a Lua transform script along with
with arguments that specify the offset and byte count for the transform.
The resulting buffer is then checked with a `content` match.

Example::

alert http any any -> any any (msg:"Lua Xform example"; flow:established; \
file.data; luaxform:./lua/lua-transform.lua, bytes 12, offset 13; content: "abc"; sid: 1;)

The following Lua script shows a transform example that handles arguments: `bytes` and `offset` and uses
those values (or defaults, if there are no arguments) for applying the uppercase transform to
the buffer.

.. code-block:: lua

function init (args)
-- Initialization time steps can be
-- performed here.
end

local function get_value(item, key)
if string.find(item, key) then
local _, value = string.match(item, "(%a+)%s*(%d*)")
if value ~= "" then
return tonumber(value)
end
end

return nil
end

-- Arguments supported
local bytes_key = "bytes"
local offset_key = "offset"
function transform(input_len, input, argc, args)
local bytes = input_len
local offset = 0

-- Look for optional bytes and offset arguments
for i, item in ipairs(args) do
local value = get_value(item, bytes_key)
if value ~= nil then
bytes = value
else
value = get_value(item, offset_key)
if value ~= nil then
offset = value
end
end
end

local str_len = #input
if offset < 0 or offset > str_len then
print("offset is out of bounds: " .. offset)
return nil
end

local avail_len = str_len - offset
if bytes < 0 or bytes > avail_len then
print("invalid bytes " .. bytes .. " or bytes exceeds available length " .. avail_len)
return nil
end

local sub = string.sub(input, offset + 1, offset + bytes)
return string.upper(sub), bytes
end
1 change: 1 addition & 0 deletions doc/userguide/upgrade.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ Major changes
- sip.to
- sip.content_type
- sip.content_length
- New transform ``luaxform`` that uses a Lua script for sticky buffer transformation. More details in :ref:`lua-transform`

Removals
~~~~~~~~
Expand Down
2 changes: 2 additions & 0 deletions src/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,7 @@ noinst_HEADERS = \
detect-transform-compress-whitespace.h \
detect-transform-dotprefix.h \
detect-transform-header-lowercase.h \
detect-transform-luaxform.h \
detect-transform-md5.h \
detect-transform-pcrexform.h \
detect-transform-sha1.h \
Expand Down Expand Up @@ -884,6 +885,7 @@ libsuricata_c_a_SOURCES = \
detect-transform-compress-whitespace.c \
detect-transform-dotprefix.c \
detect-transform-header-lowercase.c \
detect-transform-luaxform.c \
detect-transform-md5.c \
detect-transform-pcrexform.c \
detect-transform-sha1.c \
Expand Down
8 changes: 4 additions & 4 deletions src/detect-dce-stub-data.c
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ static InspectionBuffer *GetSMBData(DetectEngineThreadCtx *det_ctx,
return NULL;
SCLogDebug("have data!");

InspectionBufferSetup(det_ctx, list_id, buffer, data, data_len);
InspectionBufferApplyTransforms(buffer, transforms);
InspectionBufferSetupAndApplyTransforms(
det_ctx, list_id, buffer, data, data_len, transforms);
}
return buffer;
}
Expand All @@ -105,8 +105,8 @@ static InspectionBuffer *GetDCEData(DetectEngineThreadCtx *det_ctx,
} else {
buffer->flags |= DETECT_CI_FLAGS_DCE_BE;
}
InspectionBufferSetup(det_ctx, list_id, buffer, data, data_len);
InspectionBufferApplyTransforms(buffer, transforms);
InspectionBufferSetupAndApplyTransforms(
det_ctx, list_id, buffer, data, data_len, transforms);
}
return buffer;
}
Expand Down
4 changes: 2 additions & 2 deletions src/detect-dnp3.c
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,8 @@ static InspectionBuffer *GetDNP3Data(DetectEngineThreadCtx *det_ctx,
}

SCLogDebug("tx %p data %p data_len %u", tx, tx->buffer, tx->buffer_len);
InspectionBufferSetup(det_ctx, list_id, buffer, tx->buffer, tx->buffer_len);
InspectionBufferApplyTransforms(buffer, transforms);
InspectionBufferSetupAndApplyTransforms(
det_ctx, list_id, buffer, tx->buffer, tx->buffer_len, transforms);
}
return buffer;
}
Expand Down
2 changes: 1 addition & 1 deletion src/detect-dns-answer-name.c
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ static InspectionBuffer *GetBuffer(DetectEngineThreadCtx *det_ctx,
InspectionBufferSetupMultiEmpty(buffer);
return NULL;
}
InspectionBufferSetupMulti(buffer, transforms, data, data_len);
InspectionBufferSetupMulti(det_ctx, buffer, transforms, data, data_len);
buffer->flags = DETECT_CI_FLAGS_SINGLE;
return buffer;
}
Expand Down
2 changes: 1 addition & 1 deletion src/detect-dns-query-name.c
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ static InspectionBuffer *GetBuffer(DetectEngineThreadCtx *det_ctx,
InspectionBufferSetupMultiEmpty(buffer);
return NULL;
}
InspectionBufferSetupMulti(buffer, transforms, data, data_len);
InspectionBufferSetupMulti(det_ctx, buffer, transforms, data, data_len);
buffer->flags = DETECT_CI_FLAGS_SINGLE;
return buffer;
}
Expand Down
2 changes: 1 addition & 1 deletion src/detect-dns-query.c
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ static InspectionBuffer *DnsQueryGetData(DetectEngineThreadCtx *det_ctx,
InspectionBufferSetupMultiEmpty(buffer);
return NULL;
}
InspectionBufferSetupMulti(buffer, transforms, data, data_len);
InspectionBufferSetupMulti(det_ctx, buffer, transforms, data, data_len);
buffer->flags = DETECT_CI_FLAGS_SINGLE;

SCReturnPtr(buffer, "InspectionBuffer");
Expand Down
Loading
Loading