Implementation Details¶
There are optimizations for sequence
and join
that eliminate excessive pairing of events and
enable efficient processing of a stream of events. This is different from common database relationships,
such as a SQL Join, which matches every possible pairing and can potentially be costly for
event analytics.
Sequences¶
The underlying structure of a sequence
roughly resembles a state machine of events, meaning that only one pending sequence
can be in a node at a time. If the sequence uses by
for matching fields, then multiple
pending sequences can exist in a given node as long as values the values matched with by
are distinct. When a pending sequence matches an event, it will override any pending sequences
in the next state with identical by
values.
The state changes are described for the per-user sequence
and enumeration events below.
sequence by user_name
[process where process_name == "whoami"]
[process where process_name == "hostname"]
[process where process_name == "ifconfig"]
{id: 1, event_type: "process", user_name: "root", process_name: "whoami"}
{id: 2, event_type: "process", user_name: "root", process_name: "whoami"}
{id: 3, event_type: "process", user_name: "user", process_name: "hostname"}
{id: 4, event_type: "process", user_name: "root", process_name: "hostname"}
{id: 5, event_type: "process", user_name: "root", process_name: "hostname"}
{id: 6, event_type: "process", user_name: "user", process_name: "whoami"}
{id: 7, event_type: "process", user_name: "root", process_name: "whoami"}
{id: 8, event_type: "process", user_name: "user", process_name: "hostname"}
{id: 9, event_type: "process", user_name: "root", process_name: "ifconfig"}
{id: 10, event_type: "process", user_name: "user", process_name: "ifconfig"}
{id: 11, event_type: "process", user_name: "root", process_name: "ifconfig"}
Since the sequence is separated by
user_name
, commands executed by root
and user
are independently sequenced.
{id: 1, event_type: "process", user_name: "root", process_name: "whoami"}
// sequence [1] created in root's state 1
{id: 2, event_type: "process", user_name: "root", process_name: "whoami"}
// sequence [2] overwrote root's state 1
{id: 3, event_type: "process", user_name: "user", process_name: "hostname"}
// nothing happens, because user has an empty state 1
{id: 4, event_type: "process", user_name: "root", process_name: "hostname"}
// sequence [2, 4] now in root's state 2
// root's state 1 is empty
{id: 5, event_type: "process", user_name: "root", process_name: "hostname"}
// root's state 1 is empty, so nothing happens
{id: 6, event_type: "process", user_name: "user", process_name: "whoami"}
// sequence [6] created in user's state 1
{id: 7, event_type: "process", user_name: "root", process_name: "whoami"}
// sequence [7] created in root's state 1
{id: 8, event_type: "process", user_name: "user", process_name: "hostname"}
// sequence [6, 8] now in user's state 2
// user's state 1 is now empty
{id: 9, event_type: "process", user_name: "root", process_name: "ifconfig"}
// sequence [2, 4, 9] completes the sequence for root
// root still has [6] in state 1
{id: 10, event_type: "process", user_name: "user", process_name: "ifconfig"}
// sequence [6, 8, 10] completes the sequence for user
{id: 11, event_type: "process", user_name: "root", process_name: "ifconfig"}
// nothing happens because root has an empty state 2