processes SiLK Flow records, and its goals are to automate common tasks, to get closer to "real-time" reporting of events, and to feed interesting data to a security information and event manager (SIEM).
The Analysis Pipeline supports many types of analysis, including:
- Watch list alerting (did we see traffic from a known bad IP?)
- Beacon detection
- Passive FTP detection
- IPv6 tunnel detection
- Thresholding (e.g., is total bytes over a limit?)
- Collection issues (is a sensor no longer reporting?)
Although the Analysis Pipeline can be run interactively, it is designed to be incorporated into the SiLK collection and packing infrastructure, where it can
analyze every SiLK Flow record produced by rwflowpack as the records are being added to the SiLK data repository.
When a record matches an analysis, the Analysis Pipeline may output the record in a pipe-delimited textual format. Whether a record is output depends
on how often the administrator has configured the Analysis Pipeline to issue that type of output. The administrator can easily configure a SIEM to process
the output generated by the Analysis Pipeline.