applications/system

silk - SiLK: A network flow collection and analysis package

Website: http://tools.netsa.cert.org/silk/
License: GPLv2
Vendor: CERT Network Situational Awareness <netsa-help@cert.org>
Description:
SiLK, the System for Internet-Level Knowledge, is a collection of
traffic analysis tools developed by the CERT Network Situational
Awareness Team (CERT NetSA) to facilitate security analysis of large
networks. The SiLK tool suite supports the efficient collection,
storage and analysis of network flow data, enabling network security
analysts to rapidly query large historical traffic data sets. SiLK is
ideally suited for analyzing traffic on the backbone or border of a
large, distributed enterprise or mid-sized ISP.

SiLK consists of two sets of tools: a packing system and analysis
suite. The packing system receives network flow information from
Netflow v5 or any IPFIX-based flowmeter and converts them into a more
space efficient format, recording the packed records into
service-specific, binary flat files. The analysis suite consists of
tools which can read these flat files and then perform various query
operations, ranging from per-record filtering to statistical analysis
of groups of records. The analysis tools interoperate using pipes,
allowing a user to develop a relatively sophisticated query from a
simple beginning.

Packages

silk-3.16.0-1.fc20.src [5.2 MiB] Changelog by Lawrence Rogers (2017-06-29):
* Release 3.16.0-1/2
	rwstats
		When the primary value is a distinct count, compute the number of distinct items across all bins and print each bin's percentage of the total distinct count.
		Fix bugs that may occur when computing distinct counts and not all distinct counts fit into memory.
	rwuniq
		Fix bugs that may occur when computing distinct counts and not all distinct counts fit into memory.
	flowrate plug-in
		Change how the flowrate plug-in handles flow records whose duration is zero in order to fix bizarre looking output in rwstats. The plug-in now assumes each of these flow records has a duration of 400 microseconds (0.4 milliseconds).
		Add the --flowrate-zero-duration switch which allows the user to set the duration that the plug-in uses for flow records whose given duration is zero.
	rwrandomizeip
		Read flow records from the standard input if the number of non-switch arguments is zero.
		Write the flow records to the standard output if the number of non-switch arguments is zero or one.
	rwswapbytes
		Read flow records from the standard input if the number of non-switch arguments is zero.
		Write the flow records to the standard output if the number of non-switch arguments is zero or one.
	rwflowpack, flowcap
		Change processing of NetFlow v9 records so that, when SiLK is compiled against libfixbuf 1.8.0, the OUT_BYTES and OUT_PKTS values are used when the IN_BYTES and IN_PKTS values are 0.
	flowcap
		Print the probe definitions to the log file when the log-level is set to debug.
	rwflowpack, rwflowappend, flowcap, rwsender, rwreceiver, rwpollexec
		Change how daemons invoke subprocesses in order to avoid creating subprocesses that deadlock and never complete.
		Modify start-up scripts to be more in line with the rules in the Linux Standard Base.
	Plug-ins
		Add manual pages for the cutmatch, conficker-c, and app-mismatch plug-ins.
		No longer install the uniq-distproto plug-in since its functionality is available as --values=distinct:protocol.
silk-3.15.0-1.fc20.src [5.2 MiB] Changelog by Lawrence Rogers (2017-03-24):
* Release 3.15.0-1/2
	rwaggbag
		Create a new tool similar to rwbag: a tool to bin SiLK Flow records using a key and counter that support multiple fields and store the results in a binary Aggregate Bag file.
	rwaggbagbuild
		Create a new tool to create an Aggregate Bag file from text.
	rwaggbagcat
		Create a new tool to print the contents of an Aggregate Bag file as text.
	rwaggbagtool
		Create a new tool to manipulate binary Aggregate Bag files and create a new Aggregate Bag file.
	flowkey
		Add a new plug-in that uses the same algorithm as YAF to compute a 32-bit flow key hash.
	rwpmapcat
		Add the --output-path switch to specify the output file.
		POTENTIAL INCOMPATIBILITY. Note that the shortest unique prefix for the --output-type switch is now "--output-t".
	rwfileinfo
		Add the --xargs switch to read input file names from a file.
	rwsetcat
		Add the --output-path switch to specify the output file.
		Do not use the the pager when the output contains only the count of the number of IPs in a singe IPset.
	rwsiteinfo
		Add the --output-path switch to specify the output file.
	rwtuc
		Add the --xargs switch to read input file names from a file.
		Allow multiple fields in the input to be ignored.
		At shutdown, print the number of input lines that were not parsed unless --verbose is given or an error occurs.
		Remove the --bad-input-lines file when it is empty (in accordance with the manual page).
		Fix a bug that treated white space after the final delimiter as another field.
		Fix issues in parsing the title line when --fields is given.
	rwbagcat
		Add the --site-config-file switch to select the silk.conf file.
		Do not invoke the pager when --print-statistics is the only output and a destination argument is given to the switch.
	rwip2cc
		Do not use the pager when the --output-path switch is given.
	rwscanquery
		Fix a bug that prevented use of the SQLite database driver on a case-sensitive file system and caused "make check" to fail.
	Building
		Fix a compilation error in rwsiteinfo on Ubuntu.
		Remove support for fixbuf releases prior to libfixbuf-1.7.0.
silk-3.14.0-1.fc20.src [5.1 MiB] Changelog by Lawrence Rogers (2016-11-17):
* Release 3.14.0-1/2
	IPset changes
		Add a new file format, record-version=5, for IPsets containing IPv6 addresses that should be more
		compact than record-version=4. Unless the default file format is changed at configure time, the new
		format must be explicitly requested using --record-version switch or via the SILK_IPSET_RECORD_VERSION
		environment variable.
		Fix a bug when working with IPsets that contain IPv6 addresses and have more than 44,739,242
		internal nodes. The bug may cause the tool to crash or to loop endlessly.
		Reduce how quickly memory grows when building an IPset that contains IPv6 addresses.
		Perform additional integrity checks when reading an IPset file from disk.
	rwsetbuild
		Fix a bug introduced in SiLK-3.11.0 that may occur when computing the intersection or difference of
		an IPv4 IPset with an IPv6 IPset that is in record-version=4 format. Addresses in the ::ffff:0:0/96
		netblock of the IPv6 IPset were ignored when the IPset contained clusters of addresses less then
		::ffff:0:0.
	rwsetcat
		Allow computing the count of IP addresses in an IPset without loading the IPset into memory.
	rwbag
		Fix a bug when creating a bag whose key is attributes that causes the bag to appear to have duplicate keys.
	rwfileinfo
		Rename the title of the compression field. The title was changed unintentionally in SiLK 3.12.2 and caused iSiLK to fail.
	rwstats, rwuniq
		Do not limit the maximum hash table size to a 32-bit value on a 64-bit platform.
	flowcap, rwflowpack
		In the sensor.conf file, add support for a quirk to handle NetFlow v9 records generated by a
		SonicWall device where the router up-time is reported in seconds instead of milliseconds.
	Building
		Add a configure switch, --enable-ipset-compatibility, that allows changing the default IPset
		file format written by SiLK. The argument is the version of SiLK with which IPsets are to be
		compatible. The IPset file format changes at 3.7.0 and 3.14.0.
silk-3.13.0-1.fc20.src [5.0 MiB] Changelog by Lawrence Rogers (2016-09-29):
* Release 3.13.0-1/2
	Change across all tools
		Add support for compressing files with "Snappy" compression when the Snappy library and header are found during configuration.
		Add support for the SILK_COMPRESSION_METHOD environment variable that provides a default value for the --compression-method switch.
	rwcount
		Do not limit the maximum array size to a 32-bit value on 64-bit platforms.
	rwsettool
		Add a --symmetric-difference switch to compute the set of IP addresses that occur in only one of two input IPsets.
	rwfileinfo
		Disable printing of the record count when the file's compression method is not available.
	rwfilter, rwfglob
		Fix a file-selection bug where a --start-date specified in epoch seconds that fell on a day boundary would return files
		 for that entire day instead of for that single hour.
	PySiLK
		Fix memory leaks.
		Fix a bug in the silk.site.repository_iter() where an epoch-based start-date value that fell on a day boundary would
		 return files for that entire day instead of for that single hour.
	rwsender
		Change the log messages that are written when scanning the incoming and processing directories.
silk-3.12.2-1.fc20.src [5.0 MiB] Changelog by Lawrence Rogers (2016-06-23):
* Release 3.12.2-1/2
	rwgeoip2ccmap
		Restore support for binary input that was removed in SiLK 3.12.0.
	rwbagcat
		Sort the output using the value of each key's counter when the --sort-counters switch is given.
	rwbag
		Copy the invocation history and the notes from the source files to the output file(s).
	rwbagtool
		When inverting a bag, set the key-type of the output to the counter-type of the input. Previously it was set to custom.
	rwfileinfo
		Add a --help-fields switch.
		Expand the description of rwfileinfo's output on the manual page.
	rwfilter, rwfglob, rwsiteinfo
		Fix an unexpected fatal error that would occur when the silk.conf file contained a class that did not contain any types.
		Check the validity of the silk.conf file and report such errors.
	rwipfix2silk
		Write additional log messages when --log-destination is specified.
	rwpdu2silk
		Write additional log messages when --log-destination is specified.
	rwflowpack
		Change when record counts are reported in the log file: Report the number of records written to each output file only when the files are flushed.
		Fix a bug processing the reverse side a YAF bi-flow that stored the egressInterface in both the input and output fields.
		Fix a bug processing a bi-flow record that reversed the vlan interfaces on the forward record.
	flowcap
		Fix a bug when processing the reverse side a YAF bi-flow that stored the egressInterface in both the input and output fields.
		Fix a bug processing a bi-flow record that reversed the vlan interfaces on the forward record.
	rwflowappend
		Add locking of incremental files to prevent multiple rwflowappend invocations from processing the same file.
silk-3.12.1-1.fc20.src [5.0 MiB] Changelog by Lawrence Rogers (2016-05-05):
* Release 3.12.1-1/2
	rwbagcat
		Fix a bug where the pager was not invoked when displaying keys as IPs or integers.
	rwflowpack, flowcap
		Make substantial changes to the handling of IPFIX and NetFlow v9 records to decrease per-record processing time.
silk-3.12.0-1.fc20.src [5.1 MiB] Changelog by Lawrence Rogers (2016-03-31):
* Release 3.12.0-1/2
	rwbag
		Add a new switch --bag-file that replaces the numerous bag creation switches that previously existed. Deprecate the previous bag creation switches.
		Expand the list of keys that rwbag supports (e.g., start-time, sensor, TCP flags).
		Add support for creating a bag that contains country codes.
		Add support for creating a bag whose key is derived from a prefix map that maps either IP-addresses or protocol-port pairs.
		Add a header to the Bag file that stores the command line used to create the file.
	rwbagcat
		POTENTIAL INCOMPATIBILITY. Display a key whose type represents a time using a human-readable timestamp. Using --key-format=epoch displays the integer value.
		POTENTIAL INCOMPATIBILITY. Display a key whose type represents a SiLK sensor using the the sensor name. Using --key-format=decimal displays the integer value.
		POTENTIAL INCOMPATIBILITY. Display a key whose type represents TCP flags using the standard FSRPAUEC letters. Using --key-format=decimal displays the integer value.
		POTENTIAL INCOMPATIBILITY. Display a key whose type represents SiLK attributes using the standard TCFS letters. Use --key-format=decimal to display the integer value.
		Display a key whose type represents a country code using the two letter abbreviation.
		Require a prefix map to be specified via the --pmap-file switch when attempting to display a key whose type represents a mapping from a prefix map. Require the type of the prefix map to match the key-type specified in the Bag.
		Allow the --key-format switch to accept time-formatting and timezone arguments when printing a key that represents a time. Exit with an error when a time-format is used on a Bag whose key-type is neither a time nor 'custom'.
		POTENTIAL INCOMPATIBILITY. Exit with an error when a --key-format for an IP address is used on a Bag whose key-type is neither an IP address nor 'custom'.
		POTENTIAL INCOMPATIBILITY. Exit with an error when the --network-structure switch is used on a Bag whose key-type is neither an IP address nor 'custom'.
		POTENTIAL INCOMPATIBILITY. Exit with an error when the --mask-ips switch is using on a Bag whose key-type is neither an IP address nor 'custom'.
	rwbagbuild
		Add support for creating a bag that contains country codes.
		Add support for creating a bag whose key is derived from a prefix map that maps either IP-addresses or protocol-port pairs.
		When mapping from a protocol-port pair to a prefix map value, allow the delimiter between the protocol and port to be different than that between the port and the counter.
		Add a header to the Bag file that stores the command line used to create the file.
	rwgeoip2ccmap
		Use the first line of input to determine whether to create an IPv4 or IPv6 country code map.
		Add a header to the Bag file that stores the command line used to create the file.
		Modify the tool to more closely follow other SiLK tools.
		POTENTIAL INCOMPATIBILITY. Do not read the binary form of the Legacy GeoIP country code map. Only accept the comma separated value form.
	rwstats
		Allow the --count switch to accept an argument of 0 which indicates that it should print all bins.
		Allow the --percentage switch to accept a floating point value.
	rwsort
		Do not limit the maximum sort-buffer size to a 32-bit value on 64-bit platforms.
	rwdedupe
		Do not limit the maximum sort-buffer size to a 32-bit value on 64-bit platforms.
	rwcombine
		Do not limit the maximum sort-buffer size to a 32-bit value on 64-bit platforms.
	rwpmapbuild
		Add a header to the prefix map file that stores the command line used to create the file.
	rwsilk2ipfix
		Use multiple IPFIX templates when converting SiLK flow records.
		Add a --single-template switch to mimic the previous behavior.
	rwbagtool
		Fix an issue where the --compression-method switch was not applied to the IPset created by --coverset.
	rwflowpack, flowcap
		Fix a call to abort() that would occur when processing IPFIX records and a byte-count or packet-count of zero occurred in an unexpected place.
		Fix a bug that prevented creating a TCP IPFIX listener and a UDP IPFIX listener on the same port number.
	rwsender
		Attempt to resend any file that is not transferred unless the file is explicitly rejected by the rwreceiver.
		Add the --send-attempts switch that allows setting the number of attempts that are made to transfer a file.
		If sending a file fails and another attempt is to be made, append the file's name onto the back of the send queue.
		Allow setting of the --send-attempts switch from the configuration file and system initialization script.
		Fix a memory leak that may occur when rwsender is processing a file for an rwreceiver and their network connection ends.
		Support partial reads of a message header when GnuTLS is used.
		Log the GnuTLS error message that causes a connection to close.
	rwreceiver
		Support partial reads of a message header when GnuTLS is used.
		Log the GnuTLS error message that causes a connection to close.
	Building
		Fix several "make check" failures on OS X when System Integrity Protection is enabled.
		Remove use of pthread_atfork that preventing compilation on some systems.
silk-3.11.0.1-1.fc20.src [5.1 MiB] Changelog by Lawrence Rogers (2015-10-08):
* Release 3.11.0.1-1/2
	3.11.0.1
		Fix linking issue on Ubuntu when PySiLK support is enabled.
	3.11.0
		Allow rwsiteinfo to report on date ranges of files in a SiLK repository.
		Provide a way to set the default textual timestamp format and timezone from the environment.
		Provide a way to set the default textual IP format from the environment.
		Compile the PySiLK plug-in into the tools that can use it.
		Remove support for fixbuf releases prior to libfixbuf-1.6.0.
		Make additional changes and bug fixes.

Listing created by Repoview-0.6.6-1.el6