Flow-Tools

Argus clients now support a complete set of functions and operations on flow-tools data, when reading streams and files. By specifying that the input format is the flow-tools format, argus clients can read the netflow and juniper flow records, convert them to argus record formats, and then operate on that data, using the argus client tool set.

To turn on this support, you need to have a working flow-tools library, that the argus-cients ./configure can find. There are options in ./configure for specifying where to find the flow-tools library that you would like to use.

Argus clients attempt to provide all the data processing and analytic capabilities of the flow-tools packages, through its own client programs. However, if there is a function that you discover is missing, please notify us through the argus developers mailing list, and we'll add that support.

Enabling Flow-tools Support

Download Flow-tools Library

The latest flow tools distribution can be downloaded from this link at Google Code. You will need to unbundle the distribution, make it, and then install the libraries, or provide the path to the generated libraries in the ./configure of the argus-clients distribution.

Link Flow-tools to Argus-clients

If you are configuring and compiling your argus-clients from source code, the ./configure script will attempt to find a usable flow-tools library, in system and local standard installation target directories, as well as the parent directory that the argus-clients distribution resides. When it finds a suitable distribution, argus-clients will automatically enable the use of "ft:" as a file type specifier.

You can tell ./configure where the copy of flow-tools library is using the --with-libft=DIR option:

% ./configure --with-libft=/path/to/my/flow-tools-directory

Basic flow operations on flow-tools data

Once the argus-clients distribution has been linke to a suitable flow-tools library, reading flow tools data involves specifying the flow data type in the "-r" option. By writing the file out, the flow-tools data will be converted to argus flow data.

% ra -r ft:flow-tools-data.file -w argus.file - src host 2.4.1.5

Once converted to argus-data, the flows can be processed in any number of ways. To process the files without conversion, simply read the data using the appropriate ra* analytic program, using the "-r ft:" specifier. To generate a CSV file with your own basic fields (specified in your .rarc file):

% ra -r ft:flow-tools-data.file-c , - src host 2.4.1.5

Flow-tools/ Argus-clients Capabilities

The argus-clients package includes a set of core client programs that map well to features in the flow-tools distribution. These features include printing, processing, sorting, aggregating, tallying, collecting, and distributing flow data. Here we provide basic examples of how to use these argus-client utilities; ra, rabins, racluster, racount, radium, ranonymize, rasort and rasplit, to provide flow-tools features.

flow-capture	rasplit	rasplit provides most of the funcitons of flow-capture, with the exclusion of providing big-endian / little-endian conversion support, and archive file expiration. Additional programs provide this capability.
flow-cat	rasort	all ra* programs can provide the flow concatentation feature of flow-cat, supporting the time filtering, but "rasort -m stime" provides flow-cat's "-g" option to sort the output by time. rasort does not provide integrated compression output.
flow-dscan	radark, raports, rahosts	The flow-dscan analytic to detect suspicious activity, such as port scanning and host scanning is covered by a large number of argus analytics, such as radark, rahosts, raports. However, the argus-clients approach to suspicious activity is not the same as flow-dscan, so it may not be a good fit.
flow-expire		argus-clients provides archive management software, such as flow-expire, through it mysql support. Simple file crawling and deletion, archive, etc... have been discussed on the argus mailing list.
flow-export	ra, raconvert, rasqlinsert	All argus-client programs can write flow data into a number of output formats, especially ASCII, CSV, XML. Database support is currently provided by a separate set of database programs.
flow-fanout	radium, ranonymize	Radium is the argus-clients collection and distribution system that provides all the properties of flow-fanout, except flow data manipulation (-A AS0_substitution and -m privacy_mask). Other programs provide these functions.
flow-filter	ra, rapolicy	All argus-client programs support the same filtering capabilities, which are a superset of the flow-filter filters. To provide the "-f acl_fname" functions, use rapolicy.
flow-gen		The argus web site provides a number of flow data files for test purposes.
flow-import	raconvert	raconvert reads ASCII CSV files and converts them to argus data.
flow-mask	racluster, ranonymize	racluster, the argus-clients aggregation utility is used to modify the flow key attributes to match some level of abstraction, without losing any of the data charateristics. If the purpose, however, is to anonymize data, use ranonymize.
flow-merge	rasort	All argus-client programs can merge flow files together, however to control the output so that its interleaved, use rasort -m stime.
flow-nfilter	ra	All argus-client programs can filter records based on a complex filters, which can be provided on the command line or in a rarc file. Argus-clients do not yet support the "-v variable binding" option, however.
flow-print	ra	All argus-client programs can print the contents of the records it processes, using a free format strategy.
flow-receive	ra	All argus-client programs can "receive" flow-tools data records.
flow-report		argus-clients does not provide a specific flow-report function, but the argus-clients distribution provides a number of bash, sh, and perl example programs that generate reports.
flow-send	ra, radium	All argus-client programs can "send" argus flow data to collectors, however, radium is the ra* program of choice
flow-split	rasplit	rasplit provides all the capabilities of flow-split, with the additional features of spliting data based on flow record content.
flow-stat	racluster, racount, raports, rahosts, ra.....	argus-clients does not provide a single program to provide the large number of reports that flow-stat generates. racluster, however, will generate many, if not most, of the data that flow-stat generates, through its general aggregation mechanisms. The distribution does provide a number of programs, like racount, raports, rahosts, that do provide similar information.
flow-tag	ralabel	ralabel provides for a free form metadata label per flow record that provides all the capabilities of flow-tag, including filtering and aggregation support for the generic labels.
flow-xlate	ranonymize, raconvert	ranonymize is the principal argus data field manipulation utility, however, many flow-xlate functions can be provided using raconvert and sed.

AUDITING NETWORK ACTIVITY

Using Argus

Getting Argus

Argus Wiki

Development

Documentation

Publications

Support