Ra Examples
The argus-clients package provides a set of core client programs that provide the basic functions needed to use argus flow data. This includes printing, processing, sorting, aggregating, tallying, collecting, distributing, archiving, and anonymizing data. Here we provide basic examples of how to use the utility rasplit. For these examples, we use standard sets of argus data.
Rasplit is the principal client program that splits and distributes argus data streams, writes the data to files. Like all ra* client programs, rasplit shares the complete set of functionality of ra, so it has a lot of flexibility and utility. rasplit is predominately used to divide a single argus data stream into a number of destination files, in order to scale processing and analytics, or its used as an archive, distributing argus data records into a native system file structure, suitable for search, procesing and long term storage.
% rasplit -r argus.2012.02.13.17.20.00.out.gz -M count 100 -w rasplit.out.
% ls -la .
total 280
drwxr-xr-x 6 carter wheel 204 Mar 19 14:37 .
drwxrwxrwt 20 root wheel 680 Mar 19 14:36 ..
-rw-r--r-- 1 carter wheel 37568 Mar 19 14:37 rasplit.out.aa
-rw-r--r-- 1 carter wheel 38808 Mar 19 14:37 rasplit.out.ab
-rw-r--r-- 1 carter wheel 44500 Mar 19 14:37 rasplit.out.ac
-rw-r--r-- 1 carter wheel 14552 Mar 19 14:37 rasplit.out.ad
This is the default output. All aspects of the output can be modified, generally from the command line, and completely specified using the rarc file strategy, which is highly recommeneded and installed in the home directory as ./.rarc.
Rasplit supports spliting on the resulting output file size. This is very useful for Google Big Table like data structuring, where a fixed sized data object is the desire.
% rasplit -r argus.2012.02.13.17.20.00.out.gz -M size 25000 -w rasplit.out.<br />
% ls -la .<br />
total 304
drwxr-xr-x 8 carter wheel 272 Mar 19 14:42 .
drwxrwxrwt 20 root wheel 680 Mar 19 14:36 ..
-rw-r--r-- 1 carter wheel 24960 Mar 19 14:42 rasplit.out.aa
-rw-r--r-- 1 carter wheel 24800 Mar 19 14:42 rasplit.out.ab
-rw-r--r-- 1 carter wheel 25784 Mar 19 14:42 rasplit.out.ac
-rw-r--r-- 1 carter wheel 24836 Mar 19 14:42 rasplit.out.ad
-rw-r--r-- 1 carter wheel 24956 Mar 19 14:42 rasplit.out.ae
-rw-r--r-- 1 carter wheel 10348 Mar 19 14:42 rasplit.out.af
For the purposes of data archiving, search, and analytics, rasplit supports spliting on the contents of flow records, such as the source id ('srcid'), the startime time ('stime') etc... This is very useful for building archives.
% rasplit -r argus.2012.02.13.17.20.00.out.gz -M time 5m -w %Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S<br />
% ls -lR .<br />
total 0
drwxr-xr-x 3 carter wheel 102 Mar 19 14:47 2012
./2012:
total 0
drwxr-xr-x 3 carter wheel 102 Mar 19 14:47 02
./2012/02:
total 0
drwxr-xr-x 4 carter wheel 136 Mar 19 14:47 13
./2012/02/13:
total 288
-rw-r--r-- 1 carter wheel 82052 Mar 19 14:47 argus.2012.02.13.17.45.00
-rw-r--r-- 1 carter wheel 57744 Mar 19 14:47 argus.2012.02.13.17.50.00