Racluster Examples
The argus-clients package provides a set of core client programs that provide the basic functions needed to use argus flow data. This includes printing, processing, sorting, aggregating, tallying, collecting, distributing, archiving, and anonymizing data. Here we provide basic examples of how to use the utilit racluster. For these examples, we use standard sets of argus data.
Racluster is the principal client program that aggregates argus data. Many client programs, such as rabins.1, rasqlinsert.1, share the aggregation functionality of racluster, so its a great starting example. The standard argus data set, argus.simple.data.out, has already been aggregated, to make it simple. How do we know this? Using a .rarc file that prints out the "trans" field, we see:
ra -r argus.simple.data.out
StartTime Trans Proto SrcAddr Sport Dir DstAddr Dport SrcPkts DstPkts State
17:48:36.589413 3 arp 192.168.0.68 who 192.168.0.66 3 3 CON
17:48:36.589949 1 udp 192.168.0.68.50251 <-> 192.168.0.66.domain 1 1 CON
17:48:36.590557 1 udp 192.168.0.68.53404 <-> 192.168.0.66.domain 1 1 CON
17:48:36.590954 1 arp 192.168.0.68 who 192.168.0.1 1 1 CON
17:48:36.591391 4 arp 192.168.0.66 who 192.168.0.1 4 0 INT
17:48:36.592155 3 tcp 192.168.0.68.60245 -> 128.2.129.188.http 12 15 FIN
17:48:36.632662 3 tcp 192.168.0.68.60246 -> 216.92.14.146.http 10 14 FIN
17:48:36.705481 3 tcp 192.168.0.68.60247 -> 128.2.129.188.http 10 13 FIN
17:48:36.705669 3 tcp 192.168.0.68.60248 -> 128.2.129.188.http 6 5 FIN
The tell tale sign is the "Trans" field has values greater than 1. argus.simple.data.out is generated from the standard packet file tcpdump.2012.02.13.17.20.00. The primitive argus data from the packet file contains many status records for arp, and http traffic. To generate the simple example data file, we aggregated the complete set of primitive argus data generated from tcpdump.2012.02.13.17.20.00, using the default options, and sorting the data by 'stime', and picked records 2-10.
argus -r tcpdump.2012.02.13.17.20.00 -w - | racluster -r - -N o2-10
The real utility provided by racluster is the ability to change the aggregation key. Aggregation is the principal method for doing summarization with flow data, and by changing what objects are used to generate the key, you can create a number of very useful summarizations. Demand by CIDR address, AS number, protocol, service port etc.... are the types of metrics summarizations used in many forms of network managment reports. Below we show a number of types of aggregations. For brevity, some of the examples do not show the complete command-line specification. Hopefully, these should be apparent.
Aggregated by protocol, printing out the rate and load metrics.
racluster -r argus.simple.data.out -m proto -s stime dur trans proto srate drate sload:16 dload:16 state
StartTime Dur Trans Proto SrcRate DstRate SrcLoad DstLoad State
17:48:36.589949 0.000608 2 udp 1644.736938 1644.736938 1947368.500000 3828947.500000 CON
17:48:36.592155 27.203648 12 tcp 1.360112 1.693453 1741.531128 13204.405273 FIN
17:48:36.589413 213.552917 8 arp 0.032779 0.014048 15.883651 9.590129 CON
Aggregated by session. Because the original data is aggregated to consolidate the status records into individual flow records, the "trans" field, which is contained in the aggregation dsr, has accumlated metrics for the number of records that were used to generate the record. We need to remove this aggregation dsr, so that we will be counting only unique flows.
racluster -r argus.simple.data.out -m saddr daddr proto dport -M dsrs="-agr" -s -sport
StartTime Trans Proto SrcAddr Dir DstAddr Dport SrcPkts DstPkts State
17:48:36.592155 3 tcp 192.168.0.68 -> 128.2.129.188.http 28 33 FIN
17:48:36.589949 2 udp 192.168.0.68 <-> 192.168.0.66.domain 2 2 CON
17:48:36.632662 1 tcp 192.168.0.68 -> 216.92.14.146.http 10 14 FIN
17:48:36.591391 1 arp 192.168.0.66 who 192.168.0.1 4 0 INT
17:48:36.590954 1 arp 192.168.0.68 who 192.168.0.1 1 1 CON
17:48:36.589413 1 arp 192.168.0.68 who 192.168.0.66 3 3 CON
Aggregated by ethernet and IP address, printing out the pkts transmitted and received. Because argus flow data contains metrics for two objects, such as the source and destination mac addresses, or the source and destination IP addresses, we have to modify the records to generate, what is referred to as, RMON, data. RMON metrics are oriented around a single object, tallying metrics for the inbound and outbound network traffic. We use the "-m rmon" option to convert argus data to this style of metric.
racluster -r argus.simple.data.out -M rmon dsrs="-agr" -m smac saddr
StartTime Trans Dur Mac Host OutPkts InPkts State
17:48:36.589949 6 27.205854 00:23:32:2f:ac:9c 192.168.0.68 40 49 CON
17:48:36.589413 2 213.551453 00:23:32:2f:ac:9c 192.168.0.68 4 4 CON
17:48:36.592155 3 27.203621 80:71:1f:3c:c3:88 128.2.129.188 33 28 CON
17:48:36.632662 1 27.163141 80:71:1f:3c:c3:88 216.92.14.146 14 10 CON
17:48:36.589949 2 0.000608 c8:2a:14:58:7a:55 192.168.0.66 2 2 CON
17:48:36.591391 1 213.550949 c8:2a:14:58:7a:55 192.168.0.66 4 0 INT
17:48:36.590954 2 213.551392 ff:ff:ff:ff:ff:ff 192.168.0.1 1 5 CON
17:48:36.589413 1 213.551453 ff:ff:ff:ff:ff:ff 192.168.0.66 3 3 CON