[ Home ]      [ Blog ]      [ Tutorials ]      [ Code Library ]      [ Aviation ]      [ Manuals ]      [ Grad School ]

Configuring NetFlow for Custom Reporting

Configure Sensor

A sensor listens to network traffic and exports summary data to a collector for storage. For this implementation, we will be utilizing the netflow exporting abilities of our PacketShaper in place of a dedicated sensor. To configure the packetshaper to export netflow data, perform the following steps:
  • Login to the PacketShaper and go to the Setup tab
  • Select "flow detail records" from the drop-down list
  • Input the IP address of our collector, port and select Netflow-5 as the record type
  • Click apply changes

Configure Collector

A collector receives summary data from a sensor and stores it to disk. For this implementation, we will be utilizing the popular flow-tools package. Make a directory for flow-capture to keep its records. I usually use /opt/netflow, but anywhere you have space works. On a multimegabit network, Netflow files can fill several GB of disk over a few weeks. I also recommend creating a saved subdirectory under your log directory, for the reporting system to use.

Now you need a startup script so flow-capture will run automatically at boot. I've added this line to /etc/rc.local:
 /usr/local/netflow/bin/flow-capture -p /var/run/flow-capture.pid -n 287 -N 0 -w /opt/netflow/ -S 5 0/0/9995
The -w flag tells flow-capture where to place its files. The final argument tells flow-capture which local IP to listen to, which remote IP to listen to, and which UDP port it should listen on. Flow-capture needs the -n 287, -N, and -S 5 arguments to interoperate with the reporting package, so leave them alone.

Once you start flow-capture, flow files will appear in your log directory. The names of these files come from the version of Netflow data they're gathering and the date and time at which this data began. For example, the filename tmp-v05.2005-04-28.201001-0400 indicates a temporary file, containing Netflow version 5 data, collected on April 28, 2005, starting at 20:10:01 (or 1 second after 8:10 p.m.), at -4 hours from GMT. Every five minutes, flow-capture moves the temporary file to a permanent location and starts a new temporary file. Permanent files begin with ft instead of tmp, but otherwise the names are exactly the same.

To confirm that your flow-capture install is actually collecting something, see if the temporary file grows. This should happen quickly, within a few minutes on a busy network.

Configure Cflow.pm

Go back to /usr/local/src/flow-tools-0.66/contrib and untar, make and install Cflow:
 tar zxf Cflow-1.051.tar.gz
 cd Cflow-1.051
 perl Makefile.PL
 make install
Then, test that the installation was successful by executing flowdumper:
 flowdumper  -s /opt/netflow/ft-v05...
The output of this command should look like this:
 2006/01/10 10:04:04 -> 6(PUSH|SYN|ACK) 10 1774
 2006/01/10 10:04:07 -> 6(PUSH|ACK|FIN) 6 327
 2006/01/10 10:04:06 -> 6(PUSH|SYN|ACK|FIN) 8 441
 2006/01/10 10:04:07 -> 6(PUSH|SYN|ACK|FIN) 11 9597
 2006/01/10 10:04:06 -> 6(PUSH|ACK|FIN) 8 684
 2006/01/10 10:04:07 -> 6(PUSH|SYN|ACK|FIN) 4 1174

Configure Analyzer

First, we must install RRD:
 rpm -ivh rrdtool-1.0.50-1.1.el3.rf.i386.rpm
 rpm -ivh perl-rrdtool-1.0.50-1.1.el3.rf.i386.rpm
Then, we must install Boulder::Stream:
 tar zxf Boulder-1.30.tar.gz
 cd Boulder-1.30
 perl Makefile.PL
 make install
Then, we must install Net::Patricia:
 tar zxf Net-Patricia-1.010.tar.gz
 cd Net-Patricia-1.010
 perl Makefile.PL
 make install
Then, we must install Config::Reader:
 tar zxf ConfigReader-0.5.tar.gz
 cd ConfigReader-0.5
 mkdir -p /usr/lib/perl5/site_perl/5.8.0/ConfigReader
 cp * /usr/lib/perl5/site_perl/5.8.0/ConfigReader
THEN, we must install HTML::Table:
 tar zxf HTML-Table-1.20.tar.gz
 cd HTML-Table-1.20
 perl Makefile.PL
 make install
Finally, we can install FlowScan:
 tar zxf FlowScan-1.006.tar.gz
 cd FlowScan-1.006
 make install
Then, replace /usr/local/bin/FlowScan.pm with the newer version from http://net.doit.wisc.edu/~plonka/list/flowscan/archive/att-0848/01-FlowScan.pm.

Then "install" CUFlow.pm by "cp /usr/local/src/CUFlow-1.5/CUFlow.pm /usr/local/bin".

Next we must create /usr/local/bin/flowscan.cf:
 # These are the subnets in our network
 # These are used only to determine whether a packet is inbound our 
 # outbound
 # These are networks we are particularly interested in, and want to
 # get separate rrd's for their aggregate traffic
 Network academic_network
 # Where to put the rrd's
 # Make sure this is the same as $rrddir in CUGrapher.pl
 OutputDir /cflow/reports/rrds
 # Track multicast traffic
 # Keep top N lists 
 # Show the top ten talkers, storing reports in /cflow/flows/reports
 # and keeping the current report in /etc/httpd/data/reports/topten.html
 #Scoreboard 10 /cflow/reports/scoreboard /var/www/html/topten.html
 Scoreboard 100 /cflow/reports/big_scoreboard /var/www/html/big_topten.html
 # Same, but build an over-time average top N list
 #AggregateScore 10 /cflow/reports/scoreboard/agg.dat /var/www/html/overall.html
 AggregateScore 100 /cflow/reports/big_scoreboard/agg.dat /var/www/html/big_overall.html
 # Our two netflow exporters. Produce service and protocol reports for the
 # total, and each of these.
 Router ps9500 
 # Services we are interested in
 Service 20-21/tcp ftp
 Service 22/tcp ssh
 Service 23/tcp telnet
 Service 25/tcp smtp
 Service 53/udp,53/tcp dns
 Service 80/tcp http
 Service 110/tcp pop3
 Service 119/tcp nntp
 Service 143/tcp imap
 Service 412/tcp,412/udp dc
 Service 443/tcp https
 Service 1214/tcp kazaa
 Service 4661-4662/tcp,4665/udp edonkey
 Service 5190/tcp aim
 Service 6346-6347/tcp gnutella
 Service 6665-6669/tcp irc
 Service 54320/tcp bo2k
 Service 7070/tcp,554/tcp,6970-7170/udp real
 # protocols we are interested in
 Protocol 1 icmp
 Protocol 4 ipinip
 Protocol 6 tcp
 Protocol 17 udp
 Protocol 47 gre
 Protocol 50 esp
 Protocol 51 ah
 Protocol 57 skip
 Protocol 88 eigrp
 Protocol 169
 Protocol 255
 # ToS bit percentages to graph
 TOS 0 normal
 TOS 1-255 other
 # Interested in traffic to/from AS 1
 #ASNumber 1 Genuity
Finally make sure that our reporting directory structure is in place and start flowscan:
 cd /cflow/
 mkdir reports
 cd reports/
 mkdir -p rrds/PS9500
 mkdir big_scoreboard
 mkdir /opt/netflow/saved
 /usr/local/netflow/bin/flow-capture -p /var/run/flow-capture.pid -n 287 -N 0 -w /opt/netflow -S 5 0/0/9995

Setup the Pretty Pictures

This is the easy step. First, make CUGrapher.pl accessible via the web:
 cp /usr/local/src/CUFlow-1.5/CUGrapher.pl /var/www/html
Then edit these variables in the CUGrapher.pl:
 my $rrddir = "/cflow/reports/rrds";
 my $organization = "My University Campus";
   ©Copyright 2004. All Rights Reserved