Home > Uncategorized > Implement NetFlow on CentOS 6 (el6) and SElinux

Implement NetFlow on CentOS 6 (el6) and SElinux

Note that ipt-netflow produces netflow v5 packets; it does not produce v9 or IPFIX compliant netflow packets. If anyone has these mystery patches, please feel free to share a link in the comments below.


The goal is to set up a netflow generator and sensor on a single CentOS box for session monitoring over time.  Due to lack of hardware, I am using a box with a single NIC, and will be configuring a switch to mirror traffic on a port destined for the site firewall to the NIC.  This will affect the configuration only that I will chose to exclude packets to or from the host specifically while configuring the ipt_netflow module rule set in iptables.

The best apparent solution to generate netflow data is ipt-netflow, an iptables module. For very robust features, it appears a good alternate is pmacct.

This post references the README file included with the source as well as, some information from the NST project’s docs.

Install prerequisites:

yum -y install iptables-devel gcc kernel-headers kernel-devel gcc-c++ make wget

You can choose to upgrade the kernel by also installing the package kernel.  Note the compatibility which is explained on the project page before deciding this.  If you do this, you should reboot before continuing, and boot into this newer kernel version (this is standard unless you have multiple kernels installed).

Download, decompress, build and install ipt_netflow:

cd
wget http://sourceforge.net/projects/ipt-netflow/files/latest/download
tar -zxvf ipt_netflow*.tgz
cd ipt_netflow*
./configure
make all install; depmon
ls -al /root/ipt_netflow-*/ipt_NETFLOW.ko #it really does exist
ls -al /lib/modules/$(uname -r)/extra/ipt_NETFLOW.ko #and it really does exist here too

Set default config for ipt_NETFLOW:

The following configures the destination of netflow packets to be bound ip address of interface, port 2055:

vim /etc/modprobe.d/ipt_netflow.conf
options ipt_NETFLOW destination=[bound ip address of interface, i mean write it here, not this string and definitely not the brackets]:2055

Configure the ipt_NETFLOW module to load on system boot:

Verify that iptables will start at the runlevels you wish:

chkconfig --list | grep iptables #should list the runlevels that iptables is configured to start

Add the module to the iptables startup config:

This should, in turn, start the ipt_NETFLOW module, you have to edit the iptables module config:

vim /etc/sysconfig/iptables-config
#make sure that at least the ipt_NETFLOW is listed in the IPTABLES_MODULES option
IPTABLES_MODULES="ipt_NETFLOW"

Want to know more? Check out Quick Primer: linux kernel module loading.

Relabel the kernel module for SELinux

restorecon -v /lib/xtables/libipt_NETFLOW.so

Restart iptables:

service iptables restart

Check to make sure the ipt_NETFLOW module is loaded and revealed to the iptables module:

Check that it is loaded into memory:

grep ipt_netflow /proc/slabinfo

Check to see if it was loaded by iptables:

dmesg

Add iptable rules to activate netflow:

cp /etc/sysconfig/iptables /etc/sysconfig/iptables.bak #alternative to iptables-save and iptables-restore command
service iptables start
iptables -A INPUT -j NETFLOW
iptables -A OUTPUT -j NETFLOW
iptables -A FORWARD -j NETFLOW
service iptables save
service iptables stop #stop it for now

Configure a dummy NIC and set up a network bridge

In order to configure a dummy NIC, you can follow another post.

Restart iptables again so that your rules are picked up:

service iptables start

Here are some extra things:

Make changes to the ipt_NETFLOW module at run time:

Use sysctl to configure ipt_NETFLOW during run time by accessing the namespace net.netflow.destination.  Note that changes made during runtime are lost upon restart of the module, and are not explicitly “savable” to the run config.

List all available parameters that can be set in the name space:

sysctl -a | grep net.netflow

Write parameters using:

sysctl -w [parameter]="[double-quote nested list or info]"

Monitor statistics on the ipt_NETFLOW module

You can review/watch stats for the process via proc:

cat /proc/net/stat/ipt_netflow

You can check to see if Netflow data is being passed by using iptables LOG module:

iptables -A OUTPUT -p udp --dport 2055 -j LOG #add the LOG rule
tail -f /var/log/messages
iptables -D OUTPUT -p udp --dport 2055 -j LOG #delete the LOG rule

With reference:
NST’s wiki Dummy Interface.

Advertisements
  1. June 26, 2012 at 9:34 pm

    Hello,

    I have same scheme (port mirror for tx and rx both & ipt_netflow), but it dos’not work correctly, I can get only RX traffic from monitored interface, but not TX. TX triffic don’t goes to iptables… Could u get me any comments about this?

    • June 26, 2012 at 10:32 pm

      Thanks for reading Pavel.
      Explain to me where you have your packets ingress and where your packets are egress.

      Are you mirroring/spanning a port on a switch with the target port being that of the netflow probe?

      What egress packets are you attempting to capture? Where are they coming from/generated?

      Feel free to use Gliffy to draw a diagram.

      Thanks,

      Matt

  2. June 29, 2012 at 10:31 pm

    hi, matt
    why do you think ipt_netflow is the best solution for netflow exporter?
    I found out that nfcapd or flowexport(from flowtraq) is another good solution, it capture all packets from switch’s mirror port and form netflow packets.It’s easy to deploy, do not need to touch the linux kernel.

    but I haven’t compare their performance.

    • June 30, 2012 at 10:00 am

      Hello,

      Thanks for reading.

      I actually don’t think ipt_netflow is the best solution to generate netflow packets/as a netflow probe. I can’t make that call since I’ve also never researched comparisons between the different probes (don’t forget ntop’s nProbe). What I would focus on is an increase in packet misses when the packet count increases. In this case, I do believe something resident in the kernel will perform better than something in user space 100% of the time, strictly due to the “hops through the stack” and layers of complexity and inefficiencies that come with traversing to and living in the user space.

      It’s also worth noting that the two probes you listed solve different problems. From a brief look, nfcapd creates rotating files of netflow dumps. Flow Exporter seems to rely on pcap as well, but fits into FlowTraq, which is a pretty sweet console!

      I’ve since moved on from netflow, and started focusing more energy into working with qosient’s argus (which I’ve mentioned on here a few times). Once I learn ruby and rails, I plan on working on using argus data as input for d3js to create some useful visualizations for timeline as well as flows, related to performance and security.

      Please post back with any benchmarks or findings you come across.

      Thanks,

      Matt

  3. Sergey
    October 8, 2012 at 8:40 am

    Hello Matt,

    The dummy net bridge hack doesn’t help me.

    I have a server connected to a switch with port mirroring. Port mirroring is set to copy all traffic (both inbound and outbound) from one port to the port, to which the server is connected.

    On the server tcpdump shows the mirrored traffic on both eth1 and br0. However, iptables counters for -j NETFLOW rule are zero, and ipt_netflow doesn’t see any traffic from the mirror port.

    The OS on the server is RHEL 6.1 clone with 2.6.32-131.0.15.el6.x86_64 kernel, iptables version 1.4.7, ipt_netflow version 1.8.

    Could you please give any advice?

    Best regards,
    Sergey

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: