Install, configure, and use ntop to monitor traffic

If you like ntop, you might like flow-inspector, an up and coming open source project. Leveraging d3.js to render flow statistics, it is a very useful tool to quickly get flow information, utilizing a variety of visualization techniques. flow-inspector can use argus, VERMONT, and Bro IDS connection logs (and soon snmp connection stats) as flow sources.

---

This is a work in progress:

• testing has not yet been done.
• Problem installing libgts, hence graphviz (updated December 22nd)

---

This is in complement to a previous post about the iptables module that is a Netflow generator, ipt_NETFLOW.

The current goal of this project is to use a switch to mirror traffic to a box with a single NIC, generate Netflow data for the packets, direct them back to the same box where ntop sits.  This is the latter portion of that… installing, configuring, and using ntop to gather data on network traffic.

Holistically, the final goal of the project is to produce useful information; produce a report on source and destinations, which should help identify “abnormal” network traffic.

At our perimeter we have a transparent proxy performing some application layer filtering of HTTP, FTP, SMTP, and IMAP.  There are also firewall policies in place that do what a firewall does, provides access control up to the transport layer (IP, UDP, and TCP based access control).  With these two methods combined, I have reduced the surface of extrusions from the internal network to the outside world, getting a tighter grip on protocol usage, odd destinations, and hopefully can provide further insight during postmortem analysis of malware incidents, as well as generally monitoring the usage of the Internet.

I will start by using ntop as a solution to this challenge, but may look into a network visualizer, like visualfirewall, or a “flow perspective-izer,” like argus, if and when this solution fails to produce the information I want (I suppose live monitoring and analysis of the past are two different subjects).

Alright, enough self-affirming banter!

Building and Installing ntop
Install some prerequisites:

yum -y install lsof vim gcc make wget gcc-c++ glibc-devel glib2-devel m4 autoconf automake gdbm gdbm-devel libpcap libpcap-devel libtool rrd rrdtool-devel openssl openssl-devel zlib zlib-deveyum l

Install libGeoIP prerequisite:

cd
wget http://geolite.maxmind.com/download/geoip/api/c/GeoIP.tar.gz
tar zxvf GeoIP.tar.gz
cd GeoIP-*
./configure
make
make install

Install GraphViz for IP> Local> Network Traffic Map

#install the graphviz repo
curl http://www.graphviz.org/graphviz-rhel.repo > /etc/yum.repos.d/graphviz-rhel.repo
#install graphviz and its dependencies
yum -y install graphviz
#remove the graphviz repo
rm -f /etc/yum.repos.d/graphviz-rhel.repo

Install ntop:

Install stable (no nDPI):

cd
wget http://sourceforge.net/projects/ntop/files/latest/download?source=files
tar zxvf ntop-*.tar.gz
cd ntop-*
./autogen.sh
make
make install
make install-selinux-policy
less docs/1STRUN.tx

Install unstable (with nDPI and a nicer UI):
Follow another post and return to this post.

Initialize the ntop user database:

useradd ntop #create a user to run ntop
passwd ntop #assign that user a password
chown ntop:ntop /usr/local/share/ntop
mkdir /var/opt/ntopdb
chown ntop:ntop /var/opt/ntopdb
which ntop
su - ntop -c "/usr/local/bin/ntop -P /var/opt/ntopdb -u ntop -A"
ls /var/opt/ntopdb/

Delete ntop_pw.db if you forget the admin password and run this again.

Configure ntop to run at system startup:

ntop init script:

vim /etc/init.d/ntop

The contents of /etc/init.d/ntop should be as follows:
Note that you MUST change the ifconfig lines to match the interface you are mirroring traffic to!

#!/bin/sh
#
#
#
#
# chkconfig: 345 90 10
# description:
#init script for ntop, written by
#branded on freenode
#mbrownnyc everywhere else
#

### BEGIN INIT INFO
# Provides:          ntop
# Required-Start:    $local_fs $network
# Required-Stop:     $local_fs $network
# Default-Start: 3 4 5
# Default-Stop: 0 1 2 3 4 5 6
# Description:       ntop pre-configured by OPTIONS in /etc/sysconfig/ntop
### END INIT INFO

# Source function library.
. /etc/rc.d/init.d/functions

exec="/usr/local/bin/ntop"
prog="ntop"
progname="ntop"
lockfile=/var/lock/subsys/ntop

[ -e /etc/sysconfig/$progname ] && . /etc/sysconfig/$progname

  rh_status() {
	echo "Checking status of $prog"
        status -p /var/run/$prog.pid -l $lockfile $progname
  }

  rh_status_q() {
        rh_status
	#>/dev/null 2>&1
  }

  start() {
        echo "Starting $progname:"
	echo "   $exec $OPTIONS"
	echo
        $exec $OPTIONS
	RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && touch $lockfile
        return $RETVAL
		}
  stop() {
        echo -n "Stopping $progname: "
        killproc $prog
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && rm -f $lockfile
        return $RETVAL
  }
  restart() {
        stop
        start
  }

case "$1" in
    start)
	rh_status_q && exit 0
        $1
	;;
    stop)
	rh_status_q || exit 0
	$1
	;;
    restart)
	$1
	;;
    status)
	status -p /var/run/$prog.pid -l $lockfile $progname
	;;
    *)
	echo $"Usage: $0 {start|stop|restart|status}"
	exit 1
esac
exit 0

Create a config file to pull configuration from:

vim /etc/sysconfig/ntop

Contents of /etc/sysconfig/ntop:

OPTIONS="--daemon --max-table-rows 200 --local-subnets '192.168.10.0/24' --trace-level 2 --user ntop <strong>--http-server 0 </strong>--https-server 443 --domain domain.local --use-syslog=local3 --db-file-path /var/opt/ntopdb --mapper --interface none --no-fc --ipv4"

Note that these are options that I use in my config. The following arguments should be the same according to the previous config steps: –user, –db-file-path. Read the manpage to learn all of the options.

Add the service to be managed:

chkconfig --add ntop
chkconfig --levels 345 ntop on #configure ntop to execute at the desired runlevels

Change permissions on the file and execute the init script using the service manager:

chmod +x /etc/init.d/ntop
service ntop start

You may see the following error:

**ERROR** ++++ DEMON MODE=1

After reading the manpage, it became clear that not everything logged at an ERROR level is an error. This is the case with this error, it’s just a notification that ntop is in  mode.

Locate the ports where ntop is bound

ntop serves over http and https by default. Review the manpage for more info. I have used the –https-server parameter to control the port where ntop’s https server is bound.

netstat -apn | grep ntop

Access the web UI and review the config

Hit the host in a web browser at https://%5Bhost%5D

On the menu, hover on About, and click Show Configuration to review the currently running configuration. The manpage describes each setting in detail including their default value.

Poke around in the Admin menu, authenticating with the username admin and the password you previously configured.

Configure the Netflow plugin

Go to Plugins> Netflow> View/Configure

Confirm the Netflow device name.

Make sure that the Local Collector UDP port is 2055.  This will bind ntop to collect Netflow packets at UDP 2055.

Set some whitelist entries (a comma-separated list of hosts you’d like to capture), and blacklist entry (at least [the local IP of the server]/32).

Disable debugging.  You can re-enable this later.

Note that the plugin config will be retained through cycling of the process.

Verify your Netflow probe is sending Netflow data properly

I’m going to assume that you are using a local iptables instance with the ipt_NETFLOW module to sniff packets and produce Netflow packets.

To verify the port config is correct, run the following command which reveals the runtime/current settings for ipt_NETFLOW:

sysctl -a | grep net.netflow

Configure mirroring on your switch, and test.

About these ads