Part 3: Implement inGraph for use with icinga and icinga-web on CentOS6

S&P 500 2009 low equal to that of 1997.
Tested twice. Good to go.
Updated: November 30th, 2012 (updated download to version 1.0.1)

inGraph is an opensource RDDtool replacement backed by Netways that provides both a dedicated web interface, and integration into icinga-web.

The ingraph-collector daemon parses then passes data to the ingraph daemon who then writes it to a MySQL DB.

There is a plugin called check_ingraph which works with trends, strandard variance, and averages, although trend graphs (for forecasts) are expected in the future.

Note that I write this up as if you had followed Part 1: Implement icinga and icinga-web on CentOS6 with SElinux and Part 2: Implement Lconf with icinga and icinga-web on CentOS 6 with SElinux.

Download and install inGraph daemons and site:

setenforce permissive
yum -y install python-setuptools python-devel MySQL-python
tar zxvf SQLAlchemy-0.6.9.tar.gz
cd SQLAlchemy*
python install
#get latest ingraph
wget #check for the latest version at
tar zxvf inGraph.*.tar.gz
cd ingraph
python install
useradd ingraph -s /sbin/nologin
./ingraph-web/ --install --prefix=/usr/local/ingraph --with-web-path=/ingraph --with-web-user=apache --with-web-group=apache
cp ./ingraph-web/ingraph.conf /etc/httpd/conf.d/
./icinga-web/ --install --prefix=/usr/local/icinga-web --with-web-user=apache --with-web-group=apache
chown -R apache:apache /usr/local/ingraph/pub
service httpd restart

Configure passwords:
Note the DB password ingraph_DBUSER_PASSWORD, which will be used by inGraph to access the database.

sed s/ingraph:ingraph@localhost/ingraph:ingraph_DBUSER_PASSWORD@localhost/ -i /etc/ingraph/ingraph-database.conf
sed s/'changeme'/'xmlrpc_PASSWORD'/ -i /etc/ingraph/ingraph-xmlrpc.conf
sed s@"name=\"pass\">changeme"@"name=\"pass\">xmlrpc_PASSWORD"@ -i /usr/local/icinga-web/app/modules/inGraph/config/inGraph.xml
sed s@"name=\"pass\">changeme"@"name=\"pass\">xmlrpc_PASSWORD"@ -i /usr/local/ingraph/app/modules/inGraph/config/inGraph.xml

Consider the data trunction schedule:
Take a look at /etc/ingraph/ingraph-aggregates.conf.
Easy concept.
interval is the amount of seconds to conflate (5*60 = 5 minutes of data conflation), retention-period is the number of seconds to keep the interval (7*24*60*60 = keep 7 days of data).
Therefore the default values are likely acceptable as:
“{‘interval’: 5 * 60, ‘retention-period’: 7 * 24 * 60 * 60}” means “Conflate 5 minutes snapshots for data older than 7 days.”
Since you probably won’t be getting perfdata back for your objects more frequently than once every 5 minutes, keeping the default values probably makes sense. But, if you are getting perfdata back more frequently than 5 minutes, you might want to adjust this.

Default configuration:

conflate snapshots of.. for data older than…
5 minutes 7 days
10 minutes 7 days
30 minutes 182 days
3 hours 5 years

Configure icinga to store perfdata to files

mkdir -p /usr/local/icinga/var/perfdata
chown icinga:icinga /usr/local/icinga/var/perfdata
chmod g+w /usr/local/icinga/var/perfdata
sed s/process_performance_data=0/process_performance_data=1/ -i /etc/icinga/icinga.cfg
sed s@#host_perfdata_file=/tmp/host-perfdata@host_perfdata_file=/usr/local/icinga/var/perfdata/host-perfdata@ -i /etc/icinga/icinga.cfg
sed s@#service_perfdata_file=/tmp/service-perfdata@service_perfdata_file=/usr/local/icinga/var/perfdata/service-perfdata@ -i /etc/icinga/icinga.cfg
sed s@"#host_perfdata_file_template=\[HOSTPERFDATA\]\\\t\\\$TIMET\\\$\\\t\\\$HOSTNAME\\\$\\\t\\\$HOSTEXECUTIONTIME\\\$\\\t\\\$HOSTOUTPUT\\\$\\\t\\\$HOSTPERFDATA\\\$"@"host_perfdata_file_template=DATATYPE::HOSTPERFDATA\\\tTIMET::\\\$TIMET\\\$\\\tHOSTNAME::\\\$HOSTNAME\\\$\\\tHOSTPERFDATA::\\\$HOSTPERFDATA\\\$\\\tHOSTCHECKCOMMAND::\\\$HOSTCHECKCOMMAND\\\$\\\tHOSTSTATE::\\\$HOSTSTATE\\\$\\\tHOSTSTATETYPE::\\\$HOSTSTATETYPE\\\$"@ -i /etc/icinga/icinga.cfg
sed s@#host_perfdata_file_mode=a@host_perfdata_file_mode=a@ -i /etc/icinga/icinga.cfg
sed s@#service_perfdata_file_mode=a@service_perfdata_file_mode=a@ -i /etc/icinga/icinga.cfg
sed s@#host_perfdata_file_processing_interval=0@host_perfdata_file_processing_interval=30@ -i /etc/icinga/icinga.cfg
sed s@#service_perfdata_file_processing_interval=0@service_perfdata_file_processing_interval=30@ -i /etc/icinga/icinga.cfg
sed s@#host_perfdata_file_processing_command=process-host-perfdata-file@host_perfdata_file_processing_command=process-host-perfdata-file@ -i /etc/icinga/icinga.cfg
sed s@#service_perfdata_file_processing_command=process-service-perfdata-file@service_perfdata_file_processing_command=process-service-perfdata-file@ -i /etc/icinga/icinga.cfg

Configure ingraph and ingraph-collector daemons

mkdir /var/log/ingraph
chown ingraph:icinga /var/log/ingraph
echo "INGRAPH_CHDIR=/etc/ingraph" > /etc/sysconfig/ingraph
echo "INGRAPH_PIDFILE=/var/run/ingraph/" >> /etc/sysconfig/ingraph
echo "INGRAPH_USER=ingraph" >> /etc/sysconfig/ingraph
echo "#INGRAPH_LOGFILE=/var/log/ingraph/" >> /etc/sysconfig/ingraph
echo "INGRAPH_COLLECTOR_CHDIR="/etc/ingraph"" > /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_PIDFILE="/var/run/ingraph/"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_PERFDATA_DIR="/usr/local/icinga/var/perfdata"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_PERFDATA_PATTERN="*-perfdata.*[0-9]"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_FILE_LIMIT="50"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_FILE_MODE="BACKUP"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_SLEEPSECS="60""  >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_USER="ingraph"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_GROUP="icinga"" >> /etc/sysconfig/ingraph-collector
echo "INGRAPH_COLLECTOR_LOGFILE="-" # - = stdout/stderr" >> /etc/sysconfig/ingraph-collector

Configure database for inGraph:
Note the DB password ingraph_DBUSER_PASSWORD, which will be used by inGraph to access the database.

echo "CREATE DATABASE ingraph;" > ~/create_ingraph.sql
echo "GRANT USAGE ON *.* TO 'ingraph'@'localhost' IDENTIFIED BY 'ingraph_DBUSER_PASSWORD';" >> ~/create_ingraph.sql
echo "GRANT ALL ON ingraph.* TO 'ingraph'@'localhost';" >> ~/create_ingraph.sql
mysql -p < ~/create_ingraph.sql

Create and push the command definitions to process the perfdata files:
1) Access the LConf LDAP Editor and connect to the LDAP server you’ve configured in Part 2.
2) Create the following two command definitions somewhere below IcingaConfig. My preference is to create them under the default_conf node as this setting will never if rarely change.

process-service-perfdata-file: mv /usr/local/icinga/var/perfdata/service-perfdata /usr/local/icinga/var/perfdata/service-perfdata.$TIMET$
process-host-perfdata-file: mv /usr/local/icinga/var/perfdata/host-perfdata /usr/local/icinga/var/perfdata/host-perfdata.$TIMET$

3) Export the config by clicking on the connection tile, and clicking ‘Export config”

Test configuration
Use two consoles:

service ido2db restart
service icinga restart
ingraphd -d /etc/ingraph -uingraph -f start
ingraph-collectord -d /etc/ingraph -uicinga -gicinga -f start
# if ingraph-collectord crashes with '(key, value) = nvpair.split('::', 1) / ValueError: need more than 1 value to unpack', it's related to your perfdata files being populated with perfdata that isn't separated with '::'.  Delete it all then run ingraph-collectord again.

Re/start services

chkconfig ingraph on
chkconfig ingraph-collector on
service icinga restart
service ingraph start
service ingraph-collector start

Disallow changes to .\IcingaConfig\default_conf (optional)

With reference:

  1. December 19, 2012 at 5:58 am

    i have setup all as you wrote, and my database is not filling with any data. no graphing in icinga-web…when i click on graph, it show empy graph and says no-data
    any help would be appricieted.

    • December 19, 2012 at 6:56 am

      Did you follow the section “Create and push the command definitions to process the perfdata files”?

      Upload the following files to pastebin:

      REMOVE your password(s).

  2. December 19, 2012 at 7:10 am

    i didn’t apply this to commands.cfg
    mv /usr/local/icinga/var/perfdata/service-perfdata /usr/local/icinga/var/perfdata/service-perfdata.$TIMET$

    now works.


    • December 19, 2012 at 7:11 am

      Cool. Yea man, gotta watch out for all that stuff.

  3. ak
    September 17, 2015 at 3:05 am

    I am unable to untar inGraph.Can any one please help on this?

    Thanks in advance

    • September 17, 2015 at 1:58 pm

      Sorry, I’m not sure why that would be. You have to run the command to untar the file as described. Did you try that exactly?

  4. ty_2016
    December 17, 2016 at 2:03 pm


    Can any one please provide direct link to download ingraph.1.0.2 or earlier version.

    I tried to download from Gitlab/Netways websites but its not showing for me..

    Thanks in advance.

  5. satya
    December 28, 2016 at 1:09 pm

    Ingraph is not showing graphs for 1 hour and 4 hours. I followed same steps provided by this link.

    Can any one help me on this where I did mistake?

    Thanks in advance.

  6. robert
    May 15, 2017 at 8:16 am

    Since our mysql database file gets bigger and bigger, and every icinga reload needs about 20 minutes for ido2dm to release the db to the webgui, we tried to create an own mysql instance for ingraph on port 3307. We already moved the db to the new instance and changed the mysql dsn to the new port. But at starting ingraph, it still tries to connect to the old db on 3306, although dsn says 3307. Is there anybody who could help with this?

    Thanks in advance

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: