Archive

Posts Tagged ‘monitoring’

Workflow: Palo Alto Wildfire positive on incoming SMTP => Check Ironport for delivery status => ? => Profit.

February 24, 2015 Leave a comment

Palo Alto Wildfire positive alert on incoming SMTP.
Message tracking by source server IP (provided within Wildfire positive alert) = delivery status?
If delivered, trigger email alert to user and to admins with Email subject. Possibly move email to different store with EWS.

Advertisements

Smelling the need for a Windows Event log monitor

July 9, 2014 Leave a comment

A more effective monitoring architecture

December 31, 2013 Leave a comment

Needs some work


After having a conversation with Carter Bullard of argus fame about six months ago, two points stuck with me (loosely quoted):

  • “You throttle ICMP?! Why?! ICMP has a lot of useful data for everyone!”
  • “Why are you so focused on using argus data for security? Focus on using it to monitor performance. It’ll give you something to deliver to your manager so they don’t think you’re wasting your time and their money. Then focus on security.”

But how? Well, quite easily. At boundaries, use an argus probe to:

  • watch for ICMP status that isn’t a successful ECHO-ECHO REPLY:
    ra -S 127.0.0.1:561 -s ltime saddr daddr smac dmac spkts dpkts flgs state inode - "icmp and (dst pkts eq 0 or not echo)"
    
  • watch for no “heartbeat” (needs tuning):
    rabins -S 127.0.0.1:561 -B 15s -M 5m - src bytes lt 1 or dst bytes lt 1 or src rate lt 1 or dst rate lt 1
    
  • watch for `loss`:
    rabins -S 127.0.0.1:561 -B 15s -M 5s - ploss gt 0
    
  • watch for protocol indicated problems:
    rabins -S 127.0.0.1:561 -B 15s -M 5s - frag or retrans or outoforder or winshut
    
  • watch for performance degradation below a threshold:
    #requires at least argus-clients-3.0.7.19
    rabins -S 127.0.0.1:561 -B 15s -M 5s - src jit gt N or dst jit gt N or src intpkt gt N or dst intpkt gt N
    

If you want to filter in certain addresses to use a pipeline:

ra -S 127.0.0.1:561 -w - - icmp | rafilteraddr -r - -f raaddrfilter.txt -s ltime saddr daddr sbytes dbytes flgs state

Nagios et al are useful to get resource statistics via snmp for sure. It is also better at managing alerts than logstash (specifically schedules!).

The architecture would be like this:
monitoring_arch

nagios output from logstash is already coded.

Icinga et al. should still be used to send pings to devices, but no NOTICEs should be sent on these unreachable events, as the argus probe should be taking care of reachability monitoring.

I believe the bulk of the challenge will take place with processing argus data, but I believe it is quite doable. See: Using elasticsearch for logs (will probably run logstash or logstash-forwarder (aka lumberjack) on the local argus box for caching).

This consolidates performance monitoring into a single dashboard, who’s backend can be utilized for SIEM when the time comes. Producing reports should be very easy, and a ton of work has already been done as related to layman statistics on elasticsearch data, so this is great.

Processing icinga service and host check_results into elasticsearch should be very easy. Look at:

  • service_perfdata_file_template (very important for your logstash grok definition)
  • service_perfdata_file_mode
  • service_perfdata_file_processing_interval
  • service_perfdata_file_processing_command
  • service_perfdata_command

Monitoring output of ps or monitoring process resource utilization stats in linux

June 28, 2013 Leave a comment
while true ; do echo $(date '+%H:%M:%S'): $(ps -o rss $(pgrep rasqlinsert) | grep -v RSS) ; sleep 2 ; done > ~/test.log & 

use the following to obtain the process ID of the backgrounded job:

job -p

To kill the background job:

kill -9 [pid]

Also see:
Standard Format Specifiers

References:

inGraph: Views are accessible via…

February 7, 2013 Leave a comment

Netways’ inGraph views are accessible at the path:

/usr/local/icinga-web/app/modules/inGraph/config/views/

I have a lot of trouble dealing with the UI when cloning a graph, and modify a hostname, since this is not how the perfdata is stored to be pivoted in the ingraph DB.

Instead, what I do is make one graph, then edit the json files located within ./views/ adding the additional hosts by copy and pasting the necessary json.

nxlog: configure pattern matching for Windows Events

November 8, 2012 Leave a comment
November 8th, 2012: It isn’t possible to do this with pm_pattern, but only with if and Exec. Go to the bottom section (“Nopenopenope”) to see the correct way to block on multiple condition.

 

In this example, I am using the event source/input module im_mseventlog. You could use another input module and the process would be relatively the same.
Read more…

Configuring argus clients to have access to country code

July 3, 2012 4 comments
updated June 2013: This article is pretty poorly written and organized. I apologize before hand and I will be re-writing it soon.

 

Carter has considered a few methods for dealing with IP location information. If you are looking for simply the country of origin where the IP is allocated, you can use the ARIN database lookup method. If you require more granular information about addresses, including city/region/lat/long info, you must rely on MaxMind’s GeoIP databases (as the ARIN database does not contain this information).

I will cover both of these methods.

I suggest you review Configuring argus clients with .conf and .rarc before continuing…

Read more…

%d bloggers like this: