Before we begin here, realize that there are a number of moving parts with any IDS/IPS deployment. Understanding the basics of networking (OSI/TCPIP stacks, collision domains and/or broadcast domains) really help you in understanding why snort can or cannot see traffic for certain parts of your network. If you have no way of copying traffic from the network segment you want to protect, then snort will not be able to see anything.
Now that I got that part out of the way, there are a couple of basic components in any snort deployment:
1) the network
How busy is your network? If the network isn't particularly active, it may take a while before you see any sort of alerts from snort.
2) the method you are using to pass network traffic to snort
How are you copying traffic to snort's sniffing interface? Do you have an old hub? a professional grade network tap or hobbyist network tap? A switch that can do port mirroring/span ports? Is snort sitting in a VM? does that virtual machine have a way to sniff traffic off of the vswitch?
Any IDS/IPS deployment needs a reliable method to clone/copy traffic off of the network. If you plug snort into a regular switch port or SOHO router, you're usually only going to see broadcast traffic (usually UDP broadcasts, CDP broadcasts, Spanning tree traffic, and ARP requests) and will be absolutely useless to you.
You need to have the equipment or means to copy traffic to snort somehow.
3) Snort itself and how it's configured
If you have a light ruleset (say the balanced ruleset from pulledpork, or the connectivity over security ruleset), you won't be seeing very many alerts, unless those rules are a really big deal, like big enough to be included in those rulesets, and/or if your network is particularly infested with bad traffic.
4) Barnyard2, how it's configured and/or the database it is logging to
Autosnort is configured to use barnyard2 and log to a local database. normally this is the "snort" database, but some web interfaces require their own database (e.g. Aanval uses the aanval database and snort database, snorby uses the snorby database). Does barnyard2's database user have the right access to insert events to the database? does the database user have the right credentials?
For other barnyard2 logging options, such as syslog, is the remote host allowing connections from your sensor's management interface? is the SIEM solution configured to log syslog events properly?
So, to summarize IDS/IPS deployments have a number of parts that have to work together to provide you with a working solution: The network itself, the method you are using to copy network traffic to snort, snort or the IDS/IPS solution you are using and what its configured to alert on, and the method by which you are logging intrusion events (barnyard2/database/syslog).
Here are some basic troubleshooting methods I've devised:
Basic Steps:
ps -ef | grep snort
ps -ef | grep delayed
you are looking for this output from ps -ef | grep snort:
root 1853 1258 0 17:54 pts/0 00:00:00 grep snort
snort 1883 1 0 May18 ? 00:00:47 /usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.
root 1912 1 0 May18 ? 00:00:53 /usr/local/bin/barnyard2 -c /usr/local/snort/etc/
root 1853 1258 0 17:54 pts/0 00:00:00 grep snort
snort 1883 1 0 May18 ? 00:00:47 /usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.
root 1912 1 0 May18 ? 00:00:53 /usr/local/bin/barnyard2 -c /usr/local/snort/etc/
you are looking for this output from ps -ef | grep delayed:
root 1869 1258 0 17:55 pts/0 00:00:00 grep delayed
root 2161 1 0 May18 ? 00:11:01 delayed_job
If snort and barnyard2 BOTH are not running, check /var/log/messages (cat /var/log/messages [or /var/log/syslog depending on your distro] | grep snort) to see if you can determine why. Look for error, critical or fatal messages as they indicate reasons why snort or barnyard2 refused to run. We'll talk about snorby's delayed_job process in a moment.
-- which interface is snort sniffing traffic on? is it in promiscuous mode?
You're looking for output that looks something like this
on the interface that snort is sniffing traffic on, pay attention to the
lines in bold face:
eth1 Link encap:Ethernet HWaddr 00:0c:29:8b:bb:ca
inet6 addr: fe80::20c:29ff:fe8b:bbca/64 Scope:Link
UP BROADCAST RUNNING NOARP PROMISC MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 00:0c:29:8b:bb:ca
inet6 addr: fe80::20c:29ff:fe8b:bbca/64 Scope:Link
UP BROADCAST RUNNING NOARP PROMISC MTU:1500 Metric:1
The most important lines above are UP RUNNING PROMISC, because the interface must be up, must be running, and likely should be in promiscuous mode to pick up traffic. the NOARP line isn't entirely necessary, but it's there to ensure that the sniffing interface doesn't respond to arp queries in case attackers are probing the network to try and find your IDS installations.
If you're in a corporate environment, and are a proficient network administrator, I'm going to assume you know what a switch span port is. These are also referred to as mirror ports. Most professional grade switches have an option to copy ALL traffic seen on a switch to a single interface. You would plug snort's sniffing interface into this mirror interface for snort to see traffic for that entire network segment.
If you're a security reasearcher and don't have money for a procurve or cisco switch that does port mirroring, here's some awesome alternatives:
http://routerboard.com/RB260GS
This is a 5-port switch from microtik. Very very affordable and capable of port mirroring.
Alternatively, if you want a bigger name-brand switch, I personally use netgear's GS108T switch for my home network, since it was the cheapest switch I was aware at the time that allowed port mirroring:
http://www.newegg.com/Product/Product.aspx?Item=33-122-381&IsVirtualParent=1
An alternative to span/mirror ports are network taps. On the professional/corporate side, Net Optics, Gigamon and several other vendors offer network tap solutions. Often ridiculously priced, but without all of the possible problems that come with setting of span/mirror ports. These devices are solely dedicated to creating exact clones of network traffic passing over the wire and nothing else.
For those on the cheap side, there are several results via google on how to make your own network tap, but be aware that these sorts of taps are usually for 10/100 networks and usually do not support tapping gigabit networks.
Finally for vmware environments, I posted a specialized tutorial on how to "hack" vmware player on how to bridge traffic from one physical interface or another, using the vmnetconfig.exe from vmware workstation (check the pdf document available in my autosnort github if you're interested...)
But for professional deployments or network labs running vmware ESX/ESXi, it's pretty easy to configure a vswitch to allow promiscuous mode nics. That's really all you have to do is change vswitch security settings, allow promiscuous mode NICs and just configure the snort VM to sniff off of that vswitch.
4) run tcpdump on snort's sniffing interface:
tcpdump -i ethX not arp and not port 22
X is the ethernet interface number snort is sniffing traffic on. the "not arp and not port 22" is telling tcpdump to ignore ssh traffic and arp broadcasts.
while tcpdump is running, on another system on the network, try doing something to generate network traffic (ping google.com,
go to a web page, etc.). did tcpdump see your traffic? If not, you may
need to investigate how you're mirroring traffic to snort.
5) Check and see if the snort.u2 file is greater than 0 bytes:
ls -al /var/log/snort
check the output of the ls command. You're looking for the snort.u2 file, it should look something like this:
-rw------- 1 snort snort 67261 May 19 17:39 snort.u2.1368917933
the
numbr "67261" in the middle is the file's size. If your snort.u2 file
is 0 bytes, that means snort hasn't generated any alerts.-rw------- 1 snort snort 67261 May 19 17:39 snort.u2.1368917933
So those are the basic steps. You confirm that snort is
running, confirm that the interface is in promisc mode, and confirm that
mirroring device is actually pushing traffic to where snort can see it, and confirm snort is logging events.
Snorby-specific troubleshooting steps:
Here's a couple of things you can
try to confirm that barnyard2 and snorby are working together:
1) ps -ef | grep delayed
1) ps -ef | grep delayed
- You are looking for an entry that looks something like this:
root 2161 1 0 May18 ? 00:11:00 delayed_job
root 2161 1 0 May18 ? 00:11:00 delayed_job
if the delayed_job process is NOT running, run this command:
cd /var/www/snorby && ruby script/delayed_job start
cd /var/www/snorby && ruby script/delayed_job start
then run this command to see if the sensor_cache needs to be updated:
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.
note: I do not know if you need root permission to run
the commands above, but if the above commands fail without root
privileges, run the command:
sudo su -
sudo su -
to get a shell with root privileges and re-run those commands.
2) you'll want to check the contents of barnyard2.conf and snorby's database.yml to make sure they match:
cat /usr/local/snort/etc/
on the log mysql line, check these values and make sure they match:
log,mysql, user=snort password=[password you gave the snort database user] dbname=snorby host=localhostcat /var/www/snorby/config/database.yml
verify these lines in database.yml:
snorby: &snorby
adapter: mysql
username: snort
password: [same password in the barnyard2.conf file]
host: localhost
development:
database: snorby
<<: *snorby
test:
database: snorby
<<: *snorby
production:
database: snorby
<<: *snorby
the most important lines to check are the dbname: and database: line under production are the same database, and that the password= and password: lines are the same to make sure that barnyard2 and snorby agree on the correct password to use to log to and read from that same database.
If the database lines and password lines match, at the very least, we know that barnyard2 and snorby are talking to the right database.
3) One last check you can perform is to verify that the snort user has the privileges to talk to the snorby database. run this command:
mysql -usnort -p[snort database user's password, no spaces] snorby -e "show tables;"
your output should look like this:
+-------------------+
| Tables_in_snorby |
+-------------------+
| agent_asset_names |
| aggregated_events |
| asset_names |
| caches |
| classifications |
| data |
| delayed_jobs |
| detail |
| encoding |
| event |
| events_with_join |
| favorites |
| icmphdr |
| iphdr |
| lookups |
| notes |
| notifications |
| opt |
| reference |
| reference_system |
| schema |
| search |
| sensor |
| settings |
| severities |
| sig_class |
| sig_reference |
| signature |
| tcphdr |
| udphdr |
| users |
+-------------------+
+-------------------+
| Tables_in_snorby |
+-------------------+
| agent_asset_names |
| aggregated_events |
| asset_names |
| caches |
| classifications |
| data |
| delayed_jobs |
| detail |
| encoding |
| event |
| events_with_join |
| favorites |
| icmphdr |
| iphdr |
| lookups |
| notes |
| notifications |
| opt |
| reference |
| reference_system |
| schema |
| search |
| sensor |
| settings |
| severities |
| sig_class |
| sig_reference |
| signature |
| tcphdr |
| udphdr |
| users |
+-------------------+
This confirms that the snorby database exists and that the schema was configured properly during installation.
4) If the snort.u2 file in /var/log/snort is
NOT 0 bytes, and you confirmed the snort database user was able to see
the tables in the snorby database, there's one last set of commands you
can run:
mysql -usnort -p[snort database user's password] snorby -e "select * from event\G;"
if
you get "empty set" as the return value from that command, snorby
doesn't have anything. If you get a bunch of data from this command,
snorby should have events in the "events" tab on the web interface.
you can also run this command:
mysql -usnort -p[snort database user's password] snorby -e "select count(*) from event;"
you should get something like this:
+----------+
| count(*) |
+----------+
| 5284 |
+----------+
I
have a boatload of events, because I have a testing suite that causes
snort to trigger alerts like crazy, but basically, you're looking for
the count number to NOT be 0. If the count is 0 there's no events for
snorby to report on.+----------+
| count(*) |
+----------+
| 5284 |
+----------+
Hope this helps you all out
Cheers,
DA_667
