So, this isn't strictly snort or autosnort related, but I figured I would post it anyhow. I primarily use SSH to manage my autosnort testing sensors in my virtual environment, but I also use vnc over SSH to manage my Attack system (A Kali Linux box -- was formerly Backtrack 5 r3) and thought I would share my experience on how to set up vnc server over SSH and have it look somewhat decent.
I've found a blog post here and a blog post there about how to tunnel VNC over ssh, but nothing really comprehensive. So I thought I'd share my knowledge on the bits of wisdom I've found strewn all over the net. If you have anything to contribute, or anything to add, please do so.
To begin, VNC is old and insecure; It encrypts absolutely nothing. This is why we're going to wrap it over SSH. In addition to a lack of encryption, VNC4's password functionality only allows 8-character passwords. This means the password can be broken fairly easily, even with a random password.
Tunneling VNC over SSH allows you to use stronger authentication (keys or huge long passwords, or both) in additional to the VNC password, setting up a basic form of multifactor authentication.
Most debian-based distro package sources include vnc server. It can be installed via:
apt-get install vnc4server
Afterwards, you will need to run vnc4server manually or vnc4passwd to specify a password for the current user for using VNC. again, the password can only be 8 chars long.
Additionally, in the user's .vnc folder (e.g. ~/.vnc or /home/[user]/.vnc/ ) is a file entitled xstartup
xstartup must be executable, and looks something like this:
#!/bin/sh
# Uncomment the following two lines for normal desktop:
#unset SESSION_MANAGER
#exec /etc/X11/xinit/xinitrc
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
#xsetroot -solid grey
#vncconfig -iconic &
#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &
Change the file to look something like this:
#!/bin/sh
# Uncomment the following two lines for normal desktop:
unset SESSION_MANAGER
exec /etc/X11/xinit/xinitrc
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
#xsetroot -solid grey
#vncconfig -iconic &
#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &
startx &;
if the .vnc directory doesn't exist, try running "vnc4server" to start up vnc. This should give you a .vnc directory and the file above. After starting it, immediately kill it via kill -9 or killall vnc4server.
Next, we need to make an init script to start up vnc at boot. I found this init script via
http://www.isnull.com.ar/2010/03/vnc4server-boot-script-working-in.html
I've made some minor improvements, however. Copy these contents to /etc/init.d/vncserver:
# /etc/init.d/vnc4server
#
# Some things that run always
touch /var/lock/vnc4server
# Carry out specific functions when asked to by the system
case "$1" in
start)
echo "Starting script vnc4server "
su root -c 'vnc4server -depth 24 -pixelformat RGB888 -localhost -geometry 1440x900 '
;;
stop)
echo "Stopping script vnc4server"
su root -c 'killall Xvnc4'
;;
*)
echo "Usage: /etc/init.d/vnc4server {start|stop}"
exit 1
;;
esac
exit 0
The line that is doing the all the work of running the vnc server is this line:
vnc4server -depth 24 -pixelformat RGB888 -localhost -geometry 1440x900
This line is essentially saying "start the vnc server with 24-bit color depth, with this RGB mapping and a resolution of 1440x900, and only allow it to listen on 127.0.0.1" (This last part is important! do not forget the -localhost line!) You can change the -geometry line to any resolution you wish, but I had to figure out the -depth and -pixelformat lines on my own, and mainly through trial and error.
next you'll have to make the file executable (chmod u+x /etc/init.d/vncserver)
after making the file executable, run:
update-rc.d vncserver defaults
or
update-rc.d vncserver start [runlevels you want the server to run on]
finally, the file make the xinitrc file executable. I've found that vnc won't work properly unless this file is executable:
chmod u+x /etc/X11/xinit/xinitrc
The higher the bit depth and resolution, the more packets you need to be able to push over the network (and your SSH tunnel) to keep things running smoothly. For this reason, I recommend that if you're going to use this configuration that it be over a high-speed link, or a local network. try changing the depth to 16, or making the resolution lower if you find the VNC session to be laggy.
So to summarize what we just did above we:
installed vnc4server
set up a vnc password for your user
made an init script for vnc4server
made all necessary preparations to host vnc4 server.
In this section I'll show you how to SSH tunnel to your VNC server either from windows or linux.
We'll start with windows. I use an awesome program called MRemoteNG for managing remote connectivity via a number of remote protocols -- VNC, RDP, SSH, .ICA (citrix), HTTP/HTTPS (xulrunner-- a bit buggy), etc. I can't stress how awesome MRemoteNG is for managing multiple connections. For those worried about having the app store connection information, it also allows you to encrypt the connection configuration files to keep your creds safe.
Download and Install MRemoteNG. Open it up and click Tools::Options:: and click advanced. Click "Launch Putty"
Here is where we're going to configure the portforwarding settings. Under Category, go down to Connections, click the plus sign on SSH, then click tunnels.
This page looks confusing, but it's really simple.
The only things you need to modify are
Source Port: set this to any arbitrary port number you want. Doesn't really matter. Just remember what it is. If you can't remember, you can always come back to this page.
For destination type this in exactly: 127.0.0.1:5901
The more intrepid may ask "uh.. why am I putting localhost in as the destination?"
Because this is localhost on the REMOTE server (e.g. the system where VNC is listening on loopback (127.0.0.1) only. So what this tunnel is saying is:
"After this user has successfully authenticated to me, have the client open up source port:[source port you choose]. If you get a connection on [source port you chose], funnel it over port 22. When it reaches the remote end, connect it to 127.0.0.1:5901 where the VNC server is listening."
Save your putty session by scrolling up, clicking on sessions, putting in a name under "saved sessions" then clicking save. Afterwards, you can close putty.
You'll need to create two mremoteng sessions here:
the first Mremote session, select ssh version tool as the protocol, enter the IP address (user name and/or password if you want) and under PuTTY session, click the drop down and save the session you made.
the second Mremote session, you'll choose VNC as the protocol. Enter the IP address "127.0.0.1" and the source port that you entered when you created the putty session above. For the password, enter the password you entered when you ran vnc4passwd or vnc4server previously.
Next, double click on your ssh session in mremoteng, then double click on your localhost VNC session. You should have your remote system's desktop.
If you're doing this over Linux, it's not that much harder to do. Make sure your linux connecting to the remote SSH (and VNC) server has an ssh client and a vncviewer application of some sort.
Open up a terminal window and type:
ssh -L 5901:localhost:5901 username@[hostname/ipaddress]
The -L line above is the secret sauce and works exactly the same as the what the putty session does:
[Source Port to listen on locally]:[target remote ip address]:[remote port to connect to]
So we tell SSH to open a listener on 5901 if your local workstation gets a connection on 5901, forward it over the SSH connection and send it to 5901 on the remote server's loopback address.
Next, just tell your vnc client to connect to 127.0.0.1:5901 and you're done.
Note: Some of the more intrepid among you may notice that there's now a listener on 6001/tcp. I've searched long and hard for the reasoning behind this. 6001/tcp is the default port for the first X server screen to use when X is told to listen for remote X client connections.
Some of you are probably freaking out. you don't want your server to allow connections to X windows. Don't freak out just yet; Sure enough, if you get NMAP scanned, NMAP will say the port is open. Netcat will connect and promptly get dropped. PuTTY RAW sessions stay open (probably because the remote X server has no idea what the heck the PuTTY RAW session is trying to do.) No remote X clients will be able to connect to the x server. While you are on your vnc session, If you type "xhost" in a terminal window "access control enabled, only authorized clients can connect" should be the response you get. In the olden days, way before even my time, No one thought to tunnel X over SSH. so they would use xhost to authorize remote users to connect to that system's X windows server. The format was usually xhost +{ip address} to allow a remote ip address to connect to your X server. Even more common was for most to just type xhost + (no ip address). This is the same as saying "Allow anyone to connect to my X server". You can probably imagine why this isn't a good idea.
So why am I telling you about this? Because by default, your X server (the one that VNC uses), for some stupid reason has your X11 X windows server listen on 6001/tcp. From what googling I've done, there's no reason for this, and there's no way to configure X (without modifying the source code) to just listen on localhost. I've tried to find a solution that isn't "turn on your firewall, or just configure firewall rules to block it." That's a work-around, not a solution (A viable work-around, by the way, but not the point.) and I haven't find any (other than the just a firewall rule)
For those of you who aren't firewall gurus, you're in luck. by default, xhost doesn't authorize ANY remote users at all to connect to your X server. That means, as soon as anyone tries to connect to your X server, the session is dropped by X's own authorization system. The bottom line is: Don't freak out; no one is allowed to connect to it. You're safe. But if you know how to make X listen on 127.0.0.1 only, I'd be very interested...
Final Note: On some sshd servers, TCP Forwarding is disabled. TCP forwarding is the magic by which all this tunneling happens. Meaning if it's disabled, then none of this works. Check /etc/ssh/sshd_config if you see the line "AllowTcpForwarding no" it means tcp forward is turned off. Conversely, if it is set to yes, then you're good to go. If you don't see this line in your sshd_config file, then you're good to go as well. It's implicitly allowed by default.
Saturday, June 1, 2013
Friday, May 24, 2013
Troubleshooting your Snort deployment
I recently had a discussion with a user new to snort and autosnort who
wanted to know some troubleshooting techniques that can be done to see
if snort is working and/or snorby or the web interface of your choosing
is working. After thinking about it, I decided it might be a good idea
to post my troubleshooting steps here to assist users of snort and
snorby in general to determine if things are working okay.
Before we begin here, realize that there are a number of moving parts with any IDS/IPS deployment. Understanding the basics of networking (OSI/TCPIP stacks, collision domains and/or broadcast domains) really help you in understanding why snort can or cannot see traffic for certain parts of your network. If you have no way of copying traffic from the network segment you want to protect, then snort will not be able to see anything.
Now that I got that part out of the way, there are a couple of basic components in any snort deployment:
1) the network
How busy is your network? If the network isn't particularly active, it may take a while before you see any sort of alerts from snort.
2) the method you are using to pass network traffic to snort
How are you copying traffic to snort's sniffing interface? Do you have an old hub? a professional grade network tap or hobbyist network tap? A switch that can do port mirroring/span ports? Is snort sitting in a VM? does that virtual machine have a way to sniff traffic off of the vswitch?
Any IDS/IPS deployment needs a reliable method to clone/copy traffic off of the network. If you plug snort into a regular switch port or SOHO router, you're usually only going to see broadcast traffic (usually UDP broadcasts, CDP broadcasts, Spanning tree traffic, and ARP requests) and will be absolutely useless to you.
You need to have the equipment or means to copy traffic to snort somehow.
3) Snort itself and how it's configured
If you have a light ruleset (say the balanced ruleset from pulledpork, or the connectivity over security ruleset), you won't be seeing very many alerts, unless those rules are a really big deal, like big enough to be included in those rulesets, and/or if your network is particularly infested with bad traffic.
4) Barnyard2, how it's configured and/or the database it is logging to
Autosnort is configured to use barnyard2 and log to a local database. normally this is the "snort" database, but some web interfaces require their own database (e.g. Aanval uses the aanval database and snort database, snorby uses the snorby database). Does barnyard2's database user have the right access to insert events to the database? does the database user have the right credentials?
For other barnyard2 logging options, such as syslog, is the remote host allowing connections from your sensor's management interface? is the SIEM solution configured to log syslog events properly?
So, to summarize IDS/IPS deployments have a number of parts that have to work together to provide you with a working solution: The network itself, the method you are using to copy network traffic to snort, snort or the IDS/IPS solution you are using and what its configured to alert on, and the method by which you are logging intrusion events (barnyard2/database/syslog).
Here are some basic troubleshooting methods I've devised:
1) are snort processes running? If you're using Snorby or another specialized logging interface, are the processes that interface relies on running?
ps -ef | grep snort
you are looking for this output from ps -ef | grep delayed:
root 1869 1258 0 17:55 pts/0 00:00:00 grep delayed
root 2161 1 0 May18 ? 00:11:01 delayed_job
2) ifconfig -a
-- which interface is snort sniffing traffic on? is it in promiscuous mode?
3)
How are you passing traffic to snort? is snort's sniffing interface
attached to a switch span? a hub? a network tap? if it's a virtual
machine on ESXi or something of that nature, is the vswitch snort is
sniffing off of configured to allow promiscuous mode network cards?
If you're in a corporate environment, and are a proficient network administrator, I'm going to assume you know what a switch span port is. These are also referred to as mirror ports. Most professional grade switches have an option to copy ALL traffic seen on a switch to a single interface. You would plug snort's sniffing interface into this mirror interface for snort to see traffic for that entire network segment.
If you're a security reasearcher and don't have money for a procurve or cisco switch that does port mirroring, here's some awesome alternatives:
http://routerboard.com/RB260GS
This is a 5-port switch from microtik. Very very affordable and capable of port mirroring.
Alternatively, if you want a bigger name-brand switch, I personally use netgear's GS108T switch for my home network, since it was the cheapest switch I was aware at the time that allowed port mirroring:
http://www.newegg.com/Product/Product.aspx?Item=33-122-381&IsVirtualParent=1
An alternative to span/mirror ports are network taps. On the professional/corporate side, Net Optics, Gigamon and several other vendors offer network tap solutions. Often ridiculously priced, but without all of the possible problems that come with setting of span/mirror ports. These devices are solely dedicated to creating exact clones of network traffic passing over the wire and nothing else.
For those on the cheap side, there are several results via google on how to make your own network tap, but be aware that these sorts of taps are usually for 10/100 networks and usually do not support tapping gigabit networks.
Finally for vmware environments, I posted a specialized tutorial on how to "hack" vmware player on how to bridge traffic from one physical interface or another, using the vmnetconfig.exe from vmware workstation (check the pdf document available in my autosnort github if you're interested...)
But for professional deployments or network labs running vmware ESX/ESXi, it's pretty easy to configure a vswitch to allow promiscuous mode nics. That's really all you have to do is change vswitch security settings, allow promiscuous mode NICs and just configure the snort VM to sniff off of that vswitch.
4) run tcpdump on snort's sniffing interface:
tcpdump -i ethX not arp and not port 22
X is the ethernet interface number snort is sniffing traffic on. the "not arp and not port 22" is telling tcpdump to ignore ssh traffic and arp broadcasts.
cat /var/www/snorby/config/database.yml
verify these lines in database.yml:
snorby: &snorby
adapter: mysql
username: snort
password: [same password in the barnyard2.conf file]
host: localhost
development:
database: snorby
<<: *snorby
test:
database: snorby
<<: *snorby
production:
database: snorby
<<: *snorby
the most important lines to check are the dbname: and database: line under production are the same database, and that the password= and password: lines are the same to make sure that barnyard2 and snorby agree on the correct password to use to log to and read from that same database.
If the database lines and password lines match, at the very least, we know that barnyard2 and snorby are talking to the right database.
3) One last check you can perform is to verify that the snort user has the privileges to talk to the snorby database. run this command:
mysql -usnort -p[snort database user's password, no spaces] snorby -e "show tables;"
Hope this helps you all out
Cheers,
DA_667
Before we begin here, realize that there are a number of moving parts with any IDS/IPS deployment. Understanding the basics of networking (OSI/TCPIP stacks, collision domains and/or broadcast domains) really help you in understanding why snort can or cannot see traffic for certain parts of your network. If you have no way of copying traffic from the network segment you want to protect, then snort will not be able to see anything.
Now that I got that part out of the way, there are a couple of basic components in any snort deployment:
1) the network
How busy is your network? If the network isn't particularly active, it may take a while before you see any sort of alerts from snort.
2) the method you are using to pass network traffic to snort
How are you copying traffic to snort's sniffing interface? Do you have an old hub? a professional grade network tap or hobbyist network tap? A switch that can do port mirroring/span ports? Is snort sitting in a VM? does that virtual machine have a way to sniff traffic off of the vswitch?
Any IDS/IPS deployment needs a reliable method to clone/copy traffic off of the network. If you plug snort into a regular switch port or SOHO router, you're usually only going to see broadcast traffic (usually UDP broadcasts, CDP broadcasts, Spanning tree traffic, and ARP requests) and will be absolutely useless to you.
You need to have the equipment or means to copy traffic to snort somehow.
3) Snort itself and how it's configured
If you have a light ruleset (say the balanced ruleset from pulledpork, or the connectivity over security ruleset), you won't be seeing very many alerts, unless those rules are a really big deal, like big enough to be included in those rulesets, and/or if your network is particularly infested with bad traffic.
4) Barnyard2, how it's configured and/or the database it is logging to
Autosnort is configured to use barnyard2 and log to a local database. normally this is the "snort" database, but some web interfaces require their own database (e.g. Aanval uses the aanval database and snort database, snorby uses the snorby database). Does barnyard2's database user have the right access to insert events to the database? does the database user have the right credentials?
For other barnyard2 logging options, such as syslog, is the remote host allowing connections from your sensor's management interface? is the SIEM solution configured to log syslog events properly?
So, to summarize IDS/IPS deployments have a number of parts that have to work together to provide you with a working solution: The network itself, the method you are using to copy network traffic to snort, snort or the IDS/IPS solution you are using and what its configured to alert on, and the method by which you are logging intrusion events (barnyard2/database/syslog).
Here are some basic troubleshooting methods I've devised:
Basic Steps:
ps -ef | grep snort
ps -ef | grep delayed
you are looking for this output from ps -ef | grep snort:
root 1853 1258 0 17:54 pts/0 00:00:00 grep snort
snort 1883 1 0 May18 ? 00:00:47 /usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.
root 1912 1 0 May18 ? 00:00:53 /usr/local/bin/barnyard2 -c /usr/local/snort/etc/
root 1853 1258 0 17:54 pts/0 00:00:00 grep snort
snort 1883 1 0 May18 ? 00:00:47 /usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.
root 1912 1 0 May18 ? 00:00:53 /usr/local/bin/barnyard2 -c /usr/local/snort/etc/
you are looking for this output from ps -ef | grep delayed:
root 1869 1258 0 17:55 pts/0 00:00:00 grep delayed
root 2161 1 0 May18 ? 00:11:01 delayed_job
If snort and barnyard2 BOTH are not running, check /var/log/messages (cat /var/log/messages [or /var/log/syslog depending on your distro] | grep snort) to see if you can determine why. Look for error, critical or fatal messages as they indicate reasons why snort or barnyard2 refused to run. We'll talk about snorby's delayed_job process in a moment.
-- which interface is snort sniffing traffic on? is it in promiscuous mode?
You're looking for output that looks something like this
on the interface that snort is sniffing traffic on, pay attention to the
lines in bold face:
eth1 Link encap:Ethernet HWaddr 00:0c:29:8b:bb:ca
inet6 addr: fe80::20c:29ff:fe8b:bbca/64 Scope:Link
UP BROADCAST RUNNING NOARP PROMISC MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 00:0c:29:8b:bb:ca
inet6 addr: fe80::20c:29ff:fe8b:bbca/64 Scope:Link
UP BROADCAST RUNNING NOARP PROMISC MTU:1500 Metric:1
The most important lines above are UP RUNNING PROMISC, because the interface must be up, must be running, and likely should be in promiscuous mode to pick up traffic. the NOARP line isn't entirely necessary, but it's there to ensure that the sniffing interface doesn't respond to arp queries in case attackers are probing the network to try and find your IDS installations.
If you're in a corporate environment, and are a proficient network administrator, I'm going to assume you know what a switch span port is. These are also referred to as mirror ports. Most professional grade switches have an option to copy ALL traffic seen on a switch to a single interface. You would plug snort's sniffing interface into this mirror interface for snort to see traffic for that entire network segment.
If you're a security reasearcher and don't have money for a procurve or cisco switch that does port mirroring, here's some awesome alternatives:
http://routerboard.com/RB260GS
This is a 5-port switch from microtik. Very very affordable and capable of port mirroring.
Alternatively, if you want a bigger name-brand switch, I personally use netgear's GS108T switch for my home network, since it was the cheapest switch I was aware at the time that allowed port mirroring:
http://www.newegg.com/Product/Product.aspx?Item=33-122-381&IsVirtualParent=1
An alternative to span/mirror ports are network taps. On the professional/corporate side, Net Optics, Gigamon and several other vendors offer network tap solutions. Often ridiculously priced, but without all of the possible problems that come with setting of span/mirror ports. These devices are solely dedicated to creating exact clones of network traffic passing over the wire and nothing else.
For those on the cheap side, there are several results via google on how to make your own network tap, but be aware that these sorts of taps are usually for 10/100 networks and usually do not support tapping gigabit networks.
Finally for vmware environments, I posted a specialized tutorial on how to "hack" vmware player on how to bridge traffic from one physical interface or another, using the vmnetconfig.exe from vmware workstation (check the pdf document available in my autosnort github if you're interested...)
But for professional deployments or network labs running vmware ESX/ESXi, it's pretty easy to configure a vswitch to allow promiscuous mode nics. That's really all you have to do is change vswitch security settings, allow promiscuous mode NICs and just configure the snort VM to sniff off of that vswitch.
4) run tcpdump on snort's sniffing interface:
tcpdump -i ethX not arp and not port 22
X is the ethernet interface number snort is sniffing traffic on. the "not arp and not port 22" is telling tcpdump to ignore ssh traffic and arp broadcasts.
while tcpdump is running, on another system on the network, try doing something to generate network traffic (ping google.com,
go to a web page, etc.). did tcpdump see your traffic? If not, you may
need to investigate how you're mirroring traffic to snort.
5) Check and see if the snort.u2 file is greater than 0 bytes:
ls -al /var/log/snort
check the output of the ls command. You're looking for the snort.u2 file, it should look something like this:
-rw------- 1 snort snort 67261 May 19 17:39 snort.u2.1368917933
the
numbr "67261" in the middle is the file's size. If your snort.u2 file
is 0 bytes, that means snort hasn't generated any alerts.-rw------- 1 snort snort 67261 May 19 17:39 snort.u2.1368917933
So those are the basic steps. You confirm that snort is
running, confirm that the interface is in promisc mode, and confirm that
mirroring device is actually pushing traffic to where snort can see it, and confirm snort is logging events.
Snorby-specific troubleshooting steps:
Here's a couple of things you can
try to confirm that barnyard2 and snorby are working together:
1) ps -ef | grep delayed
1) ps -ef | grep delayed
- You are looking for an entry that looks something like this:
root 2161 1 0 May18 ? 00:11:00 delayed_job
root 2161 1 0 May18 ? 00:11:00 delayed_job
if the delayed_job process is NOT running, run this command:
cd /var/www/snorby && ruby script/delayed_job start
cd /var/www/snorby && ruby script/delayed_job start
then run this command to see if the sensor_cache needs to be updated:
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.
note: I do not know if you need root permission to run
the commands above, but if the above commands fail without root
privileges, run the command:
sudo su -
sudo su -
to get a shell with root privileges and re-run those commands.
2) you'll want to check the contents of barnyard2.conf and snorby's database.yml to make sure they match:
cat /usr/local/snort/etc/
on the log mysql line, check these values and make sure they match:
log,mysql, user=snort password=[password you gave the snort database user] dbname=snorby host=localhostcat /var/www/snorby/config/database.yml
verify these lines in database.yml:
snorby: &snorby
adapter: mysql
username: snort
password: [same password in the barnyard2.conf file]
host: localhost
development:
database: snorby
<<: *snorby
test:
database: snorby
<<: *snorby
production:
database: snorby
<<: *snorby
the most important lines to check are the dbname: and database: line under production are the same database, and that the password= and password: lines are the same to make sure that barnyard2 and snorby agree on the correct password to use to log to and read from that same database.
If the database lines and password lines match, at the very least, we know that barnyard2 and snorby are talking to the right database.
3) One last check you can perform is to verify that the snort user has the privileges to talk to the snorby database. run this command:
mysql -usnort -p[snort database user's password, no spaces] snorby -e "show tables;"
your output should look like this:
+-------------------+
| Tables_in_snorby |
+-------------------+
| agent_asset_names |
| aggregated_events |
| asset_names |
| caches |
| classifications |
| data |
| delayed_jobs |
| detail |
| encoding |
| event |
| events_with_join |
| favorites |
| icmphdr |
| iphdr |
| lookups |
| notes |
| notifications |
| opt |
| reference |
| reference_system |
| schema |
| search |
| sensor |
| settings |
| severities |
| sig_class |
| sig_reference |
| signature |
| tcphdr |
| udphdr |
| users |
+-------------------+
+-------------------+
| Tables_in_snorby |
+-------------------+
| agent_asset_names |
| aggregated_events |
| asset_names |
| caches |
| classifications |
| data |
| delayed_jobs |
| detail |
| encoding |
| event |
| events_with_join |
| favorites |
| icmphdr |
| iphdr |
| lookups |
| notes |
| notifications |
| opt |
| reference |
| reference_system |
| schema |
| search |
| sensor |
| settings |
| severities |
| sig_class |
| sig_reference |
| signature |
| tcphdr |
| udphdr |
| users |
+-------------------+
This confirms that the snorby database exists and that the schema was configured properly during installation.
4) If the snort.u2 file in /var/log/snort is
NOT 0 bytes, and you confirmed the snort database user was able to see
the tables in the snorby database, there's one last set of commands you
can run:
mysql -usnort -p[snort database user's password] snorby -e "select * from event\G;"
if
you get "empty set" as the return value from that command, snorby
doesn't have anything. If you get a bunch of data from this command,
snorby should have events in the "events" tab on the web interface.
you can also run this command:
mysql -usnort -p[snort database user's password] snorby -e "select count(*) from event;"
you should get something like this:
+----------+
| count(*) |
+----------+
| 5284 |
+----------+
I
have a boatload of events, because I have a testing suite that causes
snort to trigger alerts like crazy, but basically, you're looking for
the count number to NOT be 0. If the count is 0 there's no events for
snorby to report on.+----------+
| count(*) |
+----------+
| 5284 |
+----------+
Hope this helps you all out
Cheers,
DA_667
Sunday, May 19, 2013
Autosnorby Follow-up: Support to Debian officially released, distributed installations under testing.
Hello again,
As I had mentioned a few blog posts ago updates to autosnort and/or support for snorby were probably going to be staggered out across platforms, since I have to do testing to ensure that things aren't horribly broken on release, that I have the right packages for the right platform that puts the right stuff in the right place, etc.
That being said, porting over the changes to autosnort and support for snorby to Debian went fast. Like, really fast. I should have figured this would be the case since Ubuntu and Debian share similarity in packages installed and package management utilized. It was almost quite literally a copy/paste job -- copy the code from ubuntu autosnort and it its child shell scripts, integrate it in with debian's existing scripts (and some custom things that Debian needs to work right) and just test it to make sure nothing exploded in a fit of errors and incompatibility.
Therefore, it brings me great pleasure to release a new version of autosnort for Debian as well as a snorby child shell script for Debian-based systems, and code improvements to debian autosnort and child shell scripts.
I'm not going to bother going over all of the changes I made to the script(s), since more or less the changes are practically identical to the ubuntu/autosnorby post I made yesterday (e.g. code cleanup, doing things that make sense to make the code easier to manage, installing relevant packages only when they are needed for a particular interface/function, etc.) except to note that I fixed a small bug in the remote database/experimental distributed sensor installation support (In both Ubuntu and Debian)
Speaking of the distributed installation support, I have a rough guide to doing a distributed install. This is very, very rough right now, expect some official or at least "better" guidance soon:
1. Perform a standard autosnort install on your "master console" system, and choose the interface you wish to install. Bear in mind that aanval requires licensing to support more than one snort sensor, and that snort report doesn't really clearly delinate which sensor an event came from. Reboot.
2. Edit /etc/mysql/my.cnf and reconfigure mysql to accept connections globally (e.g. listen on 0.0.0.0; also note that you'd better have a security measure of some sort up to limit connections to your other sensors (e.g. a firewall) if you plan on having this system accessible in *any* way over the internet)
3. run mysql -uroot -p[password here]
snort@'1.2.3.4' should be the snort database user on the remote sensor you plan to have store events on your master console, likewise the IDENTIFIED BY 'my_password'; should be the password you place into barnyard2.conf to talk to the master console.
4. Run the autosnort installer on your barebones sensor (choose option 2 to NOT install apache or mysql), somewhere down the line, the script will ask if you have a remote database that barnyard2 will be logging into. Enter the username (snort), the remote database name (snort), the remote database ip (e.g. the management interface for your 'master console' sensor), and the password you entered above on the mysql statement on the master console
5. finish the autosnort install (when it gets to the what interface do you want to install phase, just select option 6 for no interface at all.)
6. On the barebones sensor, install mysql-client (apt: apt-get install mysql-client) and try to connect to the database on the master console:
mysql -uroot -psnort -h [master console ip address]
use snort;
show tables;
If you get output, and a list of database tables, that's a very good sign. Otherwise you have some troubleshooting to do. note that by default autosnort installs libmysqlclient to allow barnyard2 to talk to the remote database, however, I'm having you install mysql-client in the event you want to troubleshoot and verify that you can talk to the database as a regular user. libmysqlclient won't let you test things out with manual queries, etc.
I can't re-iterate this enough, but allowing mysql to listen globally opens up some security risks (which is why I've trying to move to having the traffic SSL wrapped and tunneled between sensors/consoles...), so I would most definitely make sure the network your barebone sensor and master consoles are talking over is protected, trusted, and your master console host is firewalled to hell and back, only allowing 3306/tcp connections from your barebone sensors. Know the risks before you take this leap.
Check out the autosnort github and happy snorting!
Cheers,
DA_667
As I had mentioned a few blog posts ago updates to autosnort and/or support for snorby were probably going to be staggered out across platforms, since I have to do testing to ensure that things aren't horribly broken on release, that I have the right packages for the right platform that puts the right stuff in the right place, etc.
That being said, porting over the changes to autosnort and support for snorby to Debian went fast. Like, really fast. I should have figured this would be the case since Ubuntu and Debian share similarity in packages installed and package management utilized. It was almost quite literally a copy/paste job -- copy the code from ubuntu autosnort and it its child shell scripts, integrate it in with debian's existing scripts (and some custom things that Debian needs to work right) and just test it to make sure nothing exploded in a fit of errors and incompatibility.
Therefore, it brings me great pleasure to release a new version of autosnort for Debian as well as a snorby child shell script for Debian-based systems, and code improvements to debian autosnort and child shell scripts.
I'm not going to bother going over all of the changes I made to the script(s), since more or less the changes are practically identical to the ubuntu/autosnorby post I made yesterday (e.g. code cleanup, doing things that make sense to make the code easier to manage, installing relevant packages only when they are needed for a particular interface/function, etc.) except to note that I fixed a small bug in the remote database/experimental distributed sensor installation support (In both Ubuntu and Debian)
Speaking of the distributed installation support, I have a rough guide to doing a distributed install. This is very, very rough right now, expect some official or at least "better" guidance soon:
1. Perform a standard autosnort install on your "master console" system, and choose the interface you wish to install. Bear in mind that aanval requires licensing to support more than one snort sensor, and that snort report doesn't really clearly delinate which sensor an event came from. Reboot.
2. Edit /etc/mysql/my.cnf and reconfigure mysql to accept connections globally (e.g. listen on 0.0.0.0; also note that you'd better have a security measure of some sort up to limit connections to your other sensors (e.g. a firewall) if you plan on having this system accessible in *any* way over the internet)
3. run mysql -uroot -p[password here]
GRANT ALL ON snort.* TO snort@'1.2.3.4' IDENTIFIED BY 'my_password';
snort@'1.2.3.4' should be the snort database user on the remote sensor you plan to have store events on your master console, likewise the IDENTIFIED BY 'my_password'; should be the password you place into barnyard2.conf to talk to the master console.
4. Run the autosnort installer on your barebones sensor (choose option 2 to NOT install apache or mysql), somewhere down the line, the script will ask if you have a remote database that barnyard2 will be logging into. Enter the username (snort), the remote database name (snort), the remote database ip (e.g. the management interface for your 'master console' sensor), and the password you entered above on the mysql statement on the master console
5. finish the autosnort install (when it gets to the what interface do you want to install phase, just select option 6 for no interface at all.)
6. On the barebones sensor, install mysql-client (apt: apt-get install mysql-client) and try to connect to the database on the master console:
mysql -uroot -psnort -h [master console ip address]
use snort;
show tables;
If you get output, and a list of database tables, that's a very good sign. Otherwise you have some troubleshooting to do. note that by default autosnort installs libmysqlclient to allow barnyard2 to talk to the remote database, however, I'm having you install mysql-client in the event you want to troubleshoot and verify that you can talk to the database as a regular user. libmysqlclient won't let you test things out with manual queries, etc.
I can't re-iterate this enough, but allowing mysql to listen globally opens up some security risks (which is why I've trying to move to having the traffic SSL wrapped and tunneled between sensors/consoles...), so I would most definitely make sure the network your barebone sensor and master consoles are talking over is protected, trusted, and your master console host is firewalled to hell and back, only allowing 3306/tcp connections from your barebone sensors. Know the risks before you take this leap.
Check out the autosnort github and happy snorting!
Cheers,
DA_667
Saturday, May 18, 2013
Autosnorby, updates to autosnort, and the updated to-do list
Hello Snort and Autosnort users.
I've got a lot to talk about today, so let's get started.
First and foremost, I've finally managed to work in support for snorby into autosnort. The snorby script installs the necessary packages, a compatible version of ruby (via rvm), all the necessary gems and configures the right files with the right info needed to get snorby up and running. Additionally, I've managed to work out away to drop privileges from the root database user to another user (the snort user in our case)
One thing to be aware of: the delayed_job and the sensor cache jobs do NOT start up on boot. Initial runs of the script had me trying to put commands into rc.local to start the delayed_job and sensorcachejob commands on startup... but they aren't working. I'm working on having a reliable method to start them on boot, but until that is discovered, any time the system is restarted, you'll want to, at a minimum start the delayed_job script:
cd /var/www/snorby && ruby script/delayed_job start
optionally, these commands run the Sensor and Daily cachejobs as well:
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
There are options to start the snorby worker process via the web UI, but I find the direct approach to my liking.
Another note is that at first you may notice alerts coming in and displaying properly in the events tab, but the dashboard may not updated at first. If this is the case, on the dashboard page, there is an option to force cache update. Select this option, wait 10 seconds, and reload the page. the graphs should update.
Next up on the list, is the main autosnort script itself. I've done a bit of spring cleaning.
- I've reduced the number of packages installed by default with autosnort.
--By default all we install are the packages necessary to install daq, libdnet, snort, and barnyard 2 with the ability to write to a remote database (more on this in a minute)
--I introduced a quick check that asks the user if they plan on installing a web interface as a part of the autosnort installation, or if they just want a stripped-down barebones install. If you're not sure what option to pick, select option 1.
- I've added a couple of functions to the script near the beginning for pulled pork to facilitate code re-use. this is because:
-- I've added support to download snort rules from previous versions of snort; up to four versions prior.
Why: I ran into a little problem recently where the free rules for version 2.9.4.5 were not available yet, and snort version 2.9.4.6 had just came out. The current version of the script only checks for the current version of snort rules, and one version prior. So now, you have a choice for four versions of rules:
1) the current version (if you have a VRT subscription oink code; e.g. 2.9.4.6)
2) the previous version (the kind that registered users usually get access to; e.g. 2.9.4.5)
3) two more prior versions (in the event that they are needed and/or a similar situation arises)
- If the user has elected to not install a web interface, the script asks them if they have a remote database they want barnyard to log to. I haven't officially tested this feature yet, but theorhetically, this allows for distributed installations... Any volunteers to test this? :)
- The child shell scripts all have functionality to install the packages necessary for them to work individually Again to reduce clutter and excess packages installed by autosnort (e.g. reduce attack surface)
- thanks to DK1844, I have a couple of code cleaniness fixes that I have added, and future todo list items.
I've achieved most of the stuff I intended to do when I initially started this little project, but I still have more that I can do.
Speaking of to-do list items:
- SSL for all web interfaces (generate a default, self-signed SSL cert, use mod-rewrite and mod-ssl to force ssl usage for all web interfaces)
- Further testing with distributed installations (Can I get a master console to register events from a remote system?)
- Encrypted comms between master console and remote sensor (e.g. have barnyard dump database updates to the master console over a secure channel)
- Kali Linux testing! Since Kali is fully compliant with Debian standards, will the debian autosnort script work with it?
The updates to autosnort.sh, snorby, aanval, snortreport and base child shell scripts are being pushed to ubuntu tonight, with support for other operating systems (debian and centOS) to come soon.
Report bugs, problems, praise, feature requests, etc. to:
deusexmachina667 [at] gmail [dot] com
As always the code is on my github repo.
Happy snorting,
DA_667
I've got a lot to talk about today, so let's get started.
First and foremost, I've finally managed to work in support for snorby into autosnort. The snorby script installs the necessary packages, a compatible version of ruby (via rvm), all the necessary gems and configures the right files with the right info needed to get snorby up and running. Additionally, I've managed to work out away to drop privileges from the root database user to another user (the snort user in our case)
One thing to be aware of: the delayed_job and the sensor cache jobs do NOT start up on boot. Initial runs of the script had me trying to put commands into rc.local to start the delayed_job and sensorcachejob commands on startup... but they aren't working. I'm working on having a reliable method to start them on boot, but until that is discovered, any time the system is restarted, you'll want to, at a minimum start the delayed_job script:
cd /var/www/snorby && ruby script/delayed_job start
optionally, these commands run the Sensor and Daily cachejobs as well:
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
There are options to start the snorby worker process via the web UI, but I find the direct approach to my liking.
Another note is that at first you may notice alerts coming in and displaying properly in the events tab, but the dashboard may not updated at first. If this is the case, on the dashboard page, there is an option to force cache update. Select this option, wait 10 seconds, and reload the page. the graphs should update.
Next up on the list, is the main autosnort script itself. I've done a bit of spring cleaning.
- I've reduced the number of packages installed by default with autosnort.
--By default all we install are the packages necessary to install daq, libdnet, snort, and barnyard 2 with the ability to write to a remote database (more on this in a minute)
--I introduced a quick check that asks the user if they plan on installing a web interface as a part of the autosnort installation, or if they just want a stripped-down barebones install. If you're not sure what option to pick, select option 1.
- I've added a couple of functions to the script near the beginning for pulled pork to facilitate code re-use. this is because:
-- I've added support to download snort rules from previous versions of snort; up to four versions prior.
Why: I ran into a little problem recently where the free rules for version 2.9.4.5 were not available yet, and snort version 2.9.4.6 had just came out. The current version of the script only checks for the current version of snort rules, and one version prior. So now, you have a choice for four versions of rules:
1) the current version (if you have a VRT subscription oink code; e.g. 2.9.4.6)
2) the previous version (the kind that registered users usually get access to; e.g. 2.9.4.5)
3) two more prior versions (in the event that they are needed and/or a similar situation arises)
- If the user has elected to not install a web interface, the script asks them if they have a remote database they want barnyard to log to. I haven't officially tested this feature yet, but theorhetically, this allows for distributed installations... Any volunteers to test this? :)
- The child shell scripts all have functionality to install the packages necessary for them to work individually Again to reduce clutter and excess packages installed by autosnort (e.g. reduce attack surface)
- thanks to DK1844, I have a couple of code cleaniness fixes that I have added, and future todo list items.
I've achieved most of the stuff I intended to do when I initially started this little project, but I still have more that I can do.
Speaking of to-do list items:
- SSL for all web interfaces (generate a default, self-signed SSL cert, use mod-rewrite and mod-ssl to force ssl usage for all web interfaces)
- Further testing with distributed installations (Can I get a master console to register events from a remote system?)
- Encrypted comms between master console and remote sensor (e.g. have barnyard dump database updates to the master console over a secure channel)
- Kali Linux testing! Since Kali is fully compliant with Debian standards, will the debian autosnort script work with it?
The updates to autosnort.sh, snorby, aanval, snortreport and base child shell scripts are being pushed to ubuntu tonight, with support for other operating systems (debian and centOS) to come soon.
Report bugs, problems, praise, feature requests, etc. to:
deusexmachina667 [at] gmail [dot] com
As always the code is on my github repo.
Happy snorting,
DA_667
Monday, May 13, 2013
Another quick post - fix to aanval installer script
Hello AS users,
A recent message from an Autosnort user, sm00th brought a problem to my attention regarding the aanval installer script. I've discovered that there is a problem with the wget command used to download the aanval installer. This is what happens when I attempt to wget the file manually:
root@CG:~# wget https://www.aanval.com/download/pickup -O aanval.tar.gz
--2013-05-13 16:13:59-- https://www.aanval.com/download/pickup
Resolving www.aanval.com (www.aanval.com)... 173.160.180.147
Connecting to www.aanval.com (www.aanval.com)|173.160.180.147|:443... connected.
ERROR: no certificate subject alternative name matches
requested host name `www.aanval.com'.
To connect to www.aanval.com insecurely, use `--no-check-certificate'.
There is some sort of a certificate problem with aanval.com. The wget command says to try using --no-check-certificate:
root@CG:~# wget https://www.aanval.com/download/pickup -O aanval.tar.gz --no-check-certificate
--2013-05-13 16:14:27-- https://www.aanval.com/download/pickup
Resolving www.aanval.com (www.aanval.com)... 173.160.180.147
Connecting to www.aanval.com (www.aanval.com)|173.160.180.147|:443... connected.
WARNING: no certificate subject alternative name matches
requested host name `www.aanval.com'.
HTTP request sent, awaiting response... 200 OK
Length: 6703589 (6.4M) [application/octet-stream]
Saving to: `aanval.tar.gz'
100%[=========================================================================================================================================================>] 6,703,589 586K/s in 12s
2013-05-13 16:14:40 (529 KB/s) - `aanval.tar.gz' saved [6703589/6703589]
root@CG:~# echo $?
0
good news: This allows you to pick up the tarball for installing aanval
bad news: The certificate for aanval.com isn't being checked -- this means we're still using SSL to connect to aanval.com and pick up the package, but we aren't checking the certificate to see who signed it, and/or what site it was signed for.
Unfortnately, the problem is out of my hands. I submitted a quick fix to github.com that adds the --no-check-certificate option to allow us to fix the package.
Regards,
DA_667
p.s. Yes, I'm still working on snorby. Hit a bit of a road bump. I thought I had the full installation down, and now, for some reason the snorby database refuses to update. at all. So, as I said before, I want to run through the entire snorby installation process without a single error. Once I can do that... I'll release the Ubuntu snorby script, then Debian, Then probably CentOS last.
Cheers!
A recent message from an Autosnort user, sm00th brought a problem to my attention regarding the aanval installer script. I've discovered that there is a problem with the wget command used to download the aanval installer. This is what happens when I attempt to wget the file manually:
root@CG:~# wget https://www.aanval.com/download/pickup -O aanval.tar.gz
--2013-05-13 16:13:59-- https://www.aanval.com/download/pickup
Resolving www.aanval.com (www.aanval.com)... 173.160.180.147
Connecting to www.aanval.com (www.aanval.com)|173.160.180.147|:443... connected.
ERROR: no certificate subject alternative name matches
requested host name `www.aanval.com'.
To connect to www.aanval.com insecurely, use `--no-check-certificate'.
There is some sort of a certificate problem with aanval.com. The wget command says to try using --no-check-certificate:
root@CG:~# wget https://www.aanval.com/download/pickup -O aanval.tar.gz --no-check-certificate
--2013-05-13 16:14:27-- https://www.aanval.com/download/pickup
Resolving www.aanval.com (www.aanval.com)... 173.160.180.147
Connecting to www.aanval.com (www.aanval.com)|173.160.180.147|:443... connected.
WARNING: no certificate subject alternative name matches
requested host name `www.aanval.com'.
HTTP request sent, awaiting response... 200 OK
Length: 6703589 (6.4M) [application/octet-stream]
Saving to: `aanval.tar.gz'
100%[=========================================================================================================================================================>] 6,703,589 586K/s in 12s
2013-05-13 16:14:40 (529 KB/s) - `aanval.tar.gz' saved [6703589/6703589]
root@CG:~# echo $?
0
good news: This allows you to pick up the tarball for installing aanval
bad news: The certificate for aanval.com isn't being checked -- this means we're still using SSL to connect to aanval.com and pick up the package, but we aren't checking the certificate to see who signed it, and/or what site it was signed for.
Unfortnately, the problem is out of my hands. I submitted a quick fix to github.com that adds the --no-check-certificate option to allow us to fix the package.
Regards,
DA_667
p.s. Yes, I'm still working on snorby. Hit a bit of a road bump. I thought I had the full installation down, and now, for some reason the snorby database refuses to update. at all. So, as I said before, I want to run through the entire snorby installation process without a single error. Once I can do that... I'll release the Ubuntu snorby script, then Debian, Then probably CentOS last.
Cheers!
Sunday, May 5, 2013
Snorby script progress, and a rundown of how to perform the installation.
Hello Autosnort users,
Still working on the Snorby release for Autosnort, and even then, It's probably going to be a staggered release -- first Ubuntu, then Debian, and Finally moving to CentOS. I imagine that based on some of the issues I have been experiencing, Getting Snorby to play with with SELinux is going to be a lot of fun.
I managed to get it to work with Aanval, snortreport and BASE, so I can do it again.
Anyhow, for those of you who can't wait for me to release a script, here's a basic run-down of the commands I'm running in Ubuntu 12.04 that I've used that work. this assumes you've ran autosnort up to the point where web interface installation takes place.
Don't be alarmed when it says you're downloading 400m of packages. that's about right:
apt-get install libyaml-dev git-core imagemagick libmagickwand-dev wkhtmltopdf libssl-dev libxslt1-dev libsqlite3-dev libmysql++-dev libcurl4-openssl-dev apache2-prefork-dev default-jre-headless
this installs rvm. rvm is a nice little program that can be used to automatically install ruby:
\curl -\#L https://get.rvm.io | sudo bash -s stable
/usr/local/rvm/bin/rvm autolibs enable
source /etc/profile.d/rvm.sh
this is a cute little hack that I've used for installing the latest stable version of snort, DAQ, and determine what version of snort rules are available via snort.org. From what I gather ruby updates frequently. very, very frequently. This string of commands tells us the latest ruby 1.9.x release available for installation. Why not install ruby 2.0.0? because snorby doesn't support it yet. Submitted an issue to snorby's github. We'll see where that takes us:
wget http://ruby-lang.org/en/index.html -O /tmp/index.html
rubyver=`cat /tmp/index.html | grep -e "ruby-" | head -4 | grep released | grep -v Continue | cut -d"/" -f8 | cut -d" " -f2 | tail -1`
rvm install ruby-$rubyver
After all the packages you need to install, there's also a boatload of gems that need installation. and oh, we haven't pulled down the snorby web UI yet:
gem install thor i18n bundler tzinfo builder memcache-client rack rack-test rack-mount rails rake rubygems-update erubis mail text-format sqlite3 daemon_controller
gem install passenger
update_rubygems
cd /var/www/
git clone http://github.com/Snorby/snorby.git
cd snorby/config
Make copies of database.yml.example and snorby_config.yml.example and rename them to database.yml and snorby_config.yml:
cp database.yml.example database.yml
cp snorby_config.yml.example snorby_config.yml
modify snorby_config.yml. this should be the only change that needs to be made:
sed -i 's/usr\/local\/bin/usr\/bin/' snorby_config.yml
modify database.yml the snorby section at the top, you MUST enter the root database user's password. this is necessary for the rake snorby:setup portion
Next, we have to compile the mod_passenger apache module. this will take a little bit of time.
passenger-install-apache2-module
If the passenger-install command above bombs, try running "gem install passenger --pre" and running the command again.
The passenger-install command will ask you to modify a couple of files. First we have to point apache to the passenger module:
echo "" >> /etc/apache2/apache2.conf
echo "# This stuff is to make Snorby work properly mod_passenger is required for snorby to work." >> /etc/apache2/apache2.conf
echo "" >> /etc/apache2/apache2.conf
echo "LoadModule passenger_module /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6/libout/apache2/mod_passenger.so" >> /etc/apache2/apache2.conf
echo "PassengerRoot /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6" >> /etc/apache2/apache2.conf
echo "PassengerDefaultRuby /usr/local/rvm/wrappers/ruby-$rubyver/ruby" >> /etc/apache2/apache2.conf
Next, we create a virtualhost for snorby and set the documentroot to snorby's public directory, then disable the default site. This effect means that if you navigate to http://[ip address] you automatically get the index page for snorby:
echo "<VirtualHost *:80>" >> /etc/apache2/sites-available/snorby
echo " ServerName snorby.localhost" >> /etc/apache2/sites-available/snorby
echo " # !!! Be sure to point DocumentRoot to 'public'!" >> /etc/apache2/sites-available/snorby
echo " DocumentRoot /var/www/snorby/public" >> /etc/apache2/sites-available/snorby
echo " <Directory /var/www/snorby/public>" >> /etc/apache2/sites-available/snorby
echo " # This relaxes Apache security settings." >> /etc/apache2/sites-available/snorby
echo " AllowOverride all" >> /etc/apache2/sites-available/snorby
echo " # MultiViews must be turned off." >> /etc/apache2/sites-available/snorby
echo " Options -MultiViews" >> /etc/apache2/sites-available/snorby
echo " </Directory>" >> /etc/apache2/sites-available/snorby
echo "</VirtualHost>" >> /etc/apache2/sites-available/snorby
All this has just been setup so we can successfully run bundler and rake to actually install snorby:
cd /var/www/snorby
bundle install --deployment
note: if the bundle install command fails or gives you an error referencing to the gemlock file or a gem called psych shield, you may have to run "bundle install --no-deployment" then run "bundle install --deployment" for things to work right.
finally, the moment you've been waiting for...:
rake snorby:setup
note: do not try to create the snorby database prior to running rake snorby:setup. If the database already exists, rake will bomb and exit.
At this point, provided rake snorby:setup ran, you should have a web interface that you can log into. default creds for snorby are: snorby@snorby.org with a password of snorby.
If you reboot your sensor, the worker jobs and the delayed_run jobs, processes necessary for populating the dashboard on the web UI do not automatically start with the system. You need to run these commands on reboot:
cd /var/www/snorby && ruby script/delayed_job start
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
What I am working on:
- Every time I've ran the installation, I've varied one thing (e.g. the first time through I tried installing with ruby 2.0.0, the second time around, I installed and found out daemon_controller was a needed gem that no install guide referenced, the third time around I tried installing by creating the snorby database and granting the snort database user access to the database. This is also when I encountered the problem with bundler complaining about the gemlock file and the psych shield gem and had to re-run bundler with the --no-deployment option.
I need to re-run the installation until I get absolutely zero errors. I want this fully repeatable with no errors.
I want to investigate the possibility of editing the database.yml post-installation: using the snort database user, and using mysql commands to grant the necessary privs to the snort mysql user instead of root. Having the root mysql user's privs in plaintext in a file that's world readable by default sounds like a terrible idea to me.
possibly adding the delayed_job and SensorCacheJob commands to rc.local to ensure they are ran upon reboot.
Getting support for Snorby is my highest priority right now. After that is done, I have some efficiencies and changes to autosnort I'll talk about in another post. Until then,
Cheers from DA.
Still working on the Snorby release for Autosnort, and even then, It's probably going to be a staggered release -- first Ubuntu, then Debian, and Finally moving to CentOS. I imagine that based on some of the issues I have been experiencing, Getting Snorby to play with with SELinux is going to be a lot of fun.
I managed to get it to work with Aanval, snortreport and BASE, so I can do it again.
Anyhow, for those of you who can't wait for me to release a script, here's a basic run-down of the commands I'm running in Ubuntu 12.04 that I've used that work. this assumes you've ran autosnort up to the point where web interface installation takes place.
Don't be alarmed when it says you're downloading 400m of packages. that's about right:
apt-get install libyaml-dev git-core imagemagick libmagickwand-dev wkhtmltopdf libssl-dev libxslt1-dev libsqlite3-dev libmysql++-dev libcurl4-openssl-dev apache2-prefork-dev default-jre-headless
this installs rvm. rvm is a nice little program that can be used to automatically install ruby:
\curl -\#L https://get.rvm.io | sudo bash -s stable
/usr/local/rvm/bin/rvm autolibs enable
source /etc/profile.d/rvm.sh
this is a cute little hack that I've used for installing the latest stable version of snort, DAQ, and determine what version of snort rules are available via snort.org. From what I gather ruby updates frequently. very, very frequently. This string of commands tells us the latest ruby 1.9.x release available for installation. Why not install ruby 2.0.0? because snorby doesn't support it yet. Submitted an issue to snorby's github. We'll see where that takes us:
wget http://ruby-lang.org/en/index.html -O /tmp/index.html
rubyver=`cat /tmp/index.html | grep -e "ruby-" | head -4 | grep released | grep -v Continue | cut -d"/" -f8 | cut -d" " -f2 | tail -1`
rvm install ruby-$rubyver
After all the packages you need to install, there's also a boatload of gems that need installation. and oh, we haven't pulled down the snorby web UI yet:
gem install thor i18n bundler tzinfo builder memcache-client rack rack-test rack-mount rails rake rubygems-update erubis mail text-format sqlite3 daemon_controller
gem install passenger
update_rubygems
cd /var/www/
git clone http://github.com/Snorby/snorby.git
cd snorby/config
Make copies of database.yml.example and snorby_config.yml.example and rename them to database.yml and snorby_config.yml:
cp database.yml.example database.yml
cp snorby_config.yml.example snorby_config.yml
modify snorby_config.yml. this should be the only change that needs to be made:
sed -i 's/usr\/local\/bin/usr\/bin/' snorby_config.yml
modify database.yml the snorby section at the top, you MUST enter the root database user's password. this is necessary for the rake snorby:setup portion
Next, we have to compile the mod_passenger apache module. this will take a little bit of time.
passenger-install-apache2-module
If the passenger-install command above bombs, try running "gem install passenger --pre" and running the command again.
The passenger-install command will ask you to modify a couple of files. First we have to point apache to the passenger module:
echo "" >> /etc/apache2/apache2.conf
echo "# This stuff is to make Snorby work properly mod_passenger is required for snorby to work." >> /etc/apache2/apache2.conf
echo "" >> /etc/apache2/apache2.conf
echo "LoadModule passenger_module /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6/libout/apache2/mod_passenger.so" >> /etc/apache2/apache2.conf
echo "PassengerRoot /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6" >> /etc/apache2/apache2.conf
echo "PassengerDefaultRuby /usr/local/rvm/wrappers/ruby-$rubyver/ruby" >> /etc/apache2/apache2.conf
Next, we create a virtualhost for snorby and set the documentroot to snorby's public directory, then disable the default site. This effect means that if you navigate to http://[ip address] you automatically get the index page for snorby:
echo "<VirtualHost *:80>" >> /etc/apache2/sites-available/snorby
echo " ServerName snorby.localhost" >> /etc/apache2/sites-available/snorby
echo " # !!! Be sure to point DocumentRoot to 'public'!" >> /etc/apache2/sites-available/snorby
echo " DocumentRoot /var/www/snorby/public" >> /etc/apache2/sites-available/snorby
echo " <Directory /var/www/snorby/public>" >> /etc/apache2/sites-available/snorby
echo " # This relaxes Apache security settings." >> /etc/apache2/sites-available/snorby
echo " AllowOverride all" >> /etc/apache2/sites-available/snorby
echo " # MultiViews must be turned off." >> /etc/apache2/sites-available/snorby
echo " Options -MultiViews" >> /etc/apache2/sites-available/snorby
echo " </Directory>" >> /etc/apache2/sites-available/snorby
echo "</VirtualHost>" >> /etc/apache2/sites-available/snorby
All this has just been setup so we can successfully run bundler and rake to actually install snorby:
cd /var/www/snorby
bundle install --deployment
note: if the bundle install command fails or gives you an error referencing to the gemlock file or a gem called psych shield, you may have to run "bundle install --no-deployment" then run "bundle install --deployment" for things to work right.
finally, the moment you've been waiting for...:
rake snorby:setup
note: do not try to create the snorby database prior to running rake snorby:setup. If the database already exists, rake will bomb and exit.
At this point, provided rake snorby:setup ran, you should have a web interface that you can log into. default creds for snorby are: snorby@snorby.org with a password of snorby.
If you reboot your sensor, the worker jobs and the delayed_run jobs, processes necessary for populating the dashboard on the web UI do not automatically start with the system. You need to run these commands on reboot:
cd /var/www/snorby && ruby script/delayed_job start
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
What I am working on:
- Every time I've ran the installation, I've varied one thing (e.g. the first time through I tried installing with ruby 2.0.0, the second time around, I installed and found out daemon_controller was a needed gem that no install guide referenced, the third time around I tried installing by creating the snorby database and granting the snort database user access to the database. This is also when I encountered the problem with bundler complaining about the gemlock file and the psych shield gem and had to re-run bundler with the --no-deployment option.
I need to re-run the installation until I get absolutely zero errors. I want this fully repeatable with no errors.
I want to investigate the possibility of editing the database.yml post-installation: using the snort database user, and using mysql commands to grant the necessary privs to the snort mysql user instead of root. Having the root mysql user's privs in plaintext in a file that's world readable by default sounds like a terrible idea to me.
possibly adding the delayed_job and SensorCacheJob commands to rc.local to ensure they are ran upon reboot.
Getting support for Snorby is my highest priority right now. After that is done, I have some efficiencies and changes to autosnort I'll talk about in another post. Until then,
Cheers from DA.
Wednesday, May 1, 2013
Short and Sweet
Autosnort users,
Two things to be aware of:
1) the pulledpork auto rule download functionality appears to be broken right now. This is due to a lack of forethought on my part on believing that there wouldn't be less than 30 days between two versions of snort being released. Working on fixing this as we speak.
2) I'm working pretty hard on automating installation of Snorby. Snorby is an ambitious project and a beautiful web interface, but the Documentation hasn't seen updates in over a year, and things that should be simple now have Ruby, rails and all the dependency hell that comes with it. It's coming together, slowly but surely.
Until next time.
Update: yesterday evening the snort 2.9.4.5 rules were made available to registered users. This sort of invalidates the fix I'm putting into autosnort, but I'm still going to do it to prevent this from happening again. Also still working on a snorby script.
Two things to be aware of:
1) the pulledpork auto rule download functionality appears to be broken right now. This is due to a lack of forethought on my part on believing that there wouldn't be less than 30 days between two versions of snort being released. Working on fixing this as we speak.
2) I'm working pretty hard on automating installation of Snorby. Snorby is an ambitious project and a beautiful web interface, but the Documentation hasn't seen updates in over a year, and things that should be simple now have Ruby, rails and all the dependency hell that comes with it. It's coming together, slowly but surely.
Until next time.
Update: yesterday evening the snort 2.9.4.5 rules were made available to registered users. This sort of invalidates the fix I'm putting into autosnort, but I'm still going to do it to prevent this from happening again. Also still working on a snorby script.
Subscribe to:
Posts (Atom)