Hello,
Lots to talk about, so let's get started.
I finally got around to finishing testing for the new, improved, and much cleaned up build of Autosnort that I had been hyping for a little while. For those who didn't see the last blog post, or the screen shot, I've done a bit of housekeeping with Autosnort.
Instead of puking the output for every single command all over the screen buffer, Autosnort's output has been significantly minimized and cleaned up where it could be cleaned up. Output is now very metasploit-like:
[*] are things that the script is doing
[*] are things the user should pay attention to
[*] are successful results
[*] are unsuccessful results
"Okay, so what if something goes wrong? All I have is a generic 'something went wrong' comment." Well, instead of hoping whatever you need was caught in the screen buffer, autosnort now logs the output of the ENTIRE installation to /var/log/autosnort_install.log, and /var/log/[interface you chose to install here -- for example, snort_install.log], So now, instead of having to hope the screen buffer caught the relevant bits of a problem with my script, you now have an entire log of what the script was actually doing for review, or to send to me to debug the script and get it to work properly.
The script will now actually tell you where I installed things, instead of you having to hunt and guess (sorry about that..)
Additionally, this release sees support for Ubuntu 13.04 and Debian 7. Currently in the process of Verifying support for Kali Linux (Backtrack's newer, younger sibling), but since it's all supposed to based off Debian and follows the Debian software methodology, it should be pretty straightforward.
The child installer scripts all got this little makeover, and as previously mentioned also have logging built in, and write to their own log files in /var/log.
The only thing of note that changed here in addition to the minimizing of output would be that I finally added a prompt to run Aanval's BPU processes on boot via rc.local. That's something I've been meaning to do for a while.
So now that this is over with, the next step would be for me to focus on Getting CentOS updated -- Snorby and the cleaned up autosnort script. That is next up on my plate before enabling anymore features on any platform. period.
After that is finished, here's what I'd like to focus on:
- All web interfaces should have SSL/HTTPS enabled by default. No exceptions.
- Remove database/barebone sensors should have a method to secure database transactions to the "Master Console"
+ some point way far down the line, I want to begin experimenting with p0f. The project has recently gotten some new blood (as mentioned in a recent shmoocon), and there are some interesting possibilities I can think of where p0f and snort could play incredibly well off of one another:
--have p0f run for a set period of time, sample traffic off the network, fingerprint operating systems and write results to a database
--script something out that takes the database results and helps write frag3/stream5 reassembly policies based off p0f results.
Same as always, the scripts are available via The Autosnort Github Repo
That's about it for now. Happy snorting, hope you enjoy the new release!
DA_667
Showing posts with label autosnort enhancements. Show all posts
Showing posts with label autosnort enhancements. Show all posts
Friday, June 14, 2013
Friday, June 7, 2013
Autosnort: MSF edition
Working on some changes to Autosnort.
After seeing Darkoperator's MSF Installer script, I wanted to change how Autosnort operates a little bit.
You see, one of the cool things about metasploit is that It will log to hell and back if things are or are not working, but while you're actually running it, It only tells you what you need to know:
"This is what I'm trying to do right now"
"This good thing happened"
"This bad thing happened"
"This is something to be aware of."
Keeping this in mind, and heavily ganking the notification printing code from darkoperator, in combination with a crazy named pipe trick I found via stack overflow, I'm changing how autosnort presents information.
As it stands currently, Autosnort, to put it bluntly, pukes all over the terminal/screen buffer. If something exploded in a bloody mess or failed to install correctly, you may see it and may be able to capture it in a screen shot and report it back to me, or you may not. Digital gods only know.
The way I've re-written the script, the only thing that gets printed to the screen are status messages indicating what the script is doing, good/bad messages to let you know if something failed or succeeded, and notifications that are sort of a "I need your input here" or "Pay attention to this!"
Apt-get (with the exception of the mysql-server installation that REQUIRES user input), configuration, compiler and all other output is redirected to log files.
For those of you who like seeing autosnort puke all over the screen or actually seeing what's going on behind the scenes, I found a nifty hack via stack overflow that uses mkfifo and named pipe magic to log everything from the script into a log file. No more running script to log output, no more screenshots, if you have a problem, you can e-mail me the output from the main autosnort script and/or the child interface installation scripts.
As of right now, I'm piloting this out on Debian, and it just looks slick. So far, I have the main script re-written and snorby's installation script re-written in this format. The other interface installers should fall in line easily enough, Snorby is the most complicated install I scripted out.
I haven't officially released either scripts out to github because I'm working on a couple other issues as well:
-- Support for both Debian 6 AND Debian 7 in the autosnort-debian main script
The only problem I'm seeing here at this point is that libmysqlclient.so (for barnyard2) installs in three different places in Debian 6 32/64 and Debian 7 32/64:
/usr/lib (Deb 6 32 and 64 bit)
/usr/lib/i386-linux-gnu (Deb 7 32)
/usr/lib/x86_64-linux-gne (Deb 7 64)
so I'm testing out a work-around involving the 'find' command and 'dirname' to fix this once and for all.
-- Support for Ubuntu 13.04
It's a short-term release, but some want to see support for it. It also appears to have an issue finding libmysqlclient.so
My theory is, if I can get find/dirname tested/working on Debian 6/7, likely I can get it to work between Ubuntu 12 and 13.04, hell I might be able to merge the Debian and Ubuntu autosnort scripts into a single script (since this was one of slight differences between the two).
So there's that.
I'm going to try and include a screenshot down here to preview what to expect, but so far, Blogger has somehow managed to not get my photos attached to my blog posts every time I've tried to use them.
First try:
Second try:
Via my twitter
After seeing Darkoperator's MSF Installer script, I wanted to change how Autosnort operates a little bit.
You see, one of the cool things about metasploit is that It will log to hell and back if things are or are not working, but while you're actually running it, It only tells you what you need to know:
"This is what I'm trying to do right now"
"This good thing happened"
"This bad thing happened"
"This is something to be aware of."
Keeping this in mind, and heavily ganking the notification printing code from darkoperator, in combination with a crazy named pipe trick I found via stack overflow, I'm changing how autosnort presents information.
As it stands currently, Autosnort, to put it bluntly, pukes all over the terminal/screen buffer. If something exploded in a bloody mess or failed to install correctly, you may see it and may be able to capture it in a screen shot and report it back to me, or you may not. Digital gods only know.
The way I've re-written the script, the only thing that gets printed to the screen are status messages indicating what the script is doing, good/bad messages to let you know if something failed or succeeded, and notifications that are sort of a "I need your input here" or "Pay attention to this!"
Apt-get (with the exception of the mysql-server installation that REQUIRES user input), configuration, compiler and all other output is redirected to log files.
For those of you who like seeing autosnort puke all over the screen or actually seeing what's going on behind the scenes, I found a nifty hack via stack overflow that uses mkfifo and named pipe magic to log everything from the script into a log file. No more running script to log output, no more screenshots, if you have a problem, you can e-mail me the output from the main autosnort script and/or the child interface installation scripts.
As of right now, I'm piloting this out on Debian, and it just looks slick. So far, I have the main script re-written and snorby's installation script re-written in this format. The other interface installers should fall in line easily enough, Snorby is the most complicated install I scripted out.
I haven't officially released either scripts out to github because I'm working on a couple other issues as well:
-- Support for both Debian 6 AND Debian 7 in the autosnort-debian main script
The only problem I'm seeing here at this point is that libmysqlclient.so (for barnyard2) installs in three different places in Debian 6 32/64 and Debian 7 32/64:
/usr/lib (Deb 6 32 and 64 bit)
/usr/lib/i386-linux-gnu (Deb 7 32)
/usr/lib/x86_64-linux-gne (Deb 7 64)
so I'm testing out a work-around involving the 'find' command and 'dirname' to fix this once and for all.
-- Support for Ubuntu 13.04
It's a short-term release, but some want to see support for it. It also appears to have an issue finding libmysqlclient.so
My theory is, if I can get find/dirname tested/working on Debian 6/7, likely I can get it to work between Ubuntu 12 and 13.04, hell I might be able to merge the Debian and Ubuntu autosnort scripts into a single script (since this was one of slight differences between the two).
So there's that.
I'm going to try and include a screenshot down here to preview what to expect, but so far, Blogger has somehow managed to not get my photos attached to my blog posts every time I've tried to use them.
First try:
Second try:
Via my twitter
Saturday, May 18, 2013
Autosnorby, updates to autosnort, and the updated to-do list
Hello Snort and Autosnort users.
I've got a lot to talk about today, so let's get started.
First and foremost, I've finally managed to work in support for snorby into autosnort. The snorby script installs the necessary packages, a compatible version of ruby (via rvm), all the necessary gems and configures the right files with the right info needed to get snorby up and running. Additionally, I've managed to work out away to drop privileges from the root database user to another user (the snort user in our case)
One thing to be aware of: the delayed_job and the sensor cache jobs do NOT start up on boot. Initial runs of the script had me trying to put commands into rc.local to start the delayed_job and sensorcachejob commands on startup... but they aren't working. I'm working on having a reliable method to start them on boot, but until that is discovered, any time the system is restarted, you'll want to, at a minimum start the delayed_job script:
cd /var/www/snorby && ruby script/delayed_job start
optionally, these commands run the Sensor and Daily cachejobs as well:
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
There are options to start the snorby worker process via the web UI, but I find the direct approach to my liking.
Another note is that at first you may notice alerts coming in and displaying properly in the events tab, but the dashboard may not updated at first. If this is the case, on the dashboard page, there is an option to force cache update. Select this option, wait 10 seconds, and reload the page. the graphs should update.
Next up on the list, is the main autosnort script itself. I've done a bit of spring cleaning.
- I've reduced the number of packages installed by default with autosnort.
--By default all we install are the packages necessary to install daq, libdnet, snort, and barnyard 2 with the ability to write to a remote database (more on this in a minute)
--I introduced a quick check that asks the user if they plan on installing a web interface as a part of the autosnort installation, or if they just want a stripped-down barebones install. If you're not sure what option to pick, select option 1.
- I've added a couple of functions to the script near the beginning for pulled pork to facilitate code re-use. this is because:
-- I've added support to download snort rules from previous versions of snort; up to four versions prior.
Why: I ran into a little problem recently where the free rules for version 2.9.4.5 were not available yet, and snort version 2.9.4.6 had just came out. The current version of the script only checks for the current version of snort rules, and one version prior. So now, you have a choice for four versions of rules:
1) the current version (if you have a VRT subscription oink code; e.g. 2.9.4.6)
2) the previous version (the kind that registered users usually get access to; e.g. 2.9.4.5)
3) two more prior versions (in the event that they are needed and/or a similar situation arises)
- If the user has elected to not install a web interface, the script asks them if they have a remote database they want barnyard to log to. I haven't officially tested this feature yet, but theorhetically, this allows for distributed installations... Any volunteers to test this? :)
- The child shell scripts all have functionality to install the packages necessary for them to work individually Again to reduce clutter and excess packages installed by autosnort (e.g. reduce attack surface)
- thanks to DK1844, I have a couple of code cleaniness fixes that I have added, and future todo list items.
I've achieved most of the stuff I intended to do when I initially started this little project, but I still have more that I can do.
Speaking of to-do list items:
- SSL for all web interfaces (generate a default, self-signed SSL cert, use mod-rewrite and mod-ssl to force ssl usage for all web interfaces)
- Further testing with distributed installations (Can I get a master console to register events from a remote system?)
- Encrypted comms between master console and remote sensor (e.g. have barnyard dump database updates to the master console over a secure channel)
- Kali Linux testing! Since Kali is fully compliant with Debian standards, will the debian autosnort script work with it?
The updates to autosnort.sh, snorby, aanval, snortreport and base child shell scripts are being pushed to ubuntu tonight, with support for other operating systems (debian and centOS) to come soon.
Report bugs, problems, praise, feature requests, etc. to:
deusexmachina667 [at] gmail [dot] com
As always the code is on my github repo.
Happy snorting,
DA_667
I've got a lot to talk about today, so let's get started.
First and foremost, I've finally managed to work in support for snorby into autosnort. The snorby script installs the necessary packages, a compatible version of ruby (via rvm), all the necessary gems and configures the right files with the right info needed to get snorby up and running. Additionally, I've managed to work out away to drop privileges from the root database user to another user (the snort user in our case)
One thing to be aware of: the delayed_job and the sensor cache jobs do NOT start up on boot. Initial runs of the script had me trying to put commands into rc.local to start the delayed_job and sensorcachejob commands on startup... but they aren't working. I'm working on having a reliable method to start them on boot, but until that is discovered, any time the system is restarted, you'll want to, at a minimum start the delayed_job script:
cd /var/www/snorby && ruby script/delayed_job start
optionally, these commands run the Sensor and Daily cachejobs as well:
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
There are options to start the snorby worker process via the web UI, but I find the direct approach to my liking.
Another note is that at first you may notice alerts coming in and displaying properly in the events tab, but the dashboard may not updated at first. If this is the case, on the dashboard page, there is an option to force cache update. Select this option, wait 10 seconds, and reload the page. the graphs should update.
Next up on the list, is the main autosnort script itself. I've done a bit of spring cleaning.
- I've reduced the number of packages installed by default with autosnort.
--By default all we install are the packages necessary to install daq, libdnet, snort, and barnyard 2 with the ability to write to a remote database (more on this in a minute)
--I introduced a quick check that asks the user if they plan on installing a web interface as a part of the autosnort installation, or if they just want a stripped-down barebones install. If you're not sure what option to pick, select option 1.
- I've added a couple of functions to the script near the beginning for pulled pork to facilitate code re-use. this is because:
-- I've added support to download snort rules from previous versions of snort; up to four versions prior.
Why: I ran into a little problem recently where the free rules for version 2.9.4.5 were not available yet, and snort version 2.9.4.6 had just came out. The current version of the script only checks for the current version of snort rules, and one version prior. So now, you have a choice for four versions of rules:
1) the current version (if you have a VRT subscription oink code; e.g. 2.9.4.6)
2) the previous version (the kind that registered users usually get access to; e.g. 2.9.4.5)
3) two more prior versions (in the event that they are needed and/or a similar situation arises)
- If the user has elected to not install a web interface, the script asks them if they have a remote database they want barnyard to log to. I haven't officially tested this feature yet, but theorhetically, this allows for distributed installations... Any volunteers to test this? :)
- The child shell scripts all have functionality to install the packages necessary for them to work individually Again to reduce clutter and excess packages installed by autosnort (e.g. reduce attack surface)
- thanks to DK1844, I have a couple of code cleaniness fixes that I have added, and future todo list items.
I've achieved most of the stuff I intended to do when I initially started this little project, but I still have more that I can do.
Speaking of to-do list items:
- SSL for all web interfaces (generate a default, self-signed SSL cert, use mod-rewrite and mod-ssl to force ssl usage for all web interfaces)
- Further testing with distributed installations (Can I get a master console to register events from a remote system?)
- Encrypted comms between master console and remote sensor (e.g. have barnyard dump database updates to the master console over a secure channel)
- Kali Linux testing! Since Kali is fully compliant with Debian standards, will the debian autosnort script work with it?
The updates to autosnort.sh, snorby, aanval, snortreport and base child shell scripts are being pushed to ubuntu tonight, with support for other operating systems (debian and centOS) to come soon.
Report bugs, problems, praise, feature requests, etc. to:
deusexmachina667 [at] gmail [dot] com
As always the code is on my github repo.
Happy snorting,
DA_667
Sunday, April 21, 2013
Sometimes the old ways are the best ways (support for BASE and Syslog_Full output)
Hello AS/snort users,
Got two new features in this latest build of autosnort.
First and foremost is support for the BASE web interface. BASE is by and far one of the older and more well-known web interfaces for snort alerting. While the interface isn't going to win for prettiest interface, it is by and far very functional. Some of the cool things include the ability to download alerts as pcap and actual tshark/pcap parsed output for each alert.
This has been added as an installation option in the interface selection menu. The installation is painless and straight forward. After the system reboots, just point your browser to http://[your ip]/base.
The web page initial setup will check for all of the required pre-requisite settings on the first page. On the second page, it will ask you where adodb, a pre-requisite package, is installed. Autosnort and most linux distros install adodb to /usr/share/php/adodb (put that in for the path to adodb). It will also, of course ask you for database information and credentials:
database name: snort
database host: localhost
database port: 3306 (can leave blank if you'd like)
database username: snort
password: (the snort database user's password)
Unless you have an archive database configured, leave that part blank.
On the last page, it will ask if you want to set up auth to get into the web page to review events. That is entirely up to you.
After that, you should be all set to get events to the BASE web interface.
The next feature I have to announce is support for syslog output as another output interface choice. I've added a child script that configures barnyard2 to output to syslog_full format to port 514/udp. I have not FULLY tested this feature, but preliminary tests show that splunk picks up and parses the events with no issues whatsoever. A couple of troubleshooting recommendations:
-Ensure that the sensor's management interface, if it is firewalled, has port 514/udp open and allowed outbound.
-Conversely, ensure that your SIEM or syslog aggregator has port 514/udp open inbound to accept events -- I hosted splunk on a CentOS box and was puzzled why tcpdump says we were sending events out, but they were never making it to be indexed by splunk. I ran system-config-firewall-tui and added a rule for 514/udp inbound (and port 8000/tcp inbound for the actual web UI), and everything was fine.
The child scripts have been made available to Debian, Ubuntu and CentOS.
One last thing:
I've made some minor changes to the script at the recommendation of maintainer of the barnyard2 project. Instead of using sed to modify the copy of barnyard2.conf included with the barnyard2 source, I'm making the script generate a barnyard.conf on-the-fly.
- the script keeps a copy of the original barnyard2.conf in /usr/local/snort/etc/ named barnyard2.conf.origin
- the script generates a barnyard2.conf with only the directives necessary to run barnyard2 (e.g. where are the reference and classification config files, the gen/sid-msg.map files, input format to accept, output format to output to, etc.)
- if you choose to configure output to syslog format, the child script disables output to the database automatically.
As always, the scripts are up on github, along with updated READMEs. Previous versions are available in the Previous_Rel directory as well.
Happy Hunting..
DA_667
Got two new features in this latest build of autosnort.
First and foremost is support for the BASE web interface. BASE is by and far one of the older and more well-known web interfaces for snort alerting. While the interface isn't going to win for prettiest interface, it is by and far very functional. Some of the cool things include the ability to download alerts as pcap and actual tshark/pcap parsed output for each alert.
This has been added as an installation option in the interface selection menu. The installation is painless and straight forward. After the system reboots, just point your browser to http://[your ip]/base.
The web page initial setup will check for all of the required pre-requisite settings on the first page. On the second page, it will ask you where adodb, a pre-requisite package, is installed. Autosnort and most linux distros install adodb to /usr/share/php/adodb (put that in for the path to adodb). It will also, of course ask you for database information and credentials:
database name: snort
database host: localhost
database port: 3306 (can leave blank if you'd like)
database username: snort
password: (the snort database user's password)
Unless you have an archive database configured, leave that part blank.
On the last page, it will ask if you want to set up auth to get into the web page to review events. That is entirely up to you.
After that, you should be all set to get events to the BASE web interface.
The next feature I have to announce is support for syslog output as another output interface choice. I've added a child script that configures barnyard2 to output to syslog_full format to port 514/udp. I have not FULLY tested this feature, but preliminary tests show that splunk picks up and parses the events with no issues whatsoever. A couple of troubleshooting recommendations:
-Ensure that the sensor's management interface, if it is firewalled, has port 514/udp open and allowed outbound.
-Conversely, ensure that your SIEM or syslog aggregator has port 514/udp open inbound to accept events -- I hosted splunk on a CentOS box and was puzzled why tcpdump says we were sending events out, but they were never making it to be indexed by splunk. I ran system-config-firewall-tui and added a rule for 514/udp inbound (and port 8000/tcp inbound for the actual web UI), and everything was fine.
The child scripts have been made available to Debian, Ubuntu and CentOS.
One last thing:
I've made some minor changes to the script at the recommendation of maintainer of the barnyard2 project. Instead of using sed to modify the copy of barnyard2.conf included with the barnyard2 source, I'm making the script generate a barnyard.conf on-the-fly.
- the script keeps a copy of the original barnyard2.conf in /usr/local/snort/etc/ named barnyard2.conf.origin
- the script generates a barnyard2.conf with only the directives necessary to run barnyard2 (e.g. where are the reference and classification config files, the gen/sid-msg.map files, input format to accept, output format to output to, etc.)
- if you choose to configure output to syslog format, the child script disables output to the database automatically.
As always, the scripts are up on github, along with updated READMEs. Previous versions are available in the Previous_Rel directory as well.
Happy Hunting..
DA_667
Saturday, April 13, 2013
New AS release, New init scripts, bug fixes and more!
Hello AS users,
Got a lot to talk about today, so let's get down to business:
First and foremost I've finally gotten around to writing half-decent documentation for autosnort. I've pushed a pdf file to github featuring documentation with screenshots on how to install Autosnort via vmware player, including an interesting hack on how to get bridging to work on multiple interfaces on vmware player 5.
I chose CentOS as the vmware player operating system, but the instructions should be easy to follow for Debian Ubuntu or Backtrack. Any major differences were noted. All said and done it was about 23 pages of documentation with screenshots peppered in.
I focused on using VMware player this time around, and generally speaking the VMware player guidance can be used just as well for VMware Fusion or Workstation. I may do an alternate document for ESXi and/or Virtualbox in the near future.
I've also included general troubleshooting steps and things you *should* see after running autosnort.
Next in line is an update to the autosnort script for Debian, Ubuntu, and CentOS. Banyard2.13 BETA2 has changed how barnyard2 behaves. As a result of this, I had to change how arguments are passed to barnyard2 in Autosnort. The details are available for each operating system's readme files. Suffice to say, everything works fine now.
Finally, I've gotten around to testing and posting init scripts for Ubuntu and Debian to control snort and barnyard2. The Ubuntu and Debian init scripts support start, stop and restart functions and are pretty easy to implement. Implemtation details are also available via the Readme files for Ubuntu and Debian Autosnort scripts, but suffice to say, its incredibly easy.
I may release a modification to the init script that detects if you installed aanval and automatically start/stop the BPUs along with snort and barnyard2, but that will come a little later.
The documentation, new scripts, readmes and init scripts should all be available via github as of now.
Cheers,
DA
Got a lot to talk about today, so let's get down to business:
First and foremost I've finally gotten around to writing half-decent documentation for autosnort. I've pushed a pdf file to github featuring documentation with screenshots on how to install Autosnort via vmware player, including an interesting hack on how to get bridging to work on multiple interfaces on vmware player 5.
I chose CentOS as the vmware player operating system, but the instructions should be easy to follow for Debian Ubuntu or Backtrack. Any major differences were noted. All said and done it was about 23 pages of documentation with screenshots peppered in.
I focused on using VMware player this time around, and generally speaking the VMware player guidance can be used just as well for VMware Fusion or Workstation. I may do an alternate document for ESXi and/or Virtualbox in the near future.
I've also included general troubleshooting steps and things you *should* see after running autosnort.
Next in line is an update to the autosnort script for Debian, Ubuntu, and CentOS. Banyard2.13 BETA2 has changed how barnyard2 behaves. As a result of this, I had to change how arguments are passed to barnyard2 in Autosnort. The details are available for each operating system's readme files. Suffice to say, everything works fine now.
Finally, I've gotten around to testing and posting init scripts for Ubuntu and Debian to control snort and barnyard2. The Ubuntu and Debian init scripts support start, stop and restart functions and are pretty easy to implement. Implemtation details are also available via the Readme files for Ubuntu and Debian Autosnort scripts, but suffice to say, its incredibly easy.
I may release a modification to the init script that detects if you installed aanval and automatically start/stop the BPUs along with snort and barnyard2, but that will come a little later.
The documentation, new scripts, readmes and init scripts should all be available via github as of now.
Cheers,
DA
Sunday, March 31, 2013
Redhat/CentOS init script!
Hello AS and regular snort users,
I wanted to announce that I have developed an init script compatible with CentOS/Redhat variants for autosnort installations (of course, this script can easily be adapted to other snort snort installations.
This init script can be used to replace /etc/rc.local as the primary method of starting up snort and barnyard2, and includes the added bonus of allow you to start/stop/restart snort and barnyard2 without requiring a reboot or sourcing /etc/rc.local if you need to make changes to snort or barnyard2. To add this script to CentOS 6.x perform the following tasks as root (or via sudo/root permissions):
1. Copy the snortbarn script to /etc/init.d
2. Edit the variables near the top of the script to suit your snort installation (the only variable that you should need to modify is the snort_iface variable if you installed snort/barnyard2 via autosnort)
3. Make the snortbarn script is executable for the root user (chmod 700 snortbarn)
4. Run chkconfig --add snortbarn
5. Remove the entries for ifconfig, snort, and barnyard2 from /etc/rc.local (note: you may want to make a backup of the rc.local script in case you run into bugs/problems with the init script!)
6. Kill your current snort/barnyard processes that ran from rc.local (killall snort && killall barnyard2)
7. Run the command "service snortbarn start"
8. check the process list to ensure that snort and barnyard2 are running after calling the init script. ( "ps -ef | grep snort" will return snort and barnyard2, if either/both processes are running. If only one process or the other is visible, something is wrong)
Troubleshooting steps: I'm not entirely sure why but there are CRLF/LF formatting problems with this script. If you get a bunch of errors stating that a file/command doesn't exist, try running dos2unix on the file to resolve the CRLF/LF errors.
If you install the init script and upon reboot find that only the snort process is running, it is because the init script for snortbarn ran BEFORE the init script for mysqld ran. To Determine when mysqld is configured to run it its runlevels, check /etc/init.d/mysqld. You'll want to pay attention to this line in particular:
# chkconfig: - 64 36
the first number, 64 indicates what number the rc startup script will get on startup. Linux rc scripts determine what services run or are killed on a particular run level. Every rc script as a K for Kill order, an S for Start order, followed by a number and the name of the symlinked script from /etc/init.d. RC scripts are read in numeric order. So if the rc script for snortbarn has an S number lower than 64, it will run before mysqld. snort will start up fine, but barnyard fails because it has no database to connect to.
To remedy this, you can modify the /etc/init.d/mysqld script to have a lower number than the snortbarn script in any of the /etc/rc[2-5].d directories, or modify the snortbarn script to have a higher number than the mysqld directory. This is a little confusing, so let's look at an example:
run this command: ls -al /etc/rc?.d/S*snortbarn
this command shows you each runlevel snortbarn is configured to start on.
now, run this command: ls -al /etc/rc?.d/S*mysqld
this command shows you the runlevels mysqld is set to run on. Don't worry about how many results you get.
If mysqld's number is higher than snortbarn's number, the mysqld process will not be running before snort and barnyard are configured to run. No database running means barnyard2 won't run. Let's say snortbarn had a number of 63, and mysqld has a number of 64. edit /etc/init.d/mysqld and change the chkconfig to something like this:
# chkconfig: - 62 36
save your changes and run chkconfig --add mysqld. This should fix the problem.
I've added the above documention to the other notes section of the CentOS readme. The snortbarn script is now available via github.
Special Note: Would like to thank Tactical FLEX/Aanval for hosting the initial version of this init script that I based this one off of. Would also like to thank Mike Miller for the initial idea of moving away from rc.local and building out a legitimate init script for autosnort.
Happy Snorting,
DA667
I wanted to announce that I have developed an init script compatible with CentOS/Redhat variants for autosnort installations (of course, this script can easily be adapted to other snort snort installations.
This init script can be used to replace /etc/rc.local as the primary method of starting up snort and barnyard2, and includes the added bonus of allow you to start/stop/restart snort and barnyard2 without requiring a reboot or sourcing /etc/rc.local if you need to make changes to snort or barnyard2. To add this script to CentOS 6.x perform the following tasks as root (or via sudo/root permissions):
1. Copy the snortbarn script to /etc/init.d
2. Edit the variables near the top of the script to suit your snort installation (the only variable that you should need to modify is the snort_iface variable if you installed snort/barnyard2 via autosnort)
3. Make the snortbarn script is executable for the root user (chmod 700 snortbarn)
4. Run chkconfig --add snortbarn
5. Remove the entries for ifconfig, snort, and barnyard2 from /etc/rc.local (note: you may want to make a backup of the rc.local script in case you run into bugs/problems with the init script!)
6. Kill your current snort/barnyard processes that ran from rc.local (killall snort && killall barnyard2)
7. Run the command "service snortbarn start"
8. check the process list to ensure that snort and barnyard2 are running after calling the init script. ( "ps -ef | grep snort" will return snort and barnyard2, if either/both processes are running. If only one process or the other is visible, something is wrong)
Troubleshooting steps: I'm not entirely sure why but there are CRLF/LF formatting problems with this script. If you get a bunch of errors stating that a file/command doesn't exist, try running dos2unix on the file to resolve the CRLF/LF errors.
If you install the init script and upon reboot find that only the snort process is running, it is because the init script for snortbarn ran BEFORE the init script for mysqld ran. To Determine when mysqld is configured to run it its runlevels, check /etc/init.d/mysqld. You'll want to pay attention to this line in particular:
# chkconfig: - 64 36
the first number, 64 indicates what number the rc startup script will get on startup. Linux rc scripts determine what services run or are killed on a particular run level. Every rc script as a K for Kill order, an S for Start order, followed by a number and the name of the symlinked script from /etc/init.d. RC scripts are read in numeric order. So if the rc script for snortbarn has an S number lower than 64, it will run before mysqld. snort will start up fine, but barnyard fails because it has no database to connect to.
To remedy this, you can modify the /etc/init.d/mysqld script to have a lower number than the snortbarn script in any of the /etc/rc[2-5].d directories, or modify the snortbarn script to have a higher number than the mysqld directory. This is a little confusing, so let's look at an example:
run this command: ls -al /etc/rc?.d/S*snortbarn
this command shows you each runlevel snortbarn is configured to start on.
now, run this command: ls -al /etc/rc?.d/S*mysqld
this command shows you the runlevels mysqld is set to run on. Don't worry about how many results you get.
If mysqld's number is higher than snortbarn's number, the mysqld process will not be running before snort and barnyard are configured to run. No database running means barnyard2 won't run. Let's say snortbarn had a number of 63, and mysqld has a number of 64. edit /etc/init.d/mysqld and change the chkconfig to something like this:
# chkconfig: - 62 36
save your changes and run chkconfig --add mysqld. This should fix the problem.
I've added the above documention to the other notes section of the CentOS readme. The snortbarn script is now available via github.
Special Note: Would like to thank Tactical FLEX/Aanval for hosting the initial version of this init script that I based this one off of. Would also like to thank Mike Miller for the initial idea of moving away from rc.local and building out a legitimate init script for autosnort.
Happy Snorting,
DA667
Saturday, March 30, 2013
Hammering out Aanval
Hello Autosnort Users,
Today I would like to announce support for Tactical Flex's Aanval Web Console on CentOS and Debian platforms in addition to last weekend's Ubuntu release
I've implemented the same changes for the CentOS and Debian scripts that I've implemented on the Ubuntu script, namely the modularization of autosnort, starting with the web interface choices snortreport and aanval.
I've updated the readme files in each operating system's directory to more of a "release notes" format, that will be easier to read and follow.
One issue that I'd like to write about that I encountered while working on getting Aanval running on CentOS in particular is SELinux. Most people see SELinux as an annoying hindrance that keeps them from getting done what they want to get done. I use to be one of those people as well. But with a little bit of research, most problems with SELinux can be resolved easily, without disabling it or setting it to permissive mode (making it essentially useless).
I ran into a problem during aanval installation where it failed to connect to the mysql database on localhost with the creds I gave it. Thinking I fat-fingered the creds, I did it again and it immediately failed again.
At this point, I check the logs. Nothing interesting in /var/log/messages or the httpd error_logs, so I move to the audit log. For those of you who are not aware, SELinux usually logs to /var/log/audit/audit.log. It's log format is pretty strange, but its easy enough to pick out the relevant pieces to determine if SELinux is interfering with what you you need your system to do. run tail -f on audit.log and try entering the database credentials again and an entry pops up for SELinux, denying access to 3306/tcp for the httpd process. I do a google search for a small portion of the audit log message, and immediately get a relevant result. Long story short, run this command:
This explicitly states to SELinux that the httpd process requesting connections to the database/database port are perfectly fine. Always remember: disabling SELinux is NEVER the answer.
Happy Easter and Happy Snorting,
DA667
Today I would like to announce support for Tactical Flex's Aanval Web Console on CentOS and Debian platforms in addition to last weekend's Ubuntu release
I've implemented the same changes for the CentOS and Debian scripts that I've implemented on the Ubuntu script, namely the modularization of autosnort, starting with the web interface choices snortreport and aanval.
I've updated the readme files in each operating system's directory to more of a "release notes" format, that will be easier to read and follow.
One issue that I'd like to write about that I encountered while working on getting Aanval running on CentOS in particular is SELinux. Most people see SELinux as an annoying hindrance that keeps them from getting done what they want to get done. I use to be one of those people as well. But with a little bit of research, most problems with SELinux can be resolved easily, without disabling it or setting it to permissive mode (making it essentially useless).
I ran into a problem during aanval installation where it failed to connect to the mysql database on localhost with the creds I gave it. Thinking I fat-fingered the creds, I did it again and it immediately failed again.
At this point, I check the logs. Nothing interesting in /var/log/messages or the httpd error_logs, so I move to the audit log. For those of you who are not aware, SELinux usually logs to /var/log/audit/audit.log. It's log format is pretty strange, but its easy enough to pick out the relevant pieces to determine if SELinux is interfering with what you you need your system to do. run tail -f on audit.log and try entering the database credentials again and an entry pops up for SELinux, denying access to 3306/tcp for the httpd process. I do a google search for a small portion of the audit log message, and immediately get a relevant result. Long story short, run this command:
setsebool -P httpd_can_network_connect_db 1
This explicitly states to SELinux that the httpd process requesting connections to the database/database port are perfectly fine. Always remember: disabling SELinux is NEVER the answer.
Happy Easter and Happy Snorting,
DA667
Sunday, March 17, 2013
Dropping the hammer on the Aanval (The Aanval Update)
Hello Autosnort Users,
I am proud to announce a new, pilot release of Autosnort that will give users another web interface choice: Aanval!
For those of you who have no idea who or what Aanval is, Aanval can serve as a web UI choice for snort and/or suricata as well as a SIEM for events sent to it via syslog. There's the free edition that supports one sensor, and their commercial releases that support a large number of sensors and/or varying sources.
Aanval is a product of Tactical FLEX -- They're a nice bunch of people. I have yet to meet any of them in person, but I have spoken with both the CEO and the designer of the console over twitter a few times. They are very nice, very personable, and down-to-earth. If you get around to installing Aanval, be sure to drop them a line to let them know how awesome their console is!
This Autosnort release introduces some change you need to be aware of, so let's get to it:
-As indicated above this is a PILOT release. I've tested it extensively over the weekend, but I can't plan for every contingency and problem. Try it out, let me know if you run into problems. The bugs you report help me, other Autosnort users, and the open-source community overall.
-The new version of the script begins an effort for me to attempt to modularize Autosnort a little bit. Individual interface choices are their own child shell scripts now, called from the main shell script. This is a design choice I made to make it easier to add new and troubleshoot existing functionality.
-When I refer to "main" and child shell scripts, I mean, the main script is the large Autosnort script we all know and love. The child shell scripts are much smaller shell scripts dedicated to a single purpose, like installing one particular web interface. For this update, there are two child shell scripts:
snortreport.sh -- as the name implies is responsible for installing Snort Report
aanval.sh -- responsible for installing aanval
-The main autosnort script should be able to be ran anywhere, but the child shell script for the interface you wish to install must be placed into the /root directory for the script to complete successfully.
-As a matter of good practice, I would advise having the main and the child scripts in /root when you run it. The main script expects to find the child script in /root when you make it to the interface choices menu, and call the child script for the interface you wish to install. The script checks to see that the child script completed with an exit state of 0. If it does not, the main script will NOT continue.
-As another matter of good practice, I've decided that from here on out, I will keep the latest version of Autosnort available via github as well as one previous version back. This way, if you find there is some major problem with the current version of Autosnort, and you find that I am not working fast enough to resolve your problem, you can at least try to install the previous version to see if that works for you instead. In the Ubuntu Autosnort directory, there should be a Previous_Rel directory that contains the last version of Autosnort.
I've updated the readme in the Ubuntu directory to reflect this information. Provided there are no major deal-breaking problems in this pilot release, I hope to push support for Aanval to Debian and CentOS in the near future as well.
Update:
Aanval Post-Setup notes:
- It is highly advised that you reboot your system before continuing to the aanval console to continue the installation via the web interface. I ran into a problem prior to rebooting where the aanval console would not recognize that the php mysql module did exist and was loaded until the system was rebooted.
- During the initial setup, aanval will want to know the name of the aanvaldb user and password.
Username:snort
Password:password you gave the snort database user during the autosnort installation
- Aanval has a set of processes that are used to bring events over from the snort database that barnyard2 will dump to, and bring them over to the aanvaldb that aanval reads from. The console interface will let you know if they are not running. To start them, navigate to /var/www/aanval/apps and run idsBackground.pl -start --- I plan on adding an rc.local entry that will do this for you in the near future!
- In order for Aanval to manage events for your snort sensor you need to enable it on the aanval console. click the gear symbol in the lower corner of the web interface. This will bring you to a page called configuration. Click the "Settings" option under the "Snort" section. On the next page, check the enabled checkbox and enter the information for the snort database:
database name: snort
database hostname: localhost
database username: snort
database password: the password you assigned to the snort database user during autosnort installation
then click update. It may take a few minute for intrusion events to show up on the aanval interface. Be patient, they'll start coming in shortly!
- At this time, Autosnort does not support the sensor configuration options that Aanval includes. This is a limitation on my part; I need to figure out where Aanval expects to find the rule files, snort.conf and other configuration files before this will be possible!
- For more guidance and information specific to aanval, pay the folks at Tactical FLEX a visit at aanval.com
Happy St. Patty's day and happy snortin' to ye.
I am proud to announce a new, pilot release of Autosnort that will give users another web interface choice: Aanval!
For those of you who have no idea who or what Aanval is, Aanval can serve as a web UI choice for snort and/or suricata as well as a SIEM for events sent to it via syslog. There's the free edition that supports one sensor, and their commercial releases that support a large number of sensors and/or varying sources.
Aanval is a product of Tactical FLEX -- They're a nice bunch of people. I have yet to meet any of them in person, but I have spoken with both the CEO and the designer of the console over twitter a few times. They are very nice, very personable, and down-to-earth. If you get around to installing Aanval, be sure to drop them a line to let them know how awesome their console is!
This Autosnort release introduces some change you need to be aware of, so let's get to it:
-As indicated above this is a PILOT release. I've tested it extensively over the weekend, but I can't plan for every contingency and problem. Try it out, let me know if you run into problems. The bugs you report help me, other Autosnort users, and the open-source community overall.
-The new version of the script begins an effort for me to attempt to modularize Autosnort a little bit. Individual interface choices are their own child shell scripts now, called from the main shell script. This is a design choice I made to make it easier to add new and troubleshoot existing functionality.
-When I refer to "main" and child shell scripts, I mean, the main script is the large Autosnort script we all know and love. The child shell scripts are much smaller shell scripts dedicated to a single purpose, like installing one particular web interface. For this update, there are two child shell scripts:
snortreport.sh -- as the name implies is responsible for installing Snort Report
aanval.sh -- responsible for installing aanval
-The main autosnort script should be able to be ran anywhere, but the child shell script for the interface you wish to install must be placed into the /root directory for the script to complete successfully.
-As a matter of good practice, I would advise having the main and the child scripts in /root when you run it. The main script expects to find the child script in /root when you make it to the interface choices menu, and call the child script for the interface you wish to install. The script checks to see that the child script completed with an exit state of 0. If it does not, the main script will NOT continue.
-As another matter of good practice, I've decided that from here on out, I will keep the latest version of Autosnort available via github as well as one previous version back. This way, if you find there is some major problem with the current version of Autosnort, and you find that I am not working fast enough to resolve your problem, you can at least try to install the previous version to see if that works for you instead. In the Ubuntu Autosnort directory, there should be a Previous_Rel directory that contains the last version of Autosnort.
I've updated the readme in the Ubuntu directory to reflect this information. Provided there are no major deal-breaking problems in this pilot release, I hope to push support for Aanval to Debian and CentOS in the near future as well.
Update:
Aanval Post-Setup notes:
- It is highly advised that you reboot your system before continuing to the aanval console to continue the installation via the web interface. I ran into a problem prior to rebooting where the aanval console would not recognize that the php mysql module did exist and was loaded until the system was rebooted.
- During the initial setup, aanval will want to know the name of the aanvaldb user and password.
Username:snort
Password:password you gave the snort database user during the autosnort installation
- Aanval has a set of processes that are used to bring events over from the snort database that barnyard2 will dump to, and bring them over to the aanvaldb that aanval reads from. The console interface will let you know if they are not running. To start them, navigate to /var/www/aanval/apps and run idsBackground.pl -start --- I plan on adding an rc.local entry that will do this for you in the near future!
- In order for Aanval to manage events for your snort sensor you need to enable it on the aanval console. click the gear symbol in the lower corner of the web interface. This will bring you to a page called configuration. Click the "Settings" option under the "Snort" section. On the next page, check the enabled checkbox and enter the information for the snort database:
database name: snort
database hostname: localhost
database username: snort
database password: the password you assigned to the snort database user during autosnort installation
then click update. It may take a few minute for intrusion events to show up on the aanval interface. Be patient, they'll start coming in shortly!
- At this time, Autosnort does not support the sensor configuration options that Aanval includes. This is a limitation on my part; I need to figure out where Aanval expects to find the rule files, snort.conf and other configuration files before this will be possible!
- For more guidance and information specific to aanval, pay the folks at Tactical FLEX a visit at aanval.com
Happy St. Patty's day and happy snortin' to ye.
Friday, March 8, 2013
Snort 2.9.4.1 and some Minor Enhancements
Hello Autosnort Users,
As most of you may know, snort 2.9.4.1 released a few days ago. I didn't find out until about a day after it released. Since then I've been busy testing autosnort against all supported operating systems to ensure that everything continues to work like clockwork.
Here are the new features as reported on Snort.org:
Just to serve as a reminder for new autosnort users, If you are using the registered user ruleset from snort.org, we're in the 30-day holding period before a rule tarball with compatible SO rules will be released. This comes into play when autosnort walks you through installing your rules manually or via pulled pork.
During this time you will have to use a 2.9.4.0 rule tarball and disable the SO rules. If you choose to do this via pulledpork in the autosnort script, this is done for you automatically if you download the 2.9.4.0 rule tarball -- it is pulled down and pulled pork is configured to install text-based rules only.
If you happen to have a VRT subscription oink code, you can install the latest rule tarballs with no issues whatsoever.
Normally I would have had the blog post out sooner to correspond to the release of 2.9.4.1, but I made a couple of minor tweaks to the script that I've been meaning to do as well. So here are the improvements I have made:
1. During the snortreport install for all versions of the script, instead installing snortreport to /var/www/snortreport-1.3.3 (for Ubuntu/Debian) or /var/www/html/snortreport-1.3.3 (for CentOS), removed the trailing version number.
Now the directory path is /var/www/snortreport or /var/www/html/snortreport. This has been done to make it easier to point your web browser at the snort report front-end post-install. Now all you should have to type in your web browser is http://[ip address]/snortreport to gain access to the web UI.
2. Updated the online scripts to pull the barnyard2 source tarball from the barnyard2 github.
It was brought to my attention a little while ago over e-mail that the version of barnyard2 I was using in the script, as well as where I was downloading it from was a little dated and no longer recommended, so I rectified that. autosnort should be installing the latest master tarball via github now.
That's all for now.
Happy Snorting.
p.s: Most of you will notice that new code has been posted for every version BUT the BT5r3 version of the script that's because quite frankly, the BT5r3 version of the script had no need to be updated whatsoever. I tested the script and it works perfectly fine as-is. The backtrack script does not install a web front-end, or barnyard2. That can change if there's enough of a desire for it...
As most of you may know, snort 2.9.4.1 released a few days ago. I didn't find out until about a day after it released. Since then I've been busy testing autosnort against all supported operating systems to ensure that everything continues to work like clockwork.
Here are the new features as reported on Snort.org:
[*] Improvements
* Updated File processing for partial HTTP content and MIME attachments.
* Addition of new config option max_attribute_services_per_host and improve memory usage within attribute table.
* Handle excessive overlaps in frag3.
* Stream API updates to return session key for a session.
* Reduce false positives for TCP window slam events.
* Updates to provide better encoding for TCP packets generated for respond and react.
* Disable non-ethernet decoders by default for performance reasons. If needed, use --enable-non-ether-decoders with configure.
Just to serve as a reminder for new autosnort users, If you are using the registered user ruleset from snort.org, we're in the 30-day holding period before a rule tarball with compatible SO rules will be released. This comes into play when autosnort walks you through installing your rules manually or via pulled pork.
During this time you will have to use a 2.9.4.0 rule tarball and disable the SO rules. If you choose to do this via pulledpork in the autosnort script, this is done for you automatically if you download the 2.9.4.0 rule tarball -- it is pulled down and pulled pork is configured to install text-based rules only.
If you happen to have a VRT subscription oink code, you can install the latest rule tarballs with no issues whatsoever.
Normally I would have had the blog post out sooner to correspond to the release of 2.9.4.1, but I made a couple of minor tweaks to the script that I've been meaning to do as well. So here are the improvements I have made:
1. During the snortreport install for all versions of the script, instead installing snortreport to /var/www/snortreport-1.3.3 (for Ubuntu/Debian) or /var/www/html/snortreport-1.3.3 (for CentOS), removed the trailing version number.
Now the directory path is /var/www/snortreport or /var/www/html/snortreport. This has been done to make it easier to point your web browser at the snort report front-end post-install. Now all you should have to type in your web browser is http://[ip address]/snortreport to gain access to the web UI.
2. Updated the online scripts to pull the barnyard2 source tarball from the barnyard2 github.
It was brought to my attention a little while ago over e-mail that the version of barnyard2 I was using in the script, as well as where I was downloading it from was a little dated and no longer recommended, so I rectified that. autosnort should be installing the latest master tarball via github now.
That's all for now.
Happy Snorting.
p.s: Most of you will notice that new code has been posted for every version BUT the BT5r3 version of the script that's because quite frankly, the BT5r3 version of the script had no need to be updated whatsoever. I tested the script and it works perfectly fine as-is. The backtrack script does not install a web front-end, or barnyard2. That can change if there's enough of a desire for it...
Subscribe to:
Posts (Atom)