Autosnort: MSF edition

Working on some changes to Autosnort.

After seeing Darkoperator's MSF Installer script, I wanted to change how Autosnort operates a little bit.

You see, one of the cool things about metasploit is that It will log to hell and back if things are or are not working, but while you're actually running it, It only tells you what you need to know:
 "This is what I'm trying to do right now"
"This good thing happened"
"This bad thing happened"
"This is something to be aware of."

Keeping this in mind, and heavily ganking the notification printing code from darkoperator, in combination with a crazy named pipe trick I found via stack overflow, I'm changing how autosnort presents information.

As it stands currently, Autosnort, to put it bluntly, pukes all over the terminal/screen buffer. If something exploded in a bloody mess or failed to install correctly, you may see it and may be able to capture it in a screen shot and report it back to me, or you may not. Digital gods only know.

The way I've re-written the script, the only thing that gets printed to the screen are status messages indicating what the script is doing, good/bad messages to let you know if something failed or succeeded, and notifications that are sort of a "I need your input here" or "Pay attention to this!"

Apt-get (with the exception of the mysql-server installation that REQUIRES user input), configuration, compiler and all other output is redirected to log files.

For those of you who like seeing autosnort puke all over the screen or actually seeing what's going on behind the scenes, I found a nifty hack via stack overflow that uses mkfifo and named pipe magic to log everything from the script into a log file. No more running script to log output, no more screenshots, if you have a problem, you can e-mail me the output from the main autosnort script and/or the child interface installation scripts.

As of right now, I'm piloting this out on Debian, and it just looks slick. So far, I have the main script re-written and snorby's installation script re-written in this format. The other interface installers should fall in line easily enough, Snorby is the most complicated install I scripted out.

I haven't officially released either scripts out to github because I'm working on a couple other issues as well:

-- Support for both Debian 6 AND Debian 7 in the autosnort-debian main script
The only problem I'm seeing here at this point is that libmysqlclient.so (for barnyard2) installs in three different places in Debian 6 32/64 and Debian 7 32/64:
/usr/lib (Deb 6 32 and 64 bit)
/usr/lib/i386-linux-gnu (Deb 7 32)
/usr/lib/x86_64-linux-gne (Deb 7 64)

so I'm testing out a work-around involving the 'find' command and 'dirname' to fix this once and for all.

-- Support for Ubuntu 13.04
It's a short-term release, but some want to see support for it. It also appears to have an issue finding libmysqlclient.so

My theory is, if I can get find/dirname tested/working on Debian 6/7, likely I can get it to work between Ubuntu 12 and 13.04, hell I might be able to merge the Debian and Ubuntu autosnort scripts into a single script (since this was one of slight differences between the two).

So there's that.

I'm going to try and include a screenshot down here to preview what to expect, but so far, Blogger has somehow managed to not get my photos attached to my blog posts every time I've tried to use them.

First try:

Second try:
Via my twitter

Autosnorby Follow-up: Support to Debian officially released, distributed installations under testing.

Hello again,

As I had mentioned a few blog posts ago updates to autosnort and/or support for snorby were probably going to be staggered out across platforms, since I have to do testing to ensure that things aren't horribly broken on release, that I have the right packages for the right platform that puts the right stuff in the right place, etc.

That being said, porting over the changes to autosnort and support for snorby to Debian went fast. Like, really fast. I should have figured this would be the case since Ubuntu and Debian share similarity in packages installed and package management utilized. It was almost quite literally a copy/paste job -- copy the code from ubuntu autosnort and it its child shell scripts, integrate it in with debian's existing scripts (and some custom things that Debian needs to work right) and just test it to make sure nothing exploded in a fit of errors and incompatibility.

Therefore, it brings me great pleasure to release a new version of autosnort for Debian as well as a snorby child shell script for Debian-based systems, and code improvements to debian autosnort and child shell scripts.

I'm not going to bother going over all of the changes I made to the script(s), since more or less the changes are practically identical to the ubuntu/autosnorby post I made yesterday (e.g. code cleanup, doing things that make sense to make the code easier to manage, installing relevant packages only when they are needed for a particular interface/function, etc.) except to note that I fixed a small bug in the remote database/experimental distributed sensor installation support (In both Ubuntu and Debian)

Speaking of the distributed installation support, I have a rough guide to doing a distributed install. This is very, very rough right now, expect some official or at least "better" guidance soon:

1. Perform a standard autosnort install on your "master console" system, and choose the interface you wish to install. Bear in mind that aanval requires licensing to support more than one snort sensor, and that snort report doesn't really clearly delinate which sensor an event came from. Reboot.

2. Edit /etc/mysql/my.cnf and reconfigure mysql to accept connections globally (e.g. listen on 0.0.0.0; also note that you'd better have a security measure of some sort up to limit connections to your other sensors (e.g. a firewall) if you plan on having this system accessible in *any* way over the internet)

3. run mysql -uroot -p[password here]

GRANT ALL ON snort.* TO snort@'1.2.3.4' IDENTIFIED BY 'my_password'; 

snort@'1.2.3.4' should be the snort database user on the remote sensor you plan to have store events on your master console, likewise the IDENTIFIED BY 'my_password'; should be the password you place into barnyard2.conf to talk to the master console.

4. Run the autosnort installer on your barebones sensor (choose option 2 to NOT install apache or mysql), somewhere down the line, the script will ask if you have a remote database that barnyard2 will be logging into. Enter the username (snort), the remote database name (snort), the remote database ip (e.g. the management interface for your 'master console' sensor), and the password you entered above on the mysql statement on the master console

5. finish the autosnort install (when it gets to the what interface do you want to install phase, just select option 6 for no interface at all.)

6. On the barebones sensor, install mysql-client (apt: apt-get install mysql-client) and try to connect to the database on the master console:

mysql -uroot -psnort -h [master console ip address]
use snort;
show tables;

If you get output, and a list of database tables, that's a very good sign. Otherwise you have some troubleshooting to do. note that by default autosnort installs libmysqlclient to allow barnyard2 to talk to the remote database, however, I'm having you install mysql-client in the event you want to troubleshoot and verify that you can talk to the database as a regular user. libmysqlclient won't let you test things out with manual queries, etc.

I can't re-iterate this enough, but allowing mysql to listen globally opens up some security risks (which is why I've trying to move to having the traffic SSL wrapped and tunneled between sensors/consoles...), so I would most definitely make sure the network your barebone sensor and master consoles are talking over is protected, trusted, and your master console host is firewalled to hell and back, only allowing 3306/tcp connections from your barebone sensors. Know the risks before you take this leap.

Check out the autosnort github and happy snorting!

Cheers,

DA_667

Autosnorby, updates to autosnort, and the updated to-do list

Hello Snort and Autosnort users.

I've got a lot to talk about today, so let's get started.

First and foremost, I've finally managed to work in support for snorby into autosnort. The snorby script installs the necessary packages, a compatible version of ruby (via rvm), all the necessary gems and configures the right files with the right info needed to get snorby up and running. Additionally, I've managed to work out away to drop privileges from the root database user to another user (the snort user in our case)

One thing to be aware of: the delayed_job and the sensor cache jobs do NOT start up on boot. Initial runs of the script had me trying to put commands into rc.local to start the delayed_job and sensorcachejob commands on startup... but they aren't working. I'm working on having a reliable method to start them on boot, but until that is discovered, any time the system is restarted, you'll want to, at a minimum start the delayed_job script:

cd /var/www/snorby && ruby script/delayed_job start

optionally, these commands run the Sensor and Daily cachejobs as well:

cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'

There are options to start the snorby worker process via the web UI, but I find the direct approach to my liking.

Another note is that at first you may notice alerts coming in and displaying properly in the events tab, but the dashboard may not updated at first. If this is the case, on the dashboard page, there is an option to force cache update. Select this option, wait 10 seconds, and reload the page. the graphs should update.

Next up on the list, is the main autosnort script itself. I've done a bit of spring cleaning.

- I've reduced the number of packages installed by default with autosnort.
--By default all we install are the packages necessary to install daq, libdnet, snort, and barnyard 2 with the ability to write to a remote database (more on this in a minute)
--I introduced a quick check that asks the user if they plan on installing a web interface as a part of the autosnort installation, or if they just want a stripped-down barebones install. If you're not sure what option to pick, select option 1.

- I've added a couple of functions to the script near the beginning for pulled pork to facilitate code re-use. this is because:

-- I've added support to download snort rules from previous versions of snort; up to four versions prior.

Why: I ran into a little problem recently where the free rules for version 2.9.4.5 were not available yet, and snort version 2.9.4.6 had just came out. The current version of the script only checks for the current version of snort rules, and one version prior. So now, you have a choice for four versions of rules:

1) the current version (if you have a VRT subscription oink code; e.g. 2.9.4.6)
2) the previous version (the kind that registered users usually get access to; e.g. 2.9.4.5)
3) two more prior versions (in the event that they are needed and/or a similar situation arises)

- If the user has elected to not install a web interface, the script asks them if they have a remote database they want barnyard to log to. I haven't officially tested this feature yet, but theorhetically, this allows for distributed installations... Any volunteers to test this? :)

- The child shell scripts all have functionality to install the packages necessary for them to work individually Again to reduce clutter and excess packages installed by autosnort (e.g. reduce attack surface)

- thanks to DK1844, I have a couple of code cleaniness fixes that I have added, and future todo list items.

I've achieved most of the stuff I intended to do when I initially started this little project, but I still have more that I can do.

Speaking of to-do list items:

- SSL for all web interfaces (generate a default, self-signed SSL cert, use mod-rewrite and mod-ssl to force ssl usage for all web interfaces)
- Further testing with distributed installations (Can I get a master console to register events from a remote system?)
- Encrypted comms between master console and remote sensor (e.g. have barnyard dump database updates to the master console over a secure channel)
- Kali Linux testing! Since Kali is fully compliant with Debian standards, will the debian autosnort script work with it?


The updates to autosnort.sh, snorby, aanval, snortreport and base child shell scripts are being pushed to ubuntu tonight, with support for other operating systems (debian and centOS) to come soon.

Report bugs, problems, praise, feature requests, etc. to:

deusexmachina667 [at] gmail [dot] com

As always the code is on my github repo.

Happy snorting,

DA_667

Snorby script progress, and a rundown of how to perform the installation.

Hello Autosnort users,

Still working on the Snorby release for Autosnort, and even then, It's probably going to be a staggered release -- first Ubuntu, then Debian, and Finally moving to CentOS. I imagine that based on some of the issues I have been experiencing, Getting Snorby to play with with SELinux is going to be a lot of fun.

I managed to get it to work with Aanval, snortreport and BASE, so I can do it again.

Anyhow, for those of you who can't wait for me to release a script, here's a basic run-down of the commands I'm running in Ubuntu 12.04 that  I've used that work. this assumes you've ran autosnort up to the point where web interface installation takes place.

Don't be alarmed when it says you're downloading 400m of packages. that's about right:

apt-get install libyaml-dev git-core imagemagick libmagickwand-dev wkhtmltopdf libssl-dev libxslt1-dev libsqlite3-dev libmysql++-dev libcurl4-openssl-dev apache2-prefork-dev default-jre-headless

this installs rvm. rvm is a nice little program that can be used to automatically install ruby:

\curl -\#L https://get.rvm.io | sudo bash -s stable
/usr/local/rvm/bin/rvm autolibs enable
source /etc/profile.d/rvm.sh

this is a cute little hack that I've used for installing the latest stable version of snort, DAQ, and determine what version of snort rules are available via snort.org. From what I gather ruby updates frequently. very, very frequently. This string of commands tells us the latest ruby 1.9.x release available for installation. Why not install ruby 2.0.0? because snorby doesn't support it yet. Submitted an issue to snorby's github. We'll see where that takes us:

wget http://ruby-lang.org/en/index.html -O /tmp/index.html
rubyver=`cat /tmp/index.html | grep -e "ruby-" | head -4 | grep released | grep -v Continue | cut -d"/" -f8 | cut -d" " -f2 | tail -1`
rvm install ruby-$rubyver

After all the packages you need to install, there's also a boatload of gems that need installation. and oh, we haven't pulled down the snorby web UI yet:

gem install thor i18n bundler tzinfo builder memcache-client rack rack-test rack-mount rails rake rubygems-update erubis mail text-format sqlite3 daemon_controller
gem install passenger
update_rubygems
cd /var/www/
git clone http://github.com/Snorby/snorby.git
cd snorby/config

Make copies of database.yml.example and snorby_config.yml.example and rename them to database.yml and snorby_config.yml:

cp database.yml.example database.yml
cp snorby_config.yml.example snorby_config.yml

modify snorby_config.yml. this should be the only change that needs to be made:
sed -i 's/usr\/local\/bin/usr\/bin/' snorby_config.yml

modify database.yml the snorby section at the top, you MUST enter the root database user's password. this is necessary for the rake snorby:setup portion

Next, we have to compile the mod_passenger apache module. this will take a little bit of time.
passenger-install-apache2-module

If the passenger-install command above bombs, try running "gem install passenger --pre" and running the command again.

The passenger-install command will ask you to modify a couple of files. First we have to point apache to the passenger module:

echo "" >> /etc/apache2/apache2.conf
echo "# This stuff is to make Snorby work properly mod_passenger is required for snorby to work." >> /etc/apache2/apache2.conf
echo "" >> /etc/apache2/apache2.conf
echo "LoadModule passenger_module /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6/libout/apache2/mod_passenger.so" >> /etc/apache2/apache2.conf
echo "PassengerRoot /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6" >> /etc/apache2/apache2.conf
echo "PassengerDefaultRuby /usr/local/rvm/wrappers/ruby-$rubyver/ruby" >> /etc/apache2/apache2.conf

Next, we create a virtualhost for snorby and set the documentroot to snorby's public directory, then disable the default site. This effect means that if you navigate to http://[ip address] you automatically get the index page for snorby:

echo "<VirtualHost *:80>" >> /etc/apache2/sites-available/snorby
echo "     ServerName snorby.localhost" >> /etc/apache2/sites-available/snorby
echo "     # !!! Be sure to point DocumentRoot to 'public'!" >> /etc/apache2/sites-available/snorby
echo "     DocumentRoot /var/www/snorby/public" >> /etc/apache2/sites-available/snorby
echo "     <Directory /var/www/snorby/public>" >> /etc/apache2/sites-available/snorby
echo "          # This relaxes Apache security settings." >> /etc/apache2/sites-available/snorby
echo "          AllowOverride all" >> /etc/apache2/sites-available/snorby
echo "          # MultiViews must be turned off." >> /etc/apache2/sites-available/snorby
echo "          Options -MultiViews" >> /etc/apache2/sites-available/snorby
echo "     </Directory>" >> /etc/apache2/sites-available/snorby
echo "</VirtualHost>" >> /etc/apache2/sites-available/snorby





All this has just been setup so we can successfully run bundler and rake to actually install snorby:

cd /var/www/snorby

bundle install --deployment

note: if the bundle install command fails or gives you an error referencing to the gemlock file or a gem called psych shield, you may have to run "bundle install --no-deployment" then run "bundle install --deployment" for things to work right.

finally, the moment you've been waiting for...:

rake snorby:setup

note: do not try to create the snorby database prior to running rake snorby:setup. If the database already exists, rake will bomb and exit.

At this point, provided rake snorby:setup ran, you should have a web interface that you can log into. default creds for snorby are: snorby@snorby.org with a password of snorby.

If you reboot your sensor, the worker jobs and the delayed_run jobs, processes necessary for populating the dashboard on the web UI do not automatically start with the system. You need to run these commands on reboot:

cd /var/www/snorby && ruby script/delayed_job start
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'

What I am working on:
- Every time I've ran the installation, I've varied one thing (e.g. the first time through I tried installing with ruby 2.0.0, the second time around, I installed and found out daemon_controller was a needed gem that no install guide referenced, the third time around I tried installing by creating the snorby database and granting the snort database user access to the database. This is also when I encountered the problem with bundler complaining about the gemlock file and the psych shield gem and had to re-run bundler with the --no-deployment option.

I need to re-run the installation until I get absolutely zero errors. I want this fully repeatable with no errors.
I want to investigate the possibility of editing the database.yml post-installation: using the snort database user, and using mysql commands to grant the necessary privs to the snort mysql user instead of root. Having the root mysql user's privs in plaintext  in a file that's world readable by default sounds like a terrible idea to me.

possibly adding the delayed_job and SensorCacheJob commands to rc.local to ensure they are ran upon reboot.

Getting support for Snorby is my highest priority right now. After that is done, I have some efficiencies and changes to autosnort I'll talk about in another post. Until then,

Cheers from DA.

Sometimes the old ways are the best ways (support for BASE and Syslog_Full output)

Hello AS/snort users,

Got two new features in this latest build of autosnort.

First and foremost is support for the BASE web interface. BASE is by and far one of the older and more well-known web interfaces for snort alerting. While the interface isn't going to win for prettiest interface, it is by and far very functional. Some of the cool things include the ability to download alerts as pcap and actual tshark/pcap parsed output for each alert.

This has been added as an installation option in the interface selection menu. The installation is painless and straight forward. After the system reboots, just point your browser to http://[your ip]/base.

The web page initial setup will check for all of the required pre-requisite settings on the first page. On the second page, it will ask you where adodb, a pre-requisite package, is installed. Autosnort and most linux distros install adodb to /usr/share/php/adodb (put that in for the path to adodb). It will also, of course ask you for database information and credentials:
database name: snort
database host: localhost
database port: 3306 (can leave blank if you'd like)
database username: snort
password: (the snort database user's password)

Unless you have an archive database configured, leave that part blank.
On the last page, it will ask if you want to set up auth to get into the web page to review events. That is entirely up to you.

After that, you should be all set to get events to the BASE web interface.

The next feature I have to announce is support for syslog output as another output interface choice. I've added a child script that configures barnyard2 to output to syslog_full format to port 514/udp. I have not FULLY tested this feature, but preliminary tests show that splunk picks up and parses the events with no issues whatsoever. A couple of troubleshooting recommendations:

-Ensure that the sensor's management interface, if it is firewalled, has port 514/udp open and allowed outbound.
-Conversely, ensure that your SIEM or syslog aggregator has port 514/udp open inbound to accept events -- I hosted splunk on a CentOS box and was puzzled why tcpdump says we were sending events out, but they were never making it to be indexed by splunk. I ran system-config-firewall-tui and added a rule for 514/udp inbound (and port 8000/tcp inbound for the actual web UI), and everything was fine.

The child scripts have been made available to Debian, Ubuntu and CentOS.

One last thing:

I've made some minor changes to the script at the recommendation of maintainer of the barnyard2 project. Instead of using sed to modify the copy of barnyard2.conf included with the barnyard2 source, I'm making the script generate a barnyard.conf on-the-fly.

- the script keeps a copy of the original barnyard2.conf in /usr/local/snort/etc/ named barnyard2.conf.origin
- the script generates a barnyard2.conf with only the directives necessary to run barnyard2 (e.g. where are the reference and classification config files, the gen/sid-msg.map files, input format to accept, output format to output to, etc.)
- if you choose to configure output to syslog format, the child script disables output to the database automatically.

As always, the scripts are up on github, along with updated READMEs. Previous versions are available in the Previous_Rel directory as well.

Happy Hunting..

DA_667

Redhat/CentOS init script!

Hello AS and regular snort users,

I wanted to announce that I have developed an init script compatible with CentOS/Redhat variants for autosnort installations (of course, this script can easily be adapted to other snort snort installations.

This init script can be used to replace /etc/rc.local as the primary method of starting up snort and barnyard2, and includes the added bonus of allow you to start/stop/restart snort and barnyard2 without requiring a reboot or sourcing /etc/rc.local if you need to make changes to snort or barnyard2. To add this script to CentOS 6.x perform the following tasks as root (or via sudo/root permissions):

    1. Copy the snortbarn script to /etc/init.d
    2. Edit the variables near the top of the script to suit your snort installation (the only variable that you should need to modify is the snort_iface variable if you installed snort/barnyard2 via autosnort)
    3. Make the snortbarn script is executable for the root user (chmod 700 snortbarn)
    4. Run chkconfig --add snortbarn
    5. Remove the entries for ifconfig, snort, and barnyard2 from /etc/rc.local (note: you may want to make a backup of the rc.local script in case you run into bugs/problems with the init script!)
    6. Kill your current snort/barnyard processes that ran from rc.local (killall snort && killall barnyard2)
    7. Run the command "service snortbarn start"
    8. check the process list to ensure that snort and barnyard2 are running after calling the init script. ( "ps -ef | grep snort" will return snort and barnyard2, if either/both processes are running. If only one process or the other is visible, something is wrong)

Troubleshooting steps: I'm not entirely sure why but there are CRLF/LF formatting problems with this script. If you get a bunch of errors stating that a file/command doesn't exist, try running dos2unix on the file to resolve the CRLF/LF errors.

If you install the init script and upon reboot find that only the snort process is running, it is because the init script for snortbarn ran BEFORE the init script for mysqld ran. To Determine when mysqld is configured to run it its runlevels, check /etc/init.d/mysqld. You'll want to pay attention to this line in particular:

# chkconfig: - 64 36
the first number, 64 indicates what number the rc startup script will get on startup. Linux rc scripts determine what services run or are killed on a particular run level. Every rc script as a K for Kill order, an S for Start order, followed by a number and the name of the symlinked script from /etc/init.d. RC scripts are read in numeric order. So if the rc script for snortbarn has an S number lower than 64, it will run before mysqld. snort will start up fine, but barnyard fails because it has no database to connect to.

To remedy this, you can modify the /etc/init.d/mysqld script to have a lower number than the snortbarn script in any of the /etc/rc[2-5].d directories, or modify the snortbarn script to have a higher number than the mysqld directory. This is a little confusing, so let's look at an example:

run this command: ls -al /etc/rc?.d/S*snortbarn

this command shows you each runlevel snortbarn is configured to start on.

now, run this command: ls -al /etc/rc?.d/S*mysqld

this command shows you the runlevels mysqld is set to run on. Don't worry about how many results you get.

If mysqld's number is higher than snortbarn's number, the mysqld process will not be running before snort and barnyard are configured to run. No database running means barnyard2 won't run. Let's say snortbarn had a number of 63, and mysqld has a number of 64. edit /etc/init.d/mysqld and change the chkconfig to something like this:

# chkconfig: - 62 36

save your changes and run chkconfig --add mysqld. This should fix the problem.

I've added the above documention to the other notes section of the CentOS readme. The snortbarn script is now available via github.

Special Note: Would like to thank Tactical FLEX/Aanval for hosting the initial version of this init script that I based this one off of. Would also like to thank Mike Miller for the initial idea of moving away from rc.local and building out a legitimate init script for autosnort.

Happy Snorting,

DA667

Snort 2.9.4.1 and some Minor Enhancements

Hello Autosnort Users,

As most of you may know, snort 2.9.4.1 released a few days ago. I didn't find out until about a day after it released. Since then I've been busy testing autosnort against all supported operating systems to ensure that everything continues to work like clockwork.

Here are the new features as reported on Snort.org:

[*] Improvements

* Updated File processing for partial HTTP content and MIME attachments.
* Addition of new config option max_attribute_services_per_host and improve memory usage within attribute table.
* Handle excessive overlaps in frag3.
* Stream API updates to return session key for a session.
* Reduce false positives for TCP window slam events.
* Updates to provide better encoding for TCP packets generated for respond and react. 
* Disable non-ethernet decoders by default for performance reasons. If needed, use --enable-non-ether-decoders with configure.

Just to serve as a reminder for new autosnort users, If you are using the registered user ruleset from snort.org, we're in the 30-day holding period before a rule tarball with compatible SO rules will be released.  This comes into play when autosnort walks you through installing your rules manually or via pulled pork.

During this time you will have to use a 2.9.4.0 rule tarball and disable the SO rules. If you choose to do this via pulledpork in the autosnort script, this is done for you automatically if you download the 2.9.4.0 rule tarball -- it is pulled down and pulled pork is configured to install text-based rules only.

If you happen to have a VRT subscription oink code, you can install the latest rule tarballs with no issues whatsoever.

Normally I would have had the blog post out sooner to correspond to the release of 2.9.4.1, but I made a couple of minor tweaks to the script that I've been meaning to do as well. So here are the improvements I have made:


1. During the snortreport install for all versions of the script, instead installing snortreport to /var/www/snortreport-1.3.3 (for Ubuntu/Debian) or /var/www/html/snortreport-1.3.3 (for CentOS), removed the trailing version number.

Now the directory path is /var/www/snortreport or /var/www/html/snortreport. This has been done to make it easier to point your web browser at the snort report front-end post-install. Now all you should have to type in your web browser is http://[ip address]/snortreport to gain access to the web UI.

2. Updated the online scripts to pull the barnyard2 source tarball from the barnyard2 github.

It was brought to my attention a little while ago over e-mail that the version of barnyard2 I was using in the script, as well as where I was downloading it from was a little dated and no longer recommended, so I rectified that. autosnort should be installing the latest master tarball via github now.


That's all for now.

Happy Snorting.

p.s: Most of you will notice that new code has been posted for every version BUT the BT5r3 version of the script that's because quite frankly, the BT5r3 version of the script had no need to be updated whatsoever. I tested the script and it works perfectly fine as-is. The backtrack script does not install a web front-end, or barnyard2. That can change if there's enough of a desire for it...

Autosnort offline build Ubuntu 12.04 i386/x86_64 available!

Hello Autosnort Users!

It's been a long time since I've posted. My day job keeps me plenty busy. I apologize for the last of responsiveness over e-mail. If I haven't gotten to you it's not because I'm an unsociable jerk it's that I'm quite literally exhausted at the end of the day. I do work for the federal government right now helping them with their infosec things and my days are NEVER boring, suffice to say.

That being said, I wanted to announce that after what seems like forever I have new updates to autosnort, namely an offline version of the script that I've released for ubuntu 12.04 today.

Some of you may be wondering why would I go through the trouble? What good is it? How does it help me?

Well, to the average user, it's not likely to be terribly helpful, but to users in situations where you have to sneakernet software from point A to point B, or from a network with good internet access to one without internet access or very limited internet access, even for hackers and infosec enthusiasts that are participating in "Capture the Flag" events with limited internet access, an offline build of autosnort providing the latest version of snort and snort rules easily seems like a godsend.

 The offline script, or scripts I should say, come in two stages:

Stage 1: as-offline-stage1.sh

When this script is ran on a system that is identical to your offline system (meaning a system running the same operating system (Ubuntu), Version (12.04), and architecture (32-bit/i386 || 64-bit/x86_64)), it will download all the required packages via apt-get, download them (without installing them on this system), grab the latest version of snort and daq as well as barnyard2, snortreport, and libdnet.

A copy of the perl script, create-sidmap.pl and dpkorder$arch.txt (where $arch is either x86_64 or i386) should be placed on this system as well; the script expects to find these two files so that they can be included in the tar file that has all the packages (.deb and .tar.gz) needed for the stage2/offline installer script.

Stage 2: as-offline-stage2.sh

This script is ran on the offline system you plan on installing snort to. The script requires the tarball generated from the stage1 script as well as a VRT rules tarball to work properly.

Essentially the script installs all the .deb packages in the CORRECT order, then installs all the downloaded source packages, much the same as the online version of the autosnort script (yay for code re-use!)

The end-product is a fully functional snort install, just like the online autosnort script.

Please note that create-sidmap.pl is a part of the Oinkmaster suite. In layman's terms, the Oinkmaster suite is a precursor to PulledPork. The Oinkmaster suite is released under the BSD license, by Sourcefire. I just want to make it abundantly clear here and now that this is NOT my software and if requested by the original creator of the script I will comply and remove it. It is included on my github page currently as a convenience and nothing more.

The reason the create-sidmap script is included is to assist offline users in generating a new sid-msg.map file. I noticed in the process of creating this script, that barnyard2 would log alerts just fine, but then snortreport wouldn't show me what the alert's message was only "snort alert [sid number]" which was very unhelpful. I eventually discovered this is due to the sid-msg.map file. Essentially, the sid-msg.map file is responsible for "mapping" the snort SID to the snort msg field in the rule, thus giving you the rule sid and message when you look at a rule via snortreport or the web interface of your choice instead of "snort alert" followed by the sid number. create-sidmap.pl resolves this problem quite easily.

I hope you all find the scripts useful!

Until next time, happy snorting.

New year, new ideas (introduction to while/true case statements)

Howdy Autosnort users,

It's been a while since I've posted anything meaty to the blog. My day job keeps me very busy and the projects I've been working on have been very intense. At this point I really only have time to work on autosnort over weekends. I appreciate all the support and questions and identification of bugs or problems with the script; you're helping me, the snort community and the greater open source community just by asking questions and reporting bugs, so thank you very much. I may not be very fast to respond, but I will ALWAYS do what I can as soon as I can.

That being said, I'm going to post a quick tutorial to bash while/true loops, and how they can be used to provide some fault tolerance in your shell scripts (or at least how it's helped me tremendously)

First and foremost, what is a while true loop? A while true loop is a do while statement that is configured to loop infinitely. While do loops usually go something like this:

#!/bin/bash
x=0
while [ $x -le 10 ]; do

        echo "$x"
        x=$(( $x + 1 ))

done
exit 0

x is a variable initialized to zero in the example above. The while statement reads "while x is less than or equal to 10 do the function below.

the function below prints the value of x, then increments it by 1. Once x is greater than 10, the loop, then the script exits. This script will print numbers 0-10, one on each line.

A while true loop looks something like this:

#!/bin/bash
while true; do
       echo "Hello!"
done
 exit 0

this statement is much much simpler, but lets run through it anyhow. The do/while statement is just "while true; do" which, in english, means while true is true (which will ALWAYS be the case) do the statement below

The statement below just prints "Hello!" ad infinitum. If you were to run the script above, you'd have to kill the script or Ctrl+C the script to stop it. Not very useful, huh? What if I told you that you could control whether or not the loop progresses or not with certain keywords?

The keywords break and continue in a loop are ways to control how a bash loop progresses. The break statement says "Stop the loop now, jump immediately to done, do not process anymore lines in the loop." while the continue statement says "Stop here, go back to the beginning statement in the loop."

Are you starting to see how this can be useful?


Case statements can be used to give the use of a script a menu or "branch" a script, performing a certain action when a variable, or output is a certain value.

#!/bin/bash
read -p "this is a case statement!
Your choices are:
1
2
Please enter your choice: " case1

case $case1 in
      1)
             echo "you have chosen case statement 1!"

      ;;
      2)
             echo "you have chosen case statement 2!"
      ;;
      *)
             echo "invalid choice!"
      ;;
esac
exit 0

the above statement asks the user to input 1 or 2 as an option for the case statement. that value is stored in the variable case1. The case statement is basically a large collection of if/then statements -- "if the value of case1 equals 1, then do this. if it equals 2, then do this. if it equals anything else, this is a catch-all statement to do, usually indicating the user picked an invalid choice."

Nothing too mind-breaking. This is stuff you're taught in your intro to UNIX classes in college. The fun part comes combining the concept of a case statement with a while true loop. and using continue or break keywords to continue the loop or break it:

while true; do

        read -p "this is a case statement var choices are 1 or 2." case1
        case $case1 in
                1)
                        echo "this is a result of selecting 1."
                        break;
                ;;
                2)
                        echo "this is a result of selecting 2."
                        break;
                      
                ;;
                *)
                        echo "invalid choice!"
                        continue;
                ;;
        esac
done
exit 0

 Let me break down what's going on above:

We begin with a while true statement, starting off our "infinite" loop. The next statement tells the user to select 1 or 2 as options in our case statement. This is normally where you'd print a menu (if a user is expected to interact here) and explain what the options do, or just launch straight into your case selection if this is a fully automated script. After printing the statement and taking the user's input, we go straight into our case statement. The statement says "If case1 equals 1 do this, if it equals 2, then do this, if it equals anything else, do this."

What's different between this case statement and the previous one I used as an example is the existence of the keywords break and continue in these statements. The break statement says "Terminate the loop and jump straight to done." No more processing, no more looping, just move on. The break statement at the end of option 1 and 2  is a way of identifying that these are the only valid cases for this bit of bash code, that in order to continue script execution, the value of case1 must be "1" or "2" (or the user has to cancel the script via kill or ctrl+c). the continue statement in our catch-all case is a way if forcing the user to select a valid choice to continue script execution. In this way it's a sort of safety net that doesn't allow the script to continue unless a valid choice is made. It's a safety net for accidents, fat-fingering or hitting the wrong keys.

Soon as I figured out how to do this, I littered them throughout the script to make the script more fault tolerant. Instead of throwing an error and exiting in a case statement, autosnort just loops back and informs the user their choice was invalid, or didn't work for whatever reason, instead of blindly plowing forward, or exiting, leaving the user with a mess. It's still imperfect, but MUCH improved over the original version.

Currently I'm working on nesting multiple while/true case statements together, something like this:

 #!/bin/bash

while true; do

        read -p "this is a case statement var choices are 1 or 2." case1
        case $case1 in
                1)
                        echo "this is a result of selecting 1."
                        break;
                ;;
                2)
                        while true; do
                                read -p "this is another case statement. var choices are 1 or 2." case2
                                case $case2 in
                                        1)
                                                echo "result of the second case statement"
                                                break;
                                        ;;
                                        2)      echo "NARF!"
                                                break;
                                        ;;
                                        *)
                                                echo "Invalid choice!"
                                                continue;
                                        ;;
                                esac
                        done
                        break;
                ;;
                *)
                        echo "invalid choice!"
                        continue;
                ;;
        esac
done
exit 0

See if you can figure it out on your own. I hope to implement multiple output options (web UIs, etc.) in this manner to make the script more foolproof, and offer more choices to users.

Until next time,

Happy snorting!