Sunday, May 5, 2013

Snorby script progress, and a rundown of how to perform the installation.

Hello Autosnort users,

Still working on the Snorby release for Autosnort, and even then, It's probably going to be a staggered release -- first Ubuntu, then Debian, and Finally moving to CentOS. I imagine that based on some of the issues I have been experiencing, Getting Snorby to play with with SELinux is going to be a lot of fun.

I managed to get it to work with Aanval, snortreport and BASE, so I can do it again.

Anyhow, for those of you who can't wait for me to release a script, here's a basic run-down of the commands I'm running in Ubuntu 12.04 that  I've used that work. this assumes you've ran autosnort up to the point where web interface installation takes place.

Don't be alarmed when it says you're downloading 400m of packages. that's about right:

apt-get install libyaml-dev git-core imagemagick libmagickwand-dev wkhtmltopdf libssl-dev libxslt1-dev libsqlite3-dev libmysql++-dev libcurl4-openssl-dev apache2-prefork-dev default-jre-headless

this installs rvm. rvm is a nice little program that can be used to automatically install ruby:

\curl -\#L https://get.rvm.io | sudo bash -s stable
/usr/local/rvm/bin/rvm autolibs enable
source /etc/profile.d/rvm.sh

this is a cute little hack that I've used for installing the latest stable version of snort, DAQ, and determine what version of snort rules are available via snort.org. From what I gather ruby updates frequently. very, very frequently. This string of commands tells us the latest ruby 1.9.x release available for installation. Why not install ruby 2.0.0? because snorby doesn't support it yet. Submitted an issue to snorby's github. We'll see where that takes us:

wget http://ruby-lang.org/en/index.html -O /tmp/index.html
rubyver=`cat /tmp/index.html | grep -e "ruby-" | head -4 | grep released | grep -v Continue | cut -d"/" -f8 | cut -d" " -f2 | tail -1`
rvm install ruby-$rubyver

After all the packages you need to install, there's also a boatload of gems that need installation. and oh, we haven't pulled down the snorby web UI yet:

gem install thor i18n bundler tzinfo builder memcache-client rack rack-test rack-mount rails rake rubygems-update erubis mail text-format sqlite3 daemon_controller
gem install passenger
update_rubygems
cd /var/www/
git clone http://github.com/Snorby/snorby.git
cd snorby/config

Make copies of database.yml.example and snorby_config.yml.example and rename them to database.yml and snorby_config.yml:

cp database.yml.example database.yml
cp snorby_config.yml.example snorby_config.yml

modify snorby_config.yml. this should be the only change that needs to be made:
sed -i 's/usr\/local\/bin/usr\/bin/' snorby_config.yml

modify database.yml the snorby section at the top, you MUST enter the root database user's password. this is necessary for the rake snorby:setup portion

Next, we have to compile the mod_passenger apache module. this will take a little bit of time.
passenger-install-apache2-module

If the passenger-install command above bombs, try running "gem install passenger --pre" and running the command again.

The passenger-install command will ask you to modify a couple of files. First we have to point apache to the passenger module:

echo "" >> /etc/apache2/apache2.conf
echo "# This stuff is to make Snorby work properly mod_passenger is required for snorby to work." >> /etc/apache2/apache2.conf
echo "" >> /etc/apache2/apache2.conf
echo "LoadModule passenger_module /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6/libout/apache2/mod_passenger.so" >> /etc/apache2/apache2.conf
echo "PassengerRoot /usr/local/rvm/gems/ruby-$rubyver/gems/passenger-4.0.0.rc6" >> /etc/apache2/apache2.conf
echo "PassengerDefaultRuby /usr/local/rvm/wrappers/ruby-$rubyver/ruby" >> /etc/apache2/apache2.conf

Next, we create a virtualhost for snorby and set the documentroot to snorby's public directory, then disable the default site. This effect means that if you navigate to http://[ip address] you automatically get the index page for snorby:

echo "<VirtualHost *:80>" >> /etc/apache2/sites-available/snorby
echo "     ServerName snorby.localhost" >> /etc/apache2/sites-available/snorby
echo "     # !!! Be sure to point DocumentRoot to 'public'!" >> /etc/apache2/sites-available/snorby
echo "     DocumentRoot /var/www/snorby/public" >> /etc/apache2/sites-available/snorby
echo "     <Directory /var/www/snorby/public>" >> /etc/apache2/sites-available/snorby
echo "          # This relaxes Apache security settings." >> /etc/apache2/sites-available/snorby
echo "          AllowOverride all" >> /etc/apache2/sites-available/snorby
echo "          # MultiViews must be turned off." >> /etc/apache2/sites-available/snorby
echo "          Options -MultiViews" >> /etc/apache2/sites-available/snorby
echo "     </Directory>" >> /etc/apache2/sites-available/snorby
echo "</VirtualHost>" >> /etc/apache2/sites-available/snorby





All this has just been setup so we can successfully run bundler and rake to actually install snorby:

cd /var/www/snorby

bundle install --deployment

note: if the bundle install command fails or gives you an error referencing to the gemlock file or a gem called psych shield, you may have to run "bundle install --no-deployment" then run "bundle install --deployment" for things to work right.

finally, the moment you've been waiting for...:

rake snorby:setup

note: do not try to create the snorby database prior to running rake snorby:setup. If the database already exists, rake will bomb and exit.

At this point, provided rake snorby:setup ran, you should have a web interface that you can log into. default creds for snorby are: snorby@snorby.org with a password of snorby.

If you reboot your sensor, the worker jobs and the delayed_run jobs, processes necessary for populating the dashboard on the web UI do not automatically start with the system. You need to run these commands on reboot:

cd /var/www/snorby && ruby script/delayed_job start
cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'

What I am working on:
- Every time I've ran the installation, I've varied one thing (e.g. the first time through I tried installing with ruby 2.0.0, the second time around, I installed and found out daemon_controller was a needed gem that no install guide referenced, the third time around I tried installing by creating the snorby database and granting the snort database user access to the database. This is also when I encountered the problem with bundler complaining about the gemlock file and the psych shield gem and had to re-run bundler with the --no-deployment option.

I need to re-run the installation until I get absolutely zero errors. I want this fully repeatable with no errors.
I want to investigate the possibility of editing the database.yml post-installation: using the snort database user, and using mysql commands to grant the necessary privs to the snort mysql user instead of root. Having the root mysql user's privs in plaintext  in a file that's world readable by default sounds like a terrible idea to me.

possibly adding the delayed_job and SensorCacheJob commands to rc.local to ensure they are ran upon reboot.

Getting support for Snorby is my highest priority right now. After that is done, I have some efficiencies and changes to autosnort I'll talk about in another post. Until then,

Cheers from DA.










5 comments:

  1. Hey,

    My name is Dustin and i'm the creator of Snorby. Good work thus far but wanted to add that both imagemagick and libmagickwand-dev are no longer needed. The readme on the Snorby repo should be updated to reflect that now. That should cut down on the size by a lot.

    ReplyDelete
    Replies
    1. Hi both of you.

      Dustin: Is there a script or any way to get the snorby process to start automatically at boot, or some init script that will be able to start|stop|restart it as a service?

      da_667:
      Please accept my notes on snort, snorby, barnyard2 and pulledpork installation:
      https://www.dropbox.com/s/u7s2fwbqfupksm4/snort-barnyard-pulledpork-snorby.doc

      I hope it can be to some help for you.
      99% of it is in English.

      /Rich

      Delete
  2. Thanks to your blog post and word doc, I've been able to setup Snorby 2.6.1!

    ReplyDelete
  3. @Rich:

    regarding the snorby process, If you add these lines to rc.local, they should run upon system reboot.

    cd /var/www/snorby && ruby script/delayed_job start
    cd /var/www/snorby && rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'

    these commands start the delayed_job process, and tells the system to run the Sensor and Daily CacheJobs immediately - this has the affect of immediately updating the snorby dashboard.

    ReplyDelete
  4. @da_667

    Alright, that's great, thanks a bunch.
    :)

    /Rich

    ReplyDelete