Sunday, December 30, 2012

Dude are you still there?

Hello Autosnort users and readers of this blog.

I wanted to assure you that yes, I am still out there, I haven't abandoned this blog or this project and do not intend on doing so any time soon. I have just taken a very long holiday hiatus. Autosnort dev and regularly scheduled postings as well as responses to bugs and problems regarding autosnort will continue in the new year. I'm just taking a long moment's reprieve for the holiday to enjoy it among family and friends.. I know its a little late, but I hope you all enjoyed your holidays and I wish you all a happy new year.

Cheers,

DA

Monday, December 10, 2012

We have pulled pork; the rest is just barbecue sauce.

Hello AS users,

Today I would like to announce a new autosnort build and introduce support and integration of pulled pork.

For those who do not know, pulled pork is a perl script written by JJ Cummings of Sourcefire. The script is the recommended method for open-source snort users to manage, download, and configure rules for snort deployments.

What does pulled pork integration mean for you and autosnort? Its even easier to get started.

- The old rule tarball method is still available (e.g. point the script to a VRT tarball to unpack), but now, in addition to the tarball method, you may choose to use pulled pork to build an initial ruleset based off of the Sourcefire base ruleset "Security over Connectivity".

- If you provide autosnort with an oink code (registered user or VRT subscription) you are then given a choice to try to download the latest rules tarball, or a the rules tarball from the previous version of snort.

The reason I do this instead of automatically downloading the latest rule tarball from snort.org is to work around the 30-day wait period registered users have to wait when a new version of snort comes out.

To summarize a blog post in which I had described this problem in detail, when a new version of snort comes out, a rule tarball that has compatible SO rules is immediately made available for VRT rule subscription holders. Registered users have to wait 30 days from the release to gain access to a VRT tarball with compatible SO rules. If you choose to download/process rules from the tarball for the previous version of snort, autosnort will only process text rules to ensure compatibility with the latest version of snort -- this is to prevent the dreaded "my SO rules don't work" phase for registered users.

Some may wonder "It's taken an awful long time to integrate just this functionality. Have you been slacking off?" Well, no more than I usually do, but there is a very good reason for the delay in this release -- I've taken a long look at the rest of the script and decided to re-write large portions of the script.

A significant problem most users had with autosnort was that if you fat-fingered  at any point during the script, autosnort would happily plow forward, giving you a broken install upon reboot. I finally decided to go through the script and write some fault tolerance into the script -- key portions of the script are now sort of  "encapsulated" into while/true loops -- meaning that you MUST give the script a valid answer for it to progress.

For example, as a part of the rule management routine in autosnort, I decided to keep the old VRT rule tarball import (e.g. untar it and drop everying into /usr/local/snort)

Before, if a user gave the script an invalid directory path and/or nonexistent file, the script would go forward without a care in the world.

This time around however the script does the following:
-checks to see if the file exists. if the file doesn't exist, you are told to try again
-if it exists, tries to untar it. if the file cannot be untarred (e.g. not a .tar.gz) it informs the user and the user is given the chance to try again

For the pulled pork phase we do the same kind of error checking:
-if pulled pork fails to download rules for any reason, we inform the user and loop back to the choice between using pulled pork for rule management, or importing a VRT rule tarball through the old method.

Pulled pork integration is going to gradually be released for all supported operating systems (e.g. Debian, Ubuntu, BT5 and CentOS. The pilot release is going to be on CentOS, with releases to other OSes trickling in behind it.

As always, the code has been posted to github under the CentOS directory

autosnort-centOS-ppinteg.sh
readme

Interesting, but useless statistics:
-file size of the shell script has nearly doubled from the old autosnort centOS script
-originally:24.5KB
-now: 42.3KB
-diff: 17.8

-lines of code (includes comments and blank lines for readability)
-original: 553
-now: 879
-diff: 326


Monday, December 3, 2012

Snort 2.9.4 release today and the dreaded "My SO rules don't work!" phase has begun.

Just in from the snort release team, looks like snort 2.9.4 just went out today:

Snort 2.9.4 is now available on snort.org, at
http://www.snort.org/snort-downloads/ in the Latest Release section.

************
Please note:
2.9.3.1 & later packages are signed with a new PGP key
(that key is signed with the previous key).
************

Snort 2.9.4 includes changes for the following:

[*] New additions

  * Consolidation of IPv6 -- now only a single build supports both
    IPv4 & IPv6, and removal of the IPv4 "only" code paths.

  * File API and improvements to file processing for HTTP downloads
    and email attachments via SMTP, POP, and IMAP to facilitate
    broader file support

  * Use of address space ID for tracking Frag & Stream connections
    when it is available with the DAQ

  * Logging of packet data that triggers PPM for post-analysis via
    Snort event

  * Decoding of IPv6 with PPPoE

  * Added an API call to add a service to a host in the attribute table.
    Remove the unused live attribute update code.

[*] Improvements

  * Update to Stream5 PAF for handling gaps in the sequence numbers of
    packets being reassembled.

  * Selection of the Stream TCP policy based on the server rather than
    the destination of first packet seen by Snort

  * Allow disabling of global thresholds via a count of -1

  * Prevent blocking duplicate SYNs when using inline normalization

  * Add SSLv3 backwards compatibility support for SSLv2 ClientHello
    messages

  * Allow active responses to packets without data (eg, a TCP SYN)

  * Changed logic of option evaluations for shared library rules that
    use a custom evaluation function to match that of the builtin logic
    when the NOT_FLAG is used.  The 'NOT' matching now happens within
    each of the individual rule option evaluation functions.

  * Updated SMTP preprocessor to better handle commands that have
    corresponding data on a subsequent line to reduce false positives.
    3 commands fall into this category - X-EXPS, XEXCH50, and BDAT.

  * Improve support for encapsulated & tunneling protocols to block or
    fastpath a connection within the tunnel rather applying that to
    the whole tunnel.

Please see the Release Notes and ChangeLog for more details.

Please submit bugs, questions, and feedback to bugs@snort.org.

Happy Snorting!
The Snort Release Team
What this means for you:
- autosnort is going to automatically download the latest snort version. This means that if you are new to snort today and you are running autosnort on a fresh OS install, you're getting snort 2.9.4 today.
- you can still use the 2.9.3.1 ruleset for registered users if you don't have a VRT subscription however you've now entered the dreaded "my SO rules don't work" phase:
    - Since the VRT only updates the free rule releases to registered users 30 days *after* the paying users, this means that when a new version of snort comes out you do not have VRT rules tarball for the newest version of snort immediately, and usually will be waiting 30 days for them to release a rule tarball with SO rules that are compatible with the newest version of snort, unless you have a VRT rule subscription.
    - Snort will realize that the SO rules provided do not match the version of snort installed (e.g. you provide the autosnort script a 2.9.3.1 rule tarball) and will refuse to run upon reboot, giving you an error that looks something like this in /var/log/messages or /var/log/syslog depending on your distro:
 snort[1497]: FATAL ERROR: The dynamic detection library "/usr/local/snort/lib/snort_dynamicrules/web-misc.so" version 1.0 compiled with dynamic engine library version 1.16 isn't compatible with the current dynamic engine library "/usr/local/snort/lib/snort_dynamicengine/libsf_engine.so" version 1.17.
 The only solution I can provide you, snort of purchasing a VRT subscription is for you to delete the SO rules that snort is currently using. Autosnort installs your SO rules in the /usr/local/snort/lib/snort_dynamicrules/ directory. run the command:

rm -rf /usr/local/snort/lib/snort_dynamicrules/*.so
Afterwards either restart snort, via the following command:

/usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.conf -i eth1(or the interface you are sniffing traffic on)
 or simply reboot your sensor and let rc.local take care of things on reboot.

This will mean that you will lose SO rule coverage for the next 30 days, but unfortunately, there's no way around it -- aside from buying a VRT rule subscription.

Some may wonder why the VRT does this? Well, making snort rules is not easy -- At all. I know; I've met the people that write these rules and I know the hours that they keep, the amount of work they put in, and just how little thanks they get in return -- still, Sourcefire and the VRT need to make money somehow, and they do so via provided accurate rules that have to meet strict quality assurance guidelines in a timely manner; you can't really fault them for the amount of effort that goes into making sure the rules work and work well.

With that being said, I hope to have pulledpork integration working in the near future to be able to remedy this. Happy snorting, everyone!




Sunday, December 2, 2012

What's on my table right now, and how you can help if you want

I've already been asked a couple of times how people can help or contribute to autosnort, so this blog post is a rehash of what I sent to a user in an e-mail today. If you feel the need to contribute, or want to help out, here are the main things I'm working on currently, in order of priority:

1) Pulledpork.pl integration - I want to tear out the portion of the script that asks for the VRT tarball. Its far too easy to mess it up, and if it gets messed up you end up with a snort install that won't work because it has no rules to trigger against and either have to re-run the entire script, or manually untar the rules, copy them and copy the right SO rules for your distro.

- Want to use pulled pork as a solution for *initial rule setup*
- Give the user an initial ruleset, the scope of the project does not creep into configuration management, stays limited to initial snort installation and setup
- Will not support utilizing pulled pork with ET rules or Bleeding snort rules - considering this to be configuration management and outside of the scope of autosnort.
- Want to keep the VRT tarball check and insert a failsafe check that checks if the VRT rules tarball exists where the user says it is and if it doesn't, tells the user to try again instead of pushing forward.

2) Giving users a choice of where to log events and giving them their second choice: log to syslog

- configure snort.conf to log to syslog, disable barnyard installation
- user would then configure syslog config to forward IDS events to their SIEM
- the 'lightweight' alternative to MySQL, Barnyard and a web front-end

3) Giving users a choice of web front-ends to install and giving them their third choice: the BASE web frontend

- BASE has been around for a really long time
- Not very pretty, but very well documented

4) another web front-end choice: Aanval

- Aanval is another web front-end that looks very promising
- Can work with snort, suricata and syslog messages

5) snorby

- installation process is very, very painful
- likely to be very difficult to get set up right across multiple distros
- is highly a functional web front-end and looks amazing; the effort is worth it.

6) See if autosnort can be made compatible with earlier OS releases -- CentOS/Redhat 5.x, Ubuntu 10 and 11.04 LTS releases, BT5 r2, etc.

7) Expanding OS support upon request

This doesn't include bug fixes. If bugs are found, they jump to the top of the stack and have to be dealt with first.


So if you feel the need to help out there's tons that needs to be done. Pick any one of the 7 on the to-do list above.

The scripts are all on github, same as always and freely available for you to download.

In terms of beta testing, etc here are some of my recommendations:




1. Use virtual machines. Set up installations for Operating Systems you want to Beta for or develop against. Upon initial install, Take a snapshot of the OS so you can revert back to it. This will make testing new features and fixes much easier on you.

-ensure your VMs have at least 512mb of ram (1gb recommended) about 14gb of hdd (can be less if you're pressed for space) and at a minimum of two virtual NICs -- one for snort to bind to and the other for network access.

2. Good tools for the trade
-On windows, I use notepad ++ for writing the scripts, WinSCP for SCP transfers and MRemoteNG for managing multiple putty sessions to multiple autosnort boxes
-If you use linux for writing the scripts, gedit does an amazing job for writing, the command line scp command will do you just as well, and look at terminator for a tabbed terminal interface that can be split any way you'd like
-If on a mac textwrangler comes recommended as an editor, Fugu or the cli tools for SCP transfers, and iterm2 for tabbed terminal sessions that can be split any way you'd like.



3. Get a registered user account on snort.org
-Makes it easier to get your snort rule tarballs, specially if pulled pork integration takes off.



If you want to donate money to the cause, please  just donate your money elsewhere -- I'm not try to say that I don't need your money, but there are worthier causes than mine. Donate your money to charity, use it to buy yourself something nice, or use it to buy something for someone you love. This project is just my way of trying to make network security a little bit easier and I am in no way looking to gain anything monetarily from it.

Don't feel obligated to do any of this, and if you realize midway in that you're in too deep, ask questions. If I don't know it, I'll try to point you in the right direction. Some people in the IT community would like to think that they were born with a keyboard in their hands and are too good to answer "noob" questions. Everyone starts out as a "noob" and you only become better by asking questions.

...That being said, don't be afraid to research, in fact, make sure that you research before you ask questions, as the question may have already been answered. This is called asking questions the smart way (note: while the material of that article may seem a bit scathing, the information contained within is very, very valuable).

Thank you for your continued support,

DA