Showing posts with label Noise Reduction. Show all posts
Showing posts with label Noise Reduction. Show all posts

Friday, June 7, 2013

Autosnort: MSF edition

Working on some changes to Autosnort.

After seeing Darkoperator's MSF Installer script, I wanted to change how Autosnort operates a little bit.

You see, one of the cool things about metasploit is that It will log to hell and back if things are or are not working, but while you're actually running it, It only tells you what you need to know:
 "This is what I'm trying to do right now"
"This good thing happened"
"This bad thing happened"
"This is something to be aware of."

Keeping this in mind, and heavily ganking the notification printing code from darkoperator, in combination with a crazy named pipe trick I found via stack overflow, I'm changing how autosnort presents information.

As it stands currently, Autosnort, to put it bluntly, pukes all over the terminal/screen buffer. If something exploded in a bloody mess or failed to install correctly, you may see it and may be able to capture it in a screen shot and report it back to me, or you may not. Digital gods only know.

The way I've re-written the script, the only thing that gets printed to the screen are status messages indicating what the script is doing, good/bad messages to let you know if something failed or succeeded, and notifications that are sort of a "I need your input here" or "Pay attention to this!"

Apt-get (with the exception of the mysql-server installation that REQUIRES user input), configuration, compiler and all other output is redirected to log files.

For those of you who like seeing autosnort puke all over the screen or actually seeing what's going on behind the scenes, I found a nifty hack via stack overflow that uses mkfifo and named pipe magic to log everything from the script into a log file. No more running script to log output, no more screenshots, if you have a problem, you can e-mail me the output from the main autosnort script and/or the child interface installation scripts.

As of right now, I'm piloting this out on Debian, and it just looks slick. So far, I have the main script re-written and snorby's installation script re-written in this format. The other interface installers should fall in line easily enough, Snorby is the most complicated install I scripted out.

I haven't officially released either scripts out to github because I'm working on a couple other issues as well:

-- Support for both Debian 6 AND Debian 7 in the autosnort-debian main script
The only problem I'm seeing here at this point is that libmysqlclient.so (for barnyard2) installs in three different places in Debian 6 32/64 and Debian 7 32/64:
/usr/lib (Deb 6 32 and 64 bit)
/usr/lib/i386-linux-gnu (Deb 7 32)
/usr/lib/x86_64-linux-gne (Deb 7 64)

so I'm testing out a work-around involving the 'find' command and 'dirname' to fix this once and for all.

-- Support for Ubuntu 13.04
It's a short-term release, but some want to see support for it. It also appears to have an issue finding libmysqlclient.so

My theory is, if I can get find/dirname tested/working on Debian 6/7, likely I can get it to work between Ubuntu 12 and 13.04, hell I might be able to merge the Debian and Ubuntu autosnort scripts into a single script (since this was one of slight differences between the two).

So there's that.

I'm going to try and include a screenshot down here to preview what to expect, but so far, Blogger has somehow managed to not get my photos attached to my blog posts every time I've tried to use them.

First try:

Second try:
Via my twitter

Sunday, January 6, 2013

Introduction to Rule Tuning

Hello AS users,

I wanted to post some additional content outside of autosnort, but still related to snort. Seeing as how autosnort only gives you a Security over Connectivity ruleset if you use PulledPork to set the rule tree up initially, I figured that rule tuning may be a good topic of discussion.

Key complaints I've heard regarding snort is that
1) It's hard to set up
2) It's hard to tune out the noise and get alerts that are relevant without being inundated with false positives.
Well, Autosnort, Security Onion, RedBorder and several other open-source and corporate products that integrate snort were created to resolve complaint number 1, let's take a look at number 2.

So you have a rule that is very noisy. Noisy being defined as "Holy sh&!, Why do I have so many alerts?" How do I determine whether or not the given rule trigger(s) is a false positive?

Too much noise; not enough signal.

So you have a rule that is triggering a number of alerts. Time to start investigating. As a Security Professional, this is where you have to start earning your bread and butter: Investing possible alerts, security violations and Bad Things (tm) on your network. Bear in mind, every single network is different in its own way, but even so, there are basic questions you can ask to begin your investigation:

  • What is the alert?
  • What application/operating system/protocol is the rule triggering on?
  • Can I glean more information regarding this alert?
  • How often is the rule triggering?

For this exercise we're going to be picking on a rule for timthumb.php RFI attempts. For those not familiar with what ANY of that means, This rule exploits a vulnerability in a version of timthumb.php, a wordpress plugin. The vulnerability allows the attacker to perform a Remote File Include (RFI) attack -- the attacker can point the target web server to a remote file of their choice and the server will execute that file. Usually, the attack will point the web server to a web/php shell of their choice, giving the attacker the ability to control your web server with the same rights and access as the account being used to run that web server. This rule, at least a while ago use to be VERY noisy and generated loads of false positives, and would be an example of when you would use to rule tuning:

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"WEB-PHP Wordpress timthumb.php theme remote file include attack attempt"; flow:to_server,established; content:"timthumb.php|3F|"; nocase; http_uri; content:"src=http"; distance:0; nocase; http_uri; pcre:"/timthumb\x2ephp\x3f[^\r\n]*src=http\x3a\x2f[^\x2e\x2f]+\x2e[^\x2e\x2f]+\x2e[^\x2e\x2f]+\x2e/Ui"; metadata:policy balanced-ips drop, policy security-ips drop, service http; reference:bugtraq,47374; reference:url,code.google.com/p/timthumb/issues/detail?id=212; classtype:web-application-attack; sid:19653; rev:1;)

Let's answer the questions above:

What is the alert? "WEB-PHP Wordpress timthumb.php theme remote file include attack attempt"

The message string alone gives us a ton of information:

  • This rule is a rule from the web-php.rules category
    • This means that we are looking at web server traffic -- particularly, web servers running PHP.
    • Even more specifically, we see that this rule is concerned with WordPress, a popular Blog/CMS suite used for managing web-based content and blogs. It's easy to use, easy to maintain, can be loaded with plugins to extend its functionality, and makes a Security Analyst's job a living nightmare :).
      • Even further than that, we can see that this is a rule concerning timthumb.php, a specific Wordpress plugin.

A TON of information from a single string.

From this string we can answer the other question: What is the Application/Operating System/Protocol the rule is triggering on?

  • Web traffic going to your web servers in your HOME_NET
    • Running PHP, Wordpress and the timthumb WP plugin
 

In other cases, the Message string may not give you much information, so you will want to look at the rule header, the "flow" rule option, and/or any "reference" rule options to give you more clues on how to answer the second question. Let's take a look at the rule header for our example rule:

alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS

From this, we can glean the following:

  • We are alerting on TCP traffic from EXTERNAL_NET (usually set to 'any' or !$HOME_NET -- not HOME_NET systems)
  • Going to your HOME_NET systems on HTTP_PORTS -- ports normally associated with HTTP traffic. In other words: Web Servers in your HOME_NET.

Next, let's look at the flow option. The flow option tells us whether we are looking at traffic as a part of an established TCP session, and where the traffic should be coming from in terms of an established TCP session. See the snort manual entry for flow for more information. Let's take a closer look at flow in the rule:

flow:to_server,established;

We can determine the following from this string:

  • We want established TCP connections only
  • We want TCP sessions going to a server.

It may not seem like much, but in some cases where you see a rule firing repeatedly and you know that the destination IP in your home_net is a workstation, that can definitely be an eye opener.

Finally, let's take a look at the reference option in the rule. References are essentially metadata about the rule itself. Sort of like a "works cited" for the snort rule you can refer to glean detailed information about the rule -- this is how you would answer question 3: "Can I glean more information regarding this alert?" Most rules will have more than one reference, and if they do, it only benefits you and makes research that much easier. Let's refer back to our rule and see what references we have:

reference:bugtraq,47374; reference:url,code.google.com/p/timthumb/issues/detail?id=212;

We have two references here, a bugtraq ID you can submit to search security focus to extract in-depth information regarding the vulnerability and a url references you could literally copy and paste the url into your browser and go directly to a web page that gives you some kind of information regarding this vulnerability:

  •  Code.google.com/p/timthumb/issues/detail?id=212
You can read the articles on either webpage and discern tons of juicy information on this rule. We can confirm that this affects Wordpress sites utilizing timthumb.php or themes that use timthumb.php as a part of the theme itself. There are version numbers and dates as to when the bug was found, resolved, and fixed.

This will be the meat and potatoes of your investigation. Do you use the Application/Plugin/Operating system in question? If this is a backdoor/malware rule, Have you ran a virus scan or checked common places where the backdoor or malware may persist on a given host? You have some work to here and some correlations to be made :)

Finally, let's have a look at our final question, "How often the rule is triggering?" Some additional questions to ask include:
  •  Is there a pattern to how often the rule triggers?
  • Do you have a rule that's just rapidly spewing out alerts? 
  • Are the alerts being seen in any discernible intervals? 
  • At irregular times? 
  • When did the rule start triggering?

If we see that a rule or rules are triggering repeatedly, usually its an indicator of false positives, in which case, the people who manage your ruleset and/or the vendor that supplies your rules (in our Case, Sourcefire's VRT) would want to know about this and would want PCAPS, or packet captures of your false positives to determine what is wrong.

In other cases where you see a rule triggering irregularly at odd hours (e.g. outside of business hours) or in no set interval, then you may have an attacker looking for an easy-in, randomly probing your network or maybe a "bohr bugged" rule that triggers under certain conditions that need to be pinned down yet.

Still yet in other cases where you see a rule triggering at set intervals, it can almost always be attributed to automated attack software or malware, or maybe some sort of automated task that occurs that time period that could be causing the rule to trigger. Check the source or destination IP in question for any sort of automated tasks you may have scheduled to run around that time to try and rule this out as a possibility.

In closing, let's look at the four questions we asked at the beginning of this article, and answer them:

  • What is the alert? 
    • Look at the rule message to try and draw out discernible information. What rule category does the rule come from? What operating system/malware strain/Application does the rule message refer to?
  • What application/operating system/protocol is the rule triggering on?
    • Look at the rule header. Look at the flow rule option. What protocols are involved? Are we looking at client side applications? Server side services? What ports is the rule interested in?
  • Can I glean more information regarding this alert? 
    • Check the reference metadata option. Read up on any references provided. Do they provide you particular operating system/application software versions? Do you have that particular application on your network? At the vulnerable version? Do you use the particular operating system on your network? If the rule is regarding a piece of malware, can you find a characterization of that malware (e.g. What it does and where it persists) from an A/V vendor's website if the rule doesn't have a good enough reference? The rule reference metadata is probably the most useful portion of the rule to pay attention to aside from the rule message! 
  • How often is the rule triggering?
    • Is it constantly alerting? Regular Intervals? Irregular, seemingly random times?

From these pieces of information you can make a strong judgement to determine whether or not the given rule is a false positive. Once you have made this determination, What do you do next? Most of you may know about the suppression and threshold options to reduce rule noise or just turning off the rule entirely, but did you know about pass rules or snort's ability to use Berkeley Packet Filters/BPFs?

We'll cover that next time...


Until then, happy snorting!