Hello Autosnort Users!
It's been a long time since I've posted. My day job keeps me plenty busy. I apologize for the last of responsiveness over e-mail. If I haven't gotten to you it's not because I'm an unsociable jerk it's that I'm quite literally exhausted at the end of the day. I do work for the federal government right now helping them with their infosec things and my days are NEVER boring, suffice to say.
That being said, I wanted to announce that after what seems like forever I have new updates to autosnort, namely an offline version of the script that I've released for ubuntu 12.04 today.
Some of you may be wondering why would I go through the trouble? What good is it? How does it help me?
Well, to the average user, it's not likely to be terribly helpful, but to users in situations where you have to sneakernet software from point A to point B, or from a network with good internet access to one without internet access or very limited internet access, even for hackers and infosec enthusiasts that are participating in "Capture the Flag" events with limited internet access, an offline build of autosnort providing the latest version of snort and snort rules easily seems like a godsend.
The offline script, or scripts I should say, come in two stages:
Stage 1: as-offline-stage1.sh
When this script is ran on a system that is identical to your offline system (meaning a system running the same operating system (Ubuntu), Version (12.04), and architecture (32-bit/i386 || 64-bit/x86_64)), it will download all the required packages via apt-get, download them (without installing them on this system), grab the latest version of snort and daq as well as barnyard2, snortreport, and libdnet.
A copy of the perl script, create-sidmap.pl and dpkorder$arch.txt (where $arch is either x86_64 or i386) should be placed on this system as well; the script expects to find these two files so that they can be included in the tar file that has all the packages (.deb and .tar.gz) needed for the stage2/offline installer script.
Stage 2: as-offline-stage2.sh
This script is ran on the offline system you plan on installing snort to. The script requires the tarball generated from the stage1 script as well as a VRT rules tarball to work properly.
Essentially the script installs all the .deb packages in the CORRECT order, then installs all the downloaded source packages, much the same as the online version of the autosnort script (yay for code re-use!)
The end-product is a fully functional snort install, just like the online autosnort script.
Please note that create-sidmap.pl is a part of the Oinkmaster suite. In layman's terms, the Oinkmaster suite is a precursor to PulledPork. The Oinkmaster suite is released under the BSD license, by Sourcefire. I just want to make it abundantly clear here and now that this is NOT my software and if requested by the original creator of the script I will comply and remove it. It is included on my github page currently as a convenience and nothing more.
The reason the create-sidmap script is included is to assist offline users in generating a new sid-msg.map file. I noticed in the process of creating this script, that barnyard2 would log alerts just fine, but then snortreport wouldn't show me what the alert's message was only "snort alert [sid number]" which was very unhelpful. I eventually discovered this is due to the sid-msg.map file. Essentially, the sid-msg.map file is responsible for "mapping" the snort SID to the snort msg field in the rule, thus giving you the rule sid and message when you look at a rule via snortreport or the web interface of your choice instead of "snort alert" followed by the sid number. create-sidmap.pl resolves this problem quite easily.
I hope you all find the scripts useful!
Until next time, happy snorting.
Friday, February 22, 2013
Tuesday, January 29, 2013
New year, new ideas (introduction to while/true case statements)
Howdy Autosnort users,
It's been a while since I've posted anything meaty to the blog. My day job keeps me very busy and the projects I've been working on have been very intense. At this point I really only have time to work on autosnort over weekends. I appreciate all the support and questions and identification of bugs or problems with the script; you're helping me, the snort community and the greater open source community just by asking questions and reporting bugs, so thank you very much. I may not be very fast to respond, but I will ALWAYS do what I can as soon as I can.
That being said, I'm going to post a quick tutorial to bash while/true loops, and how they can be used to provide some fault tolerance in your shell scripts (or at least how it's helped me tremendously)
First and foremost, what is a while true loop? A while true loop is a do while statement that is configured to loop infinitely. While do loops usually go something like this:
the function below prints the value of x, then increments it by 1. Once x is greater than 10, the loop, then the script exits. This script will print numbers 0-10, one on each line.
A while true loop looks something like this:
this statement is much much simpler, but lets run through it anyhow. The do/while statement is just "while true; do" which, in english, means while true is true (which will ALWAYS be the case) do the statement below
The statement below just prints "Hello!" ad infinitum. If you were to run the script above, you'd have to kill the script or Ctrl+C the script to stop it. Not very useful, huh? What if I told you that you could control whether or not the loop progresses or not with certain keywords?
The keywords break and continue in a loop are ways to control how a bash loop progresses. The break statement says "Stop the loop now, jump immediately to done, do not process anymore lines in the loop." while the continue statement says "Stop here, go back to the beginning statement in the loop."
Are you starting to see how this can be useful?
Case statements can be used to give the use of a script a menu or "branch" a script, performing a certain action when a variable, or output is a certain value.
the above statement asks the user to input 1 or 2 as an option for the case statement. that value is stored in the variable case1. The case statement is basically a large collection of if/then statements -- "if the value of case1 equals 1, then do this. if it equals 2, then do this. if it equals anything else, this is a catch-all statement to do, usually indicating the user picked an invalid choice."
Nothing too mind-breaking. This is stuff you're taught in your intro to UNIX classes in college. The fun part comes combining the concept of a case statement with a while true loop. and using continue or break keywords to continue the loop or break it:
Let me break down what's going on above:
We begin with a while true statement, starting off our "infinite" loop. The next statement tells the user to select 1 or 2 as options in our case statement. This is normally where you'd print a menu (if a user is expected to interact here) and explain what the options do, or just launch straight into your case selection if this is a fully automated script. After printing the statement and taking the user's input, we go straight into our case statement. The statement says "If case1 equals 1 do this, if it equals 2, then do this, if it equals anything else, do this."
What's different between this case statement and the previous one I used as an example is the existence of the keywords break and continue in these statements. The break statement says "Terminate the loop and jump straight to done." No more processing, no more looping, just move on. The break statement at the end of option 1 and 2 is a way of identifying that these are the only valid cases for this bit of bash code, that in order to continue script execution, the value of case1 must be "1" or "2" (or the user has to cancel the script via kill or ctrl+c). the continue statement in our catch-all case is a way if forcing the user to select a valid choice to continue script execution. In this way it's a sort of safety net that doesn't allow the script to continue unless a valid choice is made. It's a safety net for accidents, fat-fingering or hitting the wrong keys.
Soon as I figured out how to do this, I littered them throughout the script to make the script more fault tolerant. Instead of throwing an error and exiting in a case statement, autosnort just loops back and informs the user their choice was invalid, or didn't work for whatever reason, instead of blindly plowing forward, or exiting, leaving the user with a mess. It's still imperfect, but MUCH improved over the original version.
Currently I'm working on nesting multiple while/true case statements together, something like this:
See if you can figure it out on your own. I hope to implement multiple output options (web UIs, etc.) in this manner to make the script more foolproof, and offer more choices to users.
Until next time,
Happy snorting!
It's been a while since I've posted anything meaty to the blog. My day job keeps me very busy and the projects I've been working on have been very intense. At this point I really only have time to work on autosnort over weekends. I appreciate all the support and questions and identification of bugs or problems with the script; you're helping me, the snort community and the greater open source community just by asking questions and reporting bugs, so thank you very much. I may not be very fast to respond, but I will ALWAYS do what I can as soon as I can.
That being said, I'm going to post a quick tutorial to bash while/true loops, and how they can be used to provide some fault tolerance in your shell scripts (or at least how it's helped me tremendously)
First and foremost, what is a while true loop? A while true loop is a do while statement that is configured to loop infinitely. While do loops usually go something like this:
#!/bin/bashx is a variable initialized to zero in the example above. The while statement reads "while x is less than or equal to 10 do the function below.
x=0
while [ $x -le 10 ]; do
echo "$x"
x=$(( $x + 1 ))
done
exit 0
the function below prints the value of x, then increments it by 1. Once x is greater than 10, the loop, then the script exits. This script will print numbers 0-10, one on each line.
A while true loop looks something like this:
#!/bin/bash
while true; do
echo "Hello!"
done
exit 0
this statement is much much simpler, but lets run through it anyhow. The do/while statement is just "while true; do" which, in english, means while true is true (which will ALWAYS be the case) do the statement below
The statement below just prints "Hello!" ad infinitum. If you were to run the script above, you'd have to kill the script or Ctrl+C the script to stop it. Not very useful, huh? What if I told you that you could control whether or not the loop progresses or not with certain keywords?
The keywords break and continue in a loop are ways to control how a bash loop progresses. The break statement says "Stop the loop now, jump immediately to done, do not process anymore lines in the loop." while the continue statement says "Stop here, go back to the beginning statement in the loop."
Are you starting to see how this can be useful?
Case statements can be used to give the use of a script a menu or "branch" a script, performing a certain action when a variable, or output is a certain value.
#!/bin/bash
read -p "this is a case statement!
Your choices are:
1
2
Please enter your choice: " case1
case $case1 in
1)
echo "you have chosen case statement 1!"
;;
2)
echo "you have chosen case statement 2!"
;;
*)
echo "invalid choice!"
;;
esac
exit 0
the above statement asks the user to input 1 or 2 as an option for the case statement. that value is stored in the variable case1. The case statement is basically a large collection of if/then statements -- "if the value of case1 equals 1, then do this. if it equals 2, then do this. if it equals anything else, this is a catch-all statement to do, usually indicating the user picked an invalid choice."
Nothing too mind-breaking. This is stuff you're taught in your intro to UNIX classes in college. The fun part comes combining the concept of a case statement with a while true loop. and using continue or break keywords to continue the loop or break it:
while true; do
read -p "this is a case statement var choices are 1 or 2." case1
case $case1 in
1)
echo "this is a result of selecting 1."
break;
;;
2)
echo "this is a result of selecting 2."
break;
;;
*)
echo "invalid choice!"
continue;
;;
esac
done
exit 0
Let me break down what's going on above:
We begin with a while true statement, starting off our "infinite" loop. The next statement tells the user to select 1 or 2 as options in our case statement. This is normally where you'd print a menu (if a user is expected to interact here) and explain what the options do, or just launch straight into your case selection if this is a fully automated script. After printing the statement and taking the user's input, we go straight into our case statement. The statement says "If case1 equals 1 do this, if it equals 2, then do this, if it equals anything else, do this."
What's different between this case statement and the previous one I used as an example is the existence of the keywords break and continue in these statements. The break statement says "Terminate the loop and jump straight to done." No more processing, no more looping, just move on. The break statement at the end of option 1 and 2 is a way of identifying that these are the only valid cases for this bit of bash code, that in order to continue script execution, the value of case1 must be "1" or "2" (or the user has to cancel the script via kill or ctrl+c). the continue statement in our catch-all case is a way if forcing the user to select a valid choice to continue script execution. In this way it's a sort of safety net that doesn't allow the script to continue unless a valid choice is made. It's a safety net for accidents, fat-fingering or hitting the wrong keys.
Soon as I figured out how to do this, I littered them throughout the script to make the script more fault tolerant. Instead of throwing an error and exiting in a case statement, autosnort just loops back and informs the user their choice was invalid, or didn't work for whatever reason, instead of blindly plowing forward, or exiting, leaving the user with a mess. It's still imperfect, but MUCH improved over the original version.
Currently I'm working on nesting multiple while/true case statements together, something like this:
#!/bin/bash
while true; do
read -p "this is a case statement var choices are 1 or 2." case1
case $case1 in
1)
echo "this is a result of selecting 1."
break;
;;
2)
while true; do
read -p "this is another case statement. var choices are 1 or 2." case2
case $case2 in
1)
echo "result of the second case statement"
break;
;;
2) echo "NARF!"
break;
;;
*)
echo "Invalid choice!"
continue;
;;
esac
done
break;
;;
*)
echo "invalid choice!"
continue;
;;
esac
done
exit 0
See if you can figure it out on your own. I hope to implement multiple output options (web UIs, etc.) in this manner to make the script more foolproof, and offer more choices to users.
Until next time,
Happy snorting!
Tuesday, January 15, 2013
Pulled Pork for everyone!
Hello AS users,
I wanted to announce that I've finally got off my arse and finished pushing pulled pork integration to all autosnort shell scripts. As of today, all operating systems should have pulled pork integration.
I hope to have some more news and better posts soon, including a follow-up to the introduction to rule tuning (that seems to be getting quite a few page hits) as well as maybe an article on the testing environment I have in place for autosnort testing. Thank you for your continued support!
I wanted to announce that I've finally got off my arse and finished pushing pulled pork integration to all autosnort shell scripts. As of today, all operating systems should have pulled pork integration.
I hope to have some more news and better posts soon, including a follow-up to the introduction to rule tuning (that seems to be getting quite a few page hits) as well as maybe an article on the testing environment I have in place for autosnort testing. Thank you for your continued support!
Sunday, January 6, 2013
Introduction to Rule Tuning
Hello AS users,
I wanted to post some additional content outside of autosnort, but still related to snort. Seeing as how autosnort only gives you a Security over Connectivity ruleset if you use PulledPork to set the rule tree up initially, I figured that rule tuning may be a good topic of discussion.
Key complaints I've heard regarding snort is that
1) It's hard to set up
2) It's hard to tune out the noise and get alerts that are relevant without being inundated with false positives.
Well, Autosnort, Security Onion, RedBorder and several other open-source and corporate products that integrate snort were created to resolve complaint number 1, let's take a look at number 2.
So you have a rule that is very noisy. Noisy being defined as "Holy sh&!, Why do I have so many alerts?" How do I determine whether or not the given rule trigger(s) is a false positive?
For this exercise we're going to be picking on a rule for timthumb.php RFI attempts. For those not familiar with what ANY of that means, This rule exploits a vulnerability in a version of timthumb.php, a wordpress plugin. The vulnerability allows the attacker to perform a Remote File Include (RFI) attack -- the attacker can point the target web server to a remote file of their choice and the server will execute that file. Usually, the attack will point the web server to a web/php shell of their choice, giving the attacker the ability to control your web server with the same rights and access as the account being used to run that web server. This rule, at least a while ago use to be VERY noisy and generated loads of false positives, and would be an example of when you would use to rule tuning:
Let's answer the questions above:
What is the alert? "WEB-PHP Wordpress timthumb.php theme remote file include attack attempt"
The message string alone gives us a ton of information:
A TON of information from a single string.
From this string we can answer the other question: What is the Application/Operating System/Protocol the rule is triggering on?
In other cases, the Message string may not give you much information, so you will want to look at the rule header, the "flow" rule option, and/or any "reference" rule options to give you more clues on how to answer the second question. Let's take a look at the rule header for our example rule:
alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS
From this, we can glean the following:
Next, let's look at the flow option. The flow option tells us whether we are looking at traffic as a part of an established TCP session, and where the traffic should be coming from in terms of an established TCP session. See the snort manual entry for flow for more information. Let's take a closer look at flow in the rule:
We can determine the following from this string:
It may not seem like much, but in some cases where you see a rule firing repeatedly and you know that the destination IP in your home_net is a workstation, that can definitely be an eye opener.
Finally, let's take a look at the reference option in the rule. References are essentially metadata about the rule itself. Sort of like a "works cited" for the snort rule you can refer to glean detailed information about the rule -- this is how you would answer question 3: "Can I glean more information regarding this alert?" Most rules will have more than one reference, and if they do, it only benefits you and makes research that much easier. Let's refer back to our rule and see what references we have:
We have two references here, a bugtraq ID you can submit to search security focus to extract in-depth information regarding the vulnerability and a url references you could literally copy and paste the url into your browser and go directly to a web page that gives you some kind of information regarding this vulnerability:
This will be the meat and potatoes of your investigation. Do you use the Application/Plugin/Operating system in question? If this is a backdoor/malware rule, Have you ran a virus scan or checked common places where the backdoor or malware may persist on a given host? You have some work to here and some correlations to be made :)
Finally, let's have a look at our final question, "How often the rule is triggering?" Some additional questions to ask include:
If we see that a rule or rules are triggering repeatedly, usually its an indicator of false positives, in which case, the people who manage your ruleset and/or the vendor that supplies your rules (in our Case, Sourcefire's VRT) would want to know about this and would want PCAPS, or packet captures of your false positives to determine what is wrong.
In other cases where you see a rule triggering irregularly at odd hours (e.g. outside of business hours) or in no set interval, then you may have an attacker looking for an easy-in, randomly probing your network or maybe a "bohr bugged" rule that triggers under certain conditions that need to be pinned down yet.
Still yet in other cases where you see a rule triggering at set intervals, it can almost always be attributed to automated attack software or malware, or maybe some sort of automated task that occurs that time period that could be causing the rule to trigger. Check the source or destination IP in question for any sort of automated tasks you may have scheduled to run around that time to try and rule this out as a possibility.
In closing, let's look at the four questions we asked at the beginning of this article, and answer them:
From these pieces of information you can make a strong judgement to determine whether or not the given rule is a false positive. Once you have made this determination, What do you do next? Most of you may know about the suppression and threshold options to reduce rule noise or just turning off the rule entirely, but did you know about pass rules or snort's ability to use Berkeley Packet Filters/BPFs?
We'll cover that next time...
Until then, happy snorting!
I wanted to post some additional content outside of autosnort, but still related to snort. Seeing as how autosnort only gives you a Security over Connectivity ruleset if you use PulledPork to set the rule tree up initially, I figured that rule tuning may be a good topic of discussion.
Key complaints I've heard regarding snort is that
1) It's hard to set up
2) It's hard to tune out the noise and get alerts that are relevant without being inundated with false positives.
Well, Autosnort, Security Onion, RedBorder and several other open-source and corporate products that integrate snort were created to resolve complaint number 1, let's take a look at number 2.
So you have a rule that is very noisy. Noisy being defined as "Holy sh&!, Why do I have so many alerts?" How do I determine whether or not the given rule trigger(s) is a false positive?
Too much noise; not enough signal.
So you have a rule that is triggering a number of alerts. Time to start investigating. As a Security Professional, this is where you have to start earning your bread and butter: Investing possible alerts, security violations and Bad Things (tm) on your network. Bear in mind, every single network is different in its own way, but even so, there are basic questions you can ask to begin your investigation:- What is the alert?
- What application/operating system/protocol is the rule triggering on?
- Can I glean more information regarding this alert?
- How often is the rule triggering?
For this exercise we're going to be picking on a rule for timthumb.php RFI attempts. For those not familiar with what ANY of that means, This rule exploits a vulnerability in a version of timthumb.php, a wordpress plugin. The vulnerability allows the attacker to perform a Remote File Include (RFI) attack -- the attacker can point the target web server to a remote file of their choice and the server will execute that file. Usually, the attack will point the web server to a web/php shell of their choice, giving the attacker the ability to control your web server with the same rights and access as the account being used to run that web server. This rule, at least a while ago use to be VERY noisy and generated loads of false positives, and would be an example of when you would use to rule tuning:
alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"WEB-PHP Wordpress timthumb.php theme remote file include attack attempt"; flow:to_server,established; content:"timthumb.php|3F|"; nocase; http_uri; content:"src=http"; distance:0; nocase; http_uri; pcre:"/timthumb\x2ephp\x3f[^\r\n]*src=http\x3a\x2f[^\x2e\x2f]+\x2e[^\x2e\x2f]+\x2e[^\x2e\x2f]+\x2e/Ui"; metadata:policy balanced-ips drop, policy security-ips drop, service http; reference:bugtraq,47374; reference:url,code.google.com/p/timthumb/issues/detail?id=212; classtype:web-application-attack; sid:19653; rev:1;)
Let's answer the questions above:
What is the alert? "WEB-PHP Wordpress timthumb.php theme remote file include attack attempt"
The message string alone gives us a ton of information:
- This rule is a rule from the web-php.rules category
- This means that we are looking at web server traffic -- particularly, web servers running PHP.
- Even more specifically, we see that this rule is concerned with WordPress, a popular Blog/CMS suite used for managing web-based content and blogs. It's easy to use, easy to maintain, can be loaded with plugins to extend its functionality, and makes a Security Analyst's job a living nightmare :).
- Even further than that, we can see that this is a rule concerning timthumb.php, a specific Wordpress plugin.
A TON of information from a single string.
From this string we can answer the other question: What is the Application/Operating System/Protocol the rule is triggering on?
- Web traffic going to your web servers in your HOME_NET
- Running PHP, Wordpress and the timthumb WP plugin
In other cases, the Message string may not give you much information, so you will want to look at the rule header, the "flow" rule option, and/or any "reference" rule options to give you more clues on how to answer the second question. Let's take a look at the rule header for our example rule:
alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS
From this, we can glean the following:
- We are alerting on TCP traffic from EXTERNAL_NET (usually set to 'any' or !$HOME_NET -- not HOME_NET systems)
- Going to your HOME_NET systems on HTTP_PORTS -- ports normally associated with HTTP traffic. In other words: Web Servers in your HOME_NET.
Next, let's look at the flow option. The flow option tells us whether we are looking at traffic as a part of an established TCP session, and where the traffic should be coming from in terms of an established TCP session. See the snort manual entry for flow for more information. Let's take a closer look at flow in the rule:
flow:to_server,established;
We can determine the following from this string:
- We want established TCP connections only
- We want TCP sessions going to a server.
It may not seem like much, but in some cases where you see a rule firing repeatedly and you know that the destination IP in your home_net is a workstation, that can definitely be an eye opener.
Finally, let's take a look at the reference option in the rule. References are essentially metadata about the rule itself. Sort of like a "works cited" for the snort rule you can refer to glean detailed information about the rule -- this is how you would answer question 3: "Can I glean more information regarding this alert?" Most rules will have more than one reference, and if they do, it only benefits you and makes research that much easier. Let's refer back to our rule and see what references we have:
reference:bugtraq,47374; reference:url,code.google.com/p/timthumb/issues/detail?id=212;
We have two references here, a bugtraq ID you can submit to search security focus to extract in-depth information regarding the vulnerability and a url references you could literally copy and paste the url into your browser and go directly to a web page that gives you some kind of information regarding this vulnerability:
- Bugtraq id 47374
- Code.google.com/p/timthumb/issues/detail?id=212
This will be the meat and potatoes of your investigation. Do you use the Application/Plugin/Operating system in question? If this is a backdoor/malware rule, Have you ran a virus scan or checked common places where the backdoor or malware may persist on a given host? You have some work to here and some correlations to be made :)
Finally, let's have a look at our final question, "How often the rule is triggering?" Some additional questions to ask include:
- Is there a pattern to how often the rule triggers?
- Do you have a rule that's just rapidly spewing out alerts?
- Are the alerts being seen in any discernible intervals?
- At irregular times?
- When did the rule start triggering?
If we see that a rule or rules are triggering repeatedly, usually its an indicator of false positives, in which case, the people who manage your ruleset and/or the vendor that supplies your rules (in our Case, Sourcefire's VRT) would want to know about this and would want PCAPS, or packet captures of your false positives to determine what is wrong.
In other cases where you see a rule triggering irregularly at odd hours (e.g. outside of business hours) or in no set interval, then you may have an attacker looking for an easy-in, randomly probing your network or maybe a "bohr bugged" rule that triggers under certain conditions that need to be pinned down yet.
Still yet in other cases where you see a rule triggering at set intervals, it can almost always be attributed to automated attack software or malware, or maybe some sort of automated task that occurs that time period that could be causing the rule to trigger. Check the source or destination IP in question for any sort of automated tasks you may have scheduled to run around that time to try and rule this out as a possibility.
In closing, let's look at the four questions we asked at the beginning of this article, and answer them:
- What is the alert?
- Look at the rule message to try and draw out discernible information. What rule category does the rule come from? What operating system/malware strain/Application does the rule message refer to?
- What application/operating system/protocol is the rule triggering on?
- Look at the rule header. Look at the flow rule option. What protocols are involved? Are we looking at client side applications? Server side services? What ports is the rule interested in?
- Can I glean more information regarding this alert?
- Check the reference metadata option. Read up on any references provided. Do they provide you particular operating system/application software versions? Do you have that particular application on your network? At the vulnerable version? Do you use the particular operating system on your network? If the rule is regarding a piece of malware, can you find a characterization of that malware (e.g. What it does and where it persists) from an A/V vendor's website if the rule doesn't have a good enough reference? The rule reference metadata is probably the most useful portion of the rule to pay attention to aside from the rule message!
- How often is the rule triggering?
- Is it constantly alerting? Regular Intervals? Irregular, seemingly random times?
From these pieces of information you can make a strong judgement to determine whether or not the given rule is a false positive. Once you have made this determination, What do you do next? Most of you may know about the suppression and threshold options to reduce rule noise or just turning off the rule entirely, but did you know about pass rules or snort's ability to use Berkeley Packet Filters/BPFs?
We'll cover that next time...
Until then, happy snorting!
Wednesday, January 2, 2013
PulledPork Integration announcements
Hello AS users,
I'm finally back to working on Autosnort after my long holiday hiatus.
Several users reported problems with the pulled pork integration script for CentOS not working as it is supposed to, failing to install many things, create directories, etc. I think I've traced this issue down to my script attempting to wget a nonexist rpm, the EPEL repository rpm for 6.7 no longer exists and has been replaced by an EPEL repository rpm for 6.8. For the time being I've modified the name of the file downloaded, but sometime in the future I will design a check the same way I check for the latest version of snort/snort rules from snort.org to ensure the proper EPEL file is downloaded and used during the snort installation process. I've tested the fixed script against CentOS 32 and 64-bit editions and it seems to work reliably.
In addition to this I have released an autosnort script for ubuntu that also features pulledpork integration. I have tested this against 32-bit and 64-bit ubuntu as well and have found it to work reliably.
Happy Snorting!
I'm finally back to working on Autosnort after my long holiday hiatus.
Several users reported problems with the pulled pork integration script for CentOS not working as it is supposed to, failing to install many things, create directories, etc. I think I've traced this issue down to my script attempting to wget a nonexist rpm, the EPEL repository rpm for 6.7 no longer exists and has been replaced by an EPEL repository rpm for 6.8. For the time being I've modified the name of the file downloaded, but sometime in the future I will design a check the same way I check for the latest version of snort/snort rules from snort.org to ensure the proper EPEL file is downloaded and used during the snort installation process. I've tested the fixed script against CentOS 32 and 64-bit editions and it seems to work reliably.
In addition to this I have released an autosnort script for ubuntu that also features pulledpork integration. I have tested this against 32-bit and 64-bit ubuntu as well and have found it to work reliably.
Happy Snorting!
Sunday, December 30, 2012
Dude are you still there?
Hello Autosnort users and readers of this blog.
I wanted to assure you that yes, I am still out there, I haven't abandoned this blog or this project and do not intend on doing so any time soon. I have just taken a very long holiday hiatus. Autosnort dev and regularly scheduled postings as well as responses to bugs and problems regarding autosnort will continue in the new year. I'm just taking a long moment's reprieve for the holiday to enjoy it among family and friends.. I know its a little late, but I hope you all enjoyed your holidays and I wish you all a happy new year.
Cheers,
DA
I wanted to assure you that yes, I am still out there, I haven't abandoned this blog or this project and do not intend on doing so any time soon. I have just taken a very long holiday hiatus. Autosnort dev and regularly scheduled postings as well as responses to bugs and problems regarding autosnort will continue in the new year. I'm just taking a long moment's reprieve for the holiday to enjoy it among family and friends.. I know its a little late, but I hope you all enjoyed your holidays and I wish you all a happy new year.
Cheers,
DA
Monday, December 10, 2012
We have pulled pork; the rest is just barbecue sauce.
Hello AS users,
Today I would like to announce a new autosnort build and introduce support and integration of pulled pork.
For those who do not know, pulled pork is a perl script written by JJ Cummings of Sourcefire. The script is the recommended method for open-source snort users to manage, download, and configure rules for snort deployments.
What does pulled pork integration mean for you and autosnort? Its even easier to get started.
- The old rule tarball method is still available (e.g. point the script to a VRT tarball to unpack), but now, in addition to the tarball method, you may choose to use pulled pork to build an initial ruleset based off of the Sourcefire base ruleset "Security over Connectivity".
- If you provide autosnort with an oink code (registered user or VRT subscription) you are then given a choice to try to download the latest rules tarball, or a the rules tarball from the previous version of snort.
The reason I do this instead of automatically downloading the latest rule tarball from snort.org is to work around the 30-day wait period registered users have to wait when a new version of snort comes out.
To summarize a blog post in which I had described this problem in detail, when a new version of snort comes out, a rule tarball that has compatible SO rules is immediately made available for VRT rule subscription holders. Registered users have to wait 30 days from the release to gain access to a VRT tarball with compatible SO rules. If you choose to download/process rules from the tarball for the previous version of snort, autosnort will only process text rules to ensure compatibility with the latest version of snort -- this is to prevent the dreaded "my SO rules don't work" phase for registered users.
Some may wonder "It's taken an awful long time to integrate just this functionality. Have you been slacking off?" Well, no more than I usually do, but there is a very good reason for the delay in this release -- I've taken a long look at the rest of the script and decided to re-write large portions of the script.
A significant problem most users had with autosnort was that if you fat-fingered at any point during the script, autosnort would happily plow forward, giving you a broken install upon reboot. I finally decided to go through the script and write some fault tolerance into the script -- key portions of the script are now sort of "encapsulated" into while/true loops -- meaning that you MUST give the script a valid answer for it to progress.
For example, as a part of the rule management routine in autosnort, I decided to keep the old VRT rule tarball import (e.g. untar it and drop everying into /usr/local/snort)
Before, if a user gave the script an invalid directory path and/or nonexistent file, the script would go forward without a care in the world.
This time around however the script does the following:
-checks to see if the file exists. if the file doesn't exist, you are told to try again
-if it exists, tries to untar it. if the file cannot be untarred (e.g. not a .tar.gz) it informs the user and the user is given the chance to try again
For the pulled pork phase we do the same kind of error checking:
-if pulled pork fails to download rules for any reason, we inform the user and loop back to the choice between using pulled pork for rule management, or importing a VRT rule tarball through the old method.
Pulled pork integration is going to gradually be released for all supported operating systems (e.g. Debian, Ubuntu, BT5 and CentOS. The pilot release is going to be on CentOS, with releases to other OSes trickling in behind it.
As always, the code has been posted to github under the CentOS directory
autosnort-centOS-ppinteg.sh
readme
Interesting, but useless statistics:
-file size of the shell script has nearly doubled from the old autosnort centOS script
-originally:24.5KB
-now: 42.3KB
-diff: 17.8
-lines of code (includes comments and blank lines for readability)
-original: 553
-now: 879
-diff: 326
Today I would like to announce a new autosnort build and introduce support and integration of pulled pork.
For those who do not know, pulled pork is a perl script written by JJ Cummings of Sourcefire. The script is the recommended method for open-source snort users to manage, download, and configure rules for snort deployments.
What does pulled pork integration mean for you and autosnort? Its even easier to get started.
- The old rule tarball method is still available (e.g. point the script to a VRT tarball to unpack), but now, in addition to the tarball method, you may choose to use pulled pork to build an initial ruleset based off of the Sourcefire base ruleset "Security over Connectivity".
- If you provide autosnort with an oink code (registered user or VRT subscription) you are then given a choice to try to download the latest rules tarball, or a the rules tarball from the previous version of snort.
The reason I do this instead of automatically downloading the latest rule tarball from snort.org is to work around the 30-day wait period registered users have to wait when a new version of snort comes out.
To summarize a blog post in which I had described this problem in detail, when a new version of snort comes out, a rule tarball that has compatible SO rules is immediately made available for VRT rule subscription holders. Registered users have to wait 30 days from the release to gain access to a VRT tarball with compatible SO rules. If you choose to download/process rules from the tarball for the previous version of snort, autosnort will only process text rules to ensure compatibility with the latest version of snort -- this is to prevent the dreaded "my SO rules don't work" phase for registered users.
Some may wonder "It's taken an awful long time to integrate just this functionality. Have you been slacking off?" Well, no more than I usually do, but there is a very good reason for the delay in this release -- I've taken a long look at the rest of the script and decided to re-write large portions of the script.
A significant problem most users had with autosnort was that if you fat-fingered at any point during the script, autosnort would happily plow forward, giving you a broken install upon reboot. I finally decided to go through the script and write some fault tolerance into the script -- key portions of the script are now sort of "encapsulated" into while/true loops -- meaning that you MUST give the script a valid answer for it to progress.
For example, as a part of the rule management routine in autosnort, I decided to keep the old VRT rule tarball import (e.g. untar it and drop everying into /usr/local/snort)
Before, if a user gave the script an invalid directory path and/or nonexistent file, the script would go forward without a care in the world.
This time around however the script does the following:
-checks to see if the file exists. if the file doesn't exist, you are told to try again
-if it exists, tries to untar it. if the file cannot be untarred (e.g. not a .tar.gz) it informs the user and the user is given the chance to try again
For the pulled pork phase we do the same kind of error checking:
-if pulled pork fails to download rules for any reason, we inform the user and loop back to the choice between using pulled pork for rule management, or importing a VRT rule tarball through the old method.
Pulled pork integration is going to gradually be released for all supported operating systems (e.g. Debian, Ubuntu, BT5 and CentOS. The pilot release is going to be on CentOS, with releases to other OSes trickling in behind it.
As always, the code has been posted to github under the CentOS directory
autosnort-centOS-ppinteg.sh
readme
Interesting, but useless statistics:
-file size of the shell script has nearly doubled from the old autosnort centOS script
-originally:24.5KB
-now: 42.3KB
-diff: 17.8
-lines of code (includes comments and blank lines for readability)
-original: 553
-now: 879
-diff: 326
Subscribe to:
Posts (Atom)