This module focuses on log file analysis, specifically post-event analysis using Swatch and Log Parser. We explain how to install, configure, and execute Swatch and Log Parser and provide several sample configurations for each.

 

 

1.1    Swatch

The focus is on using Swatch and Log Parser as forensic analysis tools, meaning that they will be used on logs after an incident has occurred. If you are planning to use these techniques on files involved in a law enforcement investigation, please make sure you prepare the files according to established best practices before use.

1.1.1        Swatch Log Monitor

Log files are useful only if they are read. After an incident, log files often have clues as to what happened. However, many servers produce large volumes of log information, often spread out over more than one file, so sifting through this data can be tedious and time consuming. As an added problem, different servers have different log formats. If it is necessary to compare files, it can be challenging to match up fields.

Swatch, the Simple Watcher log monitoring tool, is capable of searching a file for a list of strings and then performing specific actions when one is found. It was designed to do real-time monitoring of server log files but can also be set to process a stand-alone file. Swatch was designed to work with syslog files, but it can be used on any file.

Swatch was written in Perl, and because of the way it is installed it is best used on a Linux system. It is an open source tool, and the project is maintained on SourceForge.

Throughout this module we will consider more heavily the case where Swatch is used to examine older log files as opposed to active log files.

1.1.2        Swatch Installation

Swatch has the same installation process as a Perl module. You download a tarball, uncompress it, expand it, and build it. The tool installs itself in /usr/bin, and you can use it from any directory. It also installs a manual page.

To begin, make sure that Perl 5 is installed on the machine. Later versions of Perl may come with some of the necessary modules installed already. You also need the ability to use the GNU utility make to fully install Swatch.

1.1.2.1     Installing Perl Modules

If your Linux distribution offers versions of the Perl modules needed to support Swatch, it is best to get the operating-system-specific ones. Otherwise, you will need to obtain them from either the module’s developer’s web site or from a centralized repository such as the Comprehensive Perl Archive Network (CPAN). ¹ CPAN indexes most of the Perl packages available, makes the list searchable, and has them available for download. They also have links to the developer's web site if you would prefer to get the modules straight from the source.

To install Swatch you need these modules:²

·         File::Tail – in File-Tail-0.99.1.tar.gz

·         Date::Calc – in Date-Calc-5.4.tar.gz

·         Date::Parse – in TimeDate-1.16.tar.gz

·         Time::HiRes – in Time-HiRes-1.66.tar.gz

To support these you might also need

·         Bit::Vector – in Bit-Vector-6.4.tar.gz

·         Carp::Clan – in Carp-Clan-5.3.tar.gz

Once the tar file is on the machine, you must decompress and expand it before it can be installed. Once expanded, read the INSTALL file to make sure that the module has the standard installation commands. For these modules, there is a Perl script called Makefile.PL that creates a makefile specific to the machine. Next, run the make file three times: once to initialize, once to test, and then once to install. After that, the package is ready to use. In order for other users to be able to use these modules, they must be installed by root. The commands follow in shaded boxes (the normal text is what is sent to the console):

Lists all the files in perlmod.tar

Writing Makefile for Perl::Mod

Check for errors

Look for “All tests successful”

Check for errors

Repeat for the other packages and you are ready to install Swatch itself.

1.1.2.2     Installing Swatch

Installing Swatch involves the same procedure as the Perl modules. First, download the tarball to the local machine from http://sourceforge.net/projects/swatch. There is a patch needed to enable the throttle action to fully work. Download that from http://www.cert.org. These instructions are for Swatch 3.1.1 (the normal text is what is sent to the console):

Lists all the files in swatch-X.X.X.tar

The character after the –p is a zero

Writing Makefile for swatch

If Time::HiRes, Date::Calc, or Date::Parse are missing it will say

Check for errors

Look for “All tests successful”

Check for errors

Swatch is now ready to be executed.

1.1.3        Swatch Configuration

The configuration file is the source of Swatch's power. It is a text file of rules that are used to create the script that will be run against the log file. This topic is about how to develop your own configuration file.

Before you begin adding rules, determine what you are trying to find. Perhaps you want to look for Nessus attacks in your Apache log files, or find when people try to use an SMTP server as an open relay. Make a list of strings or regular expression patterns that you might need. Keep in mind that creating a good configuration file is an iterative process, and it may take a few rounds to extract the desired information from the file.

1.1.3.1     The Configuration File Location

By default, Swatch looks for the configuration file .swatchrc in the home directory. If this file is not found, it uses a default configuration of

       watchfor /.*/

              echo

This merely echoes every message in the log file to the console. This is not any more useful than inspecting the log file by hand. To harness the power, a customized configuration file should be created (this is addressed in the next section, 1.1.3.2).

The default name is .swatchrc, but it can be whatever you want. Configuration files for different types of log files may be identified by a distinct name. One might call their Apache configuration file apache.swatchrc and their sendmail sendmail.swatchrc. The file can be stored anywhere that is accessible from the command line, not just in the home directory. If a different name or location is used for the configuration file, it is added as an argument to the command line when Swatch is executed (this is discussed in Section 1.1.4).

1.1.3.2     Adding Rules to the Configuration File

Rules are a list of keywords and values that are used to make conditional statements to check against, and actions to take if one is true. They have three parts: the event, the pattern, and the action(s).

1.1.3.2.1      Types of Events

There are two types of events in Swatch: “watchfor” and “ignore.” The keyword “watchfor” looks for the specified pattern in messages in the log file. The “ignore” keyword will skip the rest of the rules when a message matches the pattern.

By default, the first rule that matches a message will be the only rule that acts on that message. This property can be harnessed by using the “ignore” event to filter out messages. For example, since you know that the system administrator always uses the same machine to do penetration testing, you create an “ignore” rule for messages that come from a specific internal IP address, 127.0.0.5, and list the rule for Nessus scans after it so that internal scans will not cause alerts. In this case, you will want to put these two events in this order:

       ignore /127\.0\.0\.5/

watchfor /Nessus/

       throttle 0:10:00,key=nessus

echo

1.1.3.2.2      Types of Patterns

The value for the event keyword is the regular expression pattern that follows on the same line. The simplest regular expression is a string to match character by character enclosed in “/”; for example, “/Nessus/” matches only the substring “Nessus” if it appears anywhere in the line. If there are characters in the search string, the capitalization must be the same for the string to match. If you want “nEsSuS” to also match, then you need to put an “i” after the second “/” to indicate a case insensitive search.

Regular Expressions

A regular expression is a pattern that describes or matches a set of strings [Wikipedia 05d]. It is a syntax for describing more general criteria for matching strings than simply matching a word in a string. For example, with a regular search you could find the substring “cat” in “catapult.” But a regular expression would let you look for a string that starts with “ca” and ends with “t,” and you would find both “cat” and “catapult.”

For more examples of regular expressions in general, see the Wikipedia entry:

       http://en.wikipedia.org/wiki/Regular_expression

For more examples of regular expressions in Perl, see the Perl manual page:

       http://www.perl.com/doc/manual/html/pod/perlre.html

1.1.3.2.3      Types of Actions

Actions are what the script does when it matches a pattern. They range from printing the log message to the console to executing a separate script to call a pager. There can be multiple actions associated with each rule. The “ignore” event has its action built in, namely, to stop looking at the log entry. The complete list of actions is found in Table 1 [SourceForge 04]:

Table 1 :     Actions in Swatch

Action	Description
echo [modes]	Prints the log message to the console. Takes an optional value of a text color, a background color, or a font weight. Possible values are bold, underscore, inverse, blink, black, red, green, yellow, blue, magenta, cyan, white, black_h, red_h, green_h, yellow_h, blue_h, magenta_h, cyan_h, and white_h.