Note

Previous versions of this page page originally was created by @j_hen on her blog http://jentalkstoomuch.blogspot.com/2010/09/writing-custom-ossec-rules-for-your.html Some content may be the same, but examples have been updated.

Note

In the xml based examples, any text between <!-- and --> are comments. In the console based examples, anything after # may be an example. For more information on OSSEC’s non-standard regular expression (regex) syntax, refer to the regex page.

Create Custom decoder and rules

One of the main features of OSSEC is monitoring system and application logs. Many popular services have logs and decoders, but there are hundreds that are not covered. Custom applications and services will also not be covered. Adding decoders and rules for services is generally very easy.

Adding a File to be Monitored

Adding a log file to the configuration for monitoring is simple. In the system’s ossec.conf add an entry like this:

<localfile>
  <log_format>syslog</log_format>
  <location>/path/to/log/file</location>
</localfile>

syslog is a generic format, consisting of a singular line of text appended to the log file. There are other formats available, they are detailed on the localfile syntax page.

Note

Additional examples can be found in file-log-monitoring. More detailed syntax can be found in the localfile syntax.

After adding a localfile entry, the OSSEC processes must be restarted.

Create a Custom Decoder

The following log messages will be used for most of the examples in this section:

2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1
2013-11-01T10:01:05.600494-04:00 arrakis ossec-exampled[9123]: successful authentication for user test-user from 192.168.1.1 via test-protocol1

The first log message is broken down as follows:

• 2013-11-01T10:01:04.600374-04:00 - timestamp from rsyslog
• arrakis - hostname of the system
• ossec-exampled - daemon creating the log
• [9123] - process ID of the ossec-exampled instance
• test connection from 192.168.1.1 via test-protocol1 - log message

ossec-logtest will be used to test the custom decoder and any custom rules.

Custom decoders are added to the local_decoder.xml file, typically found in /var/ossec/etc on a standard installation. The basic syntax is listed here, but this page is not well documented at the moment.

Using ossec-logtest on this sample rule results in the following output:

# /var/ossec/bin/ossec-logtest
2013/11/01 10:39:07 ossec-testrule: INFO: Reading local decoder file.
2013/11/01 10:39:07 ossec-testrule: INFO: Started (pid: 32109).
ossec-testrule: Type one log per line.

2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1


**Phase 1: Completed pre-decoding.
       full event: '2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1'
       hostname: 'arrakis'
       program_name: 'ossec-exampled'
       log: 'test connection from 192.168.1.1 via test-protocol1'

**Phase 2: Completed decoding.
       No decoder matched.

There is not a lot of output here because OSSEC does not understand this log. Creating a decoder for this log message will provide OSSEC much more information.

Phase 1 “pre-decodes” some information. The hostname is the system that generated the log message, program_name is the name of the application that created the log, and log is the rest of the log message.

The following is a very basic decoder for ossec-exampled:

<decoder name="ossec-exampled">
  <program_name>ossec-exampled</program_name>
</decoder>

This decoder simply looks for any log messages generated by ossec-exampled. Using a very generic decoder like this can allow an OSSEC user to create more specific child decoders for services with less consistent log messages.

Here is the ossec-logtest output after adding this decoder:

# /var/ossec/bin/ossec-logtest
2013/11/01 10:52:09 ossec-testrule: INFO: Reading local decoder file.
2013/11/01 10:52:09 ossec-testrule: INFO: Started (pid: 25151).
ossec-testrule: Type one log per line.

2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1


**Phase 1: Completed pre-decoding.
       full event: '2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1'
       hostname: 'arrakis'
       program_name: 'ossec-exampled'
       log: 'test connection from 192.168.1.1 via test-protocol1'

**Phase 2: Completed decoding.
       decoder: 'ossec-exampled'

Phase 2 now correctly identifies this log message as coming from ossec-exampled. There is still some very important information in the log message that should be decoded, namely the source IP and test-protocol1. To decode these a child decoder will be added. It will set the ossec-exampled decoder as a parent, and use prematch to limit its use to the correct log message.

<decoder name="ossec-exampled-test-connection">
  <parent>ossec-exampled</parent>
  <prematch offset="after_parent">^test connection </prematch> <!-- offset="after_parent" makes OSSEC ignore anything matched by the parent decoder and before -->
  <regex offset="after_prematch">^from (\S+) via (\S+)$</regex> <!-- offset="after_prematch" makes OSSEC ignore anything matched by the prematch and earlier-->
  <order>srcip, protocol</order>
</decoder>

Breaking this down piece by piece:

• <decoder name="ossec-exampled-test-connection"> - Declaring this to be a decoder and giving it a name.
• <parent>ossec-exampled</parent> - This decoder will only be checked if ossec-exampled also matched.
• <prematch offset="after_parent">^test connection </prematch> - If a log message does not contain the data in the prematch, it will not use that decoder. Setting the offset tells OSSEC to only look at data after the parent (ossec-exampled[9123]: in this case), in an effort to speed up matches.
• <regex offset="after_prematch">^from (\S+) via (\S+)$</regex> - The regex line can be used to pull data out of the log message for use in rules. In this instance the first \S+ matches the IP address, and the second matches the protocol. Anything between the parenthesis will be able to be used in rules.
• <order>srcip, protocol</order> - Defines what the entries in the regex line are labeled as. The IP address will be labeled as srcip, and the protocol by proto.

ossec-logtest output after adding this decoder:

# /var/ossec/bin/ossec-logtest
2013/11/01 11:03:25 ossec-testrule: INFO: Reading local decoder file.
2013/11/01 11:03:25 ossec-testrule: INFO: Started (pid: 6290).
ossec-testrule: Type one log per line.

2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1


**Phase 1: Completed pre-decoding.
       full event: '2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1'
       hostname: 'arrakis'
       program_name: 'ossec-exampled'
       log: 'test connection from 192.168.1.1 via test-protocol1'

**Phase 2: Completed decoding.
       decoder: 'ossec-exampled'
       srcip: '192.168.1.1'
       proto: 'test-protocol1'

Note

The decoder will be labeled as the parent decoder, not the child. It’s common to think a child decoder doesn’t work because the parent decoder’s name is listed, but that may not be a problem.

Now that the first sample log message is decoded, how does the second message fare? ossec-logtest output:

2013-11-01T10:01:05.600494-04:00 arrakis ossec-exampled[9123]: successful authentication for user test-user from 192.168.1.1 via test-protocol1


**Phase 1: Completed pre-decoding.
       full event: '2013-11-01T10:01:05.600494-04:00 arrakis ossec-exampled[9123]: successful authentication for user test-user from 192.168.1.1 via test-protocol1'
       hostname: 'arrakis'
       program_name: 'ossec-exampled'
       log: 'successful authentication for user test-user from 192.168.1.1 via test-protocol1'

**Phase 2: Completed decoding.
       decoder: 'ossec-exampled'

The decoded fields added in ossec-exampled-test-connection do not get decoded in this log message. This is expected because the prematch does not match. In this log message there are 4 fields that would be useful: status (successful), srcuser, srcip, and protocol. Adding a decoder for this should also be simple:

<decoder name="ossec-exampled-auth">
  <parent>ossec-exampled</parent>
  <prematch offset="after_parent"> authentication </prematch>
  <regex offset="after_parent">^(\S+) authentication for user (\S+) from (\S+) via (\S+)$</regex> <!-- Using after_parent here because after_prematch would eliminate the possibility of matching the status (successful) -->
  <order>status, srcuser, srcip, protocol</order>
</decoder>

ossec-logtest output:

2013-11-01T10:01:05.600494-04:00 arrakis ossec-exampled[9123]: successful authentication for user test-user from 192.168.1.1 via test-protocol1


**Phase 1: Completed pre-decoding.
       full event: '2013-11-01T10:01:05.600494-04:00 arrakis ossec-exampled[9123]: successful authentication for user test-user from 192.168.1.1 via test-protocol1'
       hostname: 'arrakis'
       program_name: 'ossec-exampled'
       log: 'successful authentication for user test-user from 192.168.1.1 via test-protocol1'

**Phase 2: Completed decoding.
       decoder: 'ossec-exampled'
       status: 'successful'
       srcuser: 'test-user'
       srcip: '192.168.1.1'
       proto: 'test-protocol1'

Now the useful fields have been extracted for this log message as well. Double checking the original log message, to make sure there were no regressions:

2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1


**Phase 1: Completed pre-decoding.
       full event: '2013-11-01T10:01:04.600374-04:00 arrakis ossec-exampled[9123]: test connection from 192.168.1.1 via test-protocol1'
       hostname: 'arrakis'
       program_name: 'ossec-exampled'
       log: 'test connection from 192.168.1.1 via test-protocol1'

**Phase 2: Completed decoding.
       decoder: 'ossec-exampled'
       srcip: '192.168.1.1'
       proto: 'test-protocol1'

Historical

Previous versions of this page page originally was created by @j_hen on her blog http://jentalkstoomuch.blogspot.com/2010/09/writing-custom-ossec-rules-for-your.html This blog no longer exists (at least at that location).