AlienVault Building Collector Plugins

Building Collector Plugins Admin Guide
Copyright Alienvault 2010

All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and publisher. Any trademarks referenced herein are the property of their respective holders.
Building Collector Plugins - Admin Guide
Table of Content
1 Overview..................................................................................................................................................... 4 1.1 OSSIM Agent Role ............................................................................................................................... 4 Event Collection .......................................................................................................................... 4 Event Normalization ................................................................................................................... 4
1.1.1 1.1.2 1.2
OSSIM Server Role .............................................................................................................................. 6 Event Enrichment ....................................................................................................................... 6 Policies and Actions .................................................................................................................... 7
1.2.1 1.2.2 1.3 2
The Configuration Workflow .............................................................................................................. 8
Configuring Detector Plugins ....................................................................................................................10 2.1 Rsyslog ..............................................................................................................................................10 Configuration File .....................................................................................................................10 Listener Configuration ..............................................................................................................10 Filters ........................................................................................................................................10
2.1.1 2.1.2 2.1.3 2.2
OSSIM Agent Configuration ..............................................................................................................11 Configuration File .....................................................................................................................11 Parameters ...............................................................................................................................11
2.2.1 2.2.2 2.3
Detector Plugin Configuration ..........................................................................................................13 Configuration Files ....................................................................................................................13 Common Event Types ...............................................................................................................13 Parameters ...............................................................................................................................13 Using Local (Plugin) Variables...................................................................................................15 Using Global (Agent) Variables .................................................................................................15
2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.4
Aliases ...............................................................................................................................................16 Path...........................................................................................................................................16 Predefined Regular Expressions ...............................................................................................16
2.4.1 2.4.2 2.5
Functions .......................................................................................................................................... 16 Path...........................................................................................................................................16 Conversions ..............................................................................................................................16 Application Specific Translations ..............................................................................................17 User Defined Translations ........................................................................................................17
2.5.1 2.5.2 2.5.3 2.5.4 2.6 2.7
Event Fields....................................................................................................................................... 18 Rules .................................................................................................................................................19 Evaluation Order.......................................................................................................................19
2.7.1 Page 2
Building Collector Plugins - Admin Guide 2.7.2 2.8 Structure ...................................................................................................................................19
Loading Plugins .................................................................................................................................21 Priority and Reliability values ...................................................................................................21 SQL Statement ..........................................................................................................................21
2.8.1 2.8.2 2.9
Plugin Activation ...............................................................................................................................22 Activate the Plugin on the Server Side .....................................................................................22 Activate the Plugin on the Agent Side ......................................................................................22
2.9.1 2.9.2 3 4 5
Log files .....................................................................................................................................................22 Debugging .................................................................................................................................................22 Appendix...................................................................................................................................................23 5.1 5.2 Regular Expressions ..........................................................................................................................23 Configuration Example .....................................................................................................................25 Scenario ....................................................................................................................................25 Write a script to monitor the last status ...............................................................................25 Log sample ................................................................................................................................25 Collect the logs in a new log file ...............................................................................................25 Restart rsyslog .......................................................................................................................26 Check whether the new entries are written in the new log file...............................................26 Create a plugin file....................................................................................................................26 Register the Plugin with the OSSIM Agent ...............................................................................29 Register the Plugin with the OSSIM Server ..............................................................................30 Check whether the plugin was successfully registered ............................................................31 Restart the OSSIM Server .........................................................................................................31 Restart the OSSIM Agent ..........................................................................................................31 Check whether Events and Alarms are received ......................................................................32
5.2.1 5.2.2 5.2.3 5.2.4 5.2.5 5.2.6 5.2.7 5.2.8 5.2.9 5.2.10 5.2.11 5.2.12 5.2.13
Page 3
1 Overview
1.1 OSSIM Agent Role
1.1.1 Event Collection The collection process involves extracting the data logs from the source systems (Security, OS, RDBMS, etc.) and allows first steps for event log filtering. At this stage can be decided what is going to be read by the OSSIM Agent and what is going to be discarded before having an impact on the system performance. Before starting to write a plugin some actions to reduce the amount of events could be considered: Manage the log level settings at the application and managed device level Fix the problem that is generating events Use Pcap filters to ignore certain hosts or networks (Snort, Tcpdump...) In deployments with a big amount of analysed data, filtering at the application level should be done whenever possible
Log Files Good practice is to use one log file per plugin in order to increase performance. Having just one generic log file, all the plugins would have to read the same extensive content in order to catch the few relevant entries. Using rsyslog it is possible to filter the collected logs based on the syslog tags.
1.1.2 Event Normalization In the normalization stage a series of rules or functions applies to the data extracted from the source system in order to transform it in a common OSSIM format. o Raw Event The raw event might be a generic syslog message, an application log, an SNMP trap, the result of an SNMP or SQL Query or some other kind of information in a more or less structured form that is appended to a log file. Example: dmz01:/var/log/auth.log: May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2
Page 4
Building Collector Plugins - Admin Guide o Normalized Event There is a certain set of fields which are required in order to ensure a consistent evaluation and correlation of the events by the OSSIM server. These fields can be populated with information from the log message or statically through the plug-in. Example: ossim-sensor:/var/log/ossim/agent.log: 2010-05-30 13:15:49,441 Output [INFO]: event type="detector" date="1275239752" sensor="192.168.178.201" interface="eth0" plugin_id="4003" plugin_sid="7" src_ip="192.168.178.20" src_port="4445" dst_ip="192.168.178.200" dst_port="22" username="root" log="May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2" fdate="2010-05-30 13:15:52" tzone="0"
Page 5
1.2 OSSIM Server Role

1.2.1 Event Enrichment The OSSIM server enriches the received normalized event with the metadata stored in the OSSIM Database. o Enriched Event The OSSIM Server enriches the event with the Priority and Reliability values, which are specific to the event type (plugin_id) and subtype (plugin_sid), as well as with the Asset Value which is specific to the Source (asset_src) and the Destination (asset_dst) hosts. Example: ossim:/var/log/ossim/server.log: 2010-05-30 06:48:41 OSSIM-Message: Event received: event id="0" alarm="0" type="detector" fdate="2010-05-30 13:15:52" date="1275239752" tzone="0" plugin_id="4003" plugin_sid="7" src_ip="192.168.178.20" src_port="4445" dst_ip="192.168.178.200" dst_port="22" sensor="192.168.178.201" interface="eth0" protocol="TCP" asset_src="2" asset_dst="2" log="May 30 13:15:52 dmz01 sshd[12980]: Accepted password for root from 192.168.178.20 port 4445 ssh2" username="root" o Priority The priority is related to threats and it reflects the importance of a specific attack, having nothing to do with a specific host or environment. It only measures the relative importance of the attack itself. Range: 0 - 5 Default value: 1 Example: A Unix server running Samba gets attacked by the Sasser worm . Apart from the fact that the attack wont have an impact on the given environment, it has the potential to exploit a big security hole and for that reason the priority is considered as being high. o Reliability Classical risk-assessment would refer it as "probability ". Since it's quite difficult to determine how probable it is for a network to be exposed to certain vulnerabilities, the IDS related reliability approach was considered more appropriate. Range: 0 - 10 Default value : 1. Example: If a host connects to 5 different hosts in the same subnet using port 445, could be a normal behavior, unreliable for IDS purposes. If connecting to 15 hosts would be Page 6
Building Collector Plugins - Admin Guide suspicious, with 500 connections to different hosts in less than an hour the attack would get more and more reliable. Asset Value It is assigned to both the Source and the Destination Hosts and represents the importance the host has to the enterprise. Range: 0 - 5 Default value: 1 (also used for hosts not being defined in the asset database) Example: A database server can have an asset value of 5, a development test server an asset value of 2 and an unknown host in the Internet causing a portscan event would just have an asset value of 1. o Alarm Based on the Event Priority (0-5), Event Reliability (0-10) and the Asset Value (0-5), a Risk Value (0-10) is calculated and for values equal or greater than 1 Alerts are generated. The Risk is calculated based on the following formula: Risk = (Priority * Reliability * Asset) / 25 1.2.2 Policies and Actions Policies are defined in order to define what has to be done with the events as they reach the OSSIM Server: Correlation (i.e. checked against the correlation directives) Forwarding (i.e. one copy is sent to the forensic storage) Actions (i.e. send an email) Discard - the last filter possibility before saving the event in the database, although it is recommended to filter the events as close to the source as possible. Policies can make decisions on which events are going to be filtered based on: Source and Destination Assets (Hosts, Networks, ANY...) Ports Plugin Group Time Range
Page 7
1.3 The Configuration Workflow

o Collect a Log Sample
First thing to start with is checking which log messages the application generates and eventually identify sets of logs having a similar structure. Those logs having a similar structure will be where possible covered by a single collector rule. o Create a Plugin File
Best is to copy one existing file and modify its content to match the new application. Should a plugin exist for a similar application, it is recommended to copy such a file, as there is a good chance that rules have a similar content and are grouped in a similar way - a generic HTTP-Proxy log will always contain a URL, a generic Firewall log will contain a Source IP Address and Source Port as well as a Destination IP Address and Destination Port. Some user defined fields might be defined for a specific application and the correlation at the server level can be simplified if similar applications use the same user defined fields. o Define a Generic Rule
This is the last Rule to evaluate, which catches all the events that cannot be grouped under specific rules. o Define Specific Rules
The Specific rules are defined for specific error conditions or categories of events. There might also be that one single rule is used to generate different types or subtypes of events. o Discard Noise
Events that are considered noise can be discarded by OSSIM by excluding certain event subtypes (Plugin_SIDs) in the plugin file, by the way the regular expressions are defined or by using policies. However, the best way to discard events is by filtering them on the monitored device or at syslog level on the host running the OSSIM Agent. o Review the Evaluation Order
The rules are evaluated alphabetically, which means that all it counts is the name of a rule and not the position in the plug-in file. The Generic Rule might even be on the first position if the name is properly chosen. Having rules alphabetically placed after the Generic Rule will have as effect that the corresponding logs will be evaluated as generic events instead of having the proper event type and subtype assigned. o Register the Plugin with the OSSIM Agent
In order to have a Plugin activated and sending events to the OSSIM server, the path to the plugin file has to be specified in the Agent configuration file.
Page 8
Building Collector Plugins - Admin Guide o Register the Plugin with the OSSIM Server
This is required in order to let the server know which events should be expected and which priority and reliability values the events should get assigned. o Activate the Plugin on the Server Side
Restart the OSSIM Server process. o Activate the Plugin on the Agent Side
Restart the OSSIM Agent process. o Testing
Using the logger command sample logs can be replayed in order to test the operation of the OSSIM Agent or Server.
Page 9
2 Configuring Detector Plugins

2.1 Rsyslog
Rsyslog is the Syslog implementation shipped with OSSIM and allows configuring filtering and forwarding in a really easy way compared to the classical syslog daemon. Syslog is also the common method to send and receive logs. Before starting with the plugin configuration it is recommended to check whether the subset of logs the plugin will normalize are saved in an individual file and whether noise can be filtered before reaching the plugin rules. 2.1.1 Configuration File /etc/rsyslog.conf 2.1.2 Listener Configuration $ModLoad imudp $UDPServerRun 514 $ModLoad imtcp $InputTCPServerRun 514 2.1.3 Filters Forward certain events to a local file if $msg contains 'error' then /var/log/error if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains 'error0') then /var/log/somelog Stop processing some events if $msg contains 'error' then ~ Regex in Rsyslog http://www.rsyslog.com/user-regex.php
Page 10
2.2 OSSIM Agent Configuration

2.2.1 Configuration File /etc/ossim/agent/config.cfg 2.2.2 Parameters [daemon] daemon: pid: [event-consolidation] Enables event consolidation at agent level. It is recommended to use polices instead of this feature as consolidation at the agent level affects the correlation process. by_plugin: enable: time: Example: [event-consolidation] by_plugin=1001-1150,1501-1550,4001-4010 enable=False time=10 [log] Configures the verbose level and the path to the different log files error: file: stats: verbose: [output-plain] Writes in a log file what is being sent to the OSSIM Server (Useful for debugging and developing purposes) enable: file: [output-server] Configures the server to which events are sent enable: ip: port: Enable or disable sending events to the server (True or False) IP address of the OSSIM Server Listening port of the OSSIM Server Enable or disable (True or False) File in which the output-plain will be stored File in which the error events will be stored File in which all the agent logs will be stored File in which the agent stats will be stored (Every 5 minutes) Configures the verbose level (Debug, Info, Warning, Error or Critical) List of plugins that will be consolidated Enable or disable (True or False) Wait n seconds to consolidate the events before sending them Daemon mode (True or False) Path to the PID file (Process identifier)
Page 11
Building Collector Plugins - Admin Guide [plugin-defaults] In this category variables can be defined to be used in the plugins configuration. Example: [plugin-defaults] date_format=%Y-%m-%d %H:%M:%S interface=eth0 sensor=192.168.178.201 [plugins] Defines which plugins (detectors and monitors) are enabled name_of_the_plugin=path_to_the_plugin_config_file Example: [plugins] postfix=/etc/ossim/agent/plugins/postfix.cfg ssh=/etc/ossim/agent/plugins/ssh.cfg [watchdog] Monitor the process associated to each plugin (In case it is running in the same machine) enable: interval: restart_interval: Enable or disable (True or False) Wait X seconds between checks Restart the process every X seconds (This has to be enabled in each plugin)
Page 12
2.3 Detector Plugin Configuration

2.3.1 Configuration Files /etc/ossim/agent/plugins/*.cfg 2.3.2 Common Event Types Copy and modify the existing plugin files to create plugins of the following types. a. Log - Reading from files
Plugin statement: source=log b. Database - Reading from databases
Plugin statement: source=database mssql - Microsoft SQL
Plugin statement: source_type=mssql mysql - MySQL
Plugin statement: source_type=mysql c. SDEE - Cisco device logs
Plugin statement: source=sdee d. SnortLog - Snort logs
Plugin statement: source=snortlog e. WMI - Windows Management Instrumentation
Plugin statement: source=wmi 2.3.3 Parameters [DEFAULT] Any variable defined inside this category will be sent to the OSSIM Server if not modified by a plugin rule. User reserved range is between 9000 and 10000. plugin_id: Example: plugin_id=4003 Numerical identifier of the plugin within the OSSIM system
Page 13
Building Collector Plugins - Admin Guide [config] type: enable: source: location: create_file: process: start: stop: startup: shutdown: exclude_sids=SID List Example (hp-eva): process=snmptrapd start=yes stop=yes startup=/etc/init.d/snmpd start shutdown=/etc/init.d/snmpd stop exclude_sids=404,200,403 [translation] string=value Example (Postfix): [translation] sent=10 bounced=11 [Rule IDs Specific Rules] Here are the events collected and normalized. event_type=event regexp=Regular Expression plugin_sid=Plugin SID Event_Field=Value Example(ssh): [01 - Failed password] event_type=event regexp="(\SYSLOG_DATE)\s+(?P<sensor>[^\s]*).*?ssh.*?Failed password for inval user (?P<user>\S+)\s+from\s+.*?(?P<src>\IPV4).*?port\s+(?P<sport>\PORT)" plugin_sid=1 date={normalize_date($1)} Page 14 detector Enable or Disable the plugin (It must be enabled in config.cfg) Source of the events (log, mssql, mysql, wmi) The file(s) where the logs can be found - can contain multiple comma-separated files Create the log file in case it does not exist Name of the process generating logs (If on the same system) Start the process when the agent starts (yes/no) Stop the process when the agent stops (yes/no) Command that starts the process Command that stops the process Use this option to exclude SIDs
Used to map strings to their corresponding values
Building Collector Plugins - Admin Guide src_ip={$src} dst_ip={resolv($sensor)} src_port={$sport} username={$user} [Rule IDs Specific Rules] [Rule ID Generic Rule] Example (ssh): [99 - Generic rule] # Nov 15 11:55:35 11.1.4.9 sshd[1769702]: ********** event_type=event regexp="(\SYSLOG_DATE)\s+(?P<sensor>[^\s]*).*?ssh.*" plugin_sid=99 date={normalize_date($1)} dst_ip={resolv($sensor)} Note: As rules are ordered alphabetically the Generic Rule has to have the highest Rule ID . 2.3.4 Using Local (Plugin) Variables The different configuration variables defined in the plugin configuration file can be used with the following syntax: %()s Example: process=pads shutdown=killall -9 %(process)s 2.3.5 Using Global (Agent) Variables \_CFG() Example: In the agent configuration file (/etc/ossim/agent/config.cfg): [watchdog] restart_interval=3600 ; seconds between plugin process restart In the plugin configuration file(/etc/ossim/agent/plugins/*.cfg): restart_interval=\_CFG(watchdog,restart_interval)
Page 15
2.4 Aliases
2.4.1 Path /etc/ossim/agent/aliases.cfg 2.4.2 Predefined Regular Expressions The predefined regular expressions can be used when creating new plugins. IPV4= \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} MAC= \w{1,2}:\w{1,2}:\w{1,2}:\w{1,2}:\w{1,2}:\w{1,2} PORT= \d{1,5} TIME= \d\d:\d\d:\d\d
SYSLOG_DATE= \w{3}\s+\d{1,2}\s\d\d:\d\d:\d\d SYSLOG_WY_DATE= \w+\s+\d{1,2}\s\d{4}\s\d\d:\d\d:\d\d To use an Alias in the regular expression use the \IPV4, \MAC, \SYSLOG_DATE, etc.
2.5 Functions
2.5.1 Path /usr/share/ossim-agent/ossim_agent/ParserUtil.py 2.5.2 Conversions resolv(host): resolv_ip(addr): resolv_port(port): normalize_date(date): translates a host name to an IPv4 address translates an IPv4 address to a host name translate a port name into its number convert date strings to isoformat (must tag the regular expressions with the following: <year>, <month>, <minute>, <hour>, <minute>, <second> or <timestamp> for timestamps. To define new date formats add a new regexp to the DATE_REGEXPS array. translates the protocols to the protocol numbers, based on the PROTO_TABLE calculates the md5 checksum all upper case get the integer value of a hexadecimal number
normalize_protocol(proto): md5sum(datastring): upper(string): hextoint(string):
Page 16
Building Collector Plugins - Admin Guide 2.5.3 Application Specific Translations snort_id(id): intrushield_sid(sid,name): adds 1000 to the Snort ID all McAfee Intrushield IDs are divisible by 256, and this length doesn't fit in the OSSIM table ( mcafee_sid = hextoint(mcafee_sid)/256) translates the Netscreen messages based on the NETSCREEN_IDP_SID_TRANSLATION_TABLE translation table (defined in ParserUtil.py) translates the ISS_SiteProtector messages based on the ISS_SITEPROTECTOR_SID_TRANSLATION_MAP translation table (defined in ParserUtil.py) normalize interface name to either ext or int
netscreen_idp_sid(msg):
iss_siteprotector_sid(msg):
resolv_iface(iface):
2.5.4 User Defined Translations translate(string): translates strings based on the entries defined in the [translation] section of the plugin.
Example (from the iptables plugin): # The translation section in the plugin configuration file [translation] ACCEPT=1 REJECT=2 DROP=3 DENY=3 Inbound=4 Outbound=5 # Rule ID [0 - iptables] # Log sample # Oct 31 08:59:25 M2600001 kernel: RULE 0 -- ACCEPT IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 # TOS=0x00 PREC=0x00 TTL=64 ID=8437 DF PROTO=TCP SPT=57275 DPT=836 SEQ=2806649400 # ACK=0 WINDOW=32767 RES=0x00 SYN URGP=0 # Log Parsing regexp=(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(\S*) (\S*):.*?(\S+)\s+IN=(\S*) OUT=(\S*) SRC=(\S+) DST=(\S+) LEN=(\d+) \S+ \S+ TTL=(\d+) .*? PROTO=(\S*) SPT=(\d*) DPT=(\d*) # plugin_sid is set to 1, the translated value for ACCEPT plugin_sid={translate($4)}
Page 17
2.6 Event Fields

Mandatory no default values, have always to be set when creating a new plugin plugin_id plugin_sid date sensor interface protocol src_ip src_port dst_ip dst_port Optional username password filename The User referred in the event The Password referred in the event The Filename referred in the event Event Type Event Subtype The time the event has been collected from the device The IP Address of the sensor collecting the event The interface where the event has been collected IP Protocol (see /etc/protocols) The Source IP Address The Source Port The Destination IP Address The Destination Port
Mandatory default values are assigned by the OSSIM Agent
userdata1 userdata9 User defined fields that could be used in custom reports, correlation directives, etc. Special types of events and the list of fields that can be used in each event type: Host-os-event host os sensor interface date Host-mac-event host mac vendor sensor interface date Host-service-event host sensor interface port protocol service application date
Page 18
2.7 Rules
The Rules define the format of each event and how they are normalized. It is composed by a regular expression and the list of fields that the event will include once it is sent to the OSSIM Server. In some cases only one regular expression will collect every event coming from one application, in some other cases more than one rule will be required. 2.7.1 Evaluation Order Rules are loading in alphabetical order based on the name given to each rule (Rule ID). Once the log matches the regex of one rule the ossim agent stops processing the event, therefore generic rules must be the last to be evaluated. 2.7.2 Structure o Name / Rule ID The name of the rule is mandatory o Regular Expression The regexp field contains the regular expression that defines the format of the events, and extracts the information to normalize the event. The regular expression has to be written following Python regular expression syntax: http://docs.python.org/library/re.html The information extracted by the regular expression from the log can be accessed by: Position: (\d\d):(\d\d):(\d\d) hour={$1} minutes ={$2} seconds={$3} Tags: (?P<hour>\d\d):(?P<minutes>\d\d)(?P<seconds>\d\d) hour={$hour} minutes ={$minutes} seconds={$seconds} o Normalized Fields As the server must receive normalized events, where IP addresses for instance are using the IPV4 format and the date uses the format YYYY-MM-DD HH:MM:SS (2010-12-31 22:57:00) To simplify the process of normalizing events functions are defined (more details on functions can be found in the Functions section of this document): resolv() Translates hostnames into IPV4 addresses (DNS queries)
Page 19
Building Collector Plugins - Admin Guide normalize_date() The normalize_date function translates many date formats into the format accepted by the OSSIM Server. o Translations Used for instance when the Event ID is not numeric, but plugin_sid has to be numeric. Translations have to be defined inside the [translation] section. The actual translation is triggered by using the translate() function. o Exclusions Some events can be filtered during the collection process editing the configuration file for each plugin: Using the option exclude_sids Modifying the regular expressions to avoid matching certain events
Page 20
2.8 Loading Plugins

2.8.1 Priority and Reliability values For each Plugin_ID/Plugin_SID pair the Priority and Reliability values will have to be defined while registering the plugin with the OSSIM Server. 2.8.2 SQL Statement Similar to copying an existing plugin file and customize it in order to create a new plugin file, an SQL script can be copied and customized in order to insert the new Plugin information in the database. The sample SQL script can be found under: /usr/share/doc/ossim-mysql/contrib/plugins/*.sql Other than with the Plugin configuration file, the SQL script should be created and executed on the OSSIM Server and not where the OSSIM Agent runs. The following is performed by the SQL script: Remove the Plugin ID from the plugin table, should such an entry already exist Remove the Plugin SIDs from the plugin_sid table, should already exist Insert the new Plugin ID information into the plugin table Insert the new Plugin SIDs into the plugin_sid table
To run the script use the following command (please double-check the content of the SQL scripts and the command line syntax before applying the changes to the database): ossim-server:/usr/share/doc/ossim-mysql/contrib/plugins# ossim-db < ssh.sql Example (/usr/share/doc/ossim-mysql/contrib/plugins/ssh.sql): -- SSHd -- plugin_id: 4003 DELETE FROM plugin WHERE id = "4003"; DELETE FROM plugin_sid where plugin_id = "4003"; INSERT INTO plugin (id, type, name, description) VALUES (4003, 1, 'sshd', 'SSHd: Secure Shell daemon'); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (4003, 1, NULL, NULL, 'SSHd: Failed password', 3, 2); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (4003, 2, NULL, NULL, 'SSHd: Failed publickey', 2, 2); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority,reliability) VALUES (4003, 99, NULL, NULL, 'SSHd: Generic SSH Event', 1, 1); Page 21
2.9 Plugin Activation

2.9.1 Activate the Plugin on the Server Side Restart the OSSIM Server process: ossim-server:~#/etc/init.d/ossim-server restart 2.9.2 Activate the Plugin on the Agent Side Restart the OSSIM Agent process: ossim-sensor:~#/etc/init.d/ossim-agent restart
3 Log files
Generic Syslog /var/log/syslog (Unix) /var/adm/messages (Solaris) To identify where the logs for specific applications or certain logging levels are saved, check the /etc/syslog.conf or /etc/rsyslog.conf files. OSSIM Agent /var/log/ossim/agent.log OSSIM Server /var/log/ossim/server.log
4 Debugging
Note: Do never leave an application running in Debug mode in a production environment OSSIM Agent ossim-agent vv OSSIM Server ossim-server D6
Page 22
5 Appendix
5.1 Regular Expressions
Operator c \c ^ $ . [] [^] Meaning A non-special character matches with itself Removes the special meaning of the character c; The RE \$ matches with $ Indicates the beginning of the line Indicates the end of the line Any individual character One or any of the characters ; accepts intervals of the type a-z, 0-9, A-Z A char different from ; Accepts intervals of the type a-z, 0-9, A-Z
Regular Expression a.b a..b [abc] [aA] [aA][bB] [0123456789] [0-9] [A-Za-z] [0-9][0-9][0-9] [0-9]* [0-9][0-9]* ^.*$
Matches with axb aab abb aSb a#b ... axxb aaab abbb a4$b ... a b c (one character srtings) a A (one character srtings) ab Ab aB AB (two character srtings) 0123456789 0123456789 A B C ... Z a b c ... Z 000 001 .. 009 010 .. 019 100 .. 999 empty_chain 0 1 9 00 99 123 456 999 9999 ... 0 1 9 00 99 123 456 999 9999 99999 99999999 ... A full line
Page 23
Building Collector Plugins - Admin Guide Operator r* r+ r? r{n} r{,m} r{n,m} r1|r2 Meaning 0 or more occurrences of the RE r 1 or more occurrences of the RE r 0 or an occurrence of the RE r, and no more n occurrences of the RE r 0 or at most m occurrences of the RE r N or more occurrences of the RE r, but at most m The RE r1 or the RE r2
Regular expression [0-9]+ [0-9]? (ab)* ([0-9]+ab)*
Matches with 0 1 9 00 99 123 456 999 9999 99999 99999999 .. empty_string 0 1 2 .. 9 empty_string ab ababab abababababab empty_string 1234ab 9ab9ab9ab 9876543210ab 99ab99ab ...
Regular expression \d \D \s \S \w
Matches with Any decimal character Any non decimal character Any space character Any non space character Any alphanumeric character and _
Equals [0-9] [^0-9] [ \t\n\r\f\v] [^ \t\n\r\f\v] [a-zA-Z0-9_]
\W \Z
Any non alphanumeric character End of line
[^a-zA-Z0-9_]
Page 24
5.2 Configuration Example

5.2.1 Scenario In order to detect user logons on a Unix system, the last command output will be used. The last command displays the content of the /var/log/wtmp file, where closed and opened terminal sessions as well as system restarts are logged. To just create events on status updates, the last output will be collected periodically and compared to the similar information saved with the previous loop. The status updates will be sent by syslog, with the help of the logger command. 5.2.2 Write a script to monitor the last status
#!/bin/sh # create the file if does not exist touch /var/log/last.prev while true do # get last entries last > /var/log/last.new # send new entries to syslog diff /var/log/last.prev /var/log/last.new | grep '^>' | logger -t LOGON_EXAMPLE -p local2.info # move .new to .prev mv /var/log/last.new /var/log/last.prev sleep 5 done
5.2.3 Log sample

dmz01:~# tail -f /var/log/messages Jul 14 19:21:32 dmz01 LOGON_EXAMPLE: > root pts/3 Jul 14 19:23:28 dmz01 LOGON_EXAMPLE: > dbadmin pts/3 Jul 14 19:23:59 dmz01 LOGON_EXAMPLE: > root pts/4 Jul 14 19:24:09 dmz01 LOGON_EXAMPLE: > root pts/4 Jul 14 19:24:09 dmz01 LOGON_EXAMPLE: > dbadmin pts/3 Jul 14 19:24:09 dmz01 LOGON_EXAMPLE: > root pts/2 Jul 14 19:24:54 dmz01 LOGON_EXAMPLE: > root pts/2 Jul 14 19:26:15 dmz01 LOGON_EXAMPLE: > root pts/2 Jul 14 19:26:20 dmz01 LOGON_EXAMPLE: > ossim pts/2 Jul 14 19:26:25 dmz01 LOGON_EXAMPLE: > ossim pts/2 localhost Wed Jul 14 18:49 - 19:21 (00:31) localhost Wed Jul 14 19:23 still logged in localhost Wed Jul 14 19:23 still logged in localhost Wed Jul 14 19:23 - 19:24 (00:00) localhost Wed Jul 14 19:23 - 19:24 (00:00) 172.22.22.10 Wed Jul 14 18:38 - 19:24 (00:45) 172.22.22.10 Wed Jul 14 19:24 still logged in 172.22.22.10 Wed Jul 14 19:24 - 19:26 (00:01) 172.22.22.10 Wed Jul 14 19:26 still logged in 172.22.22.10 Wed Jul 14 19:26 - 19:26 (00:00)
5.2.4 Collect the logs in a new log file Add the following to the rsyslog.conf on the system running the OSSIM Agent:
# # LOGON_EXAMPLE # local2.info /var/log/last_logon.log
Page 25
Building Collector Plugins - Admin Guide 5.2.5 Restart rsyslog

opensourcesim:~# /etc/init.d/rsyslogd restart
5.2.6 Check whether the new entries are written in the new log file
opensourcesim:/etc/ossim/agent/plugins# tail -f /var/log/last_logon.log Jul 14 19:38:49 dmz01 LOGON_EXAMPLE: > root pts/2 localhost Wed Jul 14 19:38 still logged in Jul 14 19:38:54 dmz01 LOGON_EXAMPLE: > root pts/2 localhost Wed Jul 14 19:38 - 19:38 (00:00) Jul 14 19:38:59 dmz01 LOGON_EXAMPLE: > ossim pts/2 localhost Wed Jul 14 19:38 still logged in Jul 14 19:40:51 dmz01 LOGON_EXAMPLE: > ossim pts/2 localhost Wed Jul 14 19:38 - 19:40 (00:01) Jul 14 20:15:09 dmz01 LOGON_EXAMPLE: > reboot system boot 2.6.31.6 Wed Jul 14 17:39 - 20:15 (02:35)
5.2.7 Create a plugin file Copy an existing plugin to build the new one on the existing structure
opensourcesim:/etc/ossim/agent/plugins# cp syslog.cfg example.cfg
Set the new plugin specific parameters

;; Building Plugins Example ;; plugin_id: 9001 ;; type: detector [DEFAULT] plugin_id=9001 [config] type=detector enable=yes source=log # Enable syslog to log everything to one file. Add it to log rotation also. # echo "*.* /var/log/all.log" >> /etc/syslog.conf; killall -HUP syslogd #location=/var/log/all.log location=/var/log/last_logon.log # create log file if it does not exists, # otherwise stop processing this plugin create_file=true process= start=no stop=no startup= shutdown= ## rules [Rule 01 - Console Session Open] # Jul 14 20:36:47 dmz01 LOGON_EXAMPLE: > root tty1 Wed Jul 14 20:36 still logged in event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >\s+(?P<username>[^\s]+)\s+(?P<tty>tty\d+)\s+(?P<logged_event>.*still logged in.*))$" sensor=\_CFG(plugin-defaults,sensor)
Page 26

date={normalize_date($1)} plugin_sid=1 username={$username} dst_ip={resolv($host)} userdata1={$tty} userdata2={md5sum($logline)} userdata3={$logline} userdata4={$logged_event} [Rule 02 - Console Session Closed] # Jul 14 20:35:46 dmz01 LOGON_EXAMPLE: > root tty1 Wed Jul 14 20:18 - 20:35 (00:17) event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >\s+(?P<username>[^\s]+)\s+(?P<tty>tty\d+)\s+(?P<logged_event>.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($1)} plugin_sid=2 username={$username} dst_ip={resolv($host)} userdata1={$tty} userdata2={md5sum($logline)} userdata3={$logline} userdata4={$logged_event} [Rule 03 - New User Session - IP] # Jul 14 20:21:49 dmz01 LOGON_EXAMPLE: > root pts/1 172.22.22.10 Wed Jul 14 20:21 still logged in event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>\IPV4)\s+(?P<logged_event>.*still logged in.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($1)} plugin_sid=3 username={$username} src_ip={$source} dst_ip={resolv($host)} userdata1={$tty} userdata2={md5sum($logline)} userdata3={$logline} userdata4={$logged_event} [Rule 04 - New User Session - hostname] # Jul 14 19:23:28 dmz01 LOGON_EXAMPLE: > dbadmin pts/3 localhost Wed Jul 14 19:23 still logged in event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>localhost)\s+(?P<logged_event>.*still logged in.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($1)} plugin_sid=3 username={$username} src_ip=127.0.0.1 dst_ip={resolv($host)} userdata1={$tty} userdata2={md5sum($logline)} userdata3={$logline} userdata4={$logged_event}
Page 27

[Rule 05 - User Session Closed - IP] # Jul 14 19:26:25 dmz01 LOGON_EXAMPLE: > ossim pts/2 172.22.22.10 Wed Jul 14 19:26 - 19:26 (00:00) event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>\IPV4)\s+(?P<logged_event>.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($1)} plugin_sid=4 username={$username} src_ip={$source} dst_ip={resolv($host)} userdata1={$tty} userdata2={md5sum($logline)} userdata3={$logline} userdata4={$logged_event} [Rule 06 - User Session Closed - hostname] # Jul 14 19:33:56 dmz01 LOGON_EXAMPLE: > root pts/2 localhost Wed Jul 14 19:33 - 19:33 (00:00) event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >\s+(?P<username>[^\s]+)\s+(?P<tty>[^\s]+)\s+(?P<source>localhost)\s+(?P<logged_event>.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($1)} plugin_sid=4 username={$username} src_ip=127.0.0.1 dst_ip={resolv($host)} userdata1={$tty} userdata2={md5sum($logline)} userdata3={$logline} userdata4={$logged_event} [Rule 07 - Reboot Detected] # Jul 14 20:15:09 dmz01 LOGON_EXAMPLE: > reboot system boot 2.6.31.6 Mon May 24 13:51 - 20:15 (51+06:23) event_type=event regexp="^(?P<logline>(\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE: >reboot.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($1)} plugin_sid=5 userdata1={md5sum($logline)} userdata2={$logline} userdata3={$generator} userdata4={$logged_event} [Rule 99 - Catch all] # Whatever doesn't match the above rules event_type=event regexp="^(?P<logline>(?P<date>\S+\s+\d+\s+\d\d:\d\d:\d\d)\s+(?P<host>[^\s]+)\s+LOGON_EXAMPLE:.*))$" sensor=\_CFG(plugin-defaults,sensor) date={normalize_date($date)} plugin_sid=99 userdata1={md5sum($logline)} userdata2={$logline} userdata3={$logged_event}
Page 28
Building Collector Plugins - Admin Guide 5.2.8 Register the Plugin with the OSSIM Agent Add the example.cfg plugin to the agent configuration file
opensourcesim:~# more /etc/ossim/agent/config.cfg [plugins] syslog=/etc/ossim/agent/plugins/syslog.cfg example=/etc/ossim/agent/plugins/example.cfg wmi-system-logger=/etc/ossim/agent/plugins/wmi-system-logger.cfg Alternatively the plugin can be activated with the ossim-setup utility: opensourcesim:~# ossim-setup 1) Change Sensor Settings
2)
Select Detector Plugins
3)
Select the example Plugin
Page 29

4) Save & Exit
5.2.9 Register the Plugin with the OSSIM Server Copy an existing SQL script to build the new one on the existing structure
opensourcesim:/usr/share/doc/ossim-mysql/contrib/plugins# cp syslog.sql example.sql
Get a list of the rules defined in the Plugin configuration file.

opensourcesim:~# grep '^\[' /etc/ossim/agent/plugins/example.cfg [Rule 01 - Console Session Open] [Rule 02 - Console Session Closed] [Rule 03 - New User Session - IP] [Rule 04 - New User Session - hostname] [Rule 05 - User Session Closed - IP] [Rule 06 - User Session Closed - hostname] [Rule 07 - Reboot Detected] [Rule 99 - Catch all]
Rules having the same plugin_sid will only require one SQL statement and plugin_sid defined on the OSSIM server. Different rules where used just because both IP addresses and hostnames are returned as sources by the last command.
-- plugin_id: 9001 DELETE FROM plugin WHERE id = "9001"; DELETE FROM plugin_sid where plugin_id = "9001"; INSERT INTO plugin (id, type, name, description) VALUES (9001, 1, 'Example', 'User logons based on the last output'); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 1, NULL, NULL, 'Login: System console' , 5, 5); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 2, NULL, NULL, 'Logout: System console' , 5, 5); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 3, NULL, NULL, 'Login: Pseudo terminal' , 3, 5); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 4, NULL, NULL, 'Logout: Pseudo terminal' , 3, 5); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 5, NULL, NULL, 'System reboot: Restarted' , 5, 5); INSERT INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (9001, 99, NULL, NULL, 'Last: Generic messages' , 1, 1);
After changing the script to reflect the Plugin IDs and SIDs, load the changes with the command:
opensourcesim:/usr/share/doc/ossim-mysql/contrib/plugins# cat example.sql | ossim-db
Page 30
Building Collector Plugins - Admin Guide 5.2.10 Check whether the plugin was successfully registered
Plugin ID
Plugin SIDs
5.2.11
Restart the OSSIM Server
opensourcesim:~# /etc/init.d/ossim-server restart
5.2.12
Restart the OSSIM Agent
opensourcesim:~# /etc/init.d/ossim-agent restart
Page 31
Building Collector Plugins - Admin Guide 5.2.13 Check whether Events and Alarms are received
Events
Alarms
Page 32

AlienVault Building Collector Plugins

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

AlienVault Building Collector Plugins

Uploaded by

Copyright:

Available Formats

Building Collector Plugins Admin Guide

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

1.1.1 1.1.2 1.2

1.2.1 1.2.2 1.3 2

The Configuration Workflow .............................................................................................................. 8

2.1.1 2.1.2 2.1.3 2.2

2.2.1 2.2.2 2.3

2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.4

2.4.1 2.4.2 2.5

2.5.1 2.5.2 2.5.3 2.5.4 2.6 2.7

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide 2.7.2 2.8 Structure ...................................................................................................................................19

2.8.1 2.8.2 2.9

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

Copyright Alienvault 2010

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

1.2 OSSIM Server Role

Copyright Alienvault 2010

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

1.3 The Configuration Workflow

Copyright Alienvault 2010

Restart the OSSIM Agent process. o Testing

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

2 Configuring Detector Plugins

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

2.2 OSSIM Agent Configuration

Copyright Alienvault 2010

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

2.3 Detector Plugin Configuration

Plugin statement: source=log b. Database - Reading from databases

Plugin statement: source=database mssql - Microsoft SQL

Plugin statement: source_type=mssql mysql - MySQL

Plugin statement: source_type=mysql c. SDEE - Cisco device logs

Plugin statement: source=sdee d. SnortLog - Snort logs

Plugin statement: source=snortlog e. WMI - Windows Management Instrumentation

Copyright Alienvault 2010

Used to map strings to their corresponding values

Copyright Alienvault 2010

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

normalize_protocol(proto): md5sum(datastring): upper(string): hextoint(string):

Copyright Alienvault 2010

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

2.6 Event Fields

Mandatory default values are assigned by the OSSIM Agent

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

Copyright Alienvault 2010

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

2.8 Loading Plugins

Copyright Alienvault 2010

Building Collector Plugins - Admin Guide

2.9 Plugin Activation

Copyright Alienvault 2010