pandora fms: advanced log parser

Pandora FMSAdministrator's Manual

LogParser Monitoring

Administrator's Manual Monitorización LogParser

© Artica Soluciones Tecnológicas 20052012

Indice1Changelog...........................................................................................................................................32Introduction........................................................................................................................................43Requirements......................................................................................................................................54Compatibility Matrix .........................................................................................................................65Software Agent Modules generates....................................................................................................76Instalation...........................................................................................................................................87Monitoring..........................................................................................................................................9

7.1.General Parameters..................................................................................................................107.1.1.include..............................................................................................................................107.1.2.index_dir..........................................................................................................................107.1.3.logfile...............................................................................................................................10

7.2.Log's specific parameters ........................................................................................................107.2.1.log_begin y log_end.........................................................................................................107.2.2.log_module_name............................................................................................................107.2.3.log_description.................................................................................................................107.2.4.log_type............................................................................................................................117.2.5.log_rotate_mode...............................................................................................................117.2.6.log_force_readall.............................................................................................................117.2.7.log_location_exec............................................................................................................117.2.8.log_location_filename......................................................................................................11

7.3.Parametros específicos de la regexp.......................................................................................117.3.1.log_regexp_begin y log_regexp_end...............................................................................117.3.2.log_regexp_rule...............................................................................................................117.3.3.log_regexp_severity.........................................................................................................127.3.4.log_regexp_message........................................................................................................127.3.5.log_regexp_action............................................................................................................12

1 CHANGELOG

Date Author Change Version

02/03/11 Sancho First Version v1r1

22/11/12 Mario P. Second Revision v1r2

2 INTRODUCTION

This document describes the generic logs monitoring based in Enterprise parsing logs plugin,

different that OpenSource plugin .This plugin is designed to work with version 3.2.1 or higher.

3 REQUIREMENTS

The plugin has the requirements to work correctly:

• Create settings in a configuration file, which the plugin has access. (passed as parameter).

• You can write temporary files (for every log analyzed) to store the last position reading,

inode or md5 signature (for identification of rotated). The default directory is / tmp but this

is a parameter that can be specified in the configuration file.

• Can read the files to process with the user that runs Pandora, or call a script which in turn

call the plugin with all parámeters, so he can read log completly. If you use an external

script, this will have to have permissions to the plugin will generate its index files (see

above)

4 COMPATIBILITY MATRIX The agent compatibility matrix is the following:

Systems where it has been tested • Linux (SUSE, Debian, Ubuntu...)

Systems where it should work

• Solaris (con Perl 5.8)• HPUX (con Perl 5.8)• AIX (con Perl 5.8)• Windows

5 SOFTWARE AGENT MODULES GENERATES

It will create a module for each parameter that you specify in the configuration file. Config_file is

needed for execution.

The plugin is configured by an external configuration file. This configuration file has a number of

“general” parameters, a series of specific parameters for each log and a set of specific parameters

for each block of regular expression.

6 INSTALATION

Copy the plugins to the agent plugin directory, distribute it through file collections or copying it in

the pandora agent folder. Do the same with the additional files that they need. The call from the

agent will be similar to this, but using the paths where the plugin and the list would be installed.

For example:

module_plugin perl /var/opt/PandoraFMS/etc/pandora/plugins/pandora_logparser.pl /var/opt/PandoraFMS/etc/pandora/collections/fc_23/log_example.conf

7 MONITORING

The plugin is configured by an external configuration file. This configuration file has a number of

“general” parameters, a series of specific parameters for each log, and a set of specific parameters

for each block of regular expression.

In order to understand each element, following is a sample configuration file:

# Include, to load extenal/aditional configuration files # include /tmp/my_other_configuration.conf

# Directory where temporal indexes will be stored (/tmp by default) #index_dir /tmp

# Log problems with the logparser, (/tmp/pandora_logparser.log by default) #logfile /tmp/pandora_logparser.log

log_begin

log_module_name errores_apache

# This force to process all the log at the beginning log_force_readall

#log_location_exec /tmp/miscript.sh | cut -f 2 log_location_file /var/log/apache2/error_log

log_description This is a nice sample of how powerful is the new logparser

# log rotation detection mode (md5 or inode change), inode by default # log_rotate_mode md5 # log_rotate_mode inode

#log_type return_lines log_type return_ocurrences #log_type return_message

log_regexp_begin log_regexp_rule Critical - ($1)\-($2) log_regexp_rule Critical - ($1)

#log_regexp_severity NORMAL #log_regexp_severity WARNING log_regexp_severity CRITICAL log_return_message Encontrado error CRITICO en bloque $1 seccion $2 log_action <mycommand> log_regexp_end

log_regexp_begin log_regexp_rule Error -($1)\-($2) [0-9a-zA-Z]* log_regexp_severity WARNING log_return_message Otro bonito texto de error log_regexp_end

log_regexp_begin log_regexp_rule File\sdoes\snot\sexist log_regexp_severity WARNING log_regexp_end

log_end

log_begin log_force_readall

log_module_name hits_apache log_location_file /var/log/apache2/access_log log_description Access log from Apache, we will get the integria access log_type return_lines

log_regexp_begin log_regexp_rule pandora\.css log_regexp_severity WARNING log_return_message Dispongo de barcos log_regexp_end

log_end

7.1. General Parameters

7.1.1. includeMakes a call to another configuration file. You can nest without limit, and its load order is

sequence. It is important to call files with absolute paths.

7.1.2. index_dirUse this directory to store the index files. The plugin should be able to write and read in the

directory.

7.1.3. logfilePlugin's logfile.

7.2. Log's specific parameters

7.2.1. log_begin y log_endSet marks of the beginning and end of a file definition logparser.log

7.2.2. log_module_nameModule name generated by the plugin.

7.2.3. log_descriptionModule description referring to log file.

7.2.4. log_typeLog module type, can be of three types:

• return_ocurrences: Returns a numeric data with the number of occurrences.

• return_lines: Returns the log lines that do match.

• return_message: Returns a message specified by the configuration file.

7.2.5. log_rotate_modeCan be of inode type or md5 type. This is the type detection is done to know if a log is rotated or

not.

7.2.6. log_force_readallWhen this token is present, the log parser processes all the log from the beginning if you have not

already done (Is the first time I opened or detects a rotation). NOTE: You can generate large

volumes of data.

7.2.7. log_location_execExecutes the specified command to obtain the name (absoluto!) file to be processed.

7.2.8. log_location_filenameSpecific the log name (absoluto) file to process.

7.3. Parametros específicos de la regexp

7.3.1. log_regexp_begin y log_regexp_endSet marks of the beginning and end of a regular expression definition for the definition of the log

file in which they are.

7.3.2. log_regexp_ruleDefine the regular expression. NOTE: do not use markers / / Directly the extended regular

expression (Perl type). Examples:

File\sdoes\snot\sexist → Find “File does not exist”

[0-9]*\serrores → Find strings “043 errores”

7.3.3. log_regexp_severityIt sent in the XML a sternness, can be WARNING, CRITICAL or NORMAL (in capital letters). Is

optional.

7.3.4. log_regexp_messageText that was sending to find at least one occurrence (if it located several only send a message).

You can use the switches $ 1 .. $ 2 for fields previously identified with a regular expression to do

search field syntax ()→

7.3.5. log_regexp_actionCommand that executes to find at least one occurrence (if it located several run only once).

When defining a log can define several blocks of regular expressions. Each regular expression block may

also have several regular expressions. In the case of multiple matches, it will count each occurrence, but

only send a message or run an action. Should be defined several, will run to make the final "match".

pandora fms: advanced log parser

Technology