centralized logging with€¦ · 1. centralized logging project specification 1.1.introduction...

Centralized Logging with SYSLOG-NG

Abstract : Describes the concepts of using SYSLOG-NG in the enterprise.

Issue : 15

Date : 11/08/2007

Document History

Issue Author(s) Date Description Change

1 N Metrowsky 06/01/2006 Document Creation

2 N Metrowsky 06/07/2006 Added project plan to document and updated reference section

3 N Metrowsky 06/09/2006 Updated diagram and added a section on syslog log types and levels

4 N Metrowsky 06/26/2006 Added rc script for syslog2mysql. Minor updates to PHP-SYSLOG-NG screens. Added installation instructions for the Experimental Environment.

5 N Metrowsky 07/13/2006 Added more details surrounding the PHP-SYSLOG-NG installation. Revised installation for PHP-SYSLOG-NG V2.9.2.

6 N Metrowsky 07/31/2006 Added section on creating a SYSLOG-NG RPM and EPM kit.

7 N Metrowsky 08/02/2006 Added comments to section 1.3, as “Author’s Notes”. Also, added applications to section 1.3. Added information on Apache logging to SYSLOG-NG.

8 N Metrowsky 09/14/2006 Added information about IRIX issues discovered in testing.

9 N Metrowsky 09/20/2006 Added information about building SYSLOG-NG, creating kits via EPM and installing SYSLOG-NG via the EPM operating system specific kits. Some chapter reorganization.

10 N Metrowsky 09/25/2006 Updated information on PHP-SYSLOG-NG to reflect the new release V2.9.2a r13). Also added changes to reflect the new location for mysql.pipe from /var/adm to /var/run. Updated /etc/logrotate.d/syslog-ng, etc/init.d/syslog-ng, /usr/local/sbin/syslog2mysql.sh and /etc/init.d/syslog2mysql files accordingly, as well. Reorganized document to make it more of a technical user’s guide and less of a project specification. Added documentation for describing and maintaining syslog-ng.conf.

11 N Metrowsky 10/12/2006 Updated Centralized Logging System Flow to reflect a redundant environment.

12 N Metrowsky 08/02/2007 Updated syslog-ng.conf section to depict logging to Zenoss and to a Central/Relay host. Added additional notes to the syslog-ng.conf file to aid the reader and installer. Updated chapter 1 to include information on Zenoss’ role with the Centralized Logging environment.

13 N Metrowsky 09/05/2007 Updated section on EPM builds. Due to a change in how /usr/local is used in the environment, the startup files needed to be split off from the executables.

14 N Metrowsky 09/28/2007 Major document cleanup and reorganization.

15 N Metrowsky 11/08/2007 Added new /var/log/syslog-ng file to the various configuration files. Updated auth, auth.info, kern and kern.info levels. Fixed indentation on level 2 headers.

2

Table of Contents

1. Centralized Logging Project Specification .................................................................... 6

1.1. Introduction ....................................................................................................................... 6 1.1.1. Scope of this document ................................................................................................................. 6 1.1.2. Organization of this document ...................................................................................................... 6 1.1.3. What is SYSLOG-NG ................................................................................................................... 6 1.1.4. Using SYSLOG-NG with a database ............................................................................................ 6

1.2. DigitalGlobe Conceptual Deployment .............................................................................. 8 1.2.1. DigitalGlobe Deployment Conceptual Description ...................................................................... 9 1.2.2. Systems Management Project Notes ............................................................................................. 9 1.2.3. Initial Production Environment .................................................................................................. 10 1.2.4. Specifications for the SYSLOG-NG Central Server ................................................................... 11

1.3. Implementation Comments ............................................................................................. 12 1.3.1. Introduction ................................................................................................................................. 12 1.3.2. What would software installation entail? .................................................................................... 12 1.3.3. What to log centrally? ................................................................................................................. 12 1.3.4. How to report and notify on events? ........................................................................................... 13 1.3.5. What about local logs and log rotation? ...................................................................................... 13 1.3.6. What kind of server would be needed? ....................................................................................... 14 1.3.7. Should MS Windows be addressed? ........................................................................................... 14 1.3.8. Should Network Appliance and SAN Frames be addressed? ..................................................... 14 1.3.9. How should applications be handled? ......................................................................................... 15

1.4. Centralized Logging Project Plan ................................................................................... 16 1.4.1. Introduction ................................................................................................................................. 16

1.5. Conclusion ........................................................................................................................ 16

1.6. References ........................................................................................................................ 16

2. SYSLOG-NG Central Server Installation .................................................................... 17

2.1. Introduction ..................................................................................................................... 17

2.2. Prerequisite Software ...................................................................................................... 17

2.3. Install SYSLOG-NG ........................................................................................................ 17

2.4. Install PHP ....................................................................................................................... 18

2.5. Install PHP-SYSLOG-NG ............................................................................................... 18 2.5.1. MySQL Database Schema for SYSLOG-NG ............................................................................. 19

2.6. Completing the PHP-SYSLOG-NG Installation ........................................................... 21

3. SYSLOG-NG Client Installation .................................................................................. 22

3.1. Introduction ..................................................................................................................... 22

3.2. Installing on Linux ........................................................................................................... 22

3.3. Installing on IRIX ............................................................................................................ 23

3.4. Installing on Solaris 9 ...................................................................................................... 23

3.5. Installing on Solaris 10 .................................................................................................... 23

3

4. SYSLOG-NG Configuration Files ............................................................................... 25

4.1. Introduction ..................................................................................................................... 25

4.2. SYSLOG-NG Configuration File - Global Options ....................................................... 25

4.3. SYSLOG-NG Configuration File – Log Sources ........................................................... 26

4.4. SYSLOG-NG Configuration File – Log Destinations ................................................... 28

4.5. SYSLOG-NG Configuration File – Filters ..................................................................... 33 4.5.1. SYSLOG-NG Configuration File – Logging & Filtering ........................................................... 36

4.6. SYSLOG-NG Configuration File - syslog-ng.conf ......................................................... 37

5. SYSLOG-NG Log Rotation .......................................................................................... 43

5.1. Introduction ..................................................................................................................... 43

5.2. Log Rotation on Linux - /etc/logrotate.d/syslog-ng ....................................................... 43

5.3. Log Rotation on Solaris 9 & Solaris 10 - /etc/logadm.conf ........................................... 44

5.4. Log Rotation on IRIX - /usr/local/bin/rotatelog ............................................................ 45

5.5. Log Rotation for the SYSLOG-NG database - logrotate.php ....................................... 47

6. SYSLOG-NG Support Files .......................................................................................... 49

6.1. Introduction ..................................................................................................................... 49

6.2. Starting SYSLOG-NG and support software ................................................................ 49 6.2.1. Introduction ................................................................................................................................. 49 6.2.2. SYSLOG-NG to MySQL Communication – syslog2mysql.sh .................................................. 49 6.2.3. SYSLOG-NG to MYSQL Communication rc script - syslog2mysql ......................................... 49 6.2.4. SYSLOG-NG Startup rc script - syslog-ng ................................................................................. 50

7. Enhancements and Modifications ............................................................................... 53

7.1. Modifications to PHP-SYSLOG-NG .............................................................................. 53 7.1.1. Introduction ................................................................................................................................. 53 7.1.2. /usr/local/apache2a/phpsyslogng/index.php ................................................................................ 53 7.1.3. /usr/local/apache2a/phpsyslogng/includes/search.php ................................................................ 53 7.1.4. /usr/local/apache2a/phpsyslogng/includes/tailresult.php ............................................................ 54 7.1.5. /usr/local/apache2a/phpsyslogng/includes/regularresult.php ...................................................... 54 7.1.6. /usr/local/apache2a/phpsyslogng/includes/jpgraph/jpg-config.inc ............................................. 54

7.2. Directing Apache Logs to SYSLOG-NG ........................................................................ 55 7.2.1. Introduction ................................................................................................................................. 55 7.2.2. Installing the Apache httpd to SYSLOG-NG Filter .................................................................... 55

7.3. IRIX patch for SYSLOG-NG 1.6.12 ............................................................................... 56 7.3.1. Introduction ................................................................................................................................. 56 7.3.2. Implementation ........................................................................................................................... 56

8. Building a SYSLOG-NG kit using EPM .................................................................... 57

8.1. Introduction ..................................................................................................................... 57

8.2. Preparing for an EPM Build ........................................................................................... 57

8.3. Configuring and Building SYSLOG-NG on Linux and IRIX ...................................... 58

8.4. Performing a Linux SYSLOG-NG EPM kit Build ........................................................ 58

4

8.5. Performing an IRIX SYSLOG-NG EPM kit Build ....................................................... 58

8.6. Configuring and Building SYSLOG-NG on Solaris ...................................................... 59

8.7. Performing a Solaris 9 SYSLOG-NG EPM kit Build ................................................... 60

8.8. Performing a Solaris 10 Sparc SYSLOG-NG EPM kit Build ....................................... 60

8.9. Performing a Solaris 10 Intel SYSLOG-NG EPM kit Build ........................................ 60

8.10. Linux syslogng.list file for EPM .................................................................................... 61

8.11. IRIX syslogng.list file for EPM ..................................................................................... 62

8.12. IRIX syslogngstartup.list file for EPM ......................................................................... 63

8.13. Solaris 9 syslogng.list file for EPM ............................................................................... 63

8.14. Solaris 9 syslogngstartup.list file for EPM ................................................................... 64

8.15. Solaris 10 syslogng.list file for EPM ............................................................................. 64

8.16. Solaris 10 syslogngstartup.list file for EPM ................................................................. 65

9. Appendix - Syslog Facility Types and Log Levels ...................................................... 67

9.1. DigitalGlobe SYSLOG-NG Logging by Facility ............................................................ 69

10. Appendix - Introduction to PHP-SYSLOG-NG ........................................................ 70

10.1. Introduction ................................................................................................................... 70

10.2. Sample Search Screenshot from PHP-SYSLOG-NG .................................................. 71

10.3. Sample Results Screenshot from PHP-SYSLOG-NG ................................................. 72

10.4. Sample Tail Results Screenshot from PHP-SYSLOG-NG ......................................... 73

10.5. Sample Graph Screenshot from PHP-SYSLOG-NG .................................................. 74

10.6. Sample Facility Summary Screenshot from PHP-SYSLOG-NG ............................... 75

10.7. Sample Program Summary Screenshot from PHP-SYSLOG-NG ............................. 76

......................................................................................................................................... 76

5

1. Centralized Logging Project Specification

1.1. Introduction

1.1.1. Scope of this document

This document is meant to provide a detailed description of performing Centralized Logging in the DigitalGlobe Enterprise. The contents contained herein provide a conceptual view in regards to setting up, installing and using SYSLOG-NG.

1.1.2. Organization of this document

This document is organized as follows:

Chapter(s) Description1 Overview of the Centralized Logging Project and concepts (this chapter)2 Installation Instructions for the SYSLOG-NG Central Server3 Installation instructions for the SYSLOG-NG Clients

4 - 7 Reference Material related to SYSLOG-NG. Includes information on SYSLOG-NG configuration files, log rotation, system startup files and code enhancements.

8 Using EPM to package SYSLOG-NG9 Overview of SYSLOG-NG Logging, that is, what information will be logged to

the SYSLOG-NG Centralized Logging Server,10 PHP-SYSLOG-NG Introduction

1.1.3. What is SYSLOG-NG

SYSLOG-NG is a replacement for the standard syslog utility, which is the default system logging tool on UNIX and Linux. Unlike syslog, SYSLOG-NG has the capability of using TCP to transmit log information, as opposed to the less reliable UDP protocol utilized by syslog. Also, SYSLOG-NG provides a much improved filtering mechanism for handling information being transmitted to system logs. It also has the capability of sending log information to secondary hosts which in turn can transmit data to a centralized host while retaining the original host of origin on the log entries. While syslog can also use secondary hosts for logging purposes, it does not have the capability to retain the log entry’s host of origin.

1.1.4. Using SYSLOG-NG with a database

In addition to more robustness, SYSLOG-NG has the capability to transmit data to a database in addition to flat log files. This capability allows for a more efficient method of correlating log events from multiple hosts, in order to get a clearer picture of events related to a log entry. For example, a host goes down because it lost connection to a disk; this is because the host serving the disk also went down. Instead of one having to go to multiple systems to determine the order of events, these events would be searchable within a database. Also, if the hosts had time to send out a log event, the events from the multiple hosts, would all show up as near consecutive entries in the database during a specific time period. One could use SQL to request all the events that occurred on these hosts, during a particular time period, and SQL would return the events in the order in which they occurred.

6

In addition to using a database to do forensics on an event, the database could also be used by monitoring software to use SQL to select records during a specific time period and send out alerts based upon the messages that SQL extracts. This method eliminates the need for using utilities like grep, awk and diff to examine flat log files to obtain the same information.

Finally, a database presents the opportunity for Web Based Applications to view and display system log information, via SQL calls. Because the database entries contain a date/time stamp, host name, log level and a description of the event, a web based program can be written, or obtained via the Internet, to provide not only forensic information, but also provide metric information.

READERS NOTE: The terms: Central Server, Secondary Server, Relay Server, Database Server imply that these hosts are redundant or clustered hosts. The Centralized Logging/System Monitoring infrastructure is designed to eliminate any single points of failure and to provide the maximum uptime possible.

7

1.2. DigitalGlobe Conceptual Deployment

8

1.2.1. DigitalGlobe Deployment Conceptual Description

The diagram on the previous page provides a network overview for deployment of SYSLOG-NG in the DigitalGlobe environment. As described earlier, SYSLOG-NG has the capability of using SYSLOG-NG Secondary/Relay Servers to gather system log information, which than can be transmitted to a SYSLOG-NG Central Server Also, the capability of SYSLOG-NG to retain host of origin information throughout the transmission process makes SYSLOG-NG an ideal option for centralized logging. The SYSLOG-NG Central and SYSLOG-NG Secondary/Relay Servers should be redundant/clustered hosts, in order to provide 99.9999% uptime capability.

The SYSLOG-NG Secondary/Relay Servers will not only relay log information, but will also store flat log file data for a short duration. This is to account for the possibility of a network outage which would temporarily preclude data transmittal to the SYSLOG-NG Central Server.

Reporting of important server, network and storage events will be handled by Zenoss, the system monitoring software which is being implemented as part of the System Monitoring project.

Finally, the diagram contains some notes which provide some insight on deploying SYSLOG-NG in the DigitalGlobe network.

1.2.2. Systems Management Project Notes

1. The System Monitoring Server (Zenoss) will most likely get data from the Centralized Logging database. E-mail and other alerts based on log data will go out from Systems Monitoring after correlation, de-duplication, and other processing.

2. We will want the log messages to go through a local configuration and de-duplication process before the messages are sent to the SYSLOG-NG Central Server or MCC SYSLOG-NG Central Server.

9

1.2.3. Initial Production Environment

The initial production environment was created on clgold-a.digitalglobe.com to have a running version of SYSLOG-NG with MySQL and PHP-SYSLOG-NG. The production environment will allow members of the project team to view and experiment with SYSLOG-NG without affecting the existing DigitalGlobe enterprise. At the time this documents was last updated, many of the Information System servers and several Network Switches, Routers and Firwall systems are being monitored. The SYSLOG-NG environment is set up to run on TCP Port 5140, as part of the data gathering process.

Pease see the Introduction to PHP-SYSLOG-NG chapter for the sample screen shots of PHP-SYSLOG-NG. The screen shots were created from the experimental deployment on brutus.digitalglobe.com.

The PHP-SYSLOG-NG environment is available at: http://clgold.digitalglobe.com/phpsyslogng. The user id to use is admin and the password is also admin.

To review the environment, the following are a list of software installation locations.

The logrotate.php file is located in /etc/cron.daily.The syslog-ng.conf file is located in /usr/local/etc/sysog-ngThe SYSLOG-NG software was installed in /usr/localThe syslog2mysql.sh script is installed in /usr/local/sbinThe syslog2mysql startup script is located in /etc/init.dThe PHP-SYSLOG-NG startup script is located in /etc/init.d The PHP-SYSLOG-NG software is located in /var/www/html/phpsyslogng

A set of configuration scripts from clgold-a.digitalglobe.com are provided in the SYSLOG_NG Support Files chapter for reference purposes.

In addition to the hardware specification on the next page, clgold-a.digitalglobe.com has been allocated 550 Gb on the SAN (mount point /dev/mapper/cl_vol-log_lvol) for the purpose of storing SYSLOG-NG log files and MySQL database.

10

http://clgold.digitalglobe.com/phpsyslogng

1.2.4. Specifications for the SYSLOG-NG Central Server

At present, the hardware utilized for the SYSLOG-NG consist of two hosts each in DigitalGlobe Internal Network and Mission Control Center Network. The hosts will employ a hard failover method in the even the primary Central Server is taken out of service. Future plans call for clustering the hosts for higher availability. The operating system implemented is RedHat Enterpise Linux V4.4.

The following table describes each of the SYSLOG-NG Central Servers:

Base Unit: Dual Core 2222SE Processor 2X1MB Cache, 3.0GHz Opteron 1Ghz HyperTransport for PowerEdge 2970 (222-7941)

Processor: Dual Core Opteron 2222SE 2nd Processor, 2x1MB Cache, 3.0GHz1Ghz HyperTransport for PE 2970 (311-7153)

Memory: 4GB Memory, 4x1GB, 667MHz, Single Ranked DIMMs (311-6421)

Video Card: Broadcom Dual Port TCP/IP Offload Engine Key Not Enabled (430-1748)

Hard Drive: 73GB, SAS, 2.5-inch, 10K RPM Hard Drive (341-3055)

Hard Drive Controller:

PERC 5/i, x8 Backplane Integrated Controller Card (341-3067)

Floppy Disk Drive: No Floppy Drive w/Filler Panel (341-3078)

Operating System: No Operating System (420-6320)

Mouse: Mouse Option None (310-0024)

NIC: Embedded Broadcom NetXtreme II5708 GigabitEthernet NIC (430-1764)

CD-ROM or DVD-ROM Drive:

24X IDE CD-ROM for PowerEdge 2970 (313-5171)

Sound Card: Active ID Bezel (313-4946)

Speakers: 1x8 Backplane for 2.5-inch Hard Drives (311-7155)

Documentation Diskette:

Electronic Documentation and OpenManage CD Kit, PE2970 (310-8776)

Additional Storage Products:

73GB, SAS, 2.5-inch, 10K RPM Hard Drive (341-3055)

Feature Integrated SAS/SATA RAID 1, PERC 5/i Integrated (341-4576)

Feature Rack Chassis w/Sliding Rapid/Versa Rails and Cable Management Arm,Universal (310-7412)

Service: Dell Hardware Warranty Plus Onsite Service Initial Year (986-5867)

Service: Basic Enterprise Support: Business Hours (5X10) Next Business Day Onsite Service After Problem Diagnosis Initial Ye (982-5830)

Service: Dell Hardware Warranty, Extended Year (986-5868)

Service: Basic Enterprise Support: Business Hours (5X10) Next Business Day Onsite Service After Problem Diagnosis 2 Year Ext (982-2732)

Installation: On-Site Installation Declined (900-9997)

Misc: Redundant Power Supply with Dual Cords for PowerEdge 2950 (310-7422)

Misc: Power Cord, NEMA 5-15P to C14,15 amp, wall plug, 10 feet / 3 meter (310-8509)

Misc: Intel PRO 1000PT Cu, Single Port, PCIe NIC (430-0955)

11

1.3. Implementation Comments

1.3.1. Introduction

The discussion so far has dealt with a conceptual design of implementing SYSLOG-NG in the DigitalGlobe environment. This section will describe some of the possible issues and discussion items for going forward with the project. As mentioned earlier, this is not an exhaustive document, so additional items will be added by the project team as the project moves from the conceptual to the design stage. The remainder of this chapter is organized by topics in no particular order of importance.

1.3.2. What would software installation entail?

There are two possible scenarios for Centralized Logging deployment.

First, since SYSLOG-NG is not standard in Solaris, Linux or IRIX, so it will need to be installed at least on the SYSLOG-NG Central and Secondary/Relay Servers. The default syslog utility has the capability to send log information to a central host within the same network subnet and have the central host retain the origin of the log entry. However, SYSLOG-NG will be required to transfer log entries from to the SYSLOG-NG Central Server or the origin of a log entry’s host name will be lost (syslog will change the origin of the log entry to that of the SYSLOG-NG Secondary/Relay Server before sending the log entry on the SYSLOG-NG Central Server). Under this methodology, the filtering capabilities will be handled by SYSLOG-NG Central and Secondary/Relay Servers. However, each host participating in Centralized Logging will have to have their syslog.conf file modified to send log entries to the appropriate SYSLOG-NG server and log entries will be sent via UDP Port 514 throughout the entire enterprise.

The second, and preferred, methodology would be to replace syslog with SYSLOG-NG on all hosts participating in Centralized Logging. In this methodology, all hosts would be logging to TCP Port 5140. Also, the filtering of log entries can take place at the host level, as opposed to server level. As centralized logging can be used for tracking events, it also can be used for security forensics. It would be a sensible idea to replace the standard syslog on all DigitalGlobe hosts with SYSLOG-NG and transmit log data to the SYLOG-NG Central Server. The main issue with this suggestion is the amount of time and resources required for installing SYSLOG-NG within the enterprise; at the time this document was written, this would entail over 400 hosts.

Authors Note: It was decided by the Centralized Logging Project Team to use the methodology described in the second paragraph above.

Additional Note: Since the MCC is a separate network, it requires a separate Centralized Logging environment. The MCC will be set up identical to the DG Internal Network; therefore, the concepts presented in this section are applicable to the MCC environment as well.

1.3.3. What to log centrally?

Once SYSLOG-NG is implemented in the enterprise, the next issue is what to send to the SYSLOG-NG Central Server. Going with the premise “too much data is just as bad as having no data at all”, the project team needs to examine the various logging categories and “pick and choose’ the appropriate logging levels for each logging facility. For example, for auth log entries which deal with logins, it may be plausible to capture every log entry from a security forensics standpoint. However, in regards to mail log entries, it may be plausible to only capture log entries which indicate errors. Some of the log facilities available are: auth, authpriv, cron, daemon, ftp,

12

kern, lpr, mail, news, user, and uucp. Also, locally defined log facilities are available using the designation local0 – local7.

In addition to log facilities, the project team needs to decide upon the log levels that would be most important to capture. As log entries have the levels: debug, info, notice, warning, err, crit, alert and emerg, a matrix would need to be created which shows the various log facilities and log level thresholds which will be logged centrally.

Author’s Note: Syslog Facility Types and Log Levels, an Appendix to this document, goes into detail on the proposed standards for centralized logging via SYSLOG-NG. Please refer to the Syslog Facility Types and Log Levels table for a pictorial view of what information will be logged to the SYSLOG-NG Central Server.

1.3.4. How to report and notify on events?

Once it is determined what needs to be logged, what log thresholds are going to be used for the basis of logging, and what hosts will be doing the logging, the next area to consider is how information being sent to the SYSLOG-NG Central Server would be disseminated to the appropriate support area for action.

The PHP-SYSLOG-NG tool only displays information logged to the database and it does not send out alerts; though, the color coding for log level enables the user to zero in on a particular issue. Also, SYSLOG-NG is designed to note an event and to send the event information to the SYSLOG-NG Central Server and to the database/flat log files. There is no support area categorization capability in SYSLOG-NG.

Since this document was originally written, a tool has been chosen from System and log Monitoring; this new product is called Zenoss. Additional information about Zenoss is provided as part of the System Monitoring project and is available at: http://www.zenoss.com. Zenoss will be used to categorize and report on events generated via the Centralzied logging mechanism.

1.3.5. What about local logs and log rotation?

SYSLOG-NG has the capability, like syslog, to log events to the local host and to a SYSLOG-NG Central Server. With the addition of a database, it may make sense to limit or eliminate logging on local hosts; providing that the database resides in a redundant environment and that there are multiple paths to access the database. If the “database solution” were chosen, then a “cluster of servers” would need to be employed for the SYSLOG-NG Central Server for availability purposes. Data logged at the local server level will also be sent to the SYSLOG-NG Central server, but the retention time will be limited because of the “database solution”.

Author’s Note 1: It was decided by the project team to retain log files on local server to aid in server forensics. It was felt by the project team that log information should not only be available at a solitary source, the SYSLOG-NG Central Sever.

Author’s Note 2: It was decided by the project team that the log rotation schedule will be 5 days to start at the local host level, two days at the SYSLOG-NG Secondary/Relay Server level and 30 days at the SYSLOG-NG Central Server level. As for System Monitoring, the System Monitoring software will get notifications from the SYSLOG-NG Central Server as its primary source. If the SYSLOG-NG Central Server is unavailable, than the System Monitoring software will poll the appropriate SYSLOG-NG Secondary/Relay Server. Finally, local hosts will be polled, if the SYSLOG-NG Central Server and SYSLOG-NG Secondary/Relay Servers are unavailable.

13

http://www.zenoss.com/

The experimental environment on brutus.digitalglobe.com has in place the logrotate.php script. This script is set up to run on a daily basis, and, as described earlier in this document, rotates the tables in the database. In addition, /etc/logrotate.d/syslog-ng has been implemented to rotate the disk based log files.

Finally, log rotation will be performed by the application specific to the Linux, Solaris or IRIX operating system, as there is not a tool which runs effectively in all three operating system environments. Please see Chapter 4: SYSLOG-NG Log Rotation for more information.

1.3.6. What kind of server would be needed?

With over 400 hosts sending log information to the SYSLOG-NG, either directly or indirectly, it would be logical to theorize that the SYSLOG-NG Central Server would contain several network interface cards to accommodate the amount of traffic one would expect from the number of hosts reporting. Also, as mentioned earlier, from a reliability standpoint the SYSLOG-NG Central Server should be a “cluster of servers” to handle not only the network traffic, but also allow for maximum availability within the enterprise, i.e. 99.9999% uptime. Finally, the database as well as the flat log files should reside on a disk structure that maximizes I/O throughput due to the large amount of data which will be transmitted.

Centralized logging is an I/O and network intensive application, so the correct network, server and I/O configuration should be able to support the requirements

Additional Note: The MCC requires a separate Centralized Logging environment. The MCC will be set up identical to the DG Internal Network; therefore, the concepts presented in this section are applicable to the MCC environment as well.

Additional Note 2: Instead of a cluster, it was decided to employ a “hot spare” setup. As the Centralized Logging data will be on a Storage Area Network or Network Attached Storage, it was more cost effective to have hardware available that could be put into service in the event the primary Central Sever becomes unavailable.

1.3.7. Should MS Windows be addressed?

The focus area of this document concentrates on a centralized logging solution for the UNIX and Linux environment. This document does not address a similar solution for MS Windows servers. However, in the SAN Document Case Study: Implementing a Centralized Logging Environment, Pages 4 - 7, provides details on how to route MS Windows log messages to a SYSLOG-NG Central Server using evtsys. A copy of the document is provided in the Internet Resources folder in the Centralized Logging Project area.

Author’s Note: After a lengthy discussion on this subject, the project team believed that the MS Windows environment should be handled by the MS Windows MOM utility. It is believed that MOM would provide a better logging/alarm environment to the System Monitoring environment, than having the log information being directed from MS Windows to SYSLOG-NG. In addition, Zenoss provides a mechanism to monitor MS Windows Servers, which eliminates the need to include MS Windows in the Centralized Logging project.

1.3.8. Should Network Appliance and SAN Frames be addressed?

Log data will be sent from both environments to SYSLOG-NG. Also, the log facility, using one of the locally defined facilities, will be customized to support the appropriate storage device. The DigitalGlobe SYSLOG-NG Logging by Facility section provides a table of local facilities.

14

1.3.9. How should applications be handled?

Applications vary in the way they have implemented logging. Some applications can log to SYSLOG-NG, some can log to a proprietary log format, some can log to both, and some do not log at all. Due to the complex nature of applications, the project team must address which applications are worthy of being monitored. The team must also address which applications can utilize SYSLOG-NG and which applications can report out of the Centralized Logging infrastructure, either natively or via an external utility, to the System Monitoring environment.

Author’s Note 1: It was decided by the project team to address applications as a second phase of this project. Applications include software like databases, web servers and general purpose applications.

Author’s Note 2: This section was added for completeness on 2 August, 2006.

Author’s Note 3: It was decided by the project team to utilize Oracle Grid Control for the purpose of monitoring and reporting on databases. As with MOM on MS Windows, Oracle Grid Control provides more robustness than a SYSLOG-NG solution.

Author’s Note 4: Apache httpd provides the capability of logging to both SYSLOG-NG (via an external utility called snaretext) and to the native logs. It also should be noted that Apache httpd cannot log to SYSLOG-NG and to its native logs, without the use of snaretext. In order to implement Apache httpd syslog capability, it will require setting up filtering rules within the SYSLOG-NG configuration file.

15

1.4. Centralized Logging Project Plan

1.4.1. Introduction

In order to minimize duplication, he Centralized Logging Project Plan is located in the following directory:

\\Cofs1\fs1\BU\OPS\IS\Projects\Centralized Logging

Please refer to the above link for additional information.

1.5. Conclusion

This document presented a conception for implementing a Centralized Logging environment at DigitalGlobe. The focus of this document was present a higher level design employing SYSLOG-NG as the main logging tool in the UNIX/Linux enterprise. In addition, the concept of using a SYSLOG-NG Central Server with a MySQL database was also discussed. An experimental environment was introduced to allow project team members an opportunity to review a running environment. Finally, some questions and issues were provided in order to have the project team think about the proper methodology to implement Centralized Logging at DigitalGlobe.

1.6. References

The following resources were used in the construction of this document:

1. Building a Logging Infrastructure, SAGE Short Topics in System Administration 12, Abe Singer and Tina Bird.

2. Case Study: Implementing a Centralized Logging Environment, Richard L. DuClos, 8/6/2003.

3. Centralized Logging: Logging into a Centralized SQL Database, Adam Tauno Williams, 2006

4. HOWTO setup PHP-SYSLOG-NG (http://gemtoowiki.com/HOWTO_setup_PHP_SYSLOG_NG), Author Unknown, 05/2006.

5. http://code.google.com/p/php-syslog-ng , php-syslog-ng Project6. http://www.balabit.com/products/syslog-ng/ , SYSLOG-NG Project7. https://engineering.purdue.edu/ECN/Resources/Documents/UNIX/evtsys , evtsys Project8. http://loganalysis.org , Internet site devoted to the various aspects of Log File Analysis.9. Software Distribution Using the ESP Package Manager, Michael R. Sweet.

In addition, the case studies and additional documentation are available at:

\\Cofs1\fs1\BU\OPS\IS\Projects\Centralized Logging\Internet Resources

Some of these documents were reviewed and some concepts they presented were incorporated into this document.

16

http://loganalysis.org/

https://engineering.purdue.edu/ECN/Resources/Documents/UNIX/evtsys

http://www.balabit.com/products/syslog-ng/

http://code.google.com/p/php-syslog-ng

http://gemtoowiki.com/HOWTO_setup_PHP_SYSLOG_NG

2. SYSLOG-NG Central Server Installation

2.1. Introduction

This section will provide information on how install a SYSLOG-NG Central Server environment. The purpose of this section is to form the basis of installation documentation and to provide the reader information on the installation procedure.

2.2. Prerequisite Software

The following software products are required for a successful installation of the SYSLOG-NG Central Sever:

SYSLOG-NG – Available at http://www.habitat.com/products/syslog_ngMySQL – Available at http://www.mysql.orgPHP – Avalaible at http://www.php.netPHP-SYSLOG-NG – Available at http://php-syslog-ng.gdd.net/current.tgzzlib – Available at http://www.zlib.net.GD - Available at http://www.boutell.com/gd.OpenSSL – Available at http://www.openssl.org.Freetype – Available at http://freetype.sourceforge.net.msttcorefonts – Available at http://corefonts.sourceforge.net.

Note: gd, freetype and msttcorefonts are required for the PHP-SYSLOG-NG graphing capability to function. PHP requires OpenSSL and zlib. PHP-SYSLOG-NG requires MySQL.

NOTICE: With the exception of SYSLOG-NG and PHP-SYSLOG-NG, all of the above products are also available in RPM format as part of the RedHat Enterprise Operating System Distribution. Because the target SYSLOG-NG server is running RedHat Enterprise Linux 4, the reader can save a great deal of time by installing the RPM files instead.

2.3. Install SYSLOG-NG

To build SYSLOG-NG issue the following commands:

$ tar -xzf syslog-ng-1.6.12.tar.gz$ cd syslog-ng-1.6.12$ ./configure –-prefix=/usr/local –enable-full-static$ make$ make install

Note: The default installation will install SYSLOG-NG in the /usr/local directory structure.Note 2: syslog-ng-1.6.12 was the final release of the version 1 series of SYSLOG-NG. The final release addressed issues with AIX and a few minor bug fixes. Based upon the release notes, there were no security issues or bugs which would cause issues with implementing SYSLOG-NG 1.6.12 as the version of choice.

Please see SYSLOG-NG Configuration File - syslog-ng.conf for a sample syslog-ng.conf file. This file should be installed in /usr/local/etc/syslog-ng.

Please see SYSLOG-NG Startup rc script - syslog-ng for a sample syslog-ng rc file. This file should be installed in /etc/init.d.

17

http://corefonts.sourceforge.net/

http://freetype.sourceforge.net/

http://www.openssl.org/

http://www.boutell.com/gd

http://www.zlib.net/

http://php-syslog-ng.gdd.net/current.tgz

http://www.php.net/

http://www.mysql.org/

http://www.habitat.com/products/syslog_ng

Please see Log Rotation on Linux - /etc/logrotate.d/syslog-ng. This file should be copied to /etc/logrotate.d as syslog-ng.

2.4. Install PHP

In order for PHP-SYSLOG-NG to function, it requires that PHP be installed.

To Install PHP do the following:

$ bnunzip2 –xzf php-4.4.2.bz2$ tar –xf php-4.4.2.tar$ cd php-4.4.2

PHP should be configured as follows:

$ ./configure --with-apxs2=/usr/local/apache2/bin/apxs \--with-mysql=/usr/local/mysql --with-zlib --with-openssl --with-gd \--with-snmp --with-ttf --with-freetype-dir=/usr/local$ make$ make install

Notice: This document assumes that a previous installation of PHP was installed. If not, the installer will need to install the php.ini-recommended file (in the top level PHP installation directory) into /usr/local/lib:

$ cp php.ini-recommended /usr/local/lib/php.ini

Finally, to activate PHP, do the following:

$ /usr/local/apache2/bin/apachectl stop$ /usr/local/apache2/bin/apachectl start

2.5. Install PHP-SYSLOG-NG

To install PHP-SYSLOG-NG issue the following commands:

$ mv current.tgz phpsyslogng-2.9.2.tgz$ tar -xzf phpsyslogng-2.9.2.tar.gz$ mv html /usr/local/apache2/htdocs/phpsyslogng$ chown –R apache:apache /usr/local/apache2/htdocs/phpsyslogng$ cd /usr/local/apache2/htdocs/PHP-SYSLOG-NG/configs

New with version 2.9.2 is a GUI front end which will aid in the proper installation of PHP-SYSLOG-NG. To use the GUI interface type: http://brutus/phpsyslogng/install.

NOTICE: PHP-SYSLOG-NG requires additional software to support the new graphing capability. It also requires that PHP be built in the proper manner. Please see SYSLOG-NG Central Server Installation for this information.

If the installer does not desire to use the GUI interface, follow the procedure outlined as follows:

Make the appropriate modifications to config.php. This consists of passwords required to access the MySQL database used by PHP-SYSLOG-NG.

Next, do the following:

18

http://brutus/phpsyslogng/install

$ cd /usr/local/apache2/htdocs/phpsyslogng/scripts

Make the appropriate modifications to dbsetup.sql. This consists of modifying the passwords to the MySQL database used by PHP-SYSLOG-NG.

NOTICE: Make sure the passwords for MySQL match in config.php and dbsetup.sql.

Issue the following command to create the database for PHP-SYSLOG-NG:

$ /usr/local/mysql/bin/mysql –u root < dbsetup.sql

Next, move the syslog2mysql.sh script into production. This script interfaces SYSLOG-NG with MySQL:

$ cp /usr/local/apache2/htdocs/phpsyslogng/scripts/syslog2mysql.sh /usr/local/sbin/.

Next, move the logrotate.php script into production. This script rotates MySQL database tables used by PHP-SYSLOG-NG:

$ cp /usr/local/apache2/htdocs/phpsyslogng/scripts/logrotate.php /etc/cron.daily/.

Next, please see SYSLOG-NG to MYSQL Communication rc script - syslog2mysqlfor a sample syslog2mysql rc file. This file should be installed in /etc/init.d.

Next, move the syslog.conf script into production. This file should be appended to the end of the existing syslog-ng.conf file located in /usr/local/etc/syslog-ng.Note: The sample syslog-ng.conf file, included in this document, already has this change.

See /usr/local/apache2/htdocs/phpsyslogng/scripts/syslog.conf.

2.5.1. MySQL Database Schema for SYSLOG-NG

The following specification, which is from the PHP-SYSLOG-NG distribution, will set up SYSLOG-NG MySQL database, user accounts, etc. This following specification is run as part of the SYSLOG-NG installation process.

# Make sure you edit the passwords of the three database users!# Run it like this:# shell> mysql -uroot -p < dbsetup.sql#

CREATE DATABASE syslog;

USE syslog;

# create table logs under database syslogCREATE TABLE logs (host varchar(128) default NULL,facility varchar(10) default NULL,priority varchar(10) default NULL,level varchar(10) default NULL,tag varchar(10) default NULL,datetime datetime default NULL,program varchar(15) default NULL,msg text,seq bigint(20) unsigned NOT NULL auto_increment,PRIMARY KEY (seq), KEY host (host),KEY program (program), KEY datetime (datetime),KEY priority (priority), KEY facility (facility)) TYPE=MyISAM;

19

# create table users under database syslogCREATE TABLE users (username varchar(32) default NULL,pwhash char(40) default NULL,sessionid char(32) default NULL,exptime datetime default NULL,PRIMARY KEY (username)) TYPE=MyISAM;

# Create the table for the cache functionCREATE TABLE search_cache (tablename varchar(32) DEFAULT NULL,type ENUM('HOST','FACILITY'),value varchar(128) DEFAULT NULL,updatetime datetime DEFAULT NULL,INDEX type_name (type, tablename)) TYPE=MyISAM;

# Create the two tables used by the access control functionCREATE TABLE user_access (username varchar(32) DEFAULT NULL,actionname varchar(32) DEFAULT NULL,access ENUM('TRUE','FALSE'),INDEX user_action (username, actionname)) TYPE=MyISAM;

CREATE TABLE actions (actionname varchar(32) NOT NULL,actiondescr varchar(64) DEFAULT NULL,defaultaccess ENUM('TRUE','FALSE'),PRIMARY KEY (actionname)) TYPE=MyISAM;

# Add the available actions to the access control tableINSERT INTO actions (actionname, actiondescr, defaultaccess) VALUES ('add_user', 'Add users', 'TRUE');INSERT INTO actions (actionname, actiondescr, defaultaccess) VALUES ('edit_user', 'Edit users (delete and change password)', 'TRUE');INSERT INTO actions (actionname, actiondescr, defaultaccess) VALUES ('reload_cache', 'Reload search cache', 'TRUE');INSERT INTO actions (actionname, actiondescr, defaultaccess) VALUES ('edit_acl', 'Edit access control settings', 'TRUE');

# Create the CISCO Management Messages TableCREATE TABLE cemdb (id int(5) assigned DEFAULT NULL,name varchar(128) DEFAULT NULL,message text,explanation text,action text,datetime datetime default NULLPRIMARY KEY (id)) TYPE=MyISAM;

# Create user with admin/admin loginINSERT INTO users (username, pwhash) VALUES('admin', 'd033e22ae348aeb5660fc2140aec35850c4da997');

USE mysql;

# create usersINSERT INTO user (Host, User, Password) VALUES ('localhost','sysloguser', password('MY_PASSWD'));INSERT INTO db (Host, Db, User) VALUES ('localhost','syslog','sysloguser');

INSERT INTO user (Host, User, Password) VALUES ('localhost','syslogfeeder', password('MY_PASSWD'));INSERT INTO db (Host, Db, User) VALUES ('localhost','syslog','syslogfeeder');

INSERT INTO user (Host, User, Password) VALUES ('localhost','syslogadmin',password('MY_PASSWD'));

20

INSERT INTO db (Host, Db, User) VALUES ('localhost','syslog','syslogadmin');COMMIT;FLUSH PRIVILEGES;

# grant rights to user syslogadmin for backup purposeGRANT USAGE ON *.* TO syslogadmin@localhost;GRANT ALL ON syslog.* TO syslogadmin@localhost;GRANT RELOAD ON *.* TO syslogadmin@localhost;

REVOKE ALL PRIVILEGES ON syslog.* FROM sysloguser@localhost;GRANT USAGE ON *.* TO sysloguser@localhost;GRANT SELECT ON syslog.* TO sysloguser@localhost;GRANT UPDATE ON syslog.users TO sysloguser@localhost;

GRANT ALL ON syslog.search_cache TO sysloguser@localhost;GRANT SELECT ON syslog.user_access TO sysloguser@localhost;GRANT ALL ON syslog.user_access TO syslogadmin@localhost;GRANT SELECT ON syslog.actions TO sysloguser@localhost;GRANT ALL ON syslog.actions TO syslogadmin@localhost;

COMMIT;FLUSH PRIVILEGES;

2.6. Completing the PHP-SYSLOG-NG Installation

Make sure the mysqld_sys2mysql and syslog-ng startup scripts are in /etc/rc.local, so everything is brought up properly at reboot.

$ /etc/init.d/mysqld start$ /etc/init.d/syslog2mysql start$ /etc/init.d/syslog-ng start

Notice: The commands must be executed in the order noted above.

If everything was set up properly, SYSLOG-NG will now be logging to both flat files and to the MySQL database.

To check the installation, use a web browser and log into PHP-SYSLOG-NG. For the scope of this document, use: http://clgold.digitalglobe.com/phpsyslogng. The user id and password are set to admin.

Please see Modifications to PHP-SYSLOG-NG for modifications made to PHP-SYSLOG-NG.

21

http://clgold.digitalglobe.com/phpsyslogng

3. SYSLOG-NG Client Installation

3.1. Introduction

This section is to provide information on installing SYSLOG-NG on various operating systems. The information contained in this section implies that the operating system specific kit was already built (see Building a SYSLOG-NG Kis Using EPM for more information). Also, it is implied that the reader/installer is familiar with installing software on the desired operating systems, as it is beyond the scope of this document to thoroughly discussing operating system software installation procedures.

Therefore, before attempting the installation of the software, please refer to appropriate man page or documentation, as follows:

1. For Linux Installations: rpm

2. For IRIX Installations: inst

3. For Solaris Installations:pkgadd

NOTICE: If the target installation is on Solaris 10, then the reader/installer should also be familiar with the Sun’s Service Management Facility (SMF). Solaris 9, Linux and IRIX utilize rc scripts for startup/shutdown of SYSLOG-NG.

CAUTION: It is assumed that the reader/installer will connect to the proper system, before attempting an installation of any of the packages discussed in the upcoming sections. Attempting to install packages on incompatible operating systems will produce unpredictable results.

3.2. Installing on Linux

To install the EPM generated rpm package on Linux, please do the following:

$ cd /home/tools/syslog-ng/linux-2.4-intel-1.6.12$ rpm –Uvh syslogng-1.6.12-linux-2.4-intel.rpm \syslogng-startup-1.6.12-linux-2.4-intel-rpm

22

3.3. Installing on IRIX

To install the EPM generated tardist package on IRIX, please do the following:

$ cd /home/tools/syslog-ng/irix-6.5-mips-1.6.12$ inst -f syslogng-1.6.12-irix-6.5-mips.tardist -a -u newUnpacking tardist file .. 100% Done.Reading product descriptions .. 100% Done.Pre-installation check .. 8%Checking space requirements .. 16%Installing/removing files .. 16%Installing new versions of selected fw_agrep.man subsystems Installing/removing files .. 80% Installing new versions of selected fw_agrep.sw subsystems Installing/removing files .. 94% Running exit-commands .. 100% Done.Installations and removals were successful.Requickstarting ELF files (see rqsall(1)) .. 100% Done.$ inst -f syslogngstartup-1.6.12-irix-6.5-mips.tardist -a -u newUnpacking tardist file .. 100% Done.Reading product descriptions .. 100% Done.Pre-installation check .. 8%Checking space requirements .. 16%Installing/removing files .. 16%Installing new versions of selected fw_agrep.man subsystems Installing/removing files .. 80% Installing new versions of selected fw_agrep.sw subsystems Installing/removing files .. 94% Running exit-commands .. 100% Done.Installations and removals were successful.Requickstarting ELF files (see rqsall(1)) .. 100% Done.

3.4. Installing on Solaris 9

To install the EPM generated pkg package on Solaris 9 Sparc, please do the following:

$ cd /home/tools/syslog-ng/solaris-9-sparc-1.6.12$ unzip syslogng-1.6.12-solaris-9-sparc.pkg.gz$ pkgadd –d syslogng-1.6.12-solaris-9-sparc.pkg$ pkgadd –d syslogngstartup-1.6.12-solaris-9-sparc.pkg

3.5. Installing on Solaris 10

To install the EPM generated pkg package on Solaris 10 Sparc, please do the following:

$ cd /home/tools/solaris-10-sparc-1.6.12$ unzip syslogng-1.6.12-solaris-10-sparc.pkg.gz$ pkgadd –d syslogng-1.6.12-solaris-10-sparc.pkg$ pkgadd –d syslogngstartup-1.6.12-solaris-10-sparc.pkg

To install the EPM generated pkg package on Solaris 10 Intel, please do the following:

$ cd /home/tools/syslog-ng/solaris-10-intel-1.6.12$ unzip syslogng-1.6.12-solaris-10-intel.pkg.gz$ pkgadd –d syslogng-1.6.12-solaris-10-intel.pkg$ pkgadd –d syslogngstartup-1.6.12-solaris-10-intel.pkg

NOTICE: For all Solaris installations, it is possible to use the following pkgadd command, as well

$ pkgadd –d /home/tools/syslog-ng/solaris-##-type-1.6.12 syslogng

23

where: ## = 9 or 10where: type = sparc or intel

NOTICE: Solaris 10 uses a new feature called Service Management Facility (SMF), which was designed to replace the legacy scripts in /etc/init.d and /etc/rc#.d with a more robust programmatic model for managing services.

The Solaris 10 SYSLOG-NG package includes two files which define a SYSLOG-NG instance of the system-log service, which is meant to run SYSLOG-NG instead of the standard syslogd, which is provided by the default system-log service instance. The SYSLOG-NG service instance, system-log:syslog-ng, was created based on the existing system-log:default instance.

However, should a future patch or other change to Solaris cause the system-log:default instance’s dependencies to change, this could easily prevent system-log:syslog-ng from starting. This is due to how the SMF was designed, and while Sun is working on a new layered implementation called Enhanced SMF, for the time being (which could be a long time), system patches will need to be evaluated to determine if changes are made that could break the custom system-log:syslog-ng instance.

24

4. SYSLOG-NG Configuration Files

4.1. Introduction

This section provides information about configuration files for SYSLOG-NG. The configuration file, syslog-ng.conf, is used by SYSLOG-NG to read, process, filter and send out system log information for a given server.

4.2. SYSLOG-NG Configuration File - Global Options

SYSLOG-NG allows the capability to override certain variables and parameters within the syslog-ng.conf file. The following table provides a list of available options.

Option Name Value Description Defaultbad_hostname reg_exp A regular expression which matches hostnames which should not be

taken as suchchain_hostnames y/n Enable or disable the chained hostname format Nocreate_dirs y/n Enable or disable directory creation for destination files. Nodir_group string The group of directories created by SYSLOG-NG. rootdir_owner string The owner of directories created by SYSLOG-NG. rootdir_perm number The permission mask of directories created by SYSLOG-NG. Log

directories are only created if a file after macro expansion refers to a non-existing directory, and directory creation is enabled (see the create_dirs() option). For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.

O700

dns_cache y/n Enable or disable DNS cache. Yesdns_cache_expire number Number of seconds while a successful lookup is reached. 3600dns_cache_expire_failed number Number of seconds while a failed lookup is cached. 60dns_cache_size number Number of hostnames in the DNS cache. 1007gc_busy_threshold number Sets the threshold value for the garbage collector, when SYSLOG-NG

is busy. GC phase starts when the number of allocated objects reaches this setting.

3000

gc_idle_threshold number Sets the threshold value for the garbage collector, when SYSLOG-NG is idle. GC phase starts when the number of allocated objects reaches this setting.

100

group string Set the group of the created file to the one specified. rootkeep_hostname y/n Enable or disable hostname rewriting. This means that if the log entry

had been passed through at least one other logging system, the ORIGINAL hostname will be kept attached to the log. Otherwise the last logger will be considered the log entry owner and the log entry will appear to have come from that host.

No

log_fifo_size number Number of lines fitting to the output queue. 100log_msg_size number Maximum length of message in bytes. 2048 *owner string Set the owner of the created file to the one specified. rootperm number The permission mask of the file if it is created by SYSLOG-NG. For

octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.O600

Sanitize_filenames y/n Replace control characters in filename with the dot (“.”) character. Yesstats number The number of seconds between STATS. 600sync number The number of lines buffered before written to file. 0time_reap number The time to wait before an idle destination file is closed. 60time_reopen number The time to wait before a dead connection is reestablished. 60use_dns y/n Enable or disable DNS usage. SYSLOG-NG blocks on DNS queries,

so enabling DNS may lead to a Denial of Service attack. To prevent Denial of Service, protect your SYSLOG-NG network endpoint with firewall rules, and make sure that all hosts, which may get to SYSLOG-NG is resolvable.

Yes

use_fqdn y/n Add Fully Qualified Domain Name instead of short hostname. Nouse_time_received y/n Use the time a message is received instead of one in the message. No* log_msg_size has been set to 8192 to handle larger messages generated by some applications, e.g. Request Tracker

25

4.3. SYSLOG-NG Configuration File – Log Sources

The following table provides a list of available log file sources for SYSLOG-NG:

Source Description Optionsfifo/pipe The pipe driver opens a named pipe

with the specified name, and listens for messages. It's used as the native message getting protocol on HP-UX.

Name Type Description Optionspad_size number Specifies input

padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. (HP-UX uses 2048 bytes).

0

log_prefix string The string to prepend to log messages. Useful for logging kernel messages as they are not prefixed by kernel: by default.

Empty string

file Usually the kernel presents its messages in a special file (/dev/kmsg on BSD, /proc/kmsg on Linux), so to read such special files, you'll need the file() driver. Please note that you can't use this driver to follow a file like tail -f does.

Name Type Description Optionslog_prefix string The string to

prepend to log messages. Useful for logging kernel messages as they are not prefixed by kernel: by default

Empty string

internal All internally generated messages originate from this special source. If it is desired to receive warnings, errors and notices from SYSLOG-NG itself, then include this source in one of the source statements.

sun-streams Solaris uses its STREAMS API to send messages to the syslogd process. On Solaris, SYSLOG-NG must be compiled with this driver (see ./configure --help).

Newer versions of Solaris (2.5.1 and above), uses a new IPC in addition to STREAMS, called door to confirm delivery of a message. SSYSLOG-NG supports this new IPC mechanism with the door() option.

The sun-streams() driver has a single required argument, specifying the STREAMS device to open and a single option.

Name Type Description Optionsdoor string Specifies the

filename of a door to open, needed on Solaris above 2.5.1.

none

26

tcp/udp These drivers let SYSLOG-NG to receive messages from the network, and as the name of the drivers imply, one can use both UDP and TCP as transport.

UDP is a simple datagram oriented protocol, which provides "best effort service" to transfer messages between hosts. It may lose messages, and no attempt is made to retransmit such lost messages at the protocol level.

TCP provides connection-oriented service, which basically means a flow-controlled message pipeline.In this pipeline, each message is acknowledged, and retransmission is done for lost packets. Generally it's safer to use TCP, because lost connections can be detected, and no messages get lost, but traditionally the syslog protocol uses UDP.

Neither of tcp() and udp() drivers require positional parameters. By default they bind to 0.0.0.0:514, which means that SYSLOG-NG will listen on all viable interfaces, port 514. To limit accepted connections to one interface only, use the localip() parameter as described in the options column.

NOTICE: SYSLOG-NG needs to use TCP port 5140, as TCP 514 is assigned to another protocol.

Name Type Description OptionsIp orlocal ip

string The IP address to bind to. Note that this is not the address where messages are accepted from.

0.0.0.0

keep-alive y/n Available for tcp() only, and specifies whether to close connections upon the receipt of a SIGHUP signal.

yes

max-connections

number Specifies the maximum number of simultaneous connections

10

port or local Port number

The port number to bind to.

514

unix-stream /unix-dgram

These two drivers behave similarly: they open the given AF_UNIX socket, and start listening on them for messages. unix-stream() is primarily used on Linux, and uses SOCK_STREAM semantics (connection oriented, no messages are lost), unix-dgram() is used on BSD, and uses SOCK_DGRAM semantics, this may result in lost local messages, if the system is overloaded.

To avoid denial of service attacks when using connection-oriented protocols, the number of simultaneously accepted connections should be limited. This can be achieved using themax-connections() parameter. The default value of this parameter is quite strict; it may need to be increased on a busy system.

Both unix-stream and unix-dgram have a single required positional argument, specifying the filename of the socket to create, and several optional parameters.

Name Type Description Defaultgroup string Set the gid of

the socket.root

keep-alive y/n Selects whether to keep connections open when SYSLOG-NG is restarted. Can be only used with unix-stream().

yes

max-connections

num Limits the number of simultaneously opened connections. Can be used only with unix-stream();

10

owner string Set the uid of the socket.

root

perm permission Set the permission mask. For octal numbers prefix the number with ‘O’, e.g. O755 for rwxr-xr-x.

O666

27

4.4. SYSLOG-NG Configuration File – Log Destinations

The following table provides a list of available log file sources for SYSLOG-NG:

Destination Description Options/Macrosfifo/pipe This driver sends

messages to a named pipe like /dev/xconsole

The pipe driver has a single required parameter, specifying the filename of the pipe to open, and no options.

Name Type Description Defaultowner string Set the owner of the created file to the

one specified.root

group string Set the group of the created file to the one specified.

root

perm number The permission mask of the file if it is created by SYSLOG-NG. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.

O600

template string Specifies a template which defines the logformat to be used in this file. Possible macros are the same as for the file() destination.

A format conforming to the default logfile format.

template_escape y/n Turns on escaping ' and " in templated output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server.

yes

28

file The file driver is one of the most important destination drivers in SYSLOG-NG. It allows one to output messages to the named file, or to a set of files.

The destination filename may include macros which gets expanded when the message is written, thus a simple file() driver may result in several files to be created. Macros can be included by prefixing the macro name with a '$' sign (without the quotes), just as in Perl/PHP.

If the expanded filename refers to a directory which doesn't exist, it will be created depending on the create_dirs() setting (both global and a per destination option)

WARNING: Since the state of each created file must be tracked by SYSLOG-NG, it consumes some memory for each file. If no new messages are written to a file within 60 seconds (controlled by the time_reap global option), it's closed, and its state is freed.

Exploiting this, a Denial of Service attack can be mounted against a system. If the number of

Name Type Description Defaultlog_file_size number The number of entries in the output fifo. Use global

settingfsync y/n Forces an fsync() call on the destination

fd after each write. Note: this may seriously degrade performance.

No

sync_freq number The logfile is synced when this number of messages has been written to it.

Use global setting

owner string Set the owner of the created file to the one specified.

root

group string Set the group of the created file to the one specified.

root

perm number The permission mask of the file if it is created by SYSLOG-NG. For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.

O600

create_dir y/n Enable creating non-existing directories. nodir_perm number The permission mask of directories

created by SYSLOG-NG. Log directories are only created if a file after macro expansion refers to a non-existing directory, and directory creation is enabled (see the create_dirs() option). For octal numbers prefix the number with '0', e.g.: use 0755 for rwxr-xr-x.

O600

dir_owner string The owner of directories created by syslog-ng.

root

dir_group string The group of directories created by syslog-ng.

root

template string Specifies a template which defines the logformat to be used in this file. Possible macros are the same as for the file() destination.



yes

remove_if_older number If set to a value higher than 0, before writing to a file, SYSLOG-NG checks whether this file is older than the specified amount of time (specified in seconds). If so, it removes the existing file and the line to be written is the first line of a new file having the same name. In combination with e.g.: the $WEEKDAY macro, this can be used for simple log rotation, in case not all history has to be kept.

Never remove existing files; use append instead ( = 0).

FILE NAME MACROS:

29

possible destination files and its needed memory is more than the amount the target system’ SYSLOG-NG server has.

The most suspicious macro is $PROGRAM, where the possible variations is quite high, so in untrusted environments $PROGRAM usage should be avoided.

Name DescriptionDATE Date of the transaction.DAY The day of month the message was sent.FACILITY The name of the facility, the message is tagged as coming from.FULLDATE Long form of the date of the transaction.FULLHOST Full hostname of the system that sent the log.HOST The name of the source host where the message is originated from. If

the message traverses several hosts, and chain_hostnames() is on, the first one is used.

HOUR The hour of day the message was sent.ISODATE Date in ISO format.MIN The minute the message was sent.MONTH The month the message was sent.MSG or MESSAGE

Message contents.

PRIORITY or LEVEL

The priority of the message.

PROGRAM The name of the program the message was sent by.SEC The second the message was sent.TAG The priority and facility encoded as a 2 digit hexadecimal number.TZ The time zone or abbreviation, e.g. ‘MST’TZOFFSET The time zone as an hour offset from GMT, e.g. ‘-0700’WEEKDAY The 3-letter name of the day of week the message was sent, e.g.

‘Thu’.YEAR The year the message was sent. Time expansion macros can either

use the time specified in the log message, e.g. the time the log message is sent, or the time the message was received by the log server. This is controlled by the use_time_recvd() option.

program This driver fork()'s executes the given program with the given arguments and sends messages down to the stdin of the child.

The program driver has a single required parameter, specifying a program name to start and no options. The program is executed with the help of the current shell, so the command may include both file patterns and I/O redirection, they will be processed.

NOTICE: The program is executed once at startup, and kept running until SIGHUP or exit. The reason is to prevent starting up a large number of programs for messages,

Name Type Description Defaulttemplate string Specifies a template which defines the

logformat to be used. Possible macros are the same as with destination file().



Yes

30

which would imply an easy Denial of Service..

tcp/udp This driver sends messages to another host on the local intranet or internet using either UDP or TCP protocol.

Both drivers have a single required argument specifying the destination host address, where messages should be sent, and several optional parameters.

NOTICE: This differs from source drivers, where local bind address is implied, and none of the parameters are required.

Name Type Description Defaultlocalip string The IP address to bind to before

connecting to the target.0.0.0.0

localport num The port number to bind to. 0Port/destport

num The port number to connect to. 514

template string Specifies a template which defines the logformat to be used. Possible macros are the same as with destination file().


template_escape y/n Turns on escaping ' and " in template output files. This is useful for generating SQL statements and quoting string contents so that parts of the log message are not interpreted as commands to the SQL server.

Yes

spoof_source y/n Enables source address spoofing. This means that the host running SYSLOG-NG generates UDP packets with the source IP address matching the original sender of the message. It is useful when you want to perform some kind of preprocessing via SYSLOG-NG then forward messages to your central log management solution with the source address of the original sender. This option only works for UDP destinations though the original message can be received by TCP as well. This option is only available if SYSLOG-NG was compiled using the --enable-spoof-source configure option.

No

log_fifo_size number The number of entries in the output fifo. Use global setting

sync(tcp only)

number The messages are sent to the remote host when a certain number of messages have been collected.

0

tcp-keep-alive(tcp only)

y/n Available for tcp() only, and specifies whether to enable TCP keep alive messages using the SO_KEEPALIVE socket option.

No

usertty This driver writes messages to the terminal of a logged-in user.

The usertty driver has a single required argument, specifying a username who should receive a copy of matching messages, and no optional arguments.





Yes

31

unix-dgram /unix-stream

This driver sends messages to a unix socket in either SOCK_STREAM or SOCK_DGRAM mode.

Both drivers have a single required argument specifying the name of the socket to connect to, and no optional arguments.





Yes

32

4.5. SYSLOG-NG Configuration File – Filters

This section provides an introduction to SYSLOG-NG filters.

Functions Synopsis Descriptionfacility facility(facility[,facility]) Match messages having one of the listed facility codes.filter Call another filter rule and evaluate its valuehost host(regexp) Match messages by using a regular expression against the hostname field of

log messages.level/priority level(pri[,pri1..pri2[,pri3]]

) Match messages based on priority.

match Tries to match a regular expression to the message itself.netmask Netmask(ip/mask) Check the sender's IP address whether it is in the specified IP subnetprogram program(regexp) Match messages by using a regular expression against the program name

field of log messages

NOTES:

Getting filtering to work right can be difficult because while the syntax is fairly simple, it is not well documented. To illustrate a brief lesson on filtering and to explain the majority of the mechanics, we shall use the filter from the PostgreSQL database how-to page found at: http://www.umialumni.com/~ben/SYSLOG-DOC.html

This is a perfect and somewhat complex example to use. In its original form it resembles:

filter f_postgres { not( (host("monitor") and facility(cron) and level(info)) or (facility(user) and level(notice) and ( match(" gethostbyaddr: ") or match("last message repeated ") ) ) or ( facility(local3) and level(notice) and match(" SYSMON NORMAL ")) or ( facility(mail) and level(warning) and match(" writable directory") ) or ( ( host("dbserv1.somecompany.com") or host("dbserv2.somecompany.com") ) and facility(auth) and level(info) and match("su oracle") and match(" succeeded for root on /dev/") ) ); };

While in this form, it does not induce a tremendous amount of insight on what the specific filter is attempting to accomplish. In reformatting the filter to resemble something a bit more human readable, it would look like:

filter f_postgres { not ( ( host("monitor") and facility(cron) and level(info) ) or ( facility(user) and level(notice) and ( match(" gethostbyaddr: ") or match("last message repeated ")

33

http://www.umialumni.com/~ben/SYSLOG-DOC.html

) ) or ( facility(local3) and level(notice) and match(" SYSMON NORMAL ") ) or ( facility(mail) and level(warning) and match(" writable directory") ) or ( ( host("dbserv1.somecompany.com") or host("dbserv2.somecompany.com") ) and facility(auth) and level(info) and match("su oracle") and match(" succeeded for root on /dev/") ) ); };

Now in this form we can now begin to see what this filter has been attempting to accomplish. We can now further breakdown each logical section and explain the different methods:

[1] As in all statements in SYSLOG-NG, each of the beginnings and endings must be with a curly bracket "{" "}" to clearly denote the start and finish.

In this filter, the entire filter is preferred by a not to indicate that these are the messages that we are NOT interested in and should be the ones filtered out. All lines of logs that do not match these lines will be sent to the destination.

{ not

[2] The first major part of the filter is actually a compound filter that has two parts. Because the two parts are separated by an or, only one of the two parts must be matched for that line of log to be filtered.

[2a] In the first part of this filter there are three requirements to be met for the filter to take affect. These are the host string monitor, the facility cron, and the syslog level of info.

( ( host("monitor") and facility(cron) and level(info) ) or

[2b] In the second part of the filter, which in itself is a compound filter, there are three requirements as well. These are that the facility of user, and the log level of notice are met in addition to one of the two string matches that are shown in the example.

( facility(user) and level(notice) and ( match(" gethostbyaddr: ") or match("last message repeated ")

34

) ) or

[3] In the section of the filter there are once again three requirements to fire off a match which are a facility of level3 a log level of notice and a sting match of " SYSMON NORMAL ".

( facility(local3) and level(notice) and match(" SYSMON NORMAL ") ) or

[4] This part of the filter is very similar to the previous filter, but with different search patterns.

( facility(mail) and level(warning) and match(" writable directory") ) or

[5] The last section of the filter is also a compound filter that to take affect will require that one of two hosts are matched, the facility of auth, and log level of info occur in addition to the two string matches.

( ( host("dbserv1.somecompany.com") or host("dbserv2.somecompany.com") ) and facility(auth) and level(info) and match("su oracle") and match(" succeeded for root on /dev/") )

[6] As in all command sets in SYSLOG-NG, each of the statements must be properly closed with the correct ending punctuation AND a semi-colon. Do not forget both, or an error will be generated.

); };

While this may not be the most complete example, it does cover the majority of the options and features that are available within the current version of SYSLOG-NG.

35

4.5.1. SYSLOG-NG Configuration File – Logging & Filtering

This section provides an introduction to how SYSLOG-NG handles logging and filtering..

When applying filters, remember that each subsequent filter acts as a filter on the previous data flow. This means that if the first filter limits the flow to only data from the auth system, a subsequent filter for authpriv will cause no data to be written. An example of this would be:

log { source(s_dgram); source(s_internal); source(s_kernel); source(s_tcp); source(s_udp); filter(f_auth); filter(f_authpriv); destination(authlog); };

So, one can cancel out the other.

There are also certain flags that can be attached to each of the log statements:

Flag Descriptioncatchall This flag means that the source of the message is ignored. Only the filters are taken into account when

matching messages.fallback This flag makes a log statement 'fallback'. Being a fallback statement means that only messages not

matching any 'non-fallback' log statements will be dispatched.final This flag means that the processing of log statements ends here. Note that this doesn't necessarily mean

that matching messages will be stored once, as they can be matching log statements processed prior the current one.

36

4.6. SYSLOG-NG Configuration File - syslog-ng.conf

The following is a syslog-ng.conf file which contains examples of filtering and sending output to both text log files and a MySQL database. This syslog-ng.conf file can be used on a SYSLOG-NG Central Server. Please read the notes within the syslog-ng.conf file below for additional information.

Operating System Notes:

The syslog-ng.conf file below is specific to Linux. The following are changes which need to be made so the syslog-ng.conf file will run on Solaris and IRIX.

1. For IRIX and Solaris the statement unix-stream(“/dev/log”) needs to be changed to sun-streams(“/dev/log”).

2. For IRIX and Solaris, comment out the line pipe(“/proc/kmsg”).

3. For IRIX and Solaris 9 only, remove references to authpriv and security in the authpriv filter definition. Also, comment out the ftp filter definition.

# Syslog-ng configuration file. Originally written by anonymous (I can't find# his name) Revised, and rewrited by me (SZALAY Attila <[email protected]>)# revised again by Nate Campi <nate at campin dot net>

################################################################ First, set some global options.

options { # use_fqdn(yes); # use_dns(yes); # dns_cache(yes);

keep_hostname(yes); long_hostnames(off); sync(1); log_fifo_size(1024); log_msg_size(8192);

};

################################################################# This is the default behavior of sysklogd package# Logs may come from unix stream, but not from another machine.##source src { unix-stream("/dev/log"); internal(); };

source src { # don't read from /proc/kmsg and run klogd also (Linux)

pipe("/proc/kmsg"); # file("/proc/kmsg") log_prefix("kernel: ");

unix-stream("/dev/log"); # unix-stream("/chroot/named/dev/log");

internal(); udp();

# udp(ip("10.0.5.8") port(514)); tcp(port(5140) keep-alive(yes) max-connectios(250));

# tcp(ip("10.9.9.3") port(5140) keep-alive(yes)); };

################################################################ After that set destinations.

# Logfile definitions

37

destination authpriv { file("/var/adm/auth"); };destination authprivinfo { file("/var/adm/auth.info"); };destination cron { file("/var/adm/cron"); };destination croninfo { file("/var/adm/cron.info"); };destination daemon { file("/var/adm/daemon"); };destination daemoninfo { file("/var/adm/daemon.info"); };destination ftp { file("/var/adm/ftp"); };destination ftpinfo { file("/var/adm/ftp.info"); };destination kern { file("/var/adm/kern"); };destination kerninfo { file("/var/adm/kern.info"); };destination mail { file("/var/adm/mail"); };destination mailinfo { file("/var/adm/mail.info"); };destination syslog { file("/var/adm/syslog"); };destination sysloginfo { file("/var/adm/syslog.info"); };destination user { file("/var/adm/user"); };destination userinfo { file("/var/adm/user.info"); };destination local0 { file("/var/adm/local0"); };destination local0info { file("/var/adm/local0.info"); };destination local1 { file("/var/adm/local1"); };destination local1info { file("/var/adm/local1.info"); };destination local2 { file("/var/adm/local2"); };destination local2info { file("/var/adm/local2.info"); };destination local3 { file("/var/adm/local3"); };destination local3info { file("/var/adm/local3.info"); };destination local4 { file("/var/adm/local4"); };destination local4info { file("/var/adm/local4.info"); };destination local5 { file("/var/adm/local5"); };destination local5info { file("/var/adm/local5.info"); };destination local6 { file("/var/adm/local6"); };destination local6info { file("/var/adm/local6.info"); };destination local7 { file("/var/adm/local7"); };destination local7info { file("/var/adm/local7.info"); };destination allsyslog { file("/var/adm/syslog-mg"); };## Facilities not in use; shown as comments for documentation purposes##destination lpr { file("/var/adm/lpr"); };#destination lprinfo { file("/var/adm/lpr.info"); };#destination news { file("/var/adm/news/news"); };#destination newsinfo { file("/var/adm/news/news.info"); };#destination uucp { file("/var/adm/uucp"); };#destination uucp { file("/var/adm/uucp.info"); };

# Some `catch-all' logfiles.#destination messages { file("/var/adm/messages"); };destination messagesinfo { file("/var/adm/messages.info"); };destination debug { file("/var/adm/debug"); };

# The root's console.#destination console { usertty("root"); };

# Virtual console.##destination console_all { file("/dev/tty8"); };

# The named pipe /dev/xconsole is for the nsole' utility. To use it,# you must invoke nsole' with the -file' option:## $ xconsole -file /dev/xconsole [...]##destination xconsole { pipe("/dev/xconsole"); };

## scripts that accept syslog messages and mail them out#destination mail-alert { program("/usr/local/bin/syslog-mail"); };destination mail-alert-perl { program("/usr/local/bin/syslog-mail-perl"); };

38

## hack to get swatch to read from stdin##destination swatch { program("/usr/bin/swatch --read-pipe=\"cat /dev/fd/0\""); };

###########################################

# Here's the filter options. With this rules, we can set which # message go where.

filter f_attack_alert { match("attackalert");

};

filter f_ssh_login_attempt { program("sshd.*") and match("(Failed|Accepted)") and not match("Accepted (hostbased|publickey) for (root|zoneaxfr) from

(10.4.3.1)"); };

filter f_authpriv { level (notice .. emerg) and facility(auth, authpriv, security); };filter f_authprivinfo { level (info) and facility(auth, authpriv, security); };filter f_cron { level (warn .. emerg) and facility(cron); };filter f_croninfo { level (info .. notice) and facility(cron); };filter f_daemon { level (warn .. emerg) and facility(daemon); };filter f_daemoninfo { level (info .. notice) and facility(daemon); };filter f_ftp { level (warn .. emerg) and facility(ftp); };filter f_ftpinfo { level (info .. notice) and facility(ftp); };filter f_kern { level (crit .. emerg) and facility(kern); };filter f_kerninfo { level (info .. warn) and facility(kern); };filter f_mail { level (warn .. emerg) and facility(mail); };filter f_mailinfo { level (info .. notice) and facility(mail); };filter f_syslog { level (warn .. emerg) and facility(syslog); };filter f_sysloginfo { level (info .. notice) and facility(syslog); };filter f_user { level (warn .. emerg) and facility(user); };filter f_userinfo { level (info .. notice) and facility(user); };filter f_local0 { level (warn .. emerg) and facility(local0) and program("ApacheLog") and match("HTTP/1.1\" 4") or match ("bHTTP/1.1\" 5"); };filter f_local0info { level (info .. notice) and facility(local0) and program("ApacheLog") and match("HTTP/1.1\" 4") or match("HTTP/1.1\" 5"); };filter f_local1 { level (warn .. emerg) and facility(local1); };filter f_local1info { level (info .. notice) and facility(local1); };filter f_local2 { level (warn .. emerg) and facility(local2); };filter f_local2info { level (info .. notice) and facility(local2); };filter f_local3 { level (warn .. emerg) and facility(local3); };filter f_local3info { level (info .. notice) and facility(local3); };filter f_local4 { level (warn .. emerg) and facility(local4); };filter f_local4info { level (info .. notice) and facility(local4); };filter f_local5 { level (warn .. emerg) and facility(local5); };filter f_local5info { level (info .. notice) and facility(local5); };filter f_local6 { level (warn .. emerg) and facility(local6); };filter f_local6info { level (info .. notice) and facility(local6); };filter f_local7 { level (warn .. emerg) and facility(local7); };filter f_local7info { level (info .. notice) and facility(local7); };filter f_allsyslog { level(info .. emerg)

and facility(auth, authpriv, cron, daemon, ftp, kern, mail, syslog, user,local0, local1, local2, local3, local4, local5, local6, local7); };

filter f_messages { level(warn .. emerg) and not facility(auth, authpriv, cron, daemon, ftp, kern, mail, syslog, user,

local0, local1, local2, local3, local4, local5, local6, local7); };filter f_messagesinfo { level(info .. notice)

and not facility(auth, authpriv, cron, daemon, ftp, kern, mail, syslog, user, local0, local1, local2, local3, local4, local5, local6, local7); };

filter f_debug { level(debug) and not facility(auth, authpriv, cron, daemon, ftp, kern, mail, syslog, user,

local0, local1, local2, local3, local4, local5, local6, local7); };

39

filter f_info { level(info); };filter f_notice { level(notice); };filter f_warn { level(warn); };filter f_crit { level(crit); };filter f_err { level(err); };filter f_emergency { level(emerg); };

# Additional definitions for documentation purposes

#filter f_lpr { level (warn .. emerg) and facility(lpr); };#filter f_lprinfo { level (info .. notice) and facility(lpr); };#filter f_news { level (warn .. emerg) and facility(news); };#filter f_newsinfo { level (info .. notice) and facility(news); };#filter f_uucp { level (warn .. emerg) and facility(uucp); };#filter f_uucpinfo { level (info .. notice) and facility(uucp); };

################################################################# log statements actually send logs somewhere, to a file, across the network, etc#

log { source(src); filter(f_authpriv); destination(authpriv); };log { source(src); filter(f_authprivinfo); destination(authprivinfo); };log { source(src); filter(f_cron); destination(cron); };log { source(src); filter(f_croninfo); destination(croninfo); };log { source(src); filter(f_daemon); destination(daemon); };log { source(src); filter(f_daemoninfo); destination(daemoninfo); };log { source(src); filter(f_ftp); destination(ftp); };log { source(src); filter(f_ftpinfo); destination(ftpinfo); };log { source(src); filter(f_kern); destination(kern); };log { source(src); filter(f_kerninfo); destination(kerninfo); };log { source(src); filter(f_mail); destination(mail); };log { source(src); filter(f_mailinfo); destination(mailinfo); };log { source(src); filter(f_syslog); destination(syslog); };log { source(src); filter(f_sysloginfo); destination(sysloginfo); };log { source(src); filter(f_user); destination(user); };log { source(src); filter(f_userinfo); destination(userinfo); };log { source(src); filter(f_local0); destination(local0); };log { source(src); filter(f_local0info); destination(local0info); };log { source(src); filter(f_local1); destination(local1); };log { source(src); filter(f_local1info); destination(local1info); };log { source(src); filter(f_local2); destination(local2); };log { source(src); filter(f_local2info); destination(local2info); };log { source(src); filter(f_local3); destination(local3); };log { source(src); filter(f_local3info); destination(local3info); };log { source(src); filter(f_local4); destination(local4); };log { source(src); filter(f_local4info); destination(local4info); };log { source(src); filter(f_local5); destination(local5); };log { source(src); filter(f_local5info); destination(local5info); };log { source(src); filter(f_local6); destination(local6); };log { source(src); filter(f_local6info); destination(local6info); };log { source(src); filter(f_local7); destination(local7); };log { source(src); filter(f_local7info); destination(local7info); };log { source(src); filter(f_allsyslog); destination(allsyslog); };

log { source(src); filter(f_messages); destination(messages); };log { source(src); filter(f_messagesinfo); destination(messagesinfo); };

log { source(src); filter(f_debug); destination(debug); };

log { source(src); filter(f_emergency); destination(console); };


#log { source(src); filter(f_lpr); destination(lpr); };#log { source(src); filter(f_lprinfo); destination(lprinfo); };#log { source(src); filter(f_news); destination(news); };#log { source(src); filter(f_newsinfo); destination(newsinfo); };#log { source(src); filter(f_uucp); destination(uucp); };#log { source(src); filter(f_uucpinfo); destination(uucpinfo); };

40

# find messages with "attackalert" in them, and send to the mail-alert scriptlog {

source(src); filter(f_attack_alert); destination(authpriv);

};

# find messages reporting attempted ssh logins, and send to the mail-alert scriptlog {

source(src); filter(f_ssh_login_attempt); destination(authpriv);

};

# send all logs to swatch for (near) real-time alerts#log { # source(src); # destination(swatch); #};

NOTICE in loghost: The following area is used for logging to Zenoss, a Relay Server or to a Centralized Logging Server. In the kits, the following section does not exist in the format shown below. In addition, udp is required to log to the Zenoss server, and tcp will be used for all other purposes. Finally, when inserting a loghost, please use a fully qualified domain name and not an IP address.

## set up logging to loghost#destination loghost {# udp("nms-a.digitalglobe.com" port(514));#};

# send everything to loghost, too

#log { source(src); filter(f_authpriv); destination(loghost); };#log { source(src); filter(f_authprivinfo); destination(loghost); };#log { source(src); filter(f_cron); destination(loghost); };#log { source(src); filter(f_daemon); destination(loghost); };#log { source(src); filter(f_ftp); destination(loghost); };#log { source(src); filter(f_kern); destination(loghost); };#log { source(src); filter(f_mail); destination(loghost); };#log { source(src); filter(f_syslog); destination(loghost); };#log { source(src); filter(f_user); destination(loghost); };#log { source(src); filter(f_local0); destination(loghost); };#log { source(src); filter(f_local1); destination(loghost); };#log { source(src); filter(f_local2); destination(loghost); };#log { source(src); filter(f_local3); destination(loghost); };#log { source(src); filter(f_local4); destination(loghost); };#log { source(src); filter(f_local5); destination(loghost); };#log { source(src); filter(f_local6); destination(loghost); };#log { source(src); filter(f_local7); destination(loghost); };

NOTICE: END OF LOGHOST SECTION

## automatic host sorting (usually used on a loghost)#

# set it up#destination std { # file("/var/adm/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY_$HOST_$YEAR_$MONTH_$DAY" # owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)# ); #};

# log it#log { # source(src); # destination(std); #};

41

###############################################################

NOTICE Log to Database: The following section is only applicable to the Centralized Logging Server, this section will not be active on any other syslog-ng host.

destination d_mysqld { pipe("/var/run/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes));};

log { source(src); filter(f_authpriv); destination(d_mysqld); };log { source(src); filter(f_authprivinfo); destination(d_mysqld); };log { source(src); filter(f_cron); destination(d_mysqld); };log { source(src); filter(f_croninfo); destination(d_mysqld); };log { source(src); filter(f_daemon); destination(d_mysqld); };log { source(src); filter(f_daemoninfo); destination(d_mysqld); };log { source(src); filter(f_ftp); destination(d_mysqld); };log { source(src); filter(f_ftpinfo); destination(d_mysqld); };log { source(src); filter(f_kern); destination(d_mysqld); };log { source(src); filter(f_kerninfo); destination(d_mysqld); };log { source(src); filter(f_mail); destination(d_mysqld); };log { source(src); filter(f_mailinfo); destination(d_mysqld); };log { source(src); filter(f_syslog); destination(d_mysqld); };log { source(src); filter(f_sysloginfo); destination(d_mysqld); };log { source(src); filter(f_user); destination(d_mysqld); };log { source(src); filter(f_userinfo); destination(d_mysqld); };log { source(src); filter(f_local0); destination(d_mysqld); };log { source(src); filter(f_local0info); destination(d_mysqld); };log { source(src); filter(f_local1); destination(d_mysqld); };log { source(src); filter(f_local1info); destination(d_mysqld); };log { source(src); filter(f_local2); destination(d_mysqld); };log { source(src); filter(f_local2info); destination(d_mysqld); };log { source(src); filter(f_local3); destination(d_mysqld); };log { source(src); filter(f_local3info); destination(d_mysqld); };log { source(src); filter(f_local4); destination(d_mysqld); };log { source(src); filter(f_local4info); destination(d_mysqld); };log { source(src); filter(f_local5); destination(d_mysqld); };log { source(src); filter(f_local5info); destination(d_mysqld); };log { source(src); filter(f_local6); destination(d_mysqld); };log { source(src); filter(f_local6info); destination(d_mysqld); };log { source(src); filter(f_local7); destination(d_mysqld); };log { source(src); filter(f_local7info); destination(d_mysqld); };

log { source(src); filter(f_messages); destination(d_mysqld); };log { source(src); filter(f_messagesinfo); destination(d_mysqld); };

log { source(src); filter(f_debug); destination(d_mysqld); };


#log { source(src); filter(f_lpr); destination(d_mysqld); };#log { source(src); filter(f_lprinfo); destination(d_mysqld); };#log { source(src); filter(f_news); destination(d_mysqld); };#log { source(src); filter(f_newsinfo); destination(d_mysqld); };#log { source(src); filter(f_uucp); destination(d_mysqld); };#log { source(src); filter(f_uucpinfo); destination(d_mysqld); };

NOTICE: END OF LOG TO DATABASE SECTION

42

5. SYSLOG-NG Log Rotation

5.1. Introduction

This section is designed to provide the reader with information for setting up log rotation for the Linux, Solaris and IRIX Operating Systems. The log rotation specifications discussed in this chpater have been implemented in the SYSLOG-NG Installation Kits.

5.2. Log Rotation on Linux - /etc/logrotate.d/syslog-ng

The following are the contents of to /etc/logrotate.d/syslog-ng, which enable the clients to rotate the flat log files generated by SYSLOG-NG:

# SYSLOG-NG logrotate directives

/var/adm/auth /var/adm/auth.info /var/adm/cron /var/adm/cron.info /var/adm/daemon /var/adm/daemon.info /var/adm/ftp /var/adm/ftp.info /var/adm/kern /var/adm/kern.info /var/adm/user /var/adm/user.info /var/adm/local0 /var/adm/local0.info /var/adm/local1 /var/adm/local1.info /var/adm/local2 /var/adm/local2.info /var/adm/local3 /var/adm/local3.info /var/adm/local4 /var/adm/local4.info /var/adm/local5 /var/adm/local5.info /var/adm/local6 /var/adm/local6.info /var/adm/local7 /var/adm/local7.info /var/adm/syslog-ng { missingok compress size=1M rotate 5 weekly postrotate /etc/init.d/syslog-ng reload > /dev/null 2>&1 || true endscript}

INSTALLATION: This file needs to be installed in /etc/logrotate.d. Once the above file is installed in /etc/logrotate.d, then logrotate will process the log rotation automatically via /etc/cron.daily.

NOTICE: For hosts other than the central log server, the log rotation should be set to daily, instead of weekly, as shown above.

BUG/FEATURE ALERT: logrotate requires the file specifications to be on one line with no continuation characters, as logrotate does not parse the file specifications correctly otherwise. Also, a space is required between each file specification.

43

5.3. Log Rotation on Solaris 9 & Solaris 10 - /etc/logadm.conf

The following text would be the SYSLOG-NG related contents of /etc/logadm.conf. The program logadm is utilized by Solaris to perform the log rotation procedure.

## SYSLOG-NG Log Rotation Directives#/var/adm/auth /var/adm/auth.info /var/adm/cron /var/adm/cron.info /var/adm/daemon /var/adm/daemon.info /var/adm/ftp /var/adm/ftp.info /var/adm/kern /var/adm/kern.info /var/adm/user /var/adm/user.info /var/adm/local0 /var/adm/local0.info /var/adm/local1 /var/adm/local1.info /var/adm/local2 /var/adm/local2.info /var/adm/local3 /var/adm/local3.info /var/adm/local4 /var/adm/local4.info /var/adm/local5 /var/adm/local5.info /var/adm/local6 /var/adm/local6.info /var/adm/local7 /var/adm/local7.info /var/adm/syslog-ng –C 5 –a ‘kill –HUP `cat /var/run/syslog-ng.pid`’ –p 1d –s 1m

INSTALLATION: The above directives need to be appended to /etc/logadm.conf. When logadm is run via crontab, the above directives will be processed.

NOTES:

1. The –C directive retains a number of copies. In the above example 5 copies are saved.2. The –a directive runs a command. In the above example the syslog-ng process is

restarted.3. The –p directive is defined to be a certain time period. In the above example it is set to 1

day.4. The –s directive is defined to be a certain file size. In the above example it is set to 1

Megabyte.

44

5.4. Log Rotation on IRIX - /usr/local/bin/rotatelog

A script called rotatelog is available for download from http://www.interback.net/projects/rotatelog. This script, as with logroate and logadm, utilizes an external configuration file to set up log rotation. The script is written in Perl, so it does not require compiling or any other special considerations.

The rotatelog script and related manpage is loaded as part of the SYSLOG-NG kit built locally via EPM.

The following is a copy of /usr/local/etc/rotatelog.conf

# $Id: rotatelog.conf,v 1.2 2001/04/22 23:01:36 rowland Exp $

####### This a sample rotatelog configuration file.######

FILES:

# File (full path) Trigger Owner:Group Mode Compress Archive Limit# ================ ======= =========== ==== ======== =============

/var/adm/auth 0K root:sys 664 gz 5/var/adm/auth.info 0K root:sys 664 gz 5/var/adm/cron 0K root:sys 664 gz 5/var/adm/cron.info 0K root:sys 664 gz 5/var/adm/daemon 0K root:sys 664 gz 5/var/adm/daemon.info 0K root:sys 664 gz 5/var/adm/ftp 0K root:sys 664 gz 5/var/adm/ftp.info 0K root:sys 664 gz 5/var/adm/kern 0K root:sys 664 gz 5/var/adm/kern.info 0K root:sys 664 gz 5/var/adm/mail 0K root:sys 664 gz 5/var/adm/mail.info 0K root:sys 664 gz 5/var/adm/syslog 0K root:sys 664 gz 5/var/adm/syslog.info 0K root:sys 664 gz 5/var/adm/user 0K root:sys 664 gz 5/var/adm/user.info 0K root:sys 664 gz 5/var/adm/local0 0K root:sys 664 gz 5/var/adm/local0.info 0K root:sys 664 gz 5/var/adm/local1 0K root:sys 664 gz 5/var/adm/local1.info 0K root:sys 664 gz 5/var/adm/local2 0K root:sys 664 gz 5/var/adm/local2.info 0K root:sys 664 gz 5/var/adm/local3 0K root:sys 664 gz 5/var/adm/local3.info 0K root:sys 664 gz 5/var/adm/local4 0K root:sys 664 gz 5/var/adm/local4.info 0K root:sys 664 gz 5/var/adm/local5 0K root:sys 664 gz 5/var/adm/local5.info 0K root:sys 664 gz 5/var/adm/local6 0K root:sys 664 gz 5/var/adm/local6.info 0K root:sys 664 gz 5/var/adm/local7 0K root:sys 664 gz 5/var/adm/local7.info 0K root:sys 664 gz 5/var/adm/syslog-ng 0K root:sys 664 gz 5

ACTIONS:

45

http://www.interback.net/projects/rotatelog

# Shell Command :file1,file2,file3 ... fileN# ==========================================

kill -HUP `cat /var/run/syslog-ng.pid` : /var/adm/auth, /var/adm/auth.info, /var/adm/cron, /var/adm/cron.info, /var/adm/daemon, /var/adm/daemon.info, /var/adm/ftp, /var/adm/ftp.info, /var/adm/kern, /var/adm/kern.info, /var/adm/mail, /var/adm/mail.info, /var/adm/syslog, /var/adm/syslog.info, /var/adm/user, /var/adm/user.info, /var/adm/local0, /var/adm/local0.info, /var/adm/local1, /var/adm/local1.info, /var/adm/local2, /var/adm/local2.info, /var/adm/local3, /var/adm/local3.info, /var/adm/local4, /var/adm/local4.info, /var/adm/local5, /var/adm/local5.info, /var/adm/local6, /var/adm/local6.info, /var/adm/local7, /var/adm/local7.info,/var/adm/syslog-ng

NOTIFY:

# Person to receive email rotation notification# =============================================

root@localhost

The script should be copied to /usr/local/bin on the target host. Also, the following crontab entry needs to be created in the root crontab:

15 3 * * * /usr/local/bin/rotatelog > /var/adm/rotatelog.log 2>&1

46

5.5. Log Rotation for the SYSLOG-NG database - logrotate.php

The following script performs the log rotation for the SYSLOG-NG database (from the PHP-SYSLOG-NG distribution). This script runs only on the SYSLOG-NG Central Server.

#!/usr/local/bin/php<?php// Copyright (C) 2005 Claus Lund, [email protected] "\nStarting logrotate\n";echo date("Y-m-d H:i:s");

$basePath = "/usr/local/apache2a/htdocs/phpsyslogng";include_once "$basePath/includes/common_funcs.php";include_once "$basePath/config/config.php";

$dbLink = db_connect_syslog(DBADMIN, DBADMINPW);echo "\nStarting logrotate\n";echo date("Y-m-d H:i:s");

// Drop temp table if it exists$query = "DROP TABLE IF EXISTS temp".DEFAULTLOGTABLE;perform_query($query, $dbLink);

// Create new table$query = "SHOW CREATE TABLE ".DEFAULTLOGTABLE;

$result = perform_query($query, $dbLink);$row = mysql_fetch_array($result);$createQuery = $row[1];$search = "CREATE TABLE `".DEFAULTLOGTABLE."`";$replace = "CREATE TABLE `temp".DEFAULTLOGTABLE."`";$createQuery = str_replace($search, $replace, $createQuery);perform_query($createQuery, $dbLink);

$today = date("Ymd");

// Drop the merge tableif(defined('MERGELOGTABLE') && MERGELOGTABLE) {

$query = "FLUSH TABLES";perform_query($query, $dbLink);

$query = "DROP TABLE IF EXISTS ".MERGELOGTABLE;perform_query($query, $dbLink);

}

// Rename the two tables$query = "RENAME TABLE ".DBNAME.".".DEFAULTLOGTABLE." TO ".DBNAME.".".DEFAULTLOGTABLE.$today.", ".DBNAME.".temp".DEFAULTLOGTABLE." TO ".DBNAME.".".DEFAULTLOGTABLE;perform_query($query, $dbLink);

echo "\nLog rotate ended successfully.\n";echo "Now optimizing the old logs.\n";$query = "OPTIMIZE TABLE ".DBNAME.".".DEFAULTLOGTABLE.$today;perform_query($query, $dbLink);

if(defined('LOGROTATERETENTION') && LOGROTATERETENTION) {echo "Getting list of log tables.\n";$logTableArray = get_logtables($dbLink);echo "Searching for tables to drop.\n";foreach($logTableArray as $value) {

if(preg_match("([0-9]{8}$)", $value)) {// determine is datestamp is old enough$tableDate = strrev(substr(strrev($value), 0, 8));

47

$cutoffDate = date("Ymd", mktime(0, 0, 0, date("m"), date("d")-LOGROTATERETENTION, date("Y")));

if($cutoffDate > $tableDate) {echo "Dropping ".$value."!\n";$query = "DROP TABLE ".$value;perform_query($query, $dbLink);

}}

}}

if(defined('MERGELOGTABLE') && MERGELOGTABLE) {echo "Getting list of log tables.\n";$logTableArray = get_logtables($dbLink);echo "Creating merge table.\n";$query = "SHOW CREATE TABLE ".DEFAULTLOGTABLE;

$result = perform_query($query, $dbLink);$row = mysql_fetch_array($result);$createQuery = $row[1];

$oldStr = "CREATE TABLE `".DEFAULTLOGTABLE."`";$newStr = "CREATE TABLE `".MERGELOGTABLE."`";$createQuery = str_replace($oldStr, $newStr, $createQuery);

$oldStr = "ENGINE=MyISAM";$newStr = "ENGINE=MRG_MyISAM";$createQuery = str_replace($oldStr, $newStr, $createQuery);$oldStr = "TYPE=MyISAM";$newStr = "ENGINE=MRG_MyISAM";$createQuery = str_replace($oldStr, $newStr, $createQuery);

$createQuery = str_replace('PRIMARY KEY', 'INDEX', $createQuery);

$unionStr = " UNION=(";foreach($logTableArray as $value) {

$unionStr = $unionStr.$value.", ";}$unionStr = rtrim($unionStr, ", ");$unionStr = $unionStr.")";

$createQuery = $createQuery.$unionStr;

$query = "FLUSH TABLES";perform_query($createQuery, $dbLink);

}echo "\n".date("Y-m-d H:i:s")."\n";echo "All done!\n";

?>

48

6. SYSLOG-NG Support Files

6.1. Introduction

This section provides information about support files for SYSLOG-NG. These files are utilized to startup/shutdown SYSLOG-NG and to transmit SYSLOG-NG data to a MySQL database. The files discussed in this chapter are provided in the SYSLOG-NG Installation Kits.

The files are based upon a Linux implementation of SYSLOG-NG, except where noted. These files have been used to set up the SYSLOG-NG environment on brutus.digitalglobe.com, the experimental SYSLOG-NG environment.

6.2. Starting SYSLOG-NG and support software

6.2.1. Introduction

This section provides a copy of the files which are used to start up SYSLOG-NG and related software.

6.2.2. SYSLOG-NG to MySQL Communication – syslog2mysql.sh

The following script enables communication between MySQL and SYSLOG-NG. The script is designed to run on the SYSLOG-NG Central Server and is located in /usr/local/sbin.

#!/bin/bash

if [ ! -e /var/run/mysql.pipe ]then

/usr/bin/mkfifo /var/run/mysql.pipefiwhile [ -e /var/run/mysql.pipe ]do

/usr/local/mysql/bin/mysql -u syslogadmin --password=MY_PASSWD syslog < /var/run/mysql.pipe >/dev/nulldone

6.2.3. SYSLOG-NG to MYSQL Communication rc script - syslog2mysql

The following script starts up syslog2mysql at system boot time. It needs to run after mysqld (the MySQL startup script). This script is designed to run on the SYSLOG-NG Central Server and to run on Linux from /etc/init.d.

#!/bin/bash## /etc/init.d/syslog2mysql#

### BEGIN INIT INFO## This init script should be executed after the mysqld startup and before# syslog-ng is started. It is best to use rc.local to startup mysqld,# syslog2mysql, and syslog-ng, than to do so via the rc levels as the# execution order of these scripts must be in the order noted above.#### END INIT INFO

# Source function library.

49

. /etc/init.d/functions

mysql_pipe="/var/run/mysql.pipe"syslog2mysql_pid="/var/run/syslog2mysql.pid"syslog2mysql_lock="/var/lock/subsys/syslog2mysql"syslog2mysql="/usr/local/sbin/syslog2mysql.sh"

[ -x $syslog2mysql ] || exit 0 RETVAL=0

case "$1" in start)

/bin/echo -n "Starting syslog2mysql services"${syslog2mysql} &/bin/ps -C ${syslog2mysql} -o pid= > ${syslog2mysql_pid}

RETVAL=$? /bin/echo [ $RETVAL -eq 0 ] && touch ${syslog2mysql_lock}

;; stop)

/bin/echo -n "Shutting down syslog2mysql services"x=`/bin/ps -ef | /bin/grep syslogfeeder | /bin/grep -v /bin/grep | /bin/awk

'{ print $2 }'`if [ "$x" != "" ]; then

/bin/kill -9 ${x}fiRETVAL=$?

/bin/echox=`/bin/ps -ef | /bin/grep syslog2mysql.sh | /bin/grep -v /bin/grep | /bin/awk

'{ print $2 }'`if [ "$x" != "" ]; then

/bin/kill -9 ${x}fiRETVAL=$?

/bin/echo if [ $RETVAL -eq 0 ]; then /bin/rm -f ${syslog2mysql_lock} /bin/rm -f ${syslog2mysql_pid} /bin/rm -f ${mysql_pipe} fi;

;; *)

echo "Usage: $0 {start|stop}"exit 1;;

esac

6.2.4. SYSLOG-NG Startup rc script - syslog-ng

The following table applies to /etc/init.d and /etc/rc*.d, to startup and shutdown SYSLOG-NG Central Server software on Linux for SYSLOG-NG:

/etc/init.d /etc/rc2.d, /etc/rc3.d./etc/rc4.d, /etc/rc5.d

/etc/rc1.d

mysqld S10mysql K88mysqldsyslog2mysql S11syslog2mysql K89syslog2mysqlsyslog-ng S12syslog-ng K90syslog-ng

The following script starts up SYSLOG-NG at system boot time. It needs to run after syslog2mysql. This rc script is designed to run on Linux from /etc/init.d.

#!/bin/bash## syslog-ng This starts and stops syslog-ng

50

## chkconfig: 2345 12 88# description: syslog-ng is an alternative system logger# processname: syslog-ng# pidfile: /var/run/syslog-ng.pid### BEGIN INIT INFO# Provides: $syslog-ng### END INIT INFO

# Source function library.. /etc/init.d/functions

binary="/usr/local/sbin/syslog-ng"

[ -x $binary ] || exit 0

RETVAL=0

start() {echo -n "Starting syslog-ng: "daemon $binaryRETVAL=$?echo[ $RETVAL -eq 0 ] && touch /var/lock/subsys/syslog-ng

}

stop() {echo -n "Shutting down syslog-ng: "killproc syslog-ngRETVAL=$?echoif [ $RETVAL -eq 0 ]; then

rm -f /var/lock/subsys/syslog-ngrm -f /var/run/syslog-ng.pid

fi}

restart() {echo -n "Restarting syslog-ng: "$binary --syntax-onlyRETVAL=$?echoif [ $RETVAL -eq 0 ]; then

stopsleep 2start

fi}

reload() {echo -n "Reloading syslog-ng: "$binary --syntax-onlyRETVAL=$?if [ $RETVAL -eq 0 ]; then

killproc syslog-ng -1fi

}

case "$1" instart)

start;;

stop)stop;;

status)status syslog-ng;;

restart)restart;;

51

reload)reload;;

condrestart)[ -f /var/lock/subsys/syslog-ng ] && restart;;

*)echo "Usage: $0 {start|stop|status|restart|reload|condrestart}";;

esac

52

7. Enhancements and Modifications

7.1. Modifications to PHP-SYSLOG-NG

7.1.1. Introduction

In order to provide the Facility and Program Summary reports, a few modifications were made to the PHP-SYSLOG-NG source code. Two new modules were added: facilityresult.php and programresult.php; these modules are located in /var/www/html/phpsyslogng on clgold.digitalglobe.com.

The new modules perform the following functions:

facilityresult.php – Produces a count of log record by Log Facility and Log Level.programresult.php – Produces a count of log records by Program Name Logged (e. g. crond, syslog-ng, etc.) and Log Level.

Also, in order to support the new modules, this section provides information of changes to PHP-SYSLOG-NG that would need to be implemented.

7.1.2. /usr/local/apache2a/phpsyslogng/index.php

The following adds support for the Facility and Program Summary reports.

89a90,97> elseif(strcasecmp($pageId, "facility") == 0) {> $addTitle = "FACILITY SUMMARY RESULTS";> require 'includes/facilityresult.php';> }> elseif(strcasecmp($pageId, "program") == 0) {> $addTitle = "PROGRAM SUMMARY RESULTS";> require 'includes/programresult.php';> }

7.1.3. /usr/local/apache2a/phpsyslogng/includes/search.php

The following add options to run the Facility and Program Summary Reports.

369a370,371> <input type="submit" name="pageId" value="Facility">> <input type="submit" name="pageId" value="Program">

53

7.1.4. /usr/local/apache2a/phpsyslogng/includes/tailresult.php

The following change adds the facility to the log level, which was omitted in the distributed code.

239c239< <td class="resultsheader">FACILITY</td>---> <td class="resultsheader">FACILITY-LEVEL</td>262c262< echo "<td class=\"sev0\">".$row['facility']."</td>";---> echo "<td class=\"sev0\">".$row['facility']."-".$row['priority']."</td>";265c265< echo "<td class=\"sev1\">".$row['facility']."</td>";---> echo "<td class=\"sev1\">".$row['facility']."-".$row['priority']."</td>";268c268< echo "<td class=\"sev2\">".$row['facility']."</td>";---> echo "<td class=\"sev2\">".$row['facility']."-".$row['priority']."</td>";271c271< echo "<td class=\"sev3\">".$row['facility']."</td>";---> echo "<td class=\"sev3\">".$row['facility']."-".$row['priority']."</td>";274c274< echo "<td class=\"sev4\">".$row['facility']."</td>";---> echo "<td class=\"sev4\">".$row['facility']."-".$row['priority']."</td>";277c277< echo "<td class=\"sev5\">".$row['facility']."</td>";---> echo "<td class=\"sev5\">".$row['facility']."-".$row['priority']."</td>";

7.1.5. /usr/local/apache2a/phpsyslogng/includes/regularresult.php

The following minor change better defines the contents of the column:

394c394< <td>FACILITY</td>---> <td>FACILITY-LEVEL</td>

7.1.6. /usr/local/apache2a/phpsyslogng/includes/jpgraph/jpg-config.inc

In order for PHP-SYSLOG-NG to find the msttcorefonts, please modify jpg-config.inc, as follows:

Comment out line 42.Add the following as a new line just below:

DEFINE(“TTF_DIR”,”/usr/share/fonts/msttcorefonts/”);

54

7.2. Directing Apache Logs to SYSLOG-NG

7.2.1. Introduction

The Apache httpd web sever provides a couple of ways to log to SYSLOG-NG. The first alternative is to not log to the default Apache httpd log files (error_log and access_log), but just to SYSLOG-NG. This method, though it eliminates a set of log files, also renders various web server analysis tools inoperative. The second alternative is to use the CustomLog directive, and via pipes, execute an external utility which would log Apache httpd logs to SYSLOG-NG, but also keep the default Apache httpd logs. This section introduces the second alternative, as the preferred method to log both to the Apache httpd logs and SYSLOG-NG.

7.2.2. Installing the Apache httpd to SYSLOG-NG Filter

The utility snaretext provides the functionality to send Apache httpd log entries to SYSLOG-NG. In order to install snaretext, please do the following:

Download snaretext from http://www.intersectalliance.com/projects/SnareApache/index.html

Next, issue the following commands:

$ tar –xzf snaretext-1.1.tar.gz$ cd snaretext-1.1$ ./configure –prefix=/usr/local$ make$ make installOnce snaretext is built and installed. Add the following line to httpd.conf (/usr/local/apache2a/conf), just below the active Errorlog logs/error_log line:

CustomLog "|/usr/local/bin/snaretext -s local0.info 10.0.0.3 ApacheLog" combined

For the Experimental Environment, the syslog/level used was local0.info; the host name used was helpdesk.digitalglobe.com. It is advisable that a Fully Qualified Domain Name be used in place of the IP Address.

After making the change to httpd.conf, restart the Apache httpd webserver:

$ /etc/init.d/httpd stop$ /etc/init.d/httpd start

When the Apache httpd web server starts, the following message should appear:

$ /usr/local/bin/snaretext: sending data to ’10.11.30.107’ (IP: 10.11.30.107)

Finally, Apache httpd log entries should now start appearing in local0 and local0.info in the /var/adm directory.

55

http://www.intersectalliance.com/projects/SnareApache/index.html

7.3. IRIX patch for SYSLOG-NG 1.6.12

7.3.1. Introduction

During testing phase of the IRIX EPM kit, it was determined that a patch to SYSLOG-NG 1.6.2 was never propagated to later SYSLOG-NG 1.6 releases. Because IRIX does not place a NULL character at the end syslog style messages, SYSLOG-NG would not log messages on IRIX. This patch corrects the problem by making sure a NULL character is present at the end of every SYSLOG-NG message. A special thanks to Justin Lloyd for finding this patch. This patch has been added to DigitalGlobe’s SYSLOG-NG V1.6.12 code base.

7.3.2. Implementation

The following patch needs to be applied to src/adstreams.c. Based upon testing, this patch has no affect on other operating system implementations of SYSLOG-NG.

--- afstreams.c.20060911 Thu May 13 11:33:36 2004+++ afstreams.c Mon Sep 11 17:24:03 2006@@ -98,7 +98,7 @@

ctl.maxlen = ctl.len = sizeof(lc); ctl.buf = (char *) &lc;- data.maxlen = length;+ data.maxlen = length - 1; /* the -1 is to allow for a possible NUL at the end */ data.len = 0; data.buf = buf; flags = 0;@@ -134,6 +134,17 @@ UINT8 *eol, *bol; UINT32 length; struct log_info *li;++ /* Note: some systems do not terminate the message with anything.+ This null termination lets the memchr() call below work in+ these cases. We are not writing past the end of the buffer here,+ because we told the getmsg() call to read in a maximum of+ (bufferlength - 1) characters, to guarantee room for this+ null terminator. */+ if (n > 0 && line[n - 1] != '\0' && line[n - 1] != '\n') {+ line[n] = '\0';+ n++;+ }

bol = line; eol = memchr(bol, '\n', n);

56

8. Building a SYSLOG-NG kit using EPM

8.1. Introduction

This section documents the procedure for building an EPM kit. In order to provide a unified method for building and installing SYSLOG-NG on different operating systems, the usage of EPM (ESP Package Manager) enables the installer to maintain operating system specific build files to create SYSLOG-NG packages across various operating system platforms. Additional information about EPM can be obtained at http://www.easysw.com/epm. This web site contains documentation and the EPM source kit.

Notice 1: This section assumes that EPM has already been installed in the build environment. It is beyond the scope of this document to describe the EPM installation procedure, so refer to the web site noted above. Finally, the EPM software is not required on hosts where an EPM created package will be installed, it is only required on hosts which will be utilizing EPM to build an EPM package.

Notice 2: For the convenience of the reader/installer, operating system specific EPM images have been built and are available in /home/tools/syslog-ng.

8.2. Preparing for an EPM Build

Normally, the first step is to download the latest SYSLOG-NG distribution kit from http://www.habitat.com/products/syslog_ng. Since this has already been done at DigitalGlobe, downloading SYSLOG-NG is not necessary.

At the time of the writing of this document, a DigitalGlobe specific SYSLOG-NG V1.6.12 kit has been created and the kit resides in /home/tools/syslog-ng/syslog-ng-1.6.12.DG.tar.gz. This directory also contains operating system specific EPM syslogng.list files, syslog-ng.conf files, SYSLOG-NG startup/shutdown files and other supporting files. This directory should be used for kit builds. Eventually, this directory area will be relocated to lum as /home/tools/syslog-ng.

CAUTION 1: In the following sections, it is assumed that the reader/installer has connected to the proper host, before attempting a build. If not, unpredictable results are definitely possible.

CAUTION 2: Make sure the patch described in IRIX patch for SYSLOG-NG 1.6.12 is applied to the source code, before attempting a build. This patch REQUIRED for IRIX systems, as SYSLOG-NG will not log any information without the patch.

CAUTION 3: It is advisable to type make distclean in the syslog-ng-1.6.12 and syslog-ng-1.6.12/libol-0.3.18 directories, before attempting a configuration or build of SYSLOG-NG.

57

http://www.habitat.com/products/syslog_ng

http://www.easysw.com/epm

8.3. Configuring and Building SYSLOG-NG on Linux and IRIX

Perform the following steps for building on Linux:

Log into a Linux system and do the following:

$ cd /home/tools/syslog-ng/builds$ tar –xzf ../syslog-ng-1.6.12.DG.tar.gz$ mv syslog-ng-1.6.12 syslog-ng-1.6.12.linux$ cd /home/tools/syslog-ng/syslog-ng-1.6.12.linux$ ./configure -–prefix=/usr/local –-enable-full-static

Perform following steps for building on IRIX:

Log into an IRIX system and do the following:

$ cd /home/tools/syslog-ng/builds$ tar –xzf ../syslog-ng-1.6.12.DG.tar.gz$ mv syslog-ng-1.6.12 syslog-ng-1.6.12.irix$ cd /home/tools/syslog-ng/syslog-ng-1.6.12.irix$ ./configure -–prefix=/usr/local –-enable-full-static

NOTICE: The syslogng.list files, available later in this chapter, were written to extract files from the SYSLOG-NG build directory area. Therefore, there is no need to use mkepmlist to create a syslogng.list file.

8.4. Performing a Linux SYSLOG-NG EPM kit Build

Perform the following steps to build a SYSLOG-NG Linux rpm kit via EPM

Log into a Linux system and do the following:

$ cd /home/tools/syslog-ng$ rm –rf linux-2.4-intel$ cp syslog-ng-1.6.12/syslogng.list.linux syslogng.list$ epm-4.0.linix-intel/epm –frpm syslogng syslogng$ mv linux-2.4-intel linux-2.4-intel-1.6.12

8.5. Performing an IRIX SYSLOG-NG EPM kit Build

Perform the following steps to build a SYSLOG-NG IRIX tardist kit via EPM

Log into an IRIX system and do the following:

$ cd /home/tools/syslog-ng$ rm –rf irix-6.5-mips$ cp syslog-ng-1.6.12/syslogng.list.irix syslogng.list$ epm-4.0.irix-mips/epm –ftardist syslogng syslogng.list$ cp syslog-ng-1.6.12/syslogngstartup.list.irix syslogngstartup.list$ epm-4.0.irix-mips/epm –ftardist syslogngstartup syslogngstartup.list$ mv irix-6.5-mips irix-6.5-mips-1.6.12

58

8.6. Configuring and Building SYSLOG-NG on Solaris

Please perform the following steps for installing SYSLOG-NG on Solaris:

1. Make sure these PATHs are set up as follows:

$ LD_LIBRARY_PATH=/usr/lib:/usr/share/lib:/usr/sfw/lib:/opt/sfw/lib$ export LD_LIBRARY_PATH$PATH=/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/usr/ccs/bin:/usr/lib: \/usr/share/lib:/usr/sfw:/opt/sfw:/opt/sfw/bin:/usr/sfw/bin$ export PATH

2. Configure as follows:

Login to a Solaris 9 system and do the following:

$ cd /home/tools/syslog-ng/builds$ tar –xzf ../syslog-ng-1.6.12.DG.tar.gz$ mv syslog-ng-1.6.12 syslog-ng-1.6.12.solaris9$ cd /home/tools/syslog-ng/builds/syslog-ng-1.6.12.solaris9$ ./configure --prefix=/usr/local --enable-full-dynamic$ make

Login to a Solaris 10 Intel system and do the following:

$ cd /home/tools/syslog-ng/builds$ tar –xzf ../syslog-ng-1.6.12.DG.tar.gz$ mv syslog-ng-1.6.12 syslog-ng-1.6.12.solaris10-intel$ cd /home/tools/syslog-ng/builds/syslog-ng-1.6.12.solaris10-intel$ ./configure --prefix=/usr/local --enable-full-dynamic$ make

Login to a Solaris 10 Sparc system and do the following:

$ cd /home/tools/syslog-ng/builds$ tar –xzf ../syslog-ng-1.6.12.DG.tar.gz$ mv syslog-ng-1.6.12 syslog-ng-1.6.12.solaris10-sparc$ cd /home/tools/syslog-ng/builds/syslog-ng-1.6.12.solaris10-sparc$ ./configure --prefix=/usr/local --enable-full-dynamic$ make

NOTICE: When configure is run on Solaris 10 (Sparc only) it places the -ll (for libl.so) directive in the incorrect location in the LIBS definition in each Makefile. Please update each Makefile in the top level build directory, src, src/tests and doc, by modifying the following line:

From:

LIBS = -lpthread -lrt -lnsl -lsocket -ldoor -lresolv /home/nmetro/syslog-ng-1.6.12/libol-0.3.18/src/.libs/libol.a -lsocket -lnsl -lxnet -Wl,-Bstatic -ll -Wl,-Bdynamic

To:

59

LIBS = -lpthread -lrt -lnsl -lsocket -ldoor -lresolv /home/nmetro/syslog-ng-1.6.12/libol-0.3.18/src/.libs/libol.a -lsocket -lnsl -lxnet -ll -Wl,-Bstatic -Wl,-Bdynamic

8.7. Performing a Solaris 9 SYSLOG-NG EPM kit Build

Perform the following steps to build a SYSLOG-NG Solaris 9 Sparc pkg kit via EPM

Login to a Solaris 9 system and do the following:

$ cd /home/tools/syslog-ng$ rm –rf solaris-9-sparc$ cp syslog-ng-1.6.12/syslogng.list.solaris9 syslogng.list$ epm-4.0.solaris9-sparc/epm –fpkg syslogng syslogng.list$ cp syslog-ng-1.6.12/syslogngstartup.list.solaris9 \ syslogngstartup.list$ epm-4.0.solaris9-sparc/epm –fpkg syslogngstartup syslogngstartup.list$ mv solaris-9-sparc solaris-9-sparc-1.6.12

8.8. Performing a Solaris 10 Sparc SYSLOG-NG EPM kit Build

Perform the following steps to build a SYSLOG-NG Solaris 10 Sparc pkg kit via EPM

Login to a Solaris 10 Sparc system and do the following:

$ cd /home/tools/syslog-ng$ rm –rf solaris-10-sparc$ cp syslog-ng-1.6.12/syslogng.list.solaris10 syslogng.list$ epm-4.0.solaris10-sparc/epm –fpkg syslogng syslogng.list$ cp syslog-ng-1.6.12/syslogngstartup.list.solaris10 \ syslogngstartup.list$ epm-4.0.solaris10-sparc/epm –fpkg syslogngstartup \ syslogngstartup.list$ mv solaris-10-sparc solaris-10-sparc-1.6.12

8.9. Performing a Solaris 10 Intel SYSLOG-NG EPM kit Build

Perform the following steps to build a SYSLOG-NG Solaris 10 Intel pkg kit via EPM

Login to a Solaris 10 Intel system and do the following:

$ cd /home/tools/syslog-ng$ rm –rf solaris-10-intel$ cp syslog-ng-1.6.12/syslogng.list.solaris10 syslogng.list$ epm-4.0.solaris10-intel/epm –fpkg syslogng syslogng.list$ cp syslog-ng-1.6.12/syslogngstartup.list.solaris10 \ syslogngstartup.list$ epm-4.0.solaris10-intel/epm –fpkg syslogngstartup \ syslogngstartup.list$ mv solaris-10-intel solaris-10-intel-1.6.12

60

8.10. Linux syslogng.list file for EPM

$srcdir=./builds/syslog-ng-1.6.12.linux

# Product information%product syslogng%copyright 1999-2001 BalaBit IT Ltd.%vendor GPL%license ${srcdir}/COPYING%readme ${srcdir}/README%description syslog-ng, as the name shows, is a syslogd replacement, but with new functionality for the new generation. The original syslogd allows messages only to be sorted based on priority/facility pairs; syslog-ng adds the possibility to filter based on message contents using regular expressions. The new configuration scheme is intuitive and powerful. Forwarding logs over TCP and remembering all forwarding hops makes it ideal for firewalled environments.%version 1.6.12 106011

# Prerequiste Directory%system alld 755 root root /var/adm -

# Executablesd 755 root root /usr/local/sbin -f 755 root root /usr/local/sbin/syslog-ng /usr/local/sbin/syslog-ng

# Man Pagesd 755 root root /usr/local/man -d 755 root root /usr/local/man/man5 -f 644 root root /usr/local/man/man5/syslog-ng.conf.5 /usr/local/man/man5/syslog-ng.conf.5d 755 root root /usr/local/man/man8 -f 644 root root /usr/local/man/man8/syslog-ng.8 /usr/local/man/man8/syslog-ng.8

# Documentationd 755 root root /usr/local/share/syslog-ng-1.6.12 -d 755 root root /usr/local/share/syslog-ng-1.6.12/doc -d 755 root root /usr/local/share/syslog-ng-1.6.12/doc/sgml -f 644 root root /usr/local/share/syslog-ng-1.6.12/AUTHORS ${srcdir}/AUTHORSf 644 root root /usr/local/share/syslog-ng-1.6.12/COPYING ${srcdir}/COPYINGf 644 root root /usr/local/share/syslog-ng-1.6.12/README ${srcdir}/READMEf 644 root root /usr/local/share/syslog-ng-1.6.12/doc/ChangeLog ${srcdir}/ChangeLogf 644 root root /usr/local/share/syslog-ng-1.6.12/INSTALL ${srcdir}/INSTALLf 644 root root /usr/local/share/syslog-ng-1.6.12/doc/NEWS ${srcdir}/NEWSf 755 root root /usr/local/share/syslog-ng-1.6.12/doc/sgml/syslog-ng.txt ${srcdir}/doc/sgml/syslog-ng.txtf 755 root root /usr/local/share/syslog-ng-1.6.12/doc/syslog-ng.conf.demo ${srcdir}/doc/syslog-ng.conf.demof 755 root root /usr/local/share/syslog-ng-1.6.12/doc/syslog-ng.conf.sample ${srcdir}/doc/syslog-ng.conf.samplef 755 root root /usr/local/share/syslog-ng-1.6.12/doc/syslog-ng.conf.solaris ${srcdir}/doc/syslog-ng.conf.solaris

# Configuration Filesd 755 root root /usr/local/etc -d 755 root root /usr/local/etc/syslog-ng -f 644 root root /usr/local/etc/syslog-ng/syslog-ng.conf ${srcdir}/syslog-ng.conf.linux

%subpackage startup%description Startup Files# Log RotationF 644 root root /etc/logrotate.d/syslog-ng ${srcdir}/syslog-ng.logrotate.linux

# Startup Filesf 755 root root /etc/init.d/syslog-ng ${srcdir}/syslog-ng.init.linux

# Package removal%preremove <<EOF

61

/bin/rm /etc/logrotate.d/syslog-ng/sbin/chkconfig --del syslog-ng/sbin/chkconfig --add syslogEOF

# Post Installation%postinstall <<EOF/sbin/chkconfig --del syslog/sbin/chkconfig --add syslog-ngEOF

8.11. IRIX syslogng.list file for EPM

$srcdir=./build/syslog-ng-1.6.12.irix


# Prerequiste Directory%system alld 755 root root /var/adm –d 755 root root /usr/local/etc/syslog-ng

# Executablesd 755 root root /usr/local/sbin -f 755 root root /usr/local/sbin/syslog-ng ${srcdir}/src/syslog-ngf 755 root root /usr/local/bin/rotatelog ${srcdir}/rotatelog

# Man Pagesd 755 root root /usr/local/man -d 755 root root /usr/local/man/man1 -f 644 root root /usr/local/man/man1/rotatelog.1 ${srcdir}/rotatelog.1d 755 root root /usr/local/man/man5 -f 644 root root /usr/local/man/man5/syslog-ng.conf.5 ${srcdir}/doc/syslog-ng.conf.5d 755 root root /usr/local/man/man8 -f 644 root root /usr/local/man/man8/syslog-ng.8 ${srcdir}/doc/syslog-ng.8


# Configuration Filesd 755 root root /usr/local/etc -f 644 root root /usr/local/etc/syslog-ng/syslog-ng.conf ${srcdir}/syslog-ng.conf.irix

62

f 644 root root /usr/local/etc/rotatelog.conf ${srcdir}/rotatelog.conf

8.12. IRIX syslogngstartup.list file for EPM

$srcdir=./build/syslog-ng-1.6.12.irix

# Product information%product syslogng-startup%copyright 1999-2001 BalaBit IT Ltd.%vendor GPL%license ${srcdir}/COPYING%readme ${srcdir}/README%description syslog-ng Startup Files%version 1.6.12 106011

f 644 root root /usr/local/etc/rotatelog.irix.cron ${srcdir}/rotatelog.irix.cron

# Startup Filesi 755 root root syslog-ng ${srcdir}/syslog-ng.init.irix "runlevel(02) start(21) stop(82)"

# Chkconfig Files

f 644 root root /etc/config/syslog_ng ${srcdir}/syslog-ng.chkconfig.irixf 644 root root /etc/config/syslog_ng.options ${srcdir}/syslog-ng.options.irix

%preremove <<EOF/etc/init.d/syslog-ng stopEOF

# Post Installation%postinstall <<EOF/sbin/killall syslogd/etc/init.d/syslog-ng start/bin/cat /usr/local/etc/rotatelog.irix.cron >> /var/spool/cron/crontabs/root/etc/init.d/cron stop/etc/init.d/cron startEOF

8.13. Solaris 9 syslogng.list file for EPM

$srcdir=./build/syslog-ng-1.6.12.solaris9



# Executablesd 755 root root /usr/local/sbin -f 755 root root /usr/local/sbin/syslog-ng ${srcdir}/src/syslog-ngf 755 root root /usr/local/sbin/logadm_remove_syslogng.pl ${srcdir}/logadm_remove_syslogng.pl

# Man Pagesd 755 root root /usr/local/man -

63

d 755 root root /usr/local/man/man5 -f 644 root root /usr/local/man/man5/syslog-ng.conf.5 ${srcdir}/doc/syslog-ng.conf.5d 755 root root /usr/local/man/man8 -f 644 root root /usr/local/man/man8/syslog-ng.8 ${srcdir}/doc/syslog-ng.8


# Configuration Filesd 755 root root /usr/local/etc -d 755 root root /usr/local/etc/syslog-ng -f 644 root root /usr/local/etc/syslog-ng/syslog-ng.conf ${srcdir}/syslog-ng.conf.solaris9f 644 root root /usr/local/etc/syslog-ng/logadm.syslog-ng.conf ${srcdir}/logadm.syslog-ng.conf.solaris

8.14. Solaris 9 syslogngstartup.list file for EPM

% srcdir=./build/syslog-ng-1.6.12.solaris9


# Startup Filesi 755 root root syslog-ng ${srcdir}/syslog-ng.init.solaris9 "runlevel(012) start(74) stop(40)"

# Package removal%preremove <<EOF/usr/local/sbin/logadm_remove_syslogng.pl/bin/mv /etc/init.d/syslog.off /etc/init.d/syslog/etc/init.d/syslog startEOF

# Post Installation%postinstall <<EOF/etc/init.d/syslog stop/bin/mv /etc/init.d/syslog /etc/init.d/syslog.off/bin/cp /usr/local/etc/logadm.syslog-ng.conf >> /etc/logadm.confEOF

8.15. Solaris 10 syslogng.list file for EPM


# Product information

64

%product syslogng%copyright 1999-2001 BalaBit IT Ltd.%vendor GPL%license ${srcdir}/COPYING%readme ${srcdir}/README%description syslog-ng, as the name shows, is a syslogd replacement, but with new functionality for the new generation. The original syslogd allows messages only to be sorted based on priority/facility pairs; syslog-ng adds the possibility to filter based on message contents using regular expressions. The new configuration scheme is intuitive and powerful. Forwarding logs over TCP and remembering all forwarding hops makes it ideal for firewalled environments.%version 1.6.12 106011


# Executablesd 755 root root /usr/local/sbin -f 755 root root /usr/local/sbin/syslog-ng ${srcdir}/src/syslog-ngf 755 root root /usr/local/sbin/logadm_remove_syslogng.pl ${srcdir}/logadm_remove_syslogng.pl

# Man Pagesd 755 root root /usr/local/man -d 755 root root /usr/local/man/man5 -f 644 root root /usr/local/man/man5/syslog-ng.conf.5 ${srcdir}/doc/syslog-ng.conf.5d 755 root root /usr/local/man/man8 -f 644 root root /usr/local/man/man8/syslog-ng.8 ${srcdir}/doc/syslog-ng.8


# Configuration Filesd 755 root root /usr/local/etc –

d 755 root root /usr/local/etc/syslog-ng -f 644 root root /usr/local/etc/syslog-ng/syslog-ng.conf ${srcdir}/syslog-ng.conf.solarisf 644 root root /usr/local/etc/syslog-ng/logadm.syslog-ng.conf ${srcdir}/logadm.syslog-ng.conf.solaris10

8.16. Solaris 10 syslogngstartup.list file for EPM



65

# Startup filesf 644 root root /var/svc/manifest/system/syslog-ng.xml ${srcdir}/svc.manifest.syslog-ng.xmlf 755 root root /lib/svc/method/syslog-ng ${srcdir}/svc.method.syslog-ng

# Package removal%preremove <<EOF/usr/local/sbin/logadm_remove_syslogng.plsvcadm disable syslog-ngsvcadm enable system-logEOF

# Post Installation%postinstall <<EOFsvccfg import /var/svc/manifest/system/syslog-ng.xmlsvcadm disable system-logsvcadm enable syslog-ng/bin/cp /usr/local/etc/logadm.syslog-ng.conf >> /etc/logadm.confEOF

NOTICE: There is a variant syslogng.list file for Solaris 10. This variant is called syslogng.list.solaris10.plus-syslog. This EPM build is to be only used on the Jumpstart system, as there are additional requirements for syslog, because of the rarp software which runs on Jumpstart system. Please see the NOTICE at the end of the next chapter for a detailed explanation.

66

9. Appendix - Syslog Facility Types and Log Levels

LEVEL

FACILITY

DEBUG INFO NOTICE WARN ERR CRIT ALERT EMERG

AUTHAUTHPRIVCRONDAEMONFTPKERNLPRMAILMARKNEWSSECURITYSYSLOGUSERUUCPLOCAL0LOCAL1LOCAL2LOCAL3LOCAL4LOCAL5LOCAL6LOCAL7LEGEND Not

LoggedLog to Local Only

Log to Local & Central

NOTICE: AUTHPRIV, FTP and SECURITY are not applicable to IRIX and Solaris 9.

The following Log Levels are in order from the highest level to the lowest level:

• emerg -- System panic. Messages are flashed on every terminal. The host is severely damaged or extremely unstable. It is advisable to shut down the host as soon as possible.

• alert – Alerts are considered serious errors, but not as serious as errors at the emerg level. The host may continue to operate, but the error should be attended to immediately. Depending on the alert, it may be advisable to shut down the host.

• crit -- These are critical errors, such as hardware problems or serious software issues. The host can continue running, but it is not advisable.

• err -- Miscellaneous errors. These events require attention and should be taken care of as soon as possible.

• warn -- Miscellaneous warnings. Usually classified as recoverable errors.• notice – An unusual situation which merits investigation; a significant event that is

typically not part of normal day-to-day operations.• info -- General system information.• debug -- This level is usually only of use to programmers and occasionally to system

administrators who are trying to figure out just why a program or daemon is behaving in a certain manner. The debug facility entry contains information which the programmer felt necessary for debugging the code during the development cycle. Also, a debug facility message could contain sensitive information.

• none -- This special level means "don't log anything from this facility here." It is most commonly used to exclude information when using wildcard entries in syslog.conf.

67

The following contains a short description of the various Log Facilities:

• auth – Authentication information, but has been deprecated in favor of authpriv, although some programs may still use it.

• authpriv - Authentication information which may contain data like usernames and other privileged information.

• cron – Messages from the cron and at daemons.

• daemon – Messages from daemons, such as inetd, xinetd, etc.

• ftp – File Transport Protocol subsystem related.

• kern – Kernel subsystem related.

• lpr – Printing subsystem related.

• mail – SMTP (Simple Mail Transport Protocol) subsystem related.

• mark - An internal facility for syslog to generate timestamps.

• news – NNTP (Network News Transport Protocol) subsystem related.

• security - Messages from the security subsystem. This facility may not exist on all variants of UNIX.

• syslog – syslog subsystem related (includes SYSLOG-NG).

• user – application or user process related. This is the default facility if a facility is not specified.

• uucp – Unix to Unix Copy subsystem related.

• local0 – local7 – Facilities used by customized programs (i.e. in some programs you can tell it via a configuration file what facility to use, so one may opt to have OpenLDAP log to local0, OpenSSH to log to local1, and so forth).

68

9.1. DigitalGlobe SYSLOG-NG Logging by Facility

Facility Log WARN & Above Levels

File Name

Log DEBUG, INFO, NOTICE

LevelsFile Name

Functional Description

auth, authpriv, security auth auth.info Login, security cron cron cron.info Cron daemon daemon daemon daemon.info Defined by daemonftp ftp ftp.info ftp daemonkern kern kern.info kernelmail mail mail.info Sendmail daemonmark, syslog syslog syslog.info Syslog deamonuser misc misc.info See 3 belowlocal0 local0 local0.info Applications to be definedlocal1 local1 local1.info Applications to be definedlocal2 local2 local2.info Applications to be definedlocal3 local3 local3.info Applications to be definedlocal4 local4 local4.info Network Appliance Filerslocal5 local5 local5.info Storage Frameslocal6 local6 local6.info Storage Switcheslocal7 local7 local7.info Network Routers, Switches

NOTES:1. All log files will be located in /var/adm.2. lpr, uucp and news not applicable to DigitalGlobe.3. user is the “catch all” for facilities which do not have a defined IETF facility in

SYSLOG-NG.4. authpriv, ftp and security are not applicable to IRIX and Solaris 9.5. Please see the table at the beginning of this section for the exceptions to table

above.

69

10. Appendix - Introduction to PHP-SYSLOG-NG

10.1. Introduction

This section provides a short introduction to PHP-SYSLOG-NG. PHP-SYSLOG-NG is a web based utility which can be used to monitor log entries generated via SYSLOG-NG in real-time. In addition, PHP-SYSLOG-NG can read the SYSLOG-NG log entries from a MySQL database which has the potential of performing more detailed searches beyond what is provided with UNIX/Linux utilities such as grep and awk.

On the next few pages are sample screen shots and a short summary of the capabilities of PHP-SYSLOG-NG. The screen shots include some local modifications to PHP-SYSLOG-NG.

NOTICE 1: For information concerning the installation and configuration of PHP-SYSLOG-NG, please see the SYSLOG-NG Central Server Installation chapter.

NOTICE 2: The maintainer of PHP-SYSLOG-NG has developed a User’s Guide for PHP-SYSLOG-NG. A copy has been downloaded to:

\\cofs1\fs1\BU\OPS\IS\Projects\Centralized Logging\PHP-SYSLOG-NG-Userguide.doc

70

10.2. Sample Search Screenshot from PHP-SYSLOG-NG

The following is a sample search display from PHP-SYSLOG-NG:

Notes on the Search Screen:

1. The user is provided myriad selections from the drop down lists. RECORDS PER PAGE is only applicable to the Search and Tail options. All other selection criteria are applicable to the other reporting options.

2. Hostnames are in alphabetical order and can be selected or deselected. This field allows one to select multiple entries, if desired. Go to Config->Update Cache to obtain the most recent host list.

3. The user can choose the current open log table (log), a previous log (logYYYYMMDD) or select everything that has been logged (all_logs).

4. If no date range is selected, PHP-SYSLOG-NG will display data for the current date only. When entering date ranges, the date must be specified in YYYY-MM-DD format. Also, a time string is required (even 00:00:00 for midnight) and it must be in the format of HH:MM:SS. Finally, one can enter yesterday, today and now as valid date entry for From or To Date; also, a time string is not required in this instance.

71

5. In SEARCH MESSAGE, only one string portion is allowed for each search line. So, to search for ApacheLog messages containing 404, one must enter ApacheLog on first search line, then 404 on the second search line.

10.3. Sample Results Screenshot from PHP-SYSLOG-NG

The following is a sample results display from PHP-SYSLOG-NG:

Notice: The SQL statement used to generate the display is provided at the top. Also, the messages are color-coded depending of the level of message being sent to SYSLOG-NG.

72

10.4. Sample Tail Results Screenshot from PHP-SYSLOG-NG

The following is a sample tail results report from PHP-SYSLOG-NG:

Note: This screen is updated in real time, as it is displaying records as they come into the database.

73

10.5. Sample Graph Screenshot from PHP-SYSLOG-NG

The following is a sample graph display from PHP-SYSLOG-NG:

74

10.6. Sample Facility Summary Screenshot from PHP-SYSLOG-NG

The following is a sample facility summary report display from PHP-SYSLOG-NG:

75

10.7. Sample Program Summary Screenshot from PHP-SYSLOG-NG

The following is a sample program summary report display from PHP-SYSLOG-NG:

76