trouble shoot with linux syslog

Download Trouble shoot with linux syslog

Post on 31-Aug-2014




5 download

Embed Size (px)




  • 1|P ag eQuick HOWTO : Ch05 : Troubleshooting Linux with syslogContents[hide] 1 Introduction 2 syslog o 2.1 Table 5-1 Syslog Facilities o 2.2 The /etc/rsyslog.conf File o 2.3 Activating Changes to the syslog Configuration File o 2.4 How to View New Log Entries as They Happen o 2.5 Logging syslog Messages to a Remote Linux Server 2.5.1 Configuring the Linux Syslog Server 2.5.2 Configuring the Linux Client o 2.6 Syslog Configuration and Cisco Network Devices 3 Logrotate o 3.1 The /etc/logrotate.conf File o 3.2 Sample Contents of /etc/logrotate.conf o 3.3 The /etc/logrotate.d Directory o 3.4 Activating logrotate o 3.5 Compressing Your Log Files 4 syslog-ng o 4.1 The /etc/syslog-ng/syslog-ng.conf file 4.1.1 Simple Server Side Configuration for Remote Clients Figure 5-1 A Sample syslog-ng.conf File 4.1.2 Using syslog-ng in Large Data Centers Figure 5-2 More Specialized syslog-ng.conf Configuration o 4.2 Installing and Starting syslog-ng o 4.3 Configuring syslog-ng Clients 4.3.1 Example 5-1 - Syslog-ng Sample Client Configuration 5 Simple syslog Security 6 ConclusionIntroductionThere are hundreds of Linux applications on the market, each with their own configuration files and help pages. Thisvariety makes Linux vibrant, but it also makes Linux system administration daunting. Fortunately, in most cases, Linuxapplications use the syslog utility to export all their errors and status messages to files located in the /var/log directory.This can be invaluable in correlating the timing and causes of related events on your system. It is also important to knowthat applications frequently dont display errors on the screen, but will usually log them somewhere. Knowing the precisemessage that accompanies an error can be vital in researching malfunctions in product manuals, online documentation, andWeb searches.syslog, and the logrotate utility that cleans up log files, are both relatively easy to configure but they frequently dont gettheir fair share of coverage in most texts. Ive included syslog here as a dedicated chapter to both emphasize its importanceto your Linux knowledge and prepare you with a valuable skill that will help you troubleshoot all the Linux variousapplications that will be presented throughout the booksyslog
  • 2|P ag esyslog is a utility for tracking and logging all manner of system messages from the merely informational to the extremelycritical. Each system message sent to the syslog server has two descriptive labels associated with it that makes the messageeasier to handle. The first describes the function (facility) of the application that generated it. For example, applications such as mail and cron generate messages with easily identifiable facilities named mail and cron. The second describes the degree of severity of the message. There are eight in all and they are listed in Table 5-1:You can configure syslogs /etc/rsyslog.conf configuration file to place messages of differing severities and facilities indifferent files. This procedure will be covered next.Table 5-1 Syslog Facilities Severity Level Keyword Description 0 emergencies System unusable 1 alerts Immediate action required 2 critical Critical condition 3 errors Error conditions 4 warnings Warning conditions 5 notifications Normal but significant conditions 6 informational Informational messages 7 debugging Debugging messagesThe /etc/rsyslog.conf FileThe files to which syslog writes each type of message received is set in the /etc/rsyslog.conf configuration file. Inolder versions of Fedora this file was named /etc/syslog.conf.This file consists of two columns. The first lists the facilities and severities of messages to expect and the second lists thefiles to which they should be logged. By default, RedHat/Fedoras /etc/rsyslog.conf file is configured to put mostof the messages in the file /var/log/messages. Here is a sample:*.info;mail.none;authpriv.none;cron.none /var/log/messagesIn this case, all messages of severity "info" and above are logged, but none from the mail, cron or authenticationfacilities/subsystems. You can make this logging even more sensitive by replacing the line above with one that captures allmessages from debug severity and above in the /var/log/messages file. This example may be more suitable fortroubleshooting.*.debug /var/log/messagesIn this example, all debug severity messages; except auth, authpriv, news and mail; are logged to the /var/log/debugfile in caching mode. Notice how you can spread the configuration syntax across several lines using the slash () symbol atthe end of each line. *.=debug;
  • 3|P ag e auth,authpriv.none; news.none;mail.none -/var/log/debugHere we see the /var/log/messages file configured in caching mode to receive only info, notice and warningmessages except for the auth, authpriv, news and mail facilities.*.=info;*.=notice;*.=warn; auth,authpriv.none; cron,daemon.none; mail,news.none -/var/log/messagesYou can even have certain types of messages sent to the screen of all logged in users. In this example messages of severityemergency and above triggers this type of notification. The file definition is simply replaced by an asterisk to make thisoccur.*.emerg *Certain applications will additionally log to their own application specific log files and directories independent of thesyslog.conf file. Here are some common examples:Files:/var/log/maillog : Mail/var/log/httpd/access_log : Apache web server page access logsDirectories:/var/log/var/log/samba : Samba messages/var/log/mrtg : MRTG messages/var/log/httpd : Apache webserver messagesNote: In some older versions of Linux the /etc/rsyslog.conf file was very sensitive to spaces and would recognize onlytabs. The use of spaces in the file would cause unpredictable results. Check the formatting of your/etc/rsyslog.conf file to be safe.Activating Changes to the syslog Configuration FileChanges to /etc/rsyslog.conf will not take effect until you restart syslog. Issue these command to do so :Systems using systemd:[root@bigboy tmp]# systemctl restart rsyslog.serviceSystems using sysvinit:[root@bigboy tmp]# service rsyslog restartIn older versions of Fedora, this would be:[root@bigboy tmp]# service syslog restartHow to View New Log Entries as They Happen
  • 4|P ag eIf you want to get new log entries to scroll on the screen as they occur, then you can use this command:[root@bigboy tmp]# tail -f /var/log/messagesSimilar commands can be applied to all log files. This is probably one of the best troubleshooting tools available in Linux.Another good command to use apart from tail is grep. grep will help you search for all occurrences of a string in a log file;you can pipe it through the more command so that you only get one screen at a time. Here is an example:[root@bigboy tmp]# grep string /var/log/messages | moreYou can also just use the plain old more command to see one screen at a time of the entire log file without filtering withgrep. Here is an example:[root@bigboy tmp]# more /var/log/messagesLogging syslog Messages to a Remote Linux ServerLogging your system messages to a remote server is a good security practice. With all servers logging to a central syslogserver, it becomes easier to correlate events across your company. It also makes covering up mistakes or maliciousactivities harder because the purposeful deletion of log files on a server cannot simultaneously occur on your loggingserver, especially if you restrict the user access to the logging server.Configuring the Linux Syslog ServerBy default syslog doesnt expect to receive messages from remote clients. Heres how to configure your Linux server tostart listening for these messages.As we saw previously, syslog checks its /etc/rsyslog.conf file to determine the expected names and locations of the log filesit should create. It also checks the file /etc/sysconfig/syslog to determine the various modes in which it should operate.Syslog will not listen for remote messages unless the SYSLOGD_OPTIONS variable in this file has a -r included in it asshown below.# Options to syslogd# -m 0 disables MARK messages.# -r enables logging from remote machines# -x disables DNS lookups on messages received with -r# See syslogd(8) for more details SYSLOGD_OPTIONS="-m 0 -r"# Options to klogd# -2 prints all kernel oops messages twice; once for klogd to decode, and# once for processing with ksymoops# -x disables all klogd processing of oops messages entirely# See klogd(8) for more detailsKLOGD_OPTIONS="-2"Note: In Debian / Ubuntu systems you

View more >