troubleshoot enterprise portal 6.0 sp2 - sap · troubleshoot enterprise portal 6.0 sp2 part i:...
TRANSCRIPT
1
Troubleshoot Enterprise Portal 6.0 SP2
Part I: Master the Basics
SAP AG 2003, Title of Presentation, Speaker Name /2
What are EP 6.0 SP2 Troubleshooting Basics?
Essential knowledge, tools and methods which answer the following questions:
nOn which component should I focus troubleshooting effort?
nHow can I get more information about the problem?
nWhat to do when I identify the problem?
2
SAP AG 2003, Title of Presentation, Speaker Name /3
SAP Enterprise Portal 6.0
EP 6.0 SP2 Software Layers
Operating System
Java Virtual Machine 1.3.1 (SUN)
SAP J2EE Engine 6.20
Portal Framework
Portal Runtime - PRT
Portal Components(e.g. iViews)
Portal Services(e.g. http service)
SAP AG 2003, Title of Presentation, Speaker Name /4
EP 6.0 SP2 Architecture
3
SAP AG 2003, Title of Presentation, Speaker Name /5
User Management Architecture
SAP AG 2003, Title of Presentation, Speaker Name /6
User Management Architecture (II)
4
SAP AG 2003, Title of Presentation, Speaker Name /7
File System Structure (I)
sap j2eeusr <SID 1> j2ee_00 dispatcher
<SID 2>
j2ee_01
<SID n>
j2ee_n
server2
server
server n
J2EE-EngineFile System
cluster
admin
analyzer
configtool
ume
SAP AG 2003, Title of Presentation, Speaker Name /8
File System Structure (II)
dispatcher
services servlet_jsp
http
managers
jms
dbms
telnet
ThreadM.
Sys.Thr.M.
ClusterM.
ServiceM.
server
server2
server n
irj
J2EE-Engine
htmlb
work jspTemp
mobilehtmlb
J2EE Cluster Personality
settings
Console_logs
5
SAP AG 2003, Title of Presentation, Speaker Name /9
File System Structure (III)
root WEB-INF work portalirj lib
logs
portalapps
system
J2EE-Engine
SAP AG 2003, Title of Presentation, Speaker Name /10
Portal Support iViews
6
SAP AG 2003, Title of Presentation, Speaker Name /11
Portal Logging Console
SAP AG 2003, Title of Presentation, Speaker Name /12
What We’ve Learned So Far
nKnowing and understanding the EP 6.0 software layers and architecture helps isolate troubleshooting efforts in the integrated environment.
nObjects to assist troubleshooting exist in the file system:n SAP J2EE, java, and EP 6 output log files
n Property files which control SAP J2EE and EP 6 behavour
n SAP tools for controlling and analyzing error reporting
n SAP tools for visualizing configuration
nThe portal provides troubleshooting pages and iViews:n System Administration > Support
n System Administration > Monitoring > Logging Console
7
SAP AG 2003, Title of Presentation, Speaker Name /13
Troubleshooting Approach
Portal Error
StartupProblem?
ContentProblem?
WebserverProblem?
ClientProblem?
Auth.Problem?
Connect.Problem?
SAP J2EE Engine/Oper Sys?
Database/PCD?
UME/Persistence?
Browser/Network?
Network Config.?
Backend Sys Down?
SAP AG 2003, Title of Presentation, Speaker Name /14
Troubleshooting Approach: SAP J2EE Engine
EP inaccessible!http://portal.mycompany.com/irj/ -> ERROR
J2EE-Engine started?-> Java Processes (Nodes) running?
-> J2EE Documentation at http://portalX.mycompany.com:50000 available?
Just 1 Server or all Servers concerned?Connect to each Portal Server directly:
http://portalX.mycompany.com/irj/
Are all Nodes visible in the cluster?-> Login to J2EE-Engine via telnet („lsc“)
or J2EE Administrator Tool
-> Analysis of Console Logs of failed Node
Detailed Logging and Logfile analysis as necessary
8
SAP AG 2003, Title of Presentation, Speaker Name /15
Portal Error: Test each portal server
SAP AG 2003, Title of Presentation, Speaker Name /16
J2EE Engine Doc: http://portal.mycompany.com:50000 & java processes
9
SAP AG 2003, Title of Presentation, Speaker Name /17
Check SAP J2EE Status with Telnet Admin/Visual Admin
SAP AG 2003, Title of Presentation, Speaker Name /18
SAP J2EE Engine Console Logs
10
SAP AG 2003, Title of Presentation, Speaker Name /19
Portal/PRT LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/services/servlet_jsp/jspTemp/irj/root/WEB-INF/portal/logsExamples:acl_logger, http_logger, iview_logger, page_logger, ...
Logging in the Portal: Overview
The Portal consists of a number of components that work together.
è Different Components have separate Logging functions.
J2EE-Engine LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/console_logs/Examples:output.log, error.logPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/log/Examples:INFO.log, WARNING.log ERROR.log, EMERGENCY.log, ALERT.log
SAP AG 2003, Title of Presentation, Speaker Name /20
Logging: J2EE-Engine/Console Logs
J2EE-Engine - Console Logs:
Name: YYYY_DD_MM_at_HH_MM_SS_output.log
Path: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/console_logs/
Activation: console_logs.properties
Contains: Console Output, e.g. Startup messages, Garbage Collection Trace, Thread Dumps
Name: YYYY_DD_MM_at_HH_MM_SS_error.log
Path: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/ managers/console_logs/
Activation: console_logs.properties
Contains: Error and exceptions Messages about all kinds of errors and exceptions
Important: The setting “DaysToKeepLogs” in “console_log.properties” specifies how long (in days) to store the log files (default=7).
11
SAP AG 2003, Title of Presentation, Speaker Name /21
Logging: J2EE-Engine/Output.log (I)
Example for an output.log of a server node:
SAP J2EE Engine Version 6.20 PatchLevel 67440.20 is starting...
Loading: LogManager ... Loading: SystemThreadManager ... Loading: ThreadManager ... Loading: TimeoutManager ... Loading: MemoryManager ... Loading: PoolManager ... Loading: PolicyManager ... Loading: IpVerificationManager ... Loading: ClusterManager ... Loading: ClassLoaderManager ... Loading: SwapManager ... Loading: LockManager ... Loading: R3StartupManager ... Loading: ServiceManager ...
Loading core services:Starting core service monitor ... done.Starting core service p4 ... done.Starting core service file ... done.Starting core service log ... done.Starting core service dbms ... done.Starting core service security ... done.Starting core service naming ... done.Starting core service admin ... done.Starting core service deploy ... done.
Core services loaded successfully.
SAP AG 2003, Title of Presentation, Speaker Name /22
Logging: J2EE-Engine/Output.log (II)Loading additional services:
Starting service telnet ... done.Starting service ts ... done.Starting service appclient ... done.Starting service runtimeinfo ... done.Starting service javamail ... done.Starting service pinger ... done.Starting service logviewer ... done.Starting service shell ... done.Starting service eisconnector ... done.Starting service http ... done.Starting service PRTBridge ... done.Starting service jms ... done.Starting service dbpool ... done.Starting service rfcengine ... done.Starting service ejb ... done.
Registering Channels handler...PcdAccessService.init()PcdAccessService.afterInit()Initialization IViewsPersonalizationUpgraderService was successful
Starting service servlet_jsp ... done.Additional services loaded successfully.
SAP J2EE Engine Version 6.20 PatchLevel 67440.20 is running!
PatchLevel 67440.20 December 11, 2003 08:40 GMT
>Login :Element 3002 joined.Element 264395723 joined.Element 264395722 joined.Element 3020 joined.
12
SAP AG 2003, Title of Presentation, Speaker Name /23
Logging: J2EE-Engine/Log Manager – Log Service
J2EE-Engine – Log Manager:
Names: DEBUG.log, TRACE.log, INFO.log, NOTICE.log, WARNING.log, ERROR.log, CRITICAL.log, ALERT.log, EMERGENCY.log
Path: e.g./usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/cluster//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/memory//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/r3startup//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/service//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/timeout/
Activation: via J2EE Administrator Tool or via properties-file of manager
Contains: Specific information on Manager depending on the log level.
J2EE-Engine – Log Service:
Names: DEBUG.log, TRACE.log, INFO.log, NOTICE.log, WARNING.log, ERROR.log, CRITICAL.log, ALERT.log, EMERGENCY.log
Path:/usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/services/log/work/
Activation: via J2EE Administrator Tool or ConfigtoolContains: Specific information on Service depending on the log level.
SAP AG 2003, Title of Presentation, Speaker Name /24
Logging: J2EE-Engine/Log Level
The log system recognizes the following types of events:
• EMERGENCY (log level 0) – System is unusable
• ALERT (log level 1) – Immediate action must be taken
• CRITICAL (log level 2) – Critical conditions
• ERROR (log level 3) – Error conditions
• WARNING (log level 4) – Warning conditions
• NOTICE (log level 5) – Normal but significant events for the system
• INFO (log level 6) – Information
• TRACE (log level 7) – Events that occur as a result of application methods execution
• DEBUG (log level 8) – Debug-level messages,
And accordingly creates nine log file types with the these names.
13
SAP AG 2003, Title of Presentation, Speaker Name /25
Logging: J2EE-Engine/Logging Configuration
1) Select the Manager or the Log Service
2) Set values in Properties
SAP AG 2003, Title of Presentation, Speaker Name /26
Logging: J2EE-Engine/Built-In Log Viewer
2) Select Type
of Messages
For built-in Log Viewer:
1) Select Tab „Log Viewer“
Works for all Managers, the Services, awhole Node or the whole Cluster!
14
SAP AG 2003, Title of Presentation, Speaker Name /27
Logging: J2EE Engine/HTTP Service Logging
J2EE-Engine – HTTP Service:
Names: DEBUG.log, TRACE.log, INFO.log, NOTICE.log, WARNING.log, ERROR.log, CRITICAL.log, ALERT.log, EMERGENCY.log
Path:/usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/services/http/log/
Configuration: via J2EE Administrator Tool or Configtool
Contains: Information related to the HTTP service on the log level.
The HTTP Service is one of the most important services and it has some special issues:
2004-03-26 10:47:38 | http | INFO | | 10.20.19.37 | 10.20.19.37 - - [26/Mar/2004:10:47:38 +0100] "GET /irj/index.html" 200 0 [187] |
2004-03-26 10:48:00 | http | INFO | | 10.20.19.37 | 10.20.19.37 - - [26/Mar/2004:10:48:00 +0100] "GET /irj/servlet/prt/portal/prtroot/com.sap.portal.navigation.portallauncher.default" 200 8853 [21953] |
Example messages taken from an services/http/log/INFO.log
SAP AG 2003, Title of Presentation, Speaker Name /28
HTTP Trace
All http communication can be traced from the HTTP Service of the Dispatcher Node for problem analysis.
Configuration is done in: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/dispatcher/services/http/properties
The following parameters have to be adjusted:
HttpTrace =[disable|testFull|textHeaders|hexFull|hexHeaders]
HttpTraceRequestOutput=File name or empty for Standard Output
HttpTraceResponseOutput=File name or empty for Standard Output
The HTTP Trace is disabled by default.
For all details about the HTTP Trace please check Note 657765.
15
SAP AG 2003, Title of Presentation, Speaker Name /29
Logging: J2EE-Engine/Log Viewer iView
Viewing J2EE Log Files within the Portal:
ð System Administrationð Supportð SAP J2EE Log Viewerð Log Viewer for J2EE 620
(The corresponding Logger has to be activated.)
Portal has to be up and running!
SAP AG 2003, Title of Presentation, Speaker Name /30
Logging: J2EE-Engine/Log Viewer iView
Searching in the Log File is possible!
16
SAP AG 2003, Title of Presentation, Speaker Name /31
Logging: J2EE-Engine/Standalone Log Viewer (Server)
The Standalone Log Viewer:è Client/Server Tool to conveniently access Log Files
independently from the J2EE-Engineè Also usable if J2EE-Engine is down!
Installation: Available as “logviewer-standalone.zip” in /usr/sap/<SID>/j2ee/j2ee_00/admin/ or as download from SAP Service Marketplace.
è Extract file on the Portal Serverè Set JAVA_HOME and LV_HOMEè Customize LogViewerServer.properties, especially parameters:
Logviewer_LogDirectoryReadFromFile= andLogviewer_LogDirectoryWriteToFile=
è Execute server.shè Run client.bat or client.sh on your client machine
SAP AG 2003, Title of Presentation, Speaker Name /32
Logging: J2EE-Engine/Standalone Log Viewer (Client)
With the Standalone Log Viewer can you view, filter and merge log files from a number of nodes in one place!
17
SAP AG 2003, Title of Presentation, Speaker Name /33
Troubleshooting Approach: Portal Solution Path
EP shows Error Messages!
Check Logs of active Portal Loggers within the Portal or on the File System
Error in Customer Development or SAP Standard Functions?
Activate related Loggers and analyze the messages
Check J2EE-Engine Console Logs
SAP AG 2003, Title of Presentation, Speaker Name /34
Logging in the Portal: Overview
The Portal consists of a number of components that work together.
è Different Components have separate Logging functions.
J2EE-Engine LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/console_logs/Examples:output.log, error.logPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/log/Examples:INFO.log, WARNING.log ERROR.log, EMERGENCY.log, ALERT.log
Portal/PRT LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/services/servlet_jsp/jspTemp/irj/root/WEB-INF/portal/logsExamples:acl_logger, http_logger, iview_logger, page_logger, ...
18
SAP AG 2003, Title of Presentation, Speaker Name /35
Logging: Portal/PRT Logging - Display Mode
Viewing Portal/PRT Logs from within the Portal:ð System Administrationð Supportð Portal Runtimeð Log Admin
Portal has to be up and running!
Select NodeSelect LoggerSelect Log LevelClick „Display“
SAP AG 2003, Title of Presentation, Speaker Name /36
Logging: Portal/PRT Logging - Configuration
Configuration of Portal/PRT Logging within the Portal:ð System Administrationð Supportð Portal Runtimeð Log Adminð Configuration
Alternatively the Portal/PRT Logging can be modified in file “logger.xml” in:
cluster/server/services/servlet_jsp/work/jspTemp/irj/root/WEB-INF/portal/system/xml/
19
SAP AG 2003, Title of Presentation, Speaker Name /37
Logging: Portal/PRT Logging – Log Levels
Log Levels in Portal/PRT Logging:§ OFF§ SEVERE§WARNING§ INFO§ ALL
SAP AG 2003, Title of Presentation, Speaker Name /38
Logging: Portal/PRT Logging – Output Types
Writes log messages in a file in XML format.XmlFileLogger
Writes log messages in a file. The format can be specified.SimpleFileLogger
Writes log messages in a file with a simple hash-separated-field format.
ListFileLogger
Writes log messages to the console.ConsoleLogger
DescriptionLogging Class
Loggers can be set up to put out messages in different log levels to different destinations.
This can be easily seen in the logger.xml:
20
SAP AG 2003, Title of Presentation, Speaker Name /39
Logging: Portal/PRT Log and File Viewer
Viewing files from within the portal:ð System Administrationð Supportð Portal Runtimeð Log Viewer
Portal has to be up and running!
By browsing the file system any file can be displayed in the browser from this iView!
SAP AG 2003, Title of Presentation, Speaker Name /40
Logging: Portal/PRT Log and File Viewer
The Log and File Viewer
can make problem
analysis a lot easier!
21
SAP AG 2003, Title of Presentation, Speaker Name /41
PCD: Object Locking Tool
The Object Locking Tool enables you to locate objects being edited and currently in locked mode.
Path: System Administration -> Monitoring -> Portal -> Object Locking
SAP AG 2003, Title of Presentation, Speaker Name /42
Preventive Measures: java virtual machine parameters
nProper fine tuning of java memory parameters prevents:n SAP J2EE crashes and instabilityn Poor portal performancen java “out of memory (OOM)” errorsn Implement the SAP-provided How-To Guide “Fine-
Tuning Performance of EP 6.0 SP2” section “Tuning the Java Virtual Machine” immediately after installation.
nReview and understand CSN notes: 552522, 634689, 597187, 610134, 667841, 667711
nImplement java garbage collection output to console logs with “–verbose:gc” in SAP J2EE server (not dispatcher) startup.n Always attach console error log with verbose garbage
collection output to any CSN message.
nRemove java “–server” parameter from server startup to improve stability of SAP J2EE.
22
SAP AG 2003, Title of Presentation, Speaker Name /43
Summary
Enterprise Portal 6.0 SP2 exists in a constellation of technologies. Understanding these software layers and their implementation leads to faster problem identification and solution.
Troubleshooting a portal error can take a J2EE Engine approach or a portal approach.
SAP provides many tools and settings, especially loggers and logviewers, for analyzing problems in the portal. Fluency in thesetools improves analysis and solution timeframes.
Setting up portal environments that report useful information for problem analysis and submitting the correct information with a CSN message leads to increased problem resolution efficiency.
Preventive measures are better than curatives.
SAP AG 2003, Title of Presentation, Speaker Name /44
End of Troubleshooting Part I: Master the Basics
Q & A?