1. Monitoring Unicenter Service Deskwith Nagios Presented at the2011 Nagios World Conference North AmericabyKimbrough Henley Network Administration/Technology Supportand Solutions

2. My Background

Eighteen years in Information/Technology

3. Nagios user since 2000 4. Employed by Deloitte Services 5. for fifteen years 6. Solo I/T support person for 100+ staff 7. Unicenter Service Desk user 8. Disclaimer The views and opinions expressed in this presentation and discussion are solely those of the author and do not necessarily represent the opinions of Nagios Enterprises, LLC or any other parties. 9. Unicenter Service Desk

Primarily a help-desk incident management system

Sold by Computer Associates to diverse organizations

10. Includes support for asset and knowledge base management and change management 11.

Only ticket transfers generate notifications

12. No re-notification or auto escalation options 13. No notification based on a SLA 14. Limited three time periods available 15. Makes support staff more effective, productive, responsive, and compliant with policies Motivation 16.

Limitless ways to be notified

17. Customize checks for your precise needs 18. Some queues are checked less frequently 19. No budget necessary 20. Benefits the customer 21. Helps when out of the office or on the go More Motivation 22. The High Level OverviewAt its core, Nagios is a flexible task scheduling application.It schedules execution of plug-ins that return a result which is evaluated and when appropriate, other tasks are scheduled.The plug-in system for monitoring Service Desk simply fetches a page, parses the output, and returns a result back to Nagios .Hardware used in my environment is retired HP DL380 server, running quad Pentium 4's with 4 GB of RAM.. 23. The Result Do clever things such as:

• • • Check for and count tickets with a specific status
  • 24. Check for tickets about to expire
• 25. Check volume of tickets

26. Get notified when the CIO submits a help ticket

Simply by mimicking queries in the URL GET request to Service Desk, options for searching tickets are only limited to the SD application itself.Every organization has unique policies and needs. 27. The Players

Nagios

28. Service Desk Application 29. Your Nagios plug-ins for USD 30. The URL and session ID (SID) 31. Talking to USD

Screen Scraping the HTML (PDA) page

32. Screen Scraping the JAVA+HTML pages 33. Using Web Service/SOAP WSDL One can monitor Service Desk in different ways. 34. Active Checks 35. In Action 36. In Action 37. Service Desk JAVA+HTML 38. Service Desk PDA/HTML Only 39. Service Desk Search Form This is the form that the Nagios plug-in will submit via wget. 40. Notification Config 41. Components of a SD URL When viewing a group box, use control+right click to see properties of the SD frame.The URL is long and ugly:http://uscnt1115/CAisd/pdmweb.exe?SID=769292674&FID=858362218&OP=SEARCH&FACTORY=in&QBE.IN.status.sym=Open&QBE.EQ.priority=3&QBE.EQ.active=1&ADDITIONAL_WHERE=%28group.id%3DU%2710777DACA4C8F843BA340F152083348D%27%20AND%20type%3D%27I%27%29&KEEP.isHierSearch=0The complete database query that shows an inbox is in the URL.Note the QBE.EQ.priority=3 and WHERE group id....Connection variables in the URL that our plug-in scripts must be aware of are:

• Server's name

42. PDMWEB engine (identified by pdmweb.exe or pdmweb2.exe.)

43. SID or Session ID 44. FID or Function ID 45. Components of a SD URL Server name is self explanatory, but in some environments, it may be selected randomly from a pool when you first connect.The pdmweb.exe in the URL indicates what web engine is being used.In my environment, it may be either pdmweb.exe or pdmweb2.exe.The session ID is used to track, identify the type of client (HTML or JAVA), and expire connections to the server.The FID (Function ID) variable is not completely understood but has connections to identifying what type of client is in use as well as other purposes.However, it does not affect our plug-in's operation. 46. Setting the Connection Variables The plug in-needs a valid session.Once you have established a SD session in a browser, update a text file with the connection information to be used by the plug-ins.Values are stored in SD-PDA-Session.txt and SD-Browser-Session.txt.SID=1008374279 FID=208 server=uscnt1116 pdmweb=pdmweb2 For JAVA pages, the values needed are obtained by establishing a connection to the SD server, clicking on a group in the DASHBOARD, and viewing the properties of the framed page on the right.For HTML-only pages, switch user agents (to a bot), connect to the server, click Search Requests, and look in the URL.The FID should always be set to 1234 in SD-Browser-Session.txt. 47. Using Session ID's: HTML vs JAVA

Plug-in checks that use the simple PDA (HTML)-based web page do not require manually re-establishing a session. They recover when servers return online.Because of this, employing the PDA/HTML-only pages is preferred.

Complex searches from JAVA web pages require plug-in checks and a new session to be established in a browser.Then, connection variables should be updated in the SD session-information.txt file.

By using a user agent switcher for Firefox, establish a session that will respond with HTML only.

The plug-in is designed to parse one or the other type only.

48. Three Plug-ins In Use Today check-sdgroup-by-uuid.sh check-sdgroup-byname.sh check-sd-expiring.sh 49. Finding the Group's ID Looking at the inbox in SD, click Show Filter. 50. All the Pieces in Place 51. Check-Group-By-Name Syntax $ ./check-sdgroup-byname.sh US-Southeast-Memphis 1 ALERT: 2 open ticket(s) for US-Southeast-Memphis.|open=2 Request List 2 Matches ____________________________________________________________ Req #:4459981 End User: Doe, Lawrence Status:Open Priority: 3-Medium Summary: Outlook : PST size exceeded ____________________________________________________________ Req #:4452311 End User: Doe, Brad J Status:Open Priority: 3-Medium Summary: Unable to access FTS site. 52. Check-Group-by-UUID Syntax If the queue name has spaces or odd characters in it, the check-sdgroup-by-uuid.sh should be used instead.$ ./check-sdgroup-pda.sh 85751D18EFF2EC4C83528C636F4952CA 1 Hermitage ALERT: 1 open ticket(s) for Hermitage.|open=1 Request List 1 Match ______________________________________________________________ Req #:4371019 End User: Bronk, Benjy Status:Open Priority: 3-Medium Summary: dPrint - unable to log into dPrint pages for WFC office 53. Inside the Plug-ins In order to find tickets with an open status and a medium priority,the URL we would wget looks like this:wget -T 6 -t3 -q -O output-$3.html user=fred password=martian http:// $server /CAisd/ $pdmweb.exe ?SID= $SID +FID= $FID +OP=SEARCH+QBE.EQ.status=OP +QBE.EQ.priority=3+FACTORY=cr+KEEP.where_clause=group.id%3DU%27 $1 %27 n. b. I have explored using CURL instead of WGET.It was effective but not an improvement. 54. Cleaning The Response The HTML page returned by WGET contains what we want to know, but it is burred in HTML and /or JAVA code. The plug-in must clean it up.

For PDA page cleaning:

open=`grep -a title2 output-$3.html| cut -d ">" -f2| sed 's/[^0-9]*//g'`

For browser-based page cleaning:

grep -a rs.data output-$3.html |sed -e 's/rs.data//' | tr -d ('); | sed -e 's/ref_num_style//' | tr -d r | grep -v date_to | grep -v '^$' >output-$3.clean These work and require no modifications for different search parameters. It is not elegant, but it does yield results. 55. Notification Sample From a HTML/PDA based plug-in's results: ALERT: 1 open ticket(s) for Hermitage.|open=1 Request List 1 Match_______________________________________________________ Req #:4371019 End User: Doe, William David Status:Open Priority: 3-Medium Summary: dPrint - unable to log into dPrint pages for WFC office 56. Checking Expiring Tickets Using the Java based search pages in Service Desk, you can search for tickets created on a specific date.Implement a script (sample included) to determine what N business days ago is, you can find tickets due to expire. $ ./check-sd-expiring.sh US-Southeast-Memphis 1 Memphis HDR tickets expiring on servicedesk. ALERT: 1 HDR ticket(s) expire today. 4374185, Open 5-Negotiated US-Southeast-Memphis 57. Notification Sample Birmingham HDR tickets expiring on Service Desk. ALERT: 1 HDR ticket(s) expire today. 4222282, Open 5-Negotiated US-Southeast-Birmingham Name, Tech Assigned to 08/05/2011 HD Data Retention: Doe, John M. Doe, John M. Practice Protection.Data Retention.Hard Drive IT Request USHDC1432 MEMPHIS YES 9015551234 58. Host Definition define host{ usegeneric-host host_nameservicedesk aliasServicedesk addressservicedesk contact_groupsmem-admins check_commandcheck-host-alive parentsrouter1 } 59. Service Check Template define service{ namesd-open-tickets usegeneric-service host_nameservicedesk notification_interval240 notification_optionsc check_periodet flap_detection_enabled0 retry_interval1 max_check_attempts2 servicegroupsservicedesk register0 } 60. Service Check Definitions define service { usesd-open-tickets notification_optionsc notification_period24x7 check_period24x7 service_descriptionMemphis Open Tickets check_commandcheck-sdgroup-byname!US-Southeast-Memphis!1 contact_groupsmem-admins } define service { usesd-hdr-tickets notification_optionsc notification_period24x7 check_period24x7 service_descriptionMemphis HDR tickets expiring check_commandcheck-sd-expiring!US-Southeast-Memphis contact_groupsmem-admins } Generallynotification_options should be C only but admins may want C,U,R. 61. Customizing Notifications You may want to change how notifications appear, such as leaving out CRITICAL in the results, or the $SERVICE_STATE$ macro in the notification command.The notification_command (how a person is notified) is an attribute of the contact person definition.Therefore, if a contact is getting notifications from other service checks, those notifications would be affected too.You can't have a different notification style for one different service checks. You must setup an additional contact for Service Desk checks only. 62. Graphing with PNP4Nagios The plug-in will produce performance data that can be graphed: $ ./check-sdgroup-byname.sh US-Southeast-Memphis 1 ALERT: 2 open ticket(s) for US-Southeast-Hermitage.|open=2 63. Graphing with PNP4Nagios Here, the output of 5 check-group plug-ins is combined on one graph. For best results, poll every 5 minutes, 24 hours a day. 64. Web Services Background

• Web Services Description Language is a set of data exchange standards that enable communication between applications, independent of their platform.

65. .

Revolutionized the way B2B applications exchange data

Functions or methods are published on a Web server and can be invoked remotely

Uses XML messaging based on standards such as WSDL, UDDI, and SOAP

Complex with many protocols in place, making deployment and development difficult

66. Service Desk's WSDL Service Desk's WSDL is running Apache-Axis 1.x(updated last in 2006) http://server:8080/axis/services/USD_WebServiceSoap?wsdl

When developed, the best way to query Service Desk

67. Analogous to an API 68. Access may be restricted in a number of ways via permissions inthe application 69. To Do

• Develop a plug-in to use the SOAP/Web Services portal

70. Automate session ID renewal for the plug-ins using Java pages

71. Consolidate into one script that will use command line input to select multiple search parameters (such as priority, status, incident area, creation date). 72. Troubleshooting/Development Tools WFetch Mozilla Live HTTP Headers (Firefox plug-in) WireShark Fiddler2 SoapUI The CA Service Desk Web Services User Guide may be useful for developing a Web Services-based plug-in. 73. To Summarize

Monitoring SD queues is very possible, in several ways

74. Today, plug-ins that use the HTML only pages are preferred. 75. Customizable for your needs 76. Package is available on Nagios Exchange website 77. Package Includes all the scripts discussed