retail system software help desk - usps oig · evaluation of management’s comments ... this...

Cover

Retail System Software Help Desk

Report Number CP-AR-16-006June 27, 2016

Audit Report

BackgroundThe U.S. Postal Service started deployment of the Retail System Software (RSS) solution in fiscal year (FY) 2014, to replace the existing retail point-of-service software at about 18,000 Post Office locations. RSS was deployed to reduce the number of vendors involved in the Postal Service’s retail solutions and lower its operational and maintenance costs.

The NCR Corporation has provided help desk services for the Postal Service’s retail point-of-service software since 2008 and will continue to do so for RSS through September 2016.

In FY 2015, the U.S. Postal Service Office of Inspector General issued a report recommending the Postal Service evaluate metrics and ensure the RSS help desk is adequately staffed with knowledgeable personnel and able to support national deployment.

The Postal Service uses metrics to measure the RSS help desk. Our audit focused on metrics for the Tier One help desk (which is the point of entry for customer issues): (1) abandoned call rate, which occurs when a caller hangs up before being connected to the help desk agent, (2) average speed to answer, which measures how long it takes for the help desk to answer the call, and (3) average handle time, which is the amount of time it takes from customer initiation of the call to completion of related tasks that follow the call.

We selected the average speed to answer metric and abandoned call rate because they were the two primary metrics in the RSS help desk contract for which we could assess performance. Further, we evaluated the average handle time because it is a help desk efficiency measure.

Our objective in this follow-up audit was to determine whether the help desk is adequate to support RSS.

What The OIG FoundThe Postal Service is meeting or exceeding performance metrics for the RSS Tier One help desk; however, opportunities exist to improve performance by strengthening established metrics and adding an average handle time performance metric. The help desk showed continuous improvement in Tier One metrics from August 2015 to February 2016. Specifically:

■ Abandoned call rates have declined from a high of 36 percent in August 2015 to a low of about 1 percent in February 2016, while the target is 8 percent or less. A lower abandoned call rate could result in higher cost per contact.

■ Average speed to answer time has decreased from an average of about 26 minutes (August 2015) to an average of 44 seconds (February 2016), which exceeds the established metric of 90 seconds. Although the Postal Service is meeting the average speed to answer performance target, it could further improve customer satisfaction and help desk efficiency by reducing the performance target.

Highlights

The Postal Service is meeting

or exceeding performance

metrics for the RSS Tier

One help desk; however,

opportunities exist to improve

performance by strengthening

established metrics and

adding an average handle

time performance metric.

Retail System Software Help Desk Report Number: CP-AR-16-006 1

The Postal Service is able to track help desk handle time, but it does not have a metric to do this. Without a targeted performance metric, the Postal Service is at increased risk of under- or over-staffing the RSS help desk operation.

The RSS help desk efficiency and customer satisfaction could be negatively impacted without establishing and monitoring adequate performance targets.

What The OIG RecommendedWe recommended management re-evaluate help desk metrics to increase performance for future help desk agreements, to include establishing an average handling time performance target.


Transmittal Letter

JUNE 27, 2016

MEMORANDUM FOR: MICHAEL J. AMATO VICE PRESIDENT, ENGINEERING SYSTEMS

FROM: John E. Cihota Deputy Assistant Inspector General for Finance, Pricing, and Investments

SUBJECT: Audit Report – Retail System Software Help Desk (Report Number CP-AR-16-006)

This report presents the results of our audit of the Retail System Software Help Desk (Project Number 16TG009MI000).

We appreciate the cooperation and courtesies provided by your staff. If you have any questions or need additional information, please contact Charles Turley, director, Cost, Pricing, and Investments, or me at 703-248-2100.

Attachment

cc: Corporate Audit and Response Management

E-Signed by John CihotaVERIFY authenticity with eSign Desktop


Table of Contents


CoverHighlights ......................................................................................................1

Background ................................................................................................1What The OIG Found ................................................................................. 1What The OIG Recommended ..................................................................2

Transmittal Letter ..........................................................................................3Findings ........................................................................................................5

Introduction ................................................................................................5Summary ....................................................................................................5Abandoned Call Rate .................................................................................7Average Speed to Answer .........................................................................8Average Handling Time ..............................................................................9

Recommendations......................................................................................11Management’s Comments ......................................................................11Evaluation of Management’s Comments ................................................11

Appendices .................................................................................................12Appendix A: Additional Information ..........................................................13

Background ..........................................................................................13Objectives, Scope, and Methodology ..................................................13Prior Audit Coverage ............................................................................14

Appendix B: Management’s Comments ...................................................15Contact Information ....................................................................................16

Findings IntroductionThis report presents the results of our self-initiated audit of the Retail System Software Help Desk (Project Number 16TG009MI000). Our objective was to determine if help desk performance was adequate to support the Retail System Software (RSS). We focused our audit on three metrics for the Tier One1 help desk: abandoned call rate, average speed to answer, and average handle time. See Appendix A for additional information about this audit.

The RSS is a single software solution that provides point-of-sale business functionalities that support all physical retail channels. The U.S. Postal Service started deployment of RSS in fiscal year (FY) 2014, to replace its current software applications to about 18,000 Post Office locations. The NCR Corporation has been a strategic partner of the Postal Service since 1996, providing help desk services for the Point-of-Service (POS) and POS One Consolidated Help Desk. The NCR Consolidated Help Desk provided help desk services and support for RSS during our review.

In FY 2015, the U.S. Postal Service Office of Inspector General (OIG) issued a report2 recommending the Postal Service evaluate and establish metrics to ensure the RSS help desk is adequately staffed with knowledgeable personnel who could support the system as it is deployed nationally. This is a follow-up audit to identify whether the RSS help desk is adequately supporting RSS.

The Postal Service uses metrics to measure the RSS help desk performance. There were three Tier One metrics in which the NCR Consolidated Help Desk is measured in the contract and one metric that the Postal Service tracked informally that we believed assists in evaluating staffing needs. Our audit focused on two of three metrics for the Tier One help desk contract in which we could assess performance and one because it is a help desk efficiency measure: (1) abandoned call rate, which occurs when a caller hangs up before being connected to the help desk agent, (2) average speed to answer; (ASA)3 , which measures how long it takes for the help desk to answer the call, and (3) average handle time; which is the amount of time it takes from customer initiation of the call to completion of related tasks that follow the call.

SummaryThe Postal Service is meeting or exceeding the RSS Tier One help desk abandoned call rate and ASA performance metrics; however, opportunities exist to improve performance by strengthening established metrics and adding an average handle time (AHT)4 performance metric. The program office has focused on enhancing the RSS help desk operations, leading to the overall improvement in these metrics.

The abandoned call rate has declined from a high of 36 percent in August 2015 to a low of about 1 percent in February 2016, while the target is 8 percent or less. Although there is no impact on customer satisfaction, a lower abandoned call rate could result in higher cost per contact, if the Postal Service did not have a firm-fixed-price contract with the NCR Corporation as it does now.5

In addition, the ASA has significantly improved from an average of 26 minutes in August 2015 to an average of 44 seconds in February 2016 while converting 1,645 POS One to RSS sites in January and February 2016. Although the Postal Service is

1 The RSS help desk consists of four tiers to manage customer issues; we focused on Tier One. Tier One is the point of entry for RSS incidents and addresses issues relating to hardware, software, and policies and procedures.

2 Retail System Software Deployment and Functionality (Report Number MI-AR-15-002, dated March 3, 2015).3 A call center metric for the average amount of time it takes for calls to be answered in a call center during a specific time period.4 A call center metric for the average duration of one transaction, typically measured from the customer’s initiation of the call including any hold time, talk time and related

tasks that follow the transaction.5 NCR base contract 1BITRT-09-B-0001, effective date December 1, 2008, with Mods 76 and 77 for 2015 and 2016.

The Postal Service is meeting

or exceeding the RSS Tier One

help desk abandoned call rate

and average speed to answer

performance metrics; however,

opportunities exist to improve

performance by strengthening

established metrics and

adding an average handle

time performance metric. The

program office has focused on

enhancing the RSS help desk

operations, leading to the overall

improvement in these metrics.


meeting the ASA performance target of 90 seconds, it could improve customer satisfaction and help desk efficiency by reducing the performance target to 60 seconds.

The Postal Service is able to track help desk AHT, but does not do so. The RSS help desk AHT was 19 minutes in September 2015 and improved to 15 minutes in February 2016. The AHT measures how efficiently operators answer and close calls and is an indicator in assessing staffing needs. Using this metric would help ensure RSS help desk operations are adequately staffed to support call volume.


Abandoned Call RateThe RSS help desk abandoned call rate performance target is 8 percent or less. For the July 2015 through February 2016 period, the abandoned call rate6 went from a high of 36 percent in August 2015 to a low of about 1 percent in February 2016. Between August and December 2015, the Postal Service postponed deploying RSS and added additional help desk staffing, which contributed to the improvement in this metric.

The Help Desk Institute (HDI) 2015 report7 indicates the industry median abandoned call rate is 5 percent. A lower abandoned call rate could result in higher cost per contact. There is an opportunity with upcoming contract negotiations to align the abandoned call rate metric with the industry average. Figure 1 and Figure 2 illustrate call volume and percentage of dropped or abandoned calls.

Figure 1. Call AbandonmentAverage Daily Volume on a Monthly Basis

Jul-15 Aug-15 Sep-15 Oct-15 Nov-15 Dec-15 Jan-16 Feb-16

Call Volume 463 591 466 377 357 388 351 327

Dropped 29.09% 36.10% 25.80% 12.31% 8.31% 2.66% 7.31% 1.21%

Figure 2. Call Abandonment vs Performance


Source: Postal Service help desk performance data.

6 An abandoned call occurs when a caller hangs up before being connected to a live help desk agent.7 HDI 2015 Support Center Practices and Salary Report.

Average Speed to AnswerThe RSS help desk performance target for the ASA is 90 seconds. For August 2015, the ASA averaged about 26 minutes, with at least 4 days exceeding 40 minutes 8. Management attributed this lengthy wait time to transitioning the RSS help desk from one incident management system to another (from Remedy to ServiceNow), converting a large number of sites from POS One 9 to the RSS, and upgrading hardware (printers, Uninterrupted Power Supply). In August 2015, the Postal Service stopped deploying RSS to allow for an adjustment period to the new incident management system. In October 2015, additional staffing was provided to the help desk operations.

In January 2016, the Postal Service resumed deployment, converting an additional 1,645 locations from POS One to RSS by the end of February 2016. During the conversion period, the ASA was reduced to 44 seconds, which exceeded the performance metric by 46 seconds. Figure 3 illustrates the monthly ASA, and Figure 4 illustrates the monthly RSS conversions.

In comparison to industry standards, the HDI 2015 report shows support centers industry-wide target of 60-90 seconds for their ASA. Under the previous NCR POS One contract, the ASA performance target was 60 seconds, but was adjusted to 90 seconds10. As the Postal Service is meeting the ASA performance target, it could improve customer satisfaction and help desk efficiency by reducing the performance target.

Figure 3. RSS Help Desk ASA


Source: Postal Service.

8 Occurred on August 3, 4, 10 and 25, 2016.9 The POS One terminal is the primary hardware and software system used to conduct sales transactions during the Post Office check-out process.10 In contract modifications 76 and 77, ASA performance was set at 90 seconds.

Figure 4. RSS Conversions


Source: Postal Service Enterprise Data Distribution. 11

Average Handling TimeThe RSS help desk does not have a performance target for AHT. The call center metric for AHT is the average duration of one transaction, measured from the customer’s initiation of the call, including any hold time, talk time, and related tasks that follow the transaction. AHT is a prime factor when assessing call center staffing levels.The HDI industry AHT is 8-10 minutes.12 In September 201513, we identified a RSS AHT high point of 19 minutes. In February 2016, the AHT had improved to 15 minutes.

While the Postal Service has the ability to track AHT, it does not regularly monitor or use this performance metric because it could negatively influence first call resolution rates. For example, AHT could influence the RSS help desk staff to hurry customers off the phone without a call resolution. AHT is an efficiency indicator to assess adequate staffing needs. Using it can help ensure that the RSS help desk is adequately staffed. Figure 5 illustrates RSS AHT compared to HDI industry average.

11 A data management system used to manage updates to national data files for the Automated Postal Center, POS, RSS, and Integrated Retail Terminals.12 The depiction in Figure 4 uses the most conservative baseline average of 10 minutes.13 Analyzed data from July 2015 to February 2016.

Figure 5. Monthly AHT


Source: Postal Service HDI 2015 Support Center Practices and Salary Report.

The current NCR contract, which includes RSS help desk services, expires in September 2016. At that time, the Postal Service will have to negotiate another extension or contract to continue tier I help desk services. This is an opportunity to evaluate performance metrics and negotiate new or revised metrics. The Postal Service should consider the costs associated with incorporating new or improved metrics versus acceptable customer service levels and expectations.

Recommendations We recommend the vice president, Engineering Systems:

1. Re-evaluate help desk metrics for the upcoming contract or extension to establish an average handle time performance target and adjust the abandoned call rate and average speed to answer performance target.

Management’s Comments Management agreed, in part, with the recommendation to re-evaluate help desk metrics for the upcoming contract or extension to establish an average handle time performance target and adjust the abandoned call rate and ASA performance target.

Management agreed to consider the specific metrics recommended in this report as the RSS program continues to assess help desk functions. Management will evaluate and adjust metrics and other performance targets if necessary to balance a cost-effective help desk metrics while remaining responsive to operational needs.

Management plans to complete corrective action by October 1, 2017. See Appendix B for management’s comments in their entirety.

Evaluation of Management’s CommentsAs management agreed in part, they did not specifically identify disagreement with any aspects of the recommendation. Management corrective actions indicate metrics identified in this report, as well as existing metrics, will be taken into consideration while assessing the help desk function. The re-evaluation of metrics provide opportunities to strengthen overall help desk performance.The U.S. Postal Service Office of Inspector General (OIG) considers management’s comments responsive to the recommendation in the report and corrective actions should resolve the issues identified in the report.

The recommendation requires OIG concurrence before closure. Consequently, the OIG requests written confirmation when corrective action is completed. This recommendation should not be closed in the Postal Service’s follow-up tracking system until the OIG provides written confirmation that the recommendation can be closed.

We recommend

management re-evaluate

help desk metrics for the

upcoming contract or

extension to establish

an average handle time

performance target and

adjust the abandoned call

rate and average speed to

answer performance target.


Appendices

Click on the appendix title

to the right to navigate

to the section content.


Appendices .................................................................................................12Appendix A: Additional Information ..........................................................13

Background ..........................................................................................13Objectives, Scope, and Methodology ..................................................13Prior Audit Coverage ............................................................................14

Appendix B: Management’s Comments ...................................................15

Appendix A: Additional Information

BackgroundThe RSS is the commercial off-the-shelf software platform the Postal Service has implemented RSS to replace its existing retail point-of-service software. RSS was deployed to reduce the number of vendors involved in the Postal Service’s retail solutions and lower its operational and maintenance costs. Full deployment of RSS to all POS One sites was expected to be completed by May 2016.

The RSS help desk consists of four tiers to manage customer issues:

■ Tier One is the point of entry for RSS incidents and addresses hardware issues, well documented software issues, and policy and procedure issues.

■ Tier Two works directly with the customer to diagnose and address elevated issues that require additional investigation and application, platform, and procedural support.

■ Tier Three researches complex problems that cannot be resolved with the customer on the phone and ensures that incident tickets have sufficient detail so that developers have all information to research a complex problem.

■ Tier Four provides programmatic solutions to defects and deficiencies identified by Tier Three.

The NCR Corporation provides Tier One RSS hardware and software help desk services for the Postal Service as well as supporting the POS Tier One help desk.

Objectives, Scope, and MethodologyOur objective was to determine if help desk performance was adequate to support RSS. The audit scope was Tier One metrics, which includes abandoned call rate, average speed to answer and average handle time. The scope includes a review of RSS help desk contractual requirements, performance, process flows and staffing and analysis of 52,840 help desk incident tickets between May 2015 and February 2016.

To accomplish our objective, we:

■ Met with the Postal Service RSS program office, managers and contractors responsible for monitoring and managing help desk support.

■ Reviewed data from ServiceNow to evaluate whether help desk metrics are being achieved.

■ Reviewed the statement of work from August 2007, and consolidated base contract dated December 2008, to include modifications, regarding the RSS help desk and relevant RSS functionality reports.

■ Obtained and reviewed RSS help desk metrics from June 2015 through February 2016, tracking records, standard operating procedures, and staffing information.

■ Reviewed service level agreements.

■ Obtained and reviewed RSS help desk tickets for June 1 through December 31, 2015, to identify most recurring issues.


We conducted this performance audit from January through June 2016, in accordance with generally accepted government auditing standards and included such tests of internal controls as we considered necessary under the circumstances. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. We discussed our observations and conclusions with management on June 2, 2016, and included their comments where appropriate.

We assessed the reliability of ServiceNow and performance data by using ACL to test help desk incident ticket data for duplicates, gaps, and completeness. We determined that the data were sufficiently reliable for the purposes of this report.

Prior Audit Coverage

Report Title Report Number Final Report DateMonetary Impact

(in millions)Retail System Software Deployment and Functionality MI-AR-15-002 3/13/2015 None

Report Results: Our report determined that functionality issues delayed completion of the RSS pilot. In addition, management stopped RSS deployment in December 2014 because of an unrelated temporary national freeze and we found the RSS help desk needed improvement. The deployment schedule was aggressive, making successful deployment in 2015 challenging. In addition, existing and future functionality issues may require more funding, which would further reduce the expected return on investment. Management agreed, in part, with the recommendations, stating that they have converted an additional 348 sites since national deployment and indicated they are developing the self-service kiosk. Additionally, management stated that they prioritized functionality issues and addressing defects based on business and customer needs. Further, management indicated that help desk improvements were ongoing, that they have established metrics and increased help desk staffing, and will continue making training and knowledge-based enhancements throughout the life of the program.


https://www.uspsoig.gov/sites/default/files/document-library-files/2015/mi-ar-15-002.pdf

Appendix B: Management’s Comments


Contact Information


Contact us via our Hotline and FOIA forms. Follow us on social networks.

Stay informed.

1735 North Lynn Street Arlington, VA 22209-2020

(703) 248-2100

http://www.uspsoig.gov

http://www.uspsoig.gov/form/new-complaint-form

http://www.uspsoig.gov/form/foia-freedom-information-act

https://www.facebook.com/oig.usps

https://twitter.com/OIGUSPS

http://www.youtube.com/oigusps

retail system software help desk - usps oig · evaluation of management’s comments ... this...

Documents