2

Breakout Session # 1411

Robert F. Watts, Senior Consultant, ESI International

Date: April 16

Time: 2:40 – 3:40 PM

Buying Results Through Service Level Agreements: Best Practices in Performance- Based Acquisition

3

Performance Based Acquisition -- Still a Challenge

• Recent Acquisition Advisory Panel study determined that more than half the contracts designated as PBA in the Federal Procurement Data System-Next Generation were not performance-based

4

It’s All About Mindset…Mindset Characteristics

Process

Focused on process and activities -- how requirements need to be accomplished Often lacks confidence in contractor’s quality control practices Prefers to micromanage the company's technical processes

Personnel

Focused on personnel - who needs to accomplish requirements Often lacks confidence in contractor’s human resource and management practices Prefers to micromanage the company's personnel

Performance

Focused on results and mission accomplishment - what needs to be accomplished Has confidence in contractor’s technical, quality, human resources, and

management practices Prefers to manage outcomes and customer satisfaction and not to micromanage

the company’s processes and personnel

5

PBAIncentive

Plan

Quality Assurance

Surveillance Plan

PerformanceWork Statement

or SOO

PerformanceRequirements

Summary

6

What is a Service Level Agreement (SLA)?

• A performance-based technique where a written agreement is established between the customer and service provider that defines key service objectives, metrics, and acceptable quality levels (AQLs)– Primarily used in information technology (IT)

procurements – Expanding into other sustainment-type services

7

Example SLA OutlineService Level Agreement

(Title of Agreement)

1.0 Introduction

1.1 Statement of Need

1.2 Background

2.0 Service Objectives

4.0 Service Level Metrics/AQLs

3.0 Applicable Directives

3.1 Compliance

3.2 Reference

4.0 Constraints and Assumptions

5.0 Data Deliverables

6.0 Government Furnished Property (GFP)

Equivalent to PWS/SOO

and Performance Requirements

Summary

Equivalent to PWS/SOO

and Performance Requirements

Summary

8

Service Objectives

Service Desk

Provide, implement, operate and maintain a fully-functional, reliable Service Desk…

….for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime.

Do what?

To accomplish what?

Example

9

Service Level Metrics/AQLs

Title: Service Desk

Objective: Provide, implement, operate and maintain a fully-functional, reliable Service Desk for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime.

Metrics:

Response: Percentage of inbound calls answered within 20 seconds.

First Call Resolution: Percentage of inbound calls handled to resolution at Tier-1.

Quality: Percentage of customers surveyed expressing a “satisfactory” or better quality of service rating.

Acceptable Quality Levels:Response: 80%

First Call Resolution: 70%

Quality: 80%

10

Service Level Metrics: Possible Areas to Measure

Network Performance Hosting Application User Support

Availability: Percent of time the network is available to users (percentage)

Server Availability: Percent of time the server is available to users (percentage)

User Interactions: Percent of successful user interactions (downloads, data requests, and so forth)

Help Desk: Response time and accuracy in resolving trouble tickets

Throughput: Capacity of the backbone connections within the network’s core (bits/sec)

Administration of Servers: Acceptable response times for restoring failed servers (period of time)

Data Transfer: Acceptable time lapse between user request for data and moment data screen appears (seconds)

User Training:

Quality of user training, such as classroom, online, and so forth (customer survey)

Connection: Acceptable data losses and data latency, plus bandwidth provisioning (percent of packet delivery rates)

Data Backup and Storage: Scheduled data backups that will be conducted (percent of completion)

Ease of Use: Quality of customer’s experience in using the application (customer survey)

Needs Assessment: Quality and thoroughness of program for assessing ongoing user needs

Security: Protection of applications, data, and services in transit (levels of encryption, number of breaches, and so forth)

11

What is the QASP?

• An internal plan used by the customer that describes the performance monitoring methods they will use to evaluate the contractor’s progress and success in meeting service level metrics and AQLs contained in the SLA

12

Some Techniques to Monitor Contractor Performance

Example PerformanceMonitoringApproaches

13

Contractor Monitoring (Example)

Title: Service Desk

Objective: Provide, implement, operate and maintain a fully-functional, reliable Service Desk for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime.

Metrics:

Response: Percentage of inbound calls answered within 20 seconds.

First Call Resolution: Percentage of inbound calls handled to resolution at Tier-1.

Quality: Percentage of customers surveyed expressing a “satisfactory” or better quality of service rating.

Acceptable Quality Levels:Response: 80%

First Call Resolution: 70%

Quality: 80%

Method of Government Monitoring:

Response: Weekly audit of Service Desk reports

First Call Resolution: Weekly audit of Service Desk reports

Quality: Automated email customer satisfaction survey

Monitoring Officials:

COTR; Performance Monitors

14

Automated Email Survey How satisfied are you that the Service Desk responded quickly to your problem?

Very Dissatisfied Dissatisfied Satisfied Very Satisfied

O O O OComments: How satisfied are you that the Service Desk was professional and “customer-oriented” in resolving your problem?

Very Dissatisfied Dissatisfied Satisfied Very Satisfied

O O O OComments: How satisfied are you in the overall quality of the Service Desk’s final resolution of your problem?

Very Dissatisfied Dissatisfied Satisfied Very Satisfied

O O O OComments:

15

Incentive Plan

• Rewards and/or penalties assessed against the contractor based on their success in achieving service level metrics and AQLs in the SLA

16

Possible Incentive StructuresAward Fee

Pool of dollars awarded at specified periods to contractor based on performance of SLA

Award TermAdditional contract terms awarded to contractor based on performance of SLA

Past Performance RatingsFormally documented ratings of contractor performance of SLA

Milestone PaymentsLinkage of contractor payments to successful completion of milestones contained in SLA

OptionsLinkage of follow-on options to contractor based on successful performance of SLA

Invoice AdjustmentsAdjustments to contractor billings based on performance of SLA

17

Some Best Practices in Developing SLAs

• Keep it simple! – Look at the overall mission and develop a small set of key service level

metrics—find things that are measurable, meaningful and have real impact on the bottom line

• Stay focused on mission success and customer satisfaction– Leave measuring of activities and processes to the contractor

• Manage AQL expectations – be realistic– The closer to 100% perfection you require, then the higher the price --

be practical and realistic

• Performance standards cannot exist in a vacuum; there must be a forum for discussion and negotiation

18

Case Study

Chicago adjusts on the fly to make its contract perform