quarterly pulse check survey q2 2018 - new south wales · 2020-03-11 · 3 1.1 executive summary...

Quarterly Pulse Check Survey Q2 2018

Summary Report

June 2018

Item Page no.

1

Executive Summary- Executive Summary – Q2 2018 QPCS- Customer Satisfaction Index – Q2 2018 QPCS- Executive Summary – Business Expectations Deep Dive

2

2

Q2 2018 QPCS Customer Measures

- Customer Satisfaction Index (CSI) and Baseline Measures- Satisfaction, Expectation and Comparison to Ideal- Customer Effort Score- Customer Effort Compared to Other Industries

8

3

Insights on Satisfaction Drivers

- Introduction to 2017 Key Primary Opportunity Areas (KPOAs)- Performance of Satisfaction Drivers

14

4

Insights on Key Primary Opportunity Areas

- Key primary Opportunity Areas (KPOAs) in 2017- Performance of Key Primary Opportunity Areas- Drivers of Satisfaction and Suggested Areas of Improvement with Employee Efficiency and Effectiveness- Drivers of Satisfaction and Suggested Areas of Improvement with Access to Information

18

5Channel Usage and Preference- Channel Usage- Channel Preference

26

7

Appendices

- Appendix A: Demographic Profile of Respondents- Appendix B: Background to the QPCS- Appendix C: QPCS Qualitative Research Objectives

29

Source: Office of Customer Service Commissioner, Quarterly Pulse Check Survey Q2 2018

Table of Contents

1

1. Executive Summary

Section Contents

1.1 Executive Summary – Q2 2018 QPCS

1.2 Customer Satisfaction Index – Q2 2018 QPCS

1.3 Executive Summary – Business Expectations Deep Dive

3

1.1 Executive Summary – Q2 2018 QPCS

Q2 2018 QPCS insights - Consumers

• The Customer Satisfaction Index (CSI) has decreased marginally by 0.26 points from Q1 2018 to 79.6/100 (this increase is not statistically significant).

• All three baseline measures have remained relatively stable since last quarter and this time last year.

• QPCS data suggest that consumer results have stabilised, with no significant movement in CSI or outcome measures over the last 2 quarters. This suggests that we may see insignificant movement in 2018 CSMS results, with CSI remaining around 79.3 (as per 2017 CSMS).

• Consumer effort in interactions with NSW Government services has remained stable since Q1 2018, although the customer effort score this quarter does represent a statistically significant increase since this time last year.

• All drivers of customer satisfaction have remained stable this quarter, with the exception of honesty and integrity of employees, which has declined statistically significantly.

Q2 2018 background and context

Survey Period: 23rd – 26th April 2018

Preceding the survey period in Q2 2018 the following issues and initiatives were topical:

Green Slip Refunds: NSW motorists can claim refund on Green Slips following NSW Government reform of CTP insurance.

NDIS: Media coverage has raised concerns about NDIS leaving vulnerable citizens worse off and waiting up to 9 months for their cases to be resolved.

TFNSW announcements: Credit cards supplementing Opal on ferries and light rail following successful trial on Manly ferry line; announcement of new trains to be launched late 20178/early 2019.

New online birth registration: launch of new online registration process for births, replacing paper-based process [11 April, FACS/ Documentation Services]

Sydney light rail: Ongoing media coverage on cost blowouts and delays affecting a number of businesses in the CBD.

Stadium upgrades: Significant media coverage on the Sydney stadium redevelopment plans.

NAPLAN test: Media coverage of calls to review NAPLAN testing

Other news & events in Q2 2018:

Banking Royal Commission: media coverage into the ongoing Royal Commission into misconduct in the banking, superannuation and financial services Industry.

Public holidays: Easter 2018: 30 March – 2 April; Anzac Day: 25 April (during fieldwork)

Q2 2018 QPCS insights - Business

• The Customer Satisfaction Index (CSI) has increased marginallyby 0.1 points from Q1 2018 to 79.7/100 (this increase is not statistically significant).

• Business expectations have increased statistically significantlyfrom 7.7 in Q1 to 8.1 in Q2 2018. Other outcome measures have increased marginally (not statistically significant).

o The CSI has increased due to a larger expectation-satisfaction gap.

• Over the last 12 months, business expectations have fluctuated while satisfaction has remained relatively stable. Movement in the Business CSI in 2018 CSMS will likely be dependent on movement in expectations in the next quarter.

• Businesses reported significantly lower effort required in interactions with NSW Government services (down statistically significantly in Q2, from 6.9 to 6/10).

o In particular, the decline in effort is most marked in Car & Boat registration and Public Hospitals this quarter.

o In parallel, performance on the attribute the process was simple and efficient also improved statistically significantly this quarter, while all other attributes remained stable.

• There appears to be an inverse relationship between business confidence and business expectations, with increases in business expectations of NSW Government services coinciding with decreases in business confidence, and vice versa.

Consumers Business

QPCS results at a glance Q1 2017 Q2 2017 Q4 2017 Q1 2018 Q2 2018 Q1 2017 Q2 2017 Q4 2017 Q1 2018 Q2 2018

Satisfaction 7.6 7.6 7.7 7.7 7.7 7.4 7.3 7.5 7.6 7.8

Expectation 7.9 7.8 7.8 7.9 7.9 8.1 7.5 7.6 7.7 8.1

Ideal service 7.3 7.2 7.3 7.2 7.2 7.4 7.1 7.0 7.2 7.3

Customer Satisfaction Index (CSI) 79.5 78.6 80.2 79.9 79.6 77.6 77.6 77.8 79.6 79.7

CSI Score out of 100; all other baseline measures represent scores out of 10

Statistically significant movement from same quarter previous year at 95% confidence level

Statistically significant movement from previous quarter at 95% confidence level

1.2 Customer Satisfaction Index – Q2 2018 QPCS

76.6

78.4 78.377.6 77.6 77.8

79.6 79.7

2015(n=1,645)

2016(n=1,712)

2017(n=1,638)

Q1 2017(n=380)

Q2 2017(n=365)

Q4 2017(n=343)

Q1 2018(n=365)

Q2 2018(n=1,536)

Consumers – CSIPremier’s target: 79.02 (2019 CSMS)

Businesses – CSIPremier’s target: 81.16 (2019 CSMS)

Consumers – Baseline Measures

Businesses – Baseline Measures


Margin of Error (QPCS)Consumer CSI: ± 1.2

Margin of Error (QPCS)Business CSI: ± 1.8

76.7

78.7 79.3 79.578.6

80.2 79.9 79.6

2015(n=6,549)

2016(n=6,971)

2017(n=6,527)

Q1 2017(n=1,548)

Q2 2017(n=1,594)

Q4 2017(n=1,508)

Q1 2018(n=1,610)

Q2 2018(n=1,603)

7.3

7.57.7

7.6 7.67.7 7.7 7.7

7.77.8

7.9 7.97.8 7.8

7.9 7.9

6.9

7.2 7.27.3

7.27.3

7.27.3

6.6

7.1

7.6

8.1

8.6

2015(n=6,593)

2016(n=7,015)

2017(n=6,559)

Q1 2017(n=1,548)

Q2 2017(n=1,605)

Q4 2017(n=1,518)

Q1 2018(n=1,628)

Q2 2018(n=1,618)

Satisfaction Expectation Comparison to Ideal

7.2

7.5 7.5 7.47.3

7.57.6

7.87.6

7.97.7

8.1

7.57.6

7.7

8.1

7.0

7.37.1

7.4

7.1 7.07.2 7.3

6.6

7.1

7.6

8.1

8.6

2015(n=1,654)

2016(n=1,718)

2017(n=1,646)

Q1 2017(n=380)

Q2 2017(n=365)

Q4 2017(n=345)

Q1 2018(n=367)

Q2 2018(n=1,554)

Satisfaction Expectation Comparison to Ideal


CSMS

CSMS

QPCS

QPCS

CSMS QPCS

CSMS QPCS

Note - Sample size for CSI and Baseline measures may differ due to “Don’t Knows”

4

(1) Customer effort

Businesses in Q2 reported significantly less effort in interactions with NSW Government services (as measured by the Customer Effort Score (CES) in the QPCS survey). This is the first quarter (since the inception of the QPCS survey) where there is a marked divergence between the CES (down statistically significantly) and business expectations (up statistically significantly).

o The decline in effort is driven primarily by reduced effort in interactions with Car & Boat Registration and Public Hospitals. These are two of the largest services included in the QPCS survey (based on number of ratings received for services), and has driven the overall CES for Whole of Government down this quarter (note: results are indicative due to low base sizes for services in Q1*).

o The decline in effort is in line with other industries - the Customer Effort Score declined statistically significantly in other industries (airlines, banking and energy retailers).

o Interactions which involve very low customer effort garner the highest expectations of NSW Government services - in Q2 expectation scores are significantly higher for interactions with low effort, with little difference in expectations for moderate and high effort interactions (see table at right).

o In Q2, there was a higher proportion of low effort interactions – 27% of interactions were low effort compared with 14% in Q1 2018, as reported by business respondents.

o In Q2 QPCS, there was a statistically significant improvement in ‘the process was simple and efficient’ – the only measured driver to have increased this quarter.

5

1.3 Q2 2018 Business Expectations Deep Dive (1 of 3)

BusinessConsumer

6.3

6.96.5

6.76.9

6.0

7.6

8.1

7.5 7.6 7.78.1

Business Customer Effort Score Business ExpectationLowEffort

HighEffort

Q1 2017 (n=382)

Q2 2017 (n=366)

Q4 2017 (n=345)

Q1 2018 (n=372)

Q2 2018 (n=1,535)

Low Expectation

HighExpectation

Customer Effort Score and Business Expectations

*In Q2 2018 QPCS the business sample size was increased from 250 to 1,000 to reduce the margin of error and increase validity of the data

Low effort (1-4/10)

Mode-rate

effort (5-6/10)

High effort

(7-10/10) Base*

Expectation score (Q2 QPCS)

8.7 7.8 7.7

1,535Proportion of respondents (Q2 2018 QPCS)

27% 24% 49%

Proportion of respondents (Q1 2018 QPCS)

14% 20% 66% 373

Over the last 12 months, business expectations have fluctuated while satisfaction has remained relatively stable. Movement in the Business CSI in 2018 CSMS will likely be dependent on movements in expectations in the next quarter. The following have been evaluated as a ‘deep dive’ into the drivers of the rise in business expectations in Q2 2018: (1) Customer Effort; (2) Business Confidence; and (3) Attitudes to the NSW Government. These are summarised on the following pages.

Q4 2016 (n=378)

(2) Business confidence

There appears to be an inverse relationship between Business Confidence and expectations of NSW Government services.

Business expectations as measured in the QPCS and CSMS surveys were analyzed in relation to various business confidence indices in Australia and NSW.

o There appears to be an evident inverse relationship between the two measures. The figure at right illustrates the Roy Morgan Business Confidence Index and Business Expectations as measured in the QPCS and CSMS surveys. As business’ confidence in the Australian economy declines, expectations of NSW Government services increases, or vice versa over most QPCS/CSMS survey periods over the last 2 years.

o As business confidence is bolstered, the optimism of businesses in NSW may result in businesses requiring (or believing they require) less support from the NSW Government, and hence a decline in the expectations of Government services to deliver to business needs. Conversely, as business become less confident in the economy, the need, or perceived need, for Government support increases, resulting in higher expectations.

o Analysis of the NSW Business Chamber’s Business Conditions survey also showed a relationship with business’ expectations of NSW Government services, although the relationship was weaker than with the Roy Morgan Index, potentially because (a) the timing of the Chamber’s survey periods do not match the QPCS fieldwork periods; or (b) it measures ‘perceptions of’, rather than ‘confidence in’ the economy. Confidence and expectations are aspirational beliefs as opposed to perceived performance on the current state of the economy and hence have a stronger link.

6


BusinessConsumer

Business Confidence and Business Expectations

7.9

7.6

8.1

7.5

7.7

7.6

7.7

8.1

111.9

116.2

111.4

116.6 117

112.7

123.5

118.6

100

105

110

115

120

125

130

135

140

145

150

7

7.2

7.4

7.6

7.8

8

8.2

CSMS2016

Q4 2016 Q1 2017 Q2 2017 CSMS2017

Q4 2017 Q1 2018 Q2 2018

Bu

siness C

on

fiden

ce Ind

exBu

sin

ess

Exp

ecta

tio

ns

Business Expectations (QPCS and CSMS surveys)

Roy Morgan Business Confidence Index*

Notes about business confidence indices assessed:

About the Roy Morgan Business Confidence Index: The Index is based on 5 different attributes measuring Australian business’ expectations of the economic climate over the next 12 months. (n=~1,000 Australian businesses per month). Data points for the Roy Morgan Business Confidence Index are taken from the corresponding month of QPCS/CSMS fieldwork. http://www.roymorgan.com/morganpoll/consumer-confidence/roy-morgan-business-confidence

About the NSW Business Chamber’s Business Conditions Survey: The Chamber’s survey measures perceptions of the economy’s current and expected performance. (n=~1,000 NSW Business Chamber members per quarter: NSW business only). https://www.nswbusinesschamber.com.au/Issues/Business-Surveys/Business-Conditions

Denotes CSMS period + data

The next Roy Morgan Business Confidence survey results will be available in June 2018

(3) Attitudes to the NSW Government

There is a strong link between expectations of NSW Government services and attitudes towards the NSW Government, as measured by the Sensis Business Index survey of Small and Medium Businesses (SMB) in Australia.

Attitudes towards the NSW Government are assessed quarterly by Sensis, asking the question: “thinking about the current state/territory government, do you believe that their policies are supportive of small businesses, work against small businesses or have no real impact either way?” (the survey is conducted Australia wide).

Results mirror business expectations of NSW Government services. This supports the idea that ‘expectations’ as measured in the QPCS/CSMS are driven by broader perceptions of the NSW Government and its policies rather than service interactions alone.

According to Sensis reports, “those who say it [the NSW Government] is supportive [of small businesses] are impressed by infrastructure development much more than other factors. Excessive bureaucracy followed by policies being unhelpful for small businesses and a lack of advice and support are the main criticisms.” These factors consistently underlie improvements and declines in the Sensis measure over the past 2 years.

7


BusinessConsumer

Business Confidence and Business Expectations

7.9

7.6

8.1

7.5

7.7

7.6

7.7

8.1

8

-4

2

-2

3

1 1

5

-6

-4

-2

0

2

4

6

8

10

7

7.2

7.4

7.6

7.8

8

8.2

CSMS2016

Q4 2016 Q1 2017 Q2 2017 CSMS2017

Q4 2017 Q1 2018 Q2 2018

Sensis Su

rvey

Bu

sin

ess

Exp

ecta

tio

ns

Business Expectations (QPCS and CSMS surveys)

Sensis SMB Survey: Attitudes to NSW Government

About the Sensis Business Index Survey: The survey is a quarterly survey of Australia's Small and Medium Businesses (SMB), which represent 99% of Australian businesses (N=1000 Australian businesses per quarter).

The index is the net balance of the % of SMBs that regard their state or territory government to be supportive and the % that believe it is working against small business interests.

Note: the Sensis survey fieldwork is conducted in the month prior to the QPCS/CSMS fieldwork.

https://www.sensis.com.au/about/our-reports/sensis-business-index

Denotes CSMS period + data

The next Sensis Business Index survey results will be available in July 2018

2. Customer Satisfaction Measures

Section Contents

2.1 Customer Satisfaction Measures – Customer Satisfaction Index (CSI)

2.2 Satisfaction, Expectation and Comparison to Ideal

2.3 Customer Effort Score

2.4 Customer Effort Compared to Other Industries

76.7

78.779.3 79.5

78.6

80.2 79.9 79.6

2015(n=6,549)

2016(n=6,971)

2017(n=6,527)

Q1 2017(n=1,540)

Q2 2017(n=1,594)

Q4 2017(n=1,508)

Q1 2018(n=1,610)

Q2 2018(n=1,603)

2.1 Customer Satisfaction Measures – Customer Satisfaction Index (CSI)

9

Consumer – Customer Satisfaction Index Business – Customer Satisfaction Index

Cu

sto

mer

Sat

isfa

ctio

n In

dex

(o

ut

of

10

0)

Cu

sto

mer

Sat

isfa

ctio

n In

dex

(o

ut

of

10

0)

The CSI as measured by the QPCS in Q2 2018 is 79.6 for consumers. The CSI as measured by the QPCS in Q2 2018 is 79.7 for businesses.

• The Customer Satisfaction Index (CSI) has decreased marginally among consumers by 0.26 points from Q1 2018 to 79.6/100 in Q2 2018 (not statistically significant). The CSI for businesses has increased by 0.1 points from Q1 2018 to 79.7/100 in Q2 2018 (not statistically significant).

• Compared to this time last year (Q2 2017) the CSI has increased by 0.97 points among consumers to 79.6/100 (not statistically significant) and by 2.06 points among businesses to 79.7/100 (statistically significant).

BusinessConsumer

Margin of Error (QPCS)Consumer CSI: ± 1.2

Margin of Error (QPCS)Business CSI: ± 1.8

76.6

78.4 78.377.6 77.6 77.8

79.6 79.7

2015(n=1,645)

2016(n=1,712)

2017(n=1,638)

Q1 2017(n=380)

Q2 2017(n=365)

Q4 2017(n=343)

Q1 2018(n=365)

Q2 2018(n=1,536)

CSMS QPCS CSMS QPCS



Note: Figures in the graph are subject to rounding as some may not add up to 100%

7.8 7.8 7.9 7.9 7.6 7.7 7.7 7.7 7.2 7.3 7.2 7.3

2.2 Satisfaction, Expectation and Comparison to Ideal - Consumer

10

Consumer – Outcome Measures

Expectation

Average (out of 10)

Among consumers all three outcome measures have remained relatively stable since Q1 2018, with no statistically significant changes. When compared to Q2 2017, all outcome measures have marginally increased (no statistical significance).

BusinessConsumer

Satisfaction Comparison to ideal

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)


6% 7% 6% 7% 7% 8% 8% 9% 12% 11% 12% 12%

14% 14% 13% 12%17% 13% 14% 13%

18% 17% 19% 17%

80% 79% 81% 81% 76% 78% 78% 78%69% 72% 70% 71%

0%

20%

40%

60%

80%

100%

Q2 2017(n=1,624)

Q4 2017(n=1,537)

Q1 2018(n=1,644)

Q2 2018(n=1,642)

Q2 2017(n=1,647)

Q4 2017(n=1,541)

Q1 2018(n=1,651)

Q2 2018(n=1,638)

Q2 2017(n=1,605)

Q4 2017(n=1,518)

Q1 2018(n=1,628)

Q2 2018(n=1,618)

Low (1-4) Med (5-6) High (7-10)



2.2 Satisfaction, Expectation and Comparison to Ideal - Business

11

Business – Outcome Measures

Among businesses, expectations have increased by 0.36 points to 8.1/10 since Q1 2018 (statistically significant). Satisfaction and comparison to ideal scores have remained relatively stable since Q1 2018, with no statistically significant changes.

Compared to this time last year (Q2 2017), business expectations and satisfaction have both increased, by 0.54 points to 8.1/10 and 0.45 points to 7.8/10, respectively. Both changes are statistically significant.

BusinessConsumer

% D

istr

ibu

tio

n (

acro

ss lo

w,

neu

tral

an

d h

igh

sco

res)

Average (out of 10) 7.5 7.6 7.7 8.1 7.3 7.5 7.6 7.8 7.1 7.0 7.2 7.3

Expectation Satisfaction Comparison to ideal


10% 6% 6% 6% 11% 8% 5% 9%14% 10% 7% 12%

16%17% 16%

9%

17% 19% 22% 11%

18% 24%20% 15%

74% 77% 78%84%

72% 73% 73%80%

68% 66%73% 73%

0%

20%

40%

60%

80%

100%

2017 Q2(n=370)

2017 Q4(n=350)

2018 Q1(n=372)

2018 Q2(n=1,555)

2017 Q2(n=372)

2017 Q4(n=349)

2018 Q1(n=375)

2018 Q2(n=1,570)

2017 Q2(n=365)

2017 Q4(n=345)

2018 Q1(n=367)

2018 Q2(n=1,554)

Low (1-4) Med (5-6) High (7-10)



2.3 Customer Effort Score

Customer Effort Score* – Service Interaction

BusinessConsumer

Service interaction Customer Effort Score.

The Customer Effort Score (CES) has seen a marginal increase (no statistical significance) among consumers, up by 0.07 points since Q1 2018 to 6.0/10, and a statistically significant decrease among businesses (down by 0.85 to 6.0/10) from Q1 2018. When considering annual trends, the CES has seen a statistically significant increase among consumers (by 0.26 points) and a statistically significant decrease among businesses (down by 0.46 points) since Q2 2017.

12



5.9

5.7

6.1

5.9

6.0

6.9

6.5

6.7

6.9

6.0

Consumers BusinessesLowerEffort

HigherEffort

Q1 2017Consumer (n=1,529)

Business (n=382)


Business (n=366)


Business (n=345)


Business (n=373)

Note: Figures in the graph are subject to rounding*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift.

Q2 2018Consumer (n=1,585)Business (n=1,535)

6.0 6.2 6.3 6.2 6.26.6 6.4

6.16.3 6.5 6.3 6.2

6.7 6.7

5.7

6.3 6.2 6.26.6

6.3

6.8

6.3 6.26.0

6.2 6.4 6.46.7

5.6 5.8 5.8 6.0 6.0 6.16.4

Airlines Banks FederalGovernment

My local council Energy retailers NSWGovernment

Services

Telephoneservice providers

Q1 2017 QPCS (n=210 to 254) Q2 2017 QPCS (n=199 to 244) Q4 2017 QPCS (n=186 to 228)

Q1 2018 QPCS (n=208 to 239) Q2 2018 QPCS (n=726 to 902)

5.65.5

5.8 5.6 5.65.9 6.1

5.7 5.6 5.8 5.7 5.76.1

6.4

5.7 5.86.0 6.0 6.2 6.1 6.4

5.5 5.7 5.7 5.9 6.1 6.16.4

5.4 5.75.9 5.8

6.0 6.06.2

Airlines Banks FederalGovernment

My local council Energy retailers NSWGovernment

Services

Telephoneservice providers

Q1 2017 QPCS (n=722 to 954) Q2 2017 QPCS (n=669 to 955) Q4 2017 QPCS (n=763 to 948)

Q1 2018 QPCS (n=701 to 940) Q2 2018 QPCS (n=711 to 943)

2.4 Customer Effort Compared to Other Industries

Customer Effort Score* – WHoG BenchmarkCustomer Effort Score* – WHoG Benchmark

BusinessConsumer

When benchmarked against other industries, NSW Government services (at a Whole of Government level) compare favorably to telephone service providers in terms of effort required in interactions among consumers in Q2 2018.When compared to Q2 2017, airlines have seen a statistically significant decrease in effort (down 0.28 points to 5.4/10) and energy retailers have seen a statistically significant increase in effort (up 0.32 points to 6.0/10).

13

Note: Figures in the graph are subject to rounding

Consumers Businesses

*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift. Data labels in green/red indicate statistically significant movement at 95% confidence level compared to previous quarters results


Among businesses in Q2 2018, customer effort has decreased across all industries in Q2 2018. NSW Government Services remain ahead of telephone service providers, and have fallen slightly behind energy retailers in terms of effort required.

Customer Effort Score benchmarked at a Whole of Government level against other industries.

Customer Effort Score benchmarked at a Whole of Government level against other industries.

3. Insights on Satisfaction Drivers

Section Contents

3.1 Introduction to 2017 Key Primary Opportunity Areas (KPOAs)

3.2 Performance of Satisfaction Drivers

15

3.1 Introduction to 2017-18 Key Primary Opportunity Areas (KPOAs)

LOW

HIG

H

Relative performance

LOW

HIGH

Imp

ort

ance

in d

rivi

ng

sati

sfac

tio

n

Efficiency and effectiveness of

employees

Honesty and integrity of employees

Empathy and Communication of

employees

Employee autonomy

Simplicity and efficiency of processes

Transparency

Access toinformation

Privacy

Strengths to ‘build on’Primary opportunities

Secondary opportunities Lower importance - monitor

Key Primary Opportunity Areas 2017-18Recap:

Key Primary Opportunity Areas were identified based on 2017 Annual CSMS data. These were derived based on analysis of the importance of drivers in determining satisfaction, and their performance. Key Primary Opportunity Areas have been further tested in the QPCS.

Analysis to identify the Key Primary Opportunity Areas reveals that:

• Efficiency and effectiveness of employees and access to information and online services remain primary opportunity areas for increasing satisfaction as they are of high importance in driving customer satisfaction and their perceived performance is lower than other Whole of Government drivers.

• A new driver category emerged in 2017 CSMS encompassing ‘softer’ attributes such as empathy and communication of employees. This category has replaced employee communication as a driver of satisfaction with NSW Government services. Empathy and communication of employees is a relative strength to build on, as it is important to customers and perceived performance is high.

Legend:Employee attributes

Goals Processes

7.87.5 7.7

7.3 7.2

8.1 7.8 7.5 7.6 7.37.8 7.6 7.7 7.6 7.7

7.37.9 7.7

7.4 7.5 7.48.0

7.6 7.8 7.6 7.4 7.4

8.27.9

7.6 7.7 7.57.8 7.5 7.77.3

8.07.8

7.57.7

Employees wereopen and honest

during the process

Employees actedefficiently and

effectively to reachthe right outcomes

Communicationswere clear, prompt

and easy tounderstand

Employees actedwith empathy

The process wassimple and efficient

Employees tookinitiative and made

decisions

My privacy wasupheld & personalinformation was

protected &respected

I understood thesteps involved with

the process

I had good accessto information andcould find what I

needed

I was provided withgood service andoutcomes I could

trust

I felt there wasaccountability forservices delivered

Q2 2017 QPCS (n=1,405 to 1,609) Q4 2017 QPCS (n=1,385 to 1,541) Q1 2018 QPCS (n=1,444 to 1,601) Q2 2018 QPCS (n=1,431 to 1,588)

Statement Asked in QPCS:

3.2 Performance of Satisfaction Drivers - Consumers

16

Drivers of SatisfactionA

vera

ge (

ou

t o

f 1

0)

BusinessConsumer

Derived Drivers (Annual CSMS): Privacy Transparency

Access to information

Honesty and Integrity of Employees

Efficiency and Effectiveness of Employees

Simplicity and Efficiency of

ProcessesEmpathy and Communication

Employee Autonomy

Service Quality

Accountability

Employees Process Goals Values

n/a

fo

r p

revi

ou

s q

uar

ters

–n

ew d

rive

r ca

tego

ry id

enti

fied

in

in 2

01

7 C

SMS

In Q2 2018, all drivers have remained stable, with the exception of honesty and integrity of employees, which has statistically significantly decreased by 0.20 points since Q1 2018.

Data labels in green/red indicate statistically significant movement at 95% confidence level compared to previous quarters results


7.47.3 7.4

7.67.4 7.5

7.3 7.27.7

7.5 7.3 7.47.1

7.97.7 7.6

7.77.4 7.4

8.07.6 7.5 7.5

7.47.9 7.6 7.8 7.5

7.17.4

8.17.8 7.5 7.7

7.4

8.0

7.5 7.7 7.67.4 7.4

8.2 7.9 7.7 7.77.4

Employees wereopen and honest

during the process

Employees actedefficiently and

effectively to reachthe right outcomes

Communicationswere clear, prompt

and easy tounderstand

Employees actedwith empathy

The process wassimple and efficient

Employees tookinitiative and made

decisions

My privacy wasupheld & personalinformation was

protected &respected

I understood thesteps involved with

the process

I had good accessto information andcould find what I

needed

I was provided withgood service andoutcomes I could

trust

I felt there wasaccountability forservices delivered

Q2 2017 QPCS (n=353 to 371) Q4 2017 QPCS (n=322 to 347) Q1 2018 QPCS (n=356 to 371) Q2 2018 QPCS (n=1,358 to 1,524)

3.3 Performance of Satisfaction Drivers - Business

17

Employees Process Goals Values

Derived Drivers (Annual CSMS): Privacy Transparency

Access to information

Statement Asked in QPCS:

Drivers of SatisfactionA

vera

ge (

ou

t o

f 1

0)

Honesty and Integrity of Employees

Efficiency and Effectiveness of Employees

Simplicity and Efficiency of

Processes

Employee Autonomy

Service Quality

BusinessConsumer

In Q2 2018, all drivers have remained stable, with the exception of the simplicity and efficiency of processes, which has statistically significantly increased since Q1 2018.

n/a

fo

r p

revi

ou

s q

uar

ters

–n

ew d

rive

r ca

tego

ry id

enti

fied

in

in 2

01

7 C

SMS

Empathy and Communication Accountability

Data labels in green/red indicate statistically significant movement at 95% confidence level compared to previous quarters results


4. Insights on Key Primary Opportunity AreasSection Contents

4.2 Performance of Key Primary Opportunity Areas

4.3 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Efficiency and Effectiveness

4.4 Drivers of Satisfaction and Suggested Areas of Improvement with Access to Information

4.1 Key Primary Opportunity Areas (KPOAs) in 2017

19

4.1 Key Primary Opportunity Areas (KPOAs)

LOW

HIG

H


LOW

HIGH

Imp

ort

ance

in d

rivi

ng

sati

sfac

tio

n


employees



employees

Employee autonomy


Transparency


Privacy



Key Primary Opportunity Areas 2017-18

The QPCS provides a deep dive into the specific drivers of satisfaction and associated attributes as identified in the 2017 Annual CSMS. These are key areas which will have the greatest impact on overall customer satisfaction of NSW Government services as they are of high importance yet are perceived to have lower performance.

The deep dive KPOAs for the Q2 2018 QPCS are:

• Efficiency and effectiveness of employees

• Access to information

LOW

HIG

H


LOW

HIGH

Imp

ort

ance

in d

rivi

ng

sati

sfac

tio

n


employees



employees

Employee autonomy


Transparency


Privacy



Legend:Employee attributes

Goals Processes

7.6

7.97.8

7.7

7.9

7.8

7.5

7.77.6

7.67.6

7.5

7.4

7.7

7.6

7.5

7.6

7.5

7.4

7.77.6

7.5

7.5

7.5

7.3

7.7

7.5

7.4

7.57.4

7.0

7.5

8.0

Q4 2016 QPCS(n=1,427 to

1,540)

Q1 2017 QPCS(n=1,358 to

1,458)

Q2 2017 QPCS(n=1,401 to

1,518)

Q4 2017 QPCS(n=1,376 to

1,445)

Q1 2018 QPCS(n=1,466 to

1,567)

Q2 2018 QPCS(n=1,422 to

1,540)

Language that is simple and widely accessible

Easier to access information

Documents that are easy to complete

Keeping information up to date

Best use of online services

4.2 Performance of Key Primary Opportunity Areas - Consumers

20

Among consumers in Q2 2018, average scores for attributes relating to employee efficiency and effectiveness have decreased (no statistical significance) since Q1 2018. Results are also stable compared with this time last year.

Performance on access to information attributes has also remained stable since Q1 2018 and this time last year.

Efficiency and Effectiveness of Employees Access to Information

7.5

7.8

7.67.6 7.6

7.5

7.2

7.6

7.3

7.5

7.4

7.3

7.2

7.5

7.3

7.57.4

7.2

7.0

7.5

8.0

Q4 2016 QPCS(n=1,357 to

1,534)

Q1 2017 QPCS(n=1,316 to

1,499)

Q2 2017 QPCS(n=1,345 to

1,520)

Q4 2017 QPCS(n=1,325 to

1,455)

Q1 2018 QPCS(n=1,437 to

1,556)

Q2 2018 QPCS(n=1,376 to

1,536)

Deliver against actions promised

Get things done quickly

Accountable

BusinessConsumer

Statistically significant movement from same quarter previous year at 95% confidence levelStatistically significant movement from previous quarter at 95% confidence level

7.6

*Variations in figures may appear due to rounding

7.7

7.9

7.6

7.9 7.9 7.9

7.47.6

7.3

7.67.8

7.5

7.2

7.6

7.4

7.5

7.3

7.7

7.2

7.57.5

7.57.6

7.0

7.4

7.7

7.4

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=342 to 365)

Q1 2017 QPCS(n=363 to 377)

Q2 2017 QPCS(n=334 to 356)

Q4 2017 QPCS(n=323 to 338)

Q1 2018 QPCS(n=347 to 368)

Q2 2018 QPCS(n=1,386 to 1,487)

Language that is simple and widely accessible

Easier to access information

Documents that are easy to complete

Keeping information up to date

Best use of online services

4.2 Performance of Key Primary Opportunity Areas - Business

21

Among businesses in Q2 2018, relating to the KPOA employee efficiency and effectiveness there has been a statistically significant decrease in accountability, down by 0.28 points to 7.2/10 since Q1 2018. When compared to Q2 2017, deliver against actions promised and get things done quickly have increased statistically significantly to 7.6/10 and 7.4/10, respectively in Q2 2018.

Relating to the KPOA access to information in Q2 2018, two attributes easier to access information and best use of online services decreased statistically significantly by 0.25 points and 0.26 points to 7.5/10 and 7.4/10, respectively since Q1 2018.

BusinessConsumer

Efficiency and Effectiveness of Employees Access to Information

7.5

7.8

7.3

7.7 7.77.6

7.0

7.6

7.0

7.2

7.4

7.4

6.6

7.5

7.1

7.5

7.5

7.2

6.5

7.0

7.5

8.0

Q4 2016 QPCS(n=327 to 362)

Q1 2017 QPCS(n=355 to 373)

Q2 2017 QPCS(n=344 to 363)

Q4 2017 QPCS(n=316 to 336)

Q1 2018 QPCS(n=346 to 369)

Q2 2018 QPCS(n=1,343 to

1,474)

Deliver against actions promised

Get things done quickly

Accountable



7.2 7.2

*Variations in figures may appear due to rounding

4.3 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Efficiency and Effectiveness - Consumer

22

BusinessConsumer

As soon as I mentioned the problem, it was dealt with right away

My questions were answered promptly and my query was resolved quicker than I expected.

• Efficient, empathetic and knowledgeable staff

• Staff able to respond and resolve the issue quickly

• Staff were able to provide correct and timely information to customers

Reasons for high satisfaction with employee efficiency and effectiveness (7-10/10)

Very slow response times to a relatively simple query

I had to deal with a Sydney trains employee who was unhelpful, uninterested in helping, rude and

argumentative.

• Long waiting periods and time to respond• Undertrained employees leading to

increase in problem-solving time• Lack of employee empathy when dealing

with complicated issues

Reasons for low satisfaction with employee efficiency and effectiveness (1-6/10)

I believe better training for staff to be equipped with all the info to cut back on the repetitiveness

and confusion.

More staff available to take calls so wait times are reduced, removal of incorrect information

• Increase number of staff to improve wait times

• Training for staff on existing process so that they are more efficient

• Follow up to customers who have earlier complained or reached out

Customer ideas to improve employee efficiency and effectiveness

28%

72%

Low satisfaction (1-6/10)

High satisfaction (7-10/10)

“ “

”

“

”

n=1,514

“ ” ”““ ”

”

4.3 Drivers of Satisfaction and Suggested Areas of Improvement with Employee Efficiency and Effectiveness - Business

23

• Excellent customer service• Staff acted quickly and efficiently• Staff are well trained and knowledgeable

Reasons for high satisfaction with employee efficiency and effectiveness (7-10/10)

Disengaged but did what they were employed to do. However knowledge seems to be limited.

The only real problem was the waiting time in being able to reach the relevant person to talk to in the

Dept.

• Long time for documentation to be processed

• Lengthy wait times

Reasons for low satisfaction with employee efficiency and effectiveness (1-6/10)

Single window resolution of queries instead of being shunted across different people.

A more welcoming and understanding attitude. Maybe actually listen to what the customer wants

instead of just trying to clear the crowds.

• Be more time efficient• Increase the number of staff to reduce wait

times• Better staff training to make them more

knowledgeable

Ideas to improve employee efficiency and effectiveness

24%

76%



“

”

“

”

“

”

n=1,419

“” ”

““

”

BusinessConsumer

They (staff) had the ability to be precise with answers and efficient with my time

The ranger I dealt with was knowledgeable, clear and empathetic to the situation I faced and the

burdens suffered

4.4 Drivers of Satisfaction and Suggested Areas of Improvement with Access to Information - Consumer

24

BusinessConsumer

• Information that is clear / self-explanatory and easy to access

• Clear and sufficient website information and websites that are easy to navigate

Reasons for high satisfaction with access to information (7-10/10)

Online information is difficult to locate and usually results on a phone call. When I called on several

occasions I received a voice mail service. Sadly my call has never been returned

Information very confusing and confronting.

• Some websites are confusing and hard to navigate for customers.

• Online information is sometimes not up to date.

Reasons for low satisfaction with access to information (1-6/10)

"I would like to have websites that had good search functionality "

So many websites that have links that lead to other websites. Just one website, one number, one

information for all would be good

• Ensure websites are up-to-date and have easy-to-use search functionalities

• Simple-to-understand & printable forms for customers on websites

• One stop source for searching & accessing information online

Ideas to improve access to information

25%

75%



“

”

“

”“

”

n=1,503

“ ”“

”

Website was good to access with information about what happens in the process of your

application. .

I was always given or pointed in the right direction to get all the information I needed to get through

the system

”“

4.4 Drivers of Satisfaction and Suggested Areas of Improvement with Access to Information - Business

25

• Information on website is up to date and easy to access

• Being able to find what was needed quickly• Helpful and knowledgeable staff

Reasons for high satisfaction with access to information (7-10/10)

Every piece of information was in contradiction to the next person I spoke to.

The online service was often clunky making it difficult to find what I wanted to know in an unequivocal way,

website often didn't deliver efficiently

• Inconsistent information across channels• Difficult to navigate online portal to access

information• Lack of information or staff not knowing

the correct information

Reasons for low satisfaction with access to information (1-6/10)

Greater access to online documents so you don’t have to wait in long ques.

Current system is good but they have to be fully committed to their vision.

Have staff that can be contacted by phone or email.

• More online resources and information• Update information regularly to keep it

current• Simple processes

Ideas to improve access to information

25%

75%



“

”

“”

“

”

n=1,456

“”

“”

They (staff) guided me for the right information with references

Information is available in a variety of ways, in-person, online etc. and is very thorough and explained in simple to understand language.

BusinessConsumer

”

“

”“

5. Channel Usage and Preference

Section Contents

5.1 Channel Usage

5.2 Channel Preference

For consumers, there has been little change year-on-year in channel usage with results for Q2 2018 being similar to Q1 2018. Face to face remains the most used channel among this group.

For businesses, face to face interactions have increased to around one third of interactions in the first two quarters of 2018, and is now the most common channel, followed by telephone, online and email.

5.1 Channel Usage

27

Contact Methods Used*

Consumer Business

BusinessConsumer

Note: Contact methods used is a multi response question; Figures in the graph does not include the response for “None of the above” option*Channel usage has been rebased on total number of responses for comparison to channel preference

Shar

e o

f co

nta

ct m

eth

od

use

d (%

)

Shar

e o

f co

nta

ct m

eth

od

use

d (%

)Business n=1,536Consumer n=1,618

35

%

20

% 22

%

11

%

6%

3%

36%

21

%

19

%

13

%

7%

4%

35

%

21

%

19

%

14

%

7%

4%

35

%

22

%

19

%

12

%

8%

4%

0%

10%

20%

30%

40%

In person,face to face

Telephone Online Email Mail,posted

letter, fax

Thirdparties suchas Australia

Post

Q2 2017 Q4 2017 Q1 2018 Q2 2018

30

%

16

%

25

%

16

%

9%

4%

24

%

26

%

17

% 19

%

9%

6%

32

%

20

%

19

%

16

%

9%

4%

34

%

20

%

20

%

14%

8%

3%

0%

10%

20%

30%

40%

50%



letter, fax


Post

Q2 2017 Q4 2017 Q1 2018 Q2 2018

5.2 Channel Preference

28

Contact Methods Preference*

Consumer Business

There remains a clear preference for in person interactions when consumers deal with NSW Government services. Businesses also prefer in person contact methods. In Q2 2018, business’ preference for online channel has increased while email has decreased1.

BusinessConsumer

Shar

e o

f co

nta

ct m

eth

od

pre

ferr

ed (%

)

Shar

e o

f co

nta

ct m

eth

od

pre

ferr

ed (%

)

Business n=1,536Consumer n=1,602

38

%

17

%

24

%

14

%

3%

1%

38

%

18

%

19

%

15

%

3% 3%

39

%

19

% 22

%

15

%

2%

1%

40

%

17

% 22%

13%

3%

1%

0%

10%

20%

30%

40%

50%



letter, fax


Post

Q2 2017 Q4 2017 Q1 2018 Q2 2018

35

%

24

%

20

%

16

%

3%

1%

33

%

24

%

19

%

11

%

7%

2%

37

%

17

%

25

%

16

%

2%

1%

37

%

17

% 22%

18%

3%

1%

0%

10%

20%

30%

40%

50%



letter, fax


Post

Q2 2017 Q4 2017 Q1 2018 Q2 2018

*Note: Contact methods preferred is a single response question; Figures in the graph does not include the response for “Don’t know/ can’t say” and therefore, may not add up to 100%1Indicative only - does not indicate statistical significance.

Appendix

Section Contents

Appendix A – Demographic Profile of Respondents

Appendix B – Background to the QPCS

Appendix C – Qualitative Research and Online Discussion Forum


30

Consumer Respondent Profile*

49% male51%

female

Gender: Age: Region:18-34

31% 35-5434%

55-6415%

65+20%

Q2 2018 Consumers (n=1,030 respondents)

75% Metro

19% Regional

6% Rural

28% 23%

13% 12%7% 7%

4% 4% 3%

0%

10%

20%

30%

Employment Status:

14%17%

29%

15%

5% 5%

14%

0%

10%

20%

30%

40%

Up to$30,000

$30,001 to$50,000

$50,001 to$100,000

$100,001 to$150,000

$150,001 to$180,000

Over$180,001

Prefer not tosay/Don't

know

Annual Income:

49% male51%

female

Gender: Age:18-34

31% 35-5434%

55-6415%

65+20%

Region: 75% Metro

19% Regional

6% Rural

Employed full time

Retired Full-time domestic

duties

Employed part time

Self-employed/ business owner

Unemployed Student Employed on a casual

basis

Other

27% 25%

12% 10%8%

5% 5% 4% 3%

0%

10%

20%

30%

Employment Status:

Employed full time

Retired Full-time domestic

duties

Employed part time

Unemployed Student Employedon a

casual basis

Self-employed /

business owner

Other

17% 17%

28%

14%

5% 4%

15%

0%

10%

20%

30%

Up to$30,000

$30,001 to$50,000

$50,001 to$100,000

$100,001 to$150,000

$150,001 to$180,000

Over$180,001

Prefer not tosay/Don't

know

Annual Income:

* Data are weighted to be representative of the NSW population (ABS) based on gender, age and region

Q1 2018 Consumers (n=999 respondents)

BusinessConsumer


31

Business Respondent Profile*

45% male55%

female

Gender: Region:

Q2 2018 Business (n=1,003 respondents)

74% Metro

19% Regional

7% Rural

50% male

Gender:Region:

74% Metro

19% Regional

7% Rural

50% female

Business size:

Business size:

13%

6%

13%

7%5%

11%

7%

3%5%

1% 1%

5%

1%3%

1% 0%2% 1%

14%

0%

4%

8%

12%

16%

Pro

fess

ion

al,

scie

nti

fic &

tech

nic

alse

rvic

es

Hea

lth

car

e &

soci

alas

sist

ance

Ret

ail T

rad

e

Educ

atio

n &

trai

nin

g

Art

s &

recr

eat

ion

serv

ices

Co

nst

ruct

ion

Tran

spo

rt,

po

stal

&w

areh

ou

sin

g

Ad

min

istr

ativ

e&

sup

po

rtse

rvic

es

Acc

omm

od

atio

n &

foo

dse

rvic

es

Fina

nci

al &

insu

ran

cese

rvic

es

Who

lesa

leTr

ade

Man

ufa

ctu

rin

g

Agr

icul

ture

,fo

rest

ry &

fish

ing

Ren

tal,

hiri

ng &

real

est

ate

serv

ices

Info

rmat

ion

me

dia

&te

leco

mm

uni

cati

on

s

Ele

ctri

city

, gas

,w

ater

& w

aste

serv

ices

Min

ing

Pub

licad

min

istr

atio

n&

saf

ety Oth

er

Industry:

12% 11%9%

7% 7%6% 4% 4% 4% 4% 3% 3% 3% 3% 2%

1% 1% 1%

14%

0%

4%

8%

12%

16%

Pro

fess

ion

al,

scie

nti

fic a

nd

tech

nic

al s

ervi

ces

Hea

lth

car

e an

dso

cial

ass

ista

nce

Ret

ail T

rad

e

Educ

atio

n an

dtr

ain

ing

Art

s an

dre

cre

atio

nse

rvic

es

Co

nst

ruct

ion

Tran

spo

rt, p

osta

lan

d w

areh

ousi

ng

Ad

min

istr

ativ

ean

d s

up

port

serv

ices

Acc

omm

od

atio

nan

d f

oo

d se

rvic

es

Fina

nci

al a

ndin

sura

nce

serv

ices

Who

lesa

le T

rade

Man

ufa

ctu

rin

g

Agr

icul

ture

,fo

rest

ry a

ndfi

shin

g

Ren

tal,

hiri

ng a

nd

real

est

ate

serv

ices

Info

rmat

ion

me

dia

and

tele

com

mu

nica

tio

ns

Ele

ctri

city

, gas

,w

ater

an

d w

aste

serv

ices M

inin

g

Pub

licad

min

istr

atio

nan

d s

afet

y

Oth

er

Industry:

Q1 2018 Business (n=252 respondents)

* Demographic profile distribution data is weighted to be representative of the NSW population (ABS) based on business size and region

BusinessConsumer

20+3%

Sole Proprietor

30%2-939%

10-1928%

20+3%

Sole Proprietor

26%2-940%

10-1931%

Improving customer satisfaction with key Government services is one of the Premier’s Priorities in the NSW State Plan.

The Quarterly Pulse Check Survey was developed in 2016 as part of the broader customer satisfaction program of work, piloted in Q4 2016 and continued into Q2 2018. In conjunction with the annual Customer Satisfaction Measurement Survey (CSMS), outputs are used to measure progress against the Premier’s Priority 12 – ‘improve customer satisfaction with key Government services every year, this term of government’.

Appendix B – Background to the Survey

32

2013CSMS

Development of the Annual Customer

Satisfaction Measurement Survey (CSMS)

jointly developed by the Customer

Service Commissioner and the Public Sector Commissioner.

2014CSMS

Annual CSMS pilot with 6,208 customers

(consumers and businesses).

Findings used to shape

improvements to instrument and

the implementation of

approach.

2015 Annual CSMS launch

completed with 4,137 consumers

and 1,126 businesses.

2016CSMS

2016 Annual CSMS completed

with 4,237 consumers and

1,132 businesses.

The QPCS methodology was

developed and piloted with 500 consumers and

businesses.

The results of the Annual CSMS and Quarterly Pulse Check Surveys are also used to complement existing Agency level research programmes and provide important information for Agencies to continue shaping and refining their strategies.

Q4 2016QPCS

Q4 2016 QPCS


and 255 businesses.

Q1 2017QPCS

Q1 2017 QPCS


and 262 businesses.

Q2 2017QPCS

Q2 2017 QPCS


and 255 businesses.

2017 CSMS

2017 CSMS completed with

4,013 consumers and 1,133 businesses.

Q42017QPCS

Q4 2017 QPCS


and 241 businesses.

Q12018QPCS

Q1 2018 QPCS

completed with 999

consumers and 252

businesses.

Q22018QPCS

Q2 2018 QPCS

completed with 1,030 consumers and 1,003 businesses.

2015CSMS

Appendix B – Background - Objectives of the QPCS across Key Outputs

33

Driver Deep Dive

Capture breadth: understand respondents’ perceptions across

the drivers of satisfaction identified in the Annual CSMS

Capture depth: deep dive into specific Key Primary Opportunity

Areas identified in the Annual CSMS

Frequency

Provide a more frequent understanding on how New South

Wales Government services are performing overall, interpreted in

the context of the Premier’s Priority 12

+

Lead indicator for Premier’s Priority 12 - respondents’ perceptions of

NSW Government services overall, captured by the Customer Satisfaction Index and its

composite measures (overall satisfaction,

expectation, ideal service)

Breadth: more frequent understanding of the 8 drivers of

satisfaction.

Depth: deeper understanding of Key Primary Opportunity Areas captured in the Annual CSMS

(Efficiency and effectiveness of employees and Access to

information)

+

Customer Satisfaction Index & Baseline Measures

Deep Dive into Driver Performance

Project Objectives:

Research Outputs:

Fixed section Fixed + rotating variable section

Appendix B – Background - Research Scope and Approach

34

• The QPCS Methodology is aligned to the Annual Customer Satisfaction

Measurement Survey (CSMS) approach:

• Captures feedback across 23 different NSW Government services (described

in the customers language).

• Feedback received from customers about each of the individual services is

aggregated to provide a view of the performance of NSW Government

services overall.

• Each respondent provides feedback regarding 1 or 2 services (as a result,

the total number of responses received across services is greater than the

total number of customers who completed the survey).

• The survey was completed from 23rd April 2018 to 26th April 2018 and results are

therefore reflective of experiences with services over the six months prior i.e. from

November 2017 to April 2018.

• The Q2 2018 QPCS was completed with:

• N = 1,030 consumers, and

• N = 1,003 businesses.

• As each respondent provides feedback regarding one or two services, the Q2 2018

QPCS number of responses:

• N = 1,659 for consumers, and

• N = 1,577 for businesses.

• All scores reported in this document are out of 10, with the exception of the

Customer Satisfaction Index which is out of 100.

In scope services

Industry• Agricultural Advice and

Funding Services• Business Advisory Services • Water Supply• TAFE Services

Justice • Police • State Emergency Services • Prisons • Courts • Fire Brigades

Family & Community Services • Public Housing • Disability Services • Child Protection Services• Services for Older People

Transport• Public Transport • Car and Boat Registration • Major Roads

Finance, Services & Innovation

• Consumer Affairs (Fair Trading)

Planning & Environment

• Environment and Wildlife Protection

• Art Galleries and Museums

Education

• Public Schools

Health

• Public Hospitals

• Ambulance Services

Multiple clusters

• Documentation Services (including certificates for births deaths and marriages; trade licenses and certificates; and drivers licenses)

Appendix B – Background - Linkage Between the Annual CSMS and QPCS

35

Key Features 2017 Annual CSMS Q2 2017 QPCS Q4 2017 QPCS Q1 2018 QPCS Q2 2018 QPCS

Qualifying Criteria and Measurement Period

• Direct dealings with NSW public services within last 12 months

• 2017 CSMS results are reflective of experiences with services between May 2016 and May 2017

• Direct dealings with NSW public services within last 6 months

• Q2 2017 QPCS results are reflective of experiences with services between November 2016 and April 2017.

• Q4 2017 QPCS results are reflective of experiences with services between April 2017 and October 2017.

• Q1 2018 QPCS results are reflective of experiences with services between August 2017 and January 2018.

• Q2 2018 QPCS results are reflective of experiences with services between November 2017 and April 2018.

Customer Satisfaction Index

• All measure expectations, satisfaction and comparison with ideal service

• All measure expectations, satisfaction and comparison with ideal service

Perceptions of Services& Satisfaction Drivers

• Measures perceptions of attributes• Drivers of satisfaction are derived

from measurement at an attribute level for all drivers

• Measures perceptions of satisfaction drivers (asked directly)• Drivers of satisfaction as identified by the CSMS

Sample Size• Consumer n=4,013;• Business n=1,133

• Consumer n=1,005;• Business n=255

• Consumer n=1,002;• Business n=241

• Consumer n=999;• Business n=252

• Consumer n=1,030;• Business n=1,003

Margin of Error for Customer Satisfaction Index (95% Confidence Interval)

• Consumer is ± 0.6• Business is ± 1.1





Recency of Experience

Consumers:• 0-3 months: 57%• 3-6 months: 22%• 6-12 months: 21%

Businesses:• 0-3 months: 50%• 3-6 months: 25%• 6-12 months: 25%

Consumers:• 0-3 months: 63%• 3-6 months: 37%

Businesses:• 0-3 months: 61%• 3-6 months: 39%







The table below provides an overview of the differences between the QPCS and CSMS:

Appendix B – Background - Key Considerations for Interpreting QPCS Insights

36

General Considerations:

• The QPCS results do not replace the Annual CSMS results, but rather

provides a directional indication of the shift in the results.

• Although the QPCS sample characteristics are closely representative

of the NSW population, different customers have been surveyed and

as such the results are directional indicators of shifts in the Annual

CSMS results only.

• The margin of error (MoE) for the QPCS needs to be considered

when interpreting the results.Q1 2017 Q2 2017 2017

CSMSQ4 2017 Q1 2018 Q2 2018

1. Set a benchmark with the first data point

Interpretation Plan for Tracking Study

2. Draw insights by comparing to previous quarter

3. Form a directional trend

We are here

Time

Mea

sure

Illustration only - not real data

Considerations for interpreting the QPCS data points:

• The QPCS results need to be interpreted in the context of the time of the year and in light of events in order to normalise seasonal trends in

the data. Therefore, overall caution should be taken when interpreting the QPCS findings until a minimum of a full year of results has been

collected, so that any seasonal impacts can be examined and adjusted accordingly.

• Contextual factors for the following results (Q2 2018) have been considered and include factors such as train delays and strikes, stadium

upgrades and light rail construction. Contextual factors will be further explored as part of the analysis of the summary report.

• In the following slides, the results of Q2 2018 QPCS have been compared to the results of Q2 2017 QPCS and Q1 2018 QPCS. Significance

testing is based on the comparison to Q2 2017 and Q1 2018 results at 95% confidence level, which was the recommended and endorsed

approach for trend analysis as part of the QPCS Methodology Report (as opposed to 99% confidence level for CSMS).

• The Annual CSMS results have been provided as additional context for the QPCS data point and should not be used as a comparison to QPCS

results.

• A longitudinal dataset will need to be built over time in order to identify 'real' trends in the QPCS results and to strengthen the reliability and

validity of any conclusions drawn.

Appendix C – QPCS Qualitative Research Objectives

37

Qualitative research complements the QPCS survey results by providing further insights and context into the results of the quantitative survey.

Q2 2018 QPCS Qualitative Research:

An online discussion forum was trialed as a new qualitative research approach in Q2 2018 QPCS. There were 2 online discussion forums, a consumer group and a business group. Participants from each group were recruited from the respondents of Q2 2018 QPCS survey with a mix of demographics.

The online discussion forum ran from 15th to 17th May 2018. Participants were required to answer all the pre-designed questions and encouraged to comment on other people’s posts. During the 3-day discussion, moderators monitored the two forums in parallel. Follow up questions were posted which prompted to participants to make sure sufficient insights were captured.

The final responses came in both text and video formats which provided an in-depth understanding of baseline measures of Q2 2018.

38

Appendix C – Demographics of Online Forum Participants

Consumers Discussion Group Business Discussion Group

Number of Participants

Days

Day 1 23 (195 posts)



Total 22 (564 posts)

Video content uploads

23

GenderFemale 10

Male 12

Age

18-24 1

25-44 8

45-54 4

55-64 3

65+ 6

Location

Metropolitan 18

Regional 4

Rural 0

Total participants over 3 days 22

Number of Participants

Days




Total 19 (456 posts)

Video content uploads

0

GenderFemale 10

Male 9

Business size (number of employees)

1-19 11

20-199 4

200+ 4

Business stage

Startup 2

Mature 9

Growth and establishment

6

Expansion 2

Location

Metropolitan 17

Regional 1

Rural 1

Total participants over 3 days 19