2019 q2 quarterly pulse check survey (qpcs) · 2020. 3. 11. · is close to meeting these...
TRANSCRIPT
2019 Q2 Quarterly Pulse Check Survey (QPCS)
Summary Report
Please note that this report is not web accessible.
The accessible version is coming soon.
Item Page no.
1
Executive Summary- Customer Satisfaction Index (CSI)- Key Insights 2
2
Customer Satisfaction Measures- Customer Satisfaction Index (CSI) - Consumer- Satisfaction, Expectation and Comparison to Ideal - Consumer- Findings on Key Findings - Qualitative Research for Consumer- Customer Satisfaction Index (CSI) - Business- Satisfaction, Expectation and Comparison to Ideal - Business- Findings on Key Findings- Qualitative Research for Business- Comparison to Business Confidence Indices - Customer Effort Score- Customer Effort Comparison Score
7
3
Insights on Satisfaction Drivers- Impact of Satisfaction Drivers – Consumer- Impact of Satisfaction Drivers – Business- Key Drivers of Satisfaction – Qualitative Research
20
4
Channel Usage and Preference- Channel Usage and Preference - Consumer- Channel Usage and Preference - Business- Impact of Channel Expectations – Qualitative Research
26
5
Appendix- Appendix A: Demographic Profile of Respondents- Appendix B: Background to the QPCS- Appendix C: QPCS Qualitative and Quantitative Research Approach- Appendix D: Historical Consumer and Business Baseline Measures- Appendix E: Customer Satisfaction Index Q1 2019 QPCS (CSMS at 99% Confidence Level)- Appendix F: Terminology Definitions
30
Table of Contents
1
1. Executive Summary
1.1 Customer Satisfaction Index Scores (CSI) - Consumer
1.2 Key Insights - Consumer
1.3 Customer Satisfaction Index Scores (CSI) – Business
1.4 Key Insights - Business
2
Consumer – CSI
Consumer – Baseline Measures
Margin of Error (QPCS)Consumer CSI: ± 1.3
Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selectionFor comparison between CSMS and QPCS results, please refer to Appendix Section for “Interpretation notes for Q2 QPCS report»For CSMS at 99% confidence level, please refer to Appendix «page 43»
76.7
78.7 79.3 79.9 79.6 78.979.9 79.4
80.9
CSMS2015
(n=6,549)
CSMS2016
(n=6,971)
CSMS2017
(n=6,527)
Q12018
(n=1,610)
Q22018
(n=1,603)
CSMS2018
(n=6,701)
Q42018
(n=1,535)
Q12019
(n=1,575)
Q22019
(n=1,555)
7.3
7.57.7 7.7 7.7
7.6
7.87.7
7.87.7
7.87.9
7.9 7.9 7.98.0
7.8
8.0
6.9
7.2 7.2 7.2 7.37.2
7.3 7.2
7.4
6.66.87.07.27.47.67.88.08.28.48.6
CSMS2015
(n=6,593)
CSMS2016
(n=7,015)
CSMS2017
(n=6,559)
Q12018
(n=1,628)
Q22018
(n=1,618)
CSMS2018
(n=6,733)
Q42018
(n=1,545)
Q12019
(n=1,590)
Q22019
(n=1,564)
Satisfaction Expectation Comparison to Ideal
1.1 Customer Satisfaction Index (CSI) - Consumer
Consumer
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
3
Figures in the graph are subject to rounding
• Consumer CSI has increased (statistically significant) to 80.9 from 79.4.
• The significant increase in CSI is above Premier’s Priority Target of 79.02. This is the highest recorded CSI indicating strong performance this quarter.
1.2 Key Insights - Consumer
1. The consumer CSI has increased (statistically significant). This indicates an overall uplift in experiences with NSW Government services.
4
ConsumerSatisfaction
Index
Consumer
• Of the employees related drivers, Honesty and integrity, Efficiency and effectiveness, Empathy and communication and Accountability all experienced a statistically significant increase since last quarter.
• All employee related drivers are highly correlated to overall satisfaction indicating that overall experience with employees is having a significant impact on improving customer experience.
• Of the values related drivers, Service quality and Accountability, have increased (statistically significant) since last quarter. Value related drivers reflect consumers’ perceptions of trust and outcomes delivered.
3a. Perceptions of employees have significantly improved this quarter.
Satisfaction Drivers
ConsumerSatisfaction
Index
2. All three measures of customer experience (satisfaction, expectation and comparison to ideal) experienced a statistically significant increase.
• The increase in the CSI can be attributed to increases (statistically significant) since last quarter across all three baseline measures: expectation, satisfaction and comparison to ideal.
• Comparison to ideal has experienced its first statistically significant increase since the 2016 CSMS result. This result indicates that NSW Government services are moving closer towards consumers’ ideal service experience.
• With satisfaction and expectation increasing significantly, this result indicates consumers have increasing positive views of NSW Government services. The satisfaction gap to expectation of 0.2 indicates that although consumers continue to have high expectations of NSW Government services, service performance is close to meeting these expectations.
3b. Perceptions of values have significantly improved this quarter.
76.678.4 78.3
79.6 79.778.2
81.3 81.2 80.7
CSMS2015
(n=1,645)
CSMS2016
(n=1,712)
CSMS2017
(n=1,638)
Q12018
(n=365)
Q22018
(n=1,536)
CSMS2018
(n=1,494)
Q42018
(n=1,523)
Q12019
(n=1,449)
Q22019
(n=1,372)
Business – CSI
Business – Baseline Measures
Margin of Error (QPCS)Business CSI: ± 1.2
Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selectionFor comparison between CSMS and QPCS results, please refer to Appendix Section for “Interpretation notes for Q2 QPCS report»For CSMS at 99% confidence level, please refer to Appendix «page 43»
7.2
7.5 7.57.6
7.8
7.4
7.9 7.8 7.87.6
7.9
7.7 7.7
8.1
7.7
8.0 8.07.9
7.0
7.37.1
7.2 7.3 7.27.4
7.6
7.4
6.6
7.1
7.6
8.1
8.6
CSMS2015
(n=1,654)
CSMS2016
(n=1,718)
CSMS2017
(n=1,646)
Q12018
(n=367)
Q22018
(n=1,554)
CSMS2018
(n=1,506)
Q42018
(n=1,535)
Q12019
(n=1,455)
Q22019
(n=1,378)
Satisfaction Expectation Comparison to Ideal
1.3 Customer Satisfaction Index (CSI) – Business
Business
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
5
Figures in the graph are subject to rounding
1.4 Key Insights – Business
• Business CSI has decreased (not statistically significant) to 80.7 from 81.2. This result is below the Premiers Priority Target of 81.16. This decrease in CSI is due to the stabilisation of overall satisfaction and the decrease in expectation and comparison to ideal scores (statistically significant) compared to last quarter.
• The satisfaction gap to expectation has decreased by 0.1 compared to last quarter. This result indicates that although perceptions of performance are moving closer to meeting expectations, businesses have decreasing expectations of NSW Government services.
• The Roy Morgan Business Confidence index* decreased by 3 points in the last quarter from 105.9 (January 2019) to 102.9 (April 2019). The decrease in business confidence mirrors the slight decrease in the business CSI seen this quarter.
4. The business CSI has seen a slight decrease in performance. Satisfaction and expectations have stabilised and comparison to ideal has decreased.
Business
Business Satisfaction
• All satisfaction drivers across employees, values, processes and goals have stabilised as compared to last quarter.
• Following the significant uplift in driver performance in Q4 2018, this performance has been maintained this quarter, with no significant decrease for any driver.
6. Perceptions of service quality (measured through driver performance) has stabilised this quarter.
Satisfaction Drivers
6
• The business customer effort score (CES) has decreased (statistically significant) to 6.1 this quarter from 6.4 last quarter resulting in NSW Government services having the 2nd lowest effort when compared to other industries This indicates that businesses are finding it easier to interact with NSW Government services.
• The result this quarter is consistent with historical trends observed since 2017, where business customer effort decreased from Q1 to Q2.
• This decrease in business effort mirrors a decrease in effort across all other sectors, except banks. Notably, banks experienced a statistically significant increase in effort when compared to Q2 2018.
• Although the decrease in effort is a positive change in businesses overall experience with NSW Government services, this decrease does not influence the CSI as the CSI and CES are not correlated.
5. The businesscustomer effort score has decreased (statistically significant).
CustomerEffort
Business Customer
Effort Score
*Source: Roy Morgan Business Confidence poll April 2019
2. Customer Satisfaction Measures
2.1 Customer Satisfaction Index (CSI) - Consumer
2.2 Satisfaction, Expectation and Comparison to Ideal - Consumer
2.3 Findings on Key Measures - Qualitative Research for Consumer
2.4 Customer Satisfaction Index (CSI) - Business
2.5 Business Confidence
2.6 Satisfaction, Expectation and Comparison to Ideal – Business
2.7 Findings on Key Measures - Qualitative Research for Business
2.8 Customer Effort Score
2.9 Customer Effort Comparison Score
7
Source: NSW Office of the Customer Service Commissioner, Customer Satisfaction Measurement Survey 2019 Source: NSW Office of the Customer Service Commissioner, Quarterly Pulse Check Survey 2019
Note: All the numbers are subject to rounding.
76.7
78.777.8
79.578.6
79.380.2 79.9 79.6
78.979.9 79.4
80.9
CSMS 2015(n=6,549)
CSMS 2016(n=6,971)
Q4 2016(n=1,612)
Q1 2017(n=1,540)
Q2 2017(n=1,594)
CSMS 2017(n=6,527)
Q4 2017(n=1,508)
Q1 2018(n=1,610)
Q2 2018(n=1,603)
CSMS 2018(n=6,701)
Q4 2018(n=1,535)
Q1 2019(n=1,575)
Q2 2019(n=1,555)
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Premier’s Target -79.02
Q2 2019 QPCS Survey Details: Number of respondents: 1,003 consumers (1,600 responses)Fieldwork period: from 15 April to 23 April 2019
Consumer
Quarter-on-quarter comparison of QPCS results (Q1 2019 vs. Q2 2019)
• Customer Satisfaction Index (CSI) has increased by 1.5 points (statistically significant) to 80.9/100 from 79.4/100 (Q1 2019).
QPCS Customer Satisfaction Index (CSI) has statistically significantly increased to 80.9/100 when compared to Q1 2019 and Q2 2018, bringing it well above
the Premier’s target of 79.2 and making it the highest CSI the QPCS and CSMS has ever seen.
Year-on-year comparison of QPCS results (Q2 2018 vs. Q2 2019)
• CSI has increased by 1.3 points (statistically significant) to 80.9/100 from 79.6/100 (Q2 2018).
2.1 Customer Satisfaction Index (CSI) - Consumer
8
7.3
7.57.4
7.6 7.67.7 7.7 7.7 7.7 7.6
7.8 7.77.8
7.7
7.8 7.8
7.97.8
7.9
7.87.9 7.9 7.9
8.0
7.8
8.0
6.9
7.2 7.2
7.3
7.2 7.27.3
7.2 7.37.2
7.3 7.2
7.4
6.7
7.2
7.7
8.2
CSMS 2015(n=6,593)
CSMS 2016(n=7,015)
Q4 2016(n=1,621)
Q1 2017(n=1,548)
Q2 2017(n=1,605)
CSMS 2017(n=6,559)
Q4 2017(n=1,518)
Q1 2018(n=1,628)
Q2 2018(n=1,618)
CSMS 2018(n=6,733)
Q4 2018(n=1,545)
Q1 2019(n=1,590)
Q2 2019(n=1,564)
Satisfaction
Expectation
Comparison toideal
2.2 Satisfaction, Expectation and Comparison to Ideal – Consumer (1/2)
Significant changes CSMS 2015*
CSMS 2016
Q4 2016 Q1 2017 Q2 2017 CSMS 2017
Q4 2017 Q1 2018 Q2 2018 CSMS 2018
Q4 2018 Q1 2019 Q2 2019
Satisfaction NA - - - - -
Expectation NA - - - - -
Comparison to ideal NA - - - - - - - -
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
No significant movement from previous quarter or same quarter previous year at 95% confidence level
* Data for comparison with previous period not available
-Note: Figures in the graph are subject to rounding
Consumer
The CSI is composite of the following key measures: expectation, satisfaction and comparison to idea service. All three key measures experienced a
statistically significant increase when compared to Q1 2019 attributing to the significant uplift in the CSI. The satisfaction gap to expectation has widened
slightly by 0.1. This is due to a larger increase in expectation than satisfaction this quarter. This result indicates that consumers continue to have high
expectation of NSW Government services but service performance is close to meeting these high expectation.
Quarter-on-quarter comparison of QPCS results (Q1 2019 vs. Q2 2019)
• Expectation has increased by 0.2 points (statistically significant) to 8.0/10 from 7.8/10 (Q1 2019).
• Satisfaction has increased by 0.1 points (statistically significant) to 7.8/10 from 7.7/10 (Q1 2019).
• Comparison to ideal service has increased by 0.2 points (statistically significant) to 7.4/10 (Q1 2019) from 7.2/10 (Q1 2019).
Year-on-year comparison of QPCS results (Q2 2018 vs. Q2 2019)
• Expectation has increased by 0.1 points (not statistically significant) to 8.0/10 from 7.9/10 (Q2 2018).
• Satisfaction has increased by 0.1 points (statistically significant) to 7.8/10 from 7.7/10 (Q2 2018).
• Comparison to ideal service has increased by 0.1 points (statistically significant) to 7.4/10 from 7.3/10 (Q2 2018).
9
2.2 Satisfaction, Expectation and Comparison to Ideal – Consumer (2/2)
Consumer – Outcome Measures
Expectation
Average (out of 10)
Satisfaction Comparison to Ideal
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
7% 6% 6% 8% 5% 9% 8% 8% 9% 8% 12% 11% 13% 13% 11%12% 13% 11% 12% 13%
13% 15% 12% 15% 13%17% 20% 16% 16% 14%
81% 81% 84% 80% 82% 78% 77% 80% 77% 80%71% 69% 72% 71% 75%
Q2 2018(n=1,642)
CSMS 2018(n=6,922)
Q4 2018(n=1,579)
Q1 2019(n=1,614)
Q2 2019(n=1,578)
Q2 2018(n=1,638)
CSMS 2018(n=7,000)
Q4 2018(n=1,583)
Q1 2019(n=1,611)
Q2 2019(n=1,577)
Q2 2018(n=1,618)
CSMS 2018(n=6,733)
Q4 2018(n=1,545)
Q1 2019(n=1,590)
Q2 2019(n=1,564)
Low (1-4) Med (5-6) High (7-10)
Consumer
7.9 7.9 8.0 8.0 7.7 7.6 7.8 7.7 7.8 7.3 7.2 7.3 7.2 7.4
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
CSMS results
10
7.8
Note: Figures in the graph are subject to roundingPlease refer to appendix section “Historical Consumer and Businesses Baseline Measures” for more historical data
Compared to the last quarter, the increase in expectation, satisfaction and comparison to ideal is being driven by a 2%, 3% and 4% increase in consumers who rate their experience as high (i.e. 7-10 out of 10). This increase is expectation, satisfaction and comparison to ideal is further driven by a 3%, 1% and 2% decrease in consumer’s who rate their experience as low (i.e. 1-4 out of 10).
Comparison to Ideal experienced its first statistically significant increase since Q1 2019 indicating that NSW Government is moving closer to delivering consumers’ ideal services.
I felt really close and warm when I met the personnel from the school, it gave me the feeling of trust. This can be maintained as
long as those people happy and their expectation from the government is met
I think there have been a lot of changes implemented in the last 12 months, change to the service centres, being open all weekend,
Green slip rebate are all good positive changes to happen. The new centres are so much brighter, open, friendly and staff are pleasant
and nice to you
• Overall, there has been positive sentiment supporting the increase
in CSI for NSW Government services
• Consumers positively mentioned that having the option to utilise
online channels and services provides them greater flexibility to
complete processes and receive status updates via their preferred
channel when interacting with government services
• Consumers have also expressed positivity to government services
due to recent improvements in infrastructure supporting service
delivery such as refurbished service centres. Customers mention
that these changes make it easier to interact with services due to
greater availability of staff and information, and increase time
efficiencies during dealings with government services
Consumers show positive sentiment towards changes in services
Insights from Q2 Qualitative Discussion
• Consumers have expressed feeling a sense of trust and reliability
with the service if they are met with staff that display empathy and
effective communication during the process
• Consumers are increasingly impressed by the product knowledge
and empathy of staff especially within the service categories of
emergency services and social services. Consumers have stated
that this has allowed them to build greater trust with services
Employee drivers have contributed to increasing consumer satisfaction
Shout out to the very helpful and delightful employee at Service NSW Brookvale who guided me through and sped up process of applying
for a replacement driver licence last month
“
“
“
“”
2.3 Findings on Key Measures – Qualitative
When I think about the changes I’ve seen to government services in last 12 months, I feel relief because, most of the information and
enquiries can be solved and done online. I really don’t want to waste tons of time by visiting centres
“
Consumer
”My experience has improved over the last few months by virtue of the move to making services more accessible through online self service portals. This reflects an understanding on behalf of the
government the importance of allowing users to access services they need in a flexible manner
“
”
”
11
”
Source: NSW Office of the Customer Service Commissioner, Customer Satisfaction Measurement Survey 2019 Source: NSW Office of the Customer Service Commissioner, Quarterly Pulse Check Survey 2019
Note: All the numbers are subject to rounding.
Q2 2019 QPCS Survey Details: Number of respondents: 1,009 businesses (1,394 total responses)Fieldwork period: from 15 April to 23 April 2019
2.4 Customer Satisfaction Index (CSI) – Business
76.6
78.4
76.1
77.6 77.678.3
77.8
79.6 79.7
78.2
81.3 81.280.7
CSMS 2015(n=1,645)
CSMS 2016(n=1,712)
Q4 2016(n=375)
Q1 2017(n=380)
Q2 2017(n=365)
CSMS 2017(n=1,638)
Q4 2017(n=343)
Q1 2018(n=365)
Q2 2018(n=1,536)
CSMS 2018(n=1,494)
Q4 2018(n=1,523)
Q1 2019(n=1,449)
Q2 2019(n=1,372)
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Premier’s Target –81.16
Business
Quarter-on-quarter comparison of QPCS results (Q1 2019 vs. Q2 2019)
• Customer Satisfaction Index (CSI) has decreased by 0.5 points (not statistically significant) to 80.7/100 from 81.2/100 (Q1 2019).
For Business, the QPCS Customer Satisfaction Index (CSI) has decreased to 80.7/100 (not statistically significant) when compared to the last quarter but
increased by 1.0 points when compared to Q2 2018. The slight decrease in CSI is due to the stabilisation of the overall performance of satisfaction and
expectation at a high level.
Year-on-year comparison of QPCS results (Q2 2018 vs. Q2 2019)
• CSI has increased by 1.0 points (not statistically significant) to 80.7/100 from 79.7/100 (Q2 2018).
12
2.5 Business Confidence
The decrease in business CSI mirrors the results of the Roy Morgan Business Confidence Index and the Sensis Business Confidence Index. The Roy Morgan index experienced a decrease of 3.0 when compared to Q1 2019 and the Sensis Index decreased by 16 when compared to Q1 2019. The trends of the Roy Morgan Index, Sensis Index and the business CSI indicates that businesses have decreasing confidence and perceptions of their ability to effectively manage their business in the current economic environment.
Business
13Note: Figures in the graph are subject to rounding
Comparison of Business CSI and Business Confidence Measures
About the Roy Morgan Business Confidence Index: The Index is based on 5 different attributes measuring Australian business’ expectations of the economic climate over the next 12 months. (n=~1,000 Australian businesses per month). Data points for the Roy Morgan Business Confidence Index are taken from the corresponding month of QPCS/CSMS fieldwork. http://www.roymorgan.com/morganpoll/consumer-confidence/roy-morgan-business-confidence.
Business Confidence (Roy Morgan Index) and Business CSI
79.59 79.69
75.92
81.2581.20
80.70
114
117 124120
115
119117
114 114
110112
113 113.8112.2
105.9 105.6106.7
102.9
100
105
110
115
120
125
130
75
82
No
v-1
7
Dec
-17
Jan
-18
Feb
-18
Mar
-18
Ap
r-18
May
-18
Jun
-18
Jul-
18
Au
g-18
Sep
-18
Oct
-18
No
v-1
8
Dec
-18
Jan
-19
Feb
-19
Mar
-19
Ap
r-19
Bu
siness C
on
fiden
ce Ind
ex
CSI
CSI Roy Morgan Business Confidence Index*
75.92
80.70
42
49
42
50
34
30
35
40
45
50
55
60
75
82
Mar
-18
Jun
-18
Sep
-18
Dec
-18
Mar
-19
Bu
siness C
on
fiden
ce Ind
ex
CSI
CSI Sensis Business Confidence Index
Business Confidence (Sensis) and Business CSI
About the Sensis Business Confidence Index:. This index is calculated by subtracting the number of respondents who are negative about the current economic climate from the number that are positive (n=1,000 small and medium Australian businesses per quarter). Data points for the Sensis index are taken from the corresponding month of QPCS/CSMS fieldwork. https://www.sensis.com.au/about/sensis-business-index
Source: NSW Office of the Customer Service Commissioner, Customer Satisfaction Measurement Survey 2019 Source: NSW Office of the Customer Service Commissioner, Quarterly Pulse Check Survey 2019
7.2
7.5 7.2
7.4
7.37.5 7.5
7.6
7.8
7.4
7.9
7.8
7.8
7.6
7.9
7.6
8.1
7.5
7.77.6
7.7
8.1
7.7
8.0 8.07.9
7.0
7.3
7.1
7.4
7.1 7.1 7.0
7.27.3
7.2
7.4
7.6
7.4
6.7
6.9
7.1
7.3
7.5
7.7
7.9
8.1
8.3
CSMS 2015(n=1,654)
CSMS 2016(n=1,718)
Q4 2016(n=375)
Q1 2017(n=380)
Q2 2017(n=365)
CSMS 2017(n=1,646)
Q4 2017(n=345)
Q1 2018(n=367)
Q2 2018(n=1,554)
CSMS 2018(n=1,506)
Q4 2018(n=1,535)
Q1 2019(n=1,455)
Q2 2019(n=1,378)
Satisfaction
Expectation
Comparison toideal
Significant changes CSMS 2015*
CSMS 2016
Q4 2016 Q1 2017 Q2 2017 CSMS 2017
Q4 2017 Q1 2018 Q2 2018 CSMS 2018
Q4 2018 Q1 2019 Q2 2019
Satisfaction NA - - - - - -
Expectation NA -
Comparison to ideal NA - - - - - - -
Quarter-on-quarter comparison of QPCS results (Q1 2019 vs. Q2 2019)
• Expectation has decreased by 0.1 points (not statistically significant) to 7.9/10 from 8.0/10 (Q1 2019).
• Satisfaction has remained stable at 7.8/10.
• Comparison to ideal service has decreased by 0.2 points (statistically significant) to 7.4/10 from 7.6/10 (Q1 2019).
Note: Figures in the graph are subject to rounding
* Data for comparison with previous period not available
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
No significant movement from previous quarter or same quarter previous year at 95% confidence level-
Business
2.6 Satisfaction, Expectation and Comparison to Ideal – Business (1/2)
The CSI is composite of the following key measures: expectation, satisfaction and comparison to idea service. The satisfaction gap to expectation has
narrowed to 0.1 as a result of satisfaction stabilising and expectation slightly increasing.
Quarter-on-quarter comparison of QPCS results (Q1 2019 vs. Q2 2019)
• Expectation has decreased by 0.1 points (not statistically significant) to 7.9/10 from 8.0/10 (Q1 2019).
• Satisfaction has remained stable at 7.8/10.
• Comparison to ideal service has decreased by 0.2 points (statistically significant) to 7.4/10 from 7.6/10 (Q1 2019).
Year-on-year comparison of QPCS results (Q2 2018 vs. Q2 2019)
• Expectation has decreased by 0.2 points (statistically significant) to 7.9/10 from 8.1/10 (Q2 2018).
• Satisfaction has remained stable at 7.8/10.
• Comparison to ideal service has increased by 0.1 points (not statistically significant) to 7.4/10 from 7.3/10 (Q2 2018).
14
7.7
2.6 Satisfaction, Expectation and Comparison to Ideal – Business (2/2)
Business – Outcome Measures
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
Average (out of 10)
Expectation Satisfaction Comparison to Ideal
6% 6% 5% 6% 5% 9% 7% 5% 9% 8% 12% 11% 9% 8% 10%9% 15% 14% 11% 14% 11% 18% 16% 12% 14%
15% 19% 18% 13%17%
84% 79% 82% 83% 81% 80% 75% 79% 79% 79% 73% 71% 73% 79% 73%
Q2 2018(n=1,555)
CSMS 2018(n=1,531)
Q4 2018(n=1,552)
Q1 2019(n=1,465)
Q2 2019(n=1,389)
Q2 2018(n=1,570)
CSMS 2018(n=1,555)
Q4 2018(n=1,553)
Q1 2019(n=1,469)
Q2 2019(n=1,387)
Q2 2018(n=1,554)
CSMS 2018(n=1,506)
Q4 2018(n=1,535)
Q1 2019(n=1,455)
Q2 2019(n=1,378)
Low (1-4) Med (5-6) High (7-10)
Business
8.0 8.0 7.9 7.8 7.4 7.9 7.8 7.8 7.3 7.2 7.4 7.6 7.4
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
CSMS results
15
8.1
Compared to the last quarter, the decrease in comparison to ideal for business is being driven by a 6% decrease in the businesses that rate their experience as high (i.e. 7-10 out of 10).
Note: Figures in the graph are subject to roundingPlease refer to appendix section “Historical Consumer and Businesses Baseline Measures” for more historical data
Duplicating steps is frustrating, a waste of time and makes me question whether the process I'm following is accurate or not. I don't want to be led down a rabbit hole where you need to go to various
sites, departments, organisations just to complete one process
I feel really happy as they have improved immensely by making customer service a place you can do a lot of different things…a lot
more things can be done online which is much less time consuming …which really helps for small business owners
• Businesses have expressed mixed sentiment towards the changes
within government services and overall satisfaction
• On one side, businesses have mentioned that the introduction of
online channels has improved their experience. It provides greater
convenience and status transparency during the process
• On the other hand, businesses have expressed the need for
effective communication between offline and online channels to
create a more seamless experience. This potentially removes
duplicative steps in the process as businesses need to upload
documents for approval and also prove these documents in person
Businesses show mixed sentiment towards recent changes
16Insights from Q2 Qualitative Discussion
• Businesses want to focus efforts on their business and customers.
Hence, a lack of integration between channels can cause
duplicative steps, reducing the time spent on growing the business
• Businesses mentioned that phone and online channels were
unable to provide a reliable seamless experience with a tailored
service aligned to their needs and situation. Although expectations
have not been met, offline channels are perceived to be more
approachable, as they allow for a more personal interaction
Businesses have a lower expectation towards government processes
Over the phone is always hard as they don't have the customer and their documents in front of them
“
“
“
“
”
”
2.7 Findings on Key Measures – Qualitative
When I think about the changes I’ve seen to government services in the last 12 months I feel in the last 12 months that the government
approach has become more personal and much more approachable. I like that you get an email from a genuine person
rather than just an auto response and a follow up call from the same person really makes you feel like your concerns matter
“
”
”
Business
I would say that a majority of the changes haven't improved my experience with dealing with the relevant Govt. Services.
Compliance has been become a top priority and trying to adhere to some of the changes has been challenging and also frustrating.
“
”
I would be angry I found any duplicative steps in a process as I would find it a waste of time – where I could spend more time on my
business. If there were duplicative steps within a process - it would make me question if it was reputable and trustworthy source or not.
“
”
NSW Government service interaction Customer Effort Score.
2.8 Customer Effort Score
In this section, customers rated the level of effort for individual services; this graph shows the average of those ratings
Q: Thinking about your direct dealing with [SERVICE], how much effort did you personally have to put forth? 1 is ‘Low effort’ and 10 is ‘High effort’
Overall, businesses have to put forth higher effort than consumers when interacting with NSW Government services.Since Q1 2019, the customer effort score (CES) has remained stable among consumers at 5.8/10 and decreased by 0.4 points (statistically significant) among businesses to 6.1/10 from 6.5/10 (Q1 2019). Comparing to Q2 2018, the CES has decreased by 0.2 points (not statistically significant) among consumers to 5.8/10 from 6.0/10 and has increased by 0.1 points (not statistically significant) among businesses to 6.1/10 from 6.0/10.Since 2017, a seasonal decrease in business CES from Q1 to Q2 has been observed.
5.9 5.9
5.75.9
6.15.9
6.0
6.0
5.75.8
5.8
6.3
6.9
6.5
4.9
6.76.9
6.0
6.6
6.1
6.5
6.1
Consumers BusinessesLowerEffort
HigherEffort
Q1 2017(n=1,529)(n=382)
Q2 2017(n=1,590)(n=366)
Q4 2017(n=1,496)(n=345)
Q1 2018(n=1,611)(n=373)
*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift.
Q2 2018(n=1,585)(n=1,535)
Q4 2018(n=1,526)(n=1,508)
Consumer Business
CSMS 2017(n=6,501)(n=1,650)
CSMS 2018(n=6,634)(n=1,496)
Statistically significant movement from same quarter previous year at 95% confidence levelStatistically significant movement from previous quarter at 95% confidence level
CSMS results
17
Q1 2019(n=1,548)(n=1,458)
Q4 2016(n=1,601)(n=378)
Note: Figures in the graph are subject to rounding
Q2 2019(n=1,542)(n=1,360)
ConsumerBusiness
• Businesses have mentioned the
efficiencies of having more digitally
available services
• These services allow businesses to
conduct multiple processes
simultaneously, saving time and
allowing them to be proactively
informed of their status via their
preferred channel
• Information about the process is
proactively provided online and
businesses are able to complete
processes at their own convenience
which reduces the effort during
dealings with government services
Generally being informed … via text works really well and I’m satisfied
”
I find it handy to just go online whilst travelling…also I can renew my
business licences online and get a discount for doing so
“
”“
Insights from Q2 Qualitative Discussion
6.26.0 5.9 6.0 5.8 5.7
5.4
6.1 6.05.8
6.05.6 5.6 5.5
6.15.9 5.8 5.9 5.7 5.5 5.5
6.2 6.0 5.9 5.9 5.8 5.6 5.6
Telephone serviceproviders
NSW Governmentservices
Federal Government Energy retailers My Local Council Banks Airlines
Q2 2018 QPCS (n=711 to 943) Q4 2018 QPCS (n=680 to 929) Q1 2019 QPCS (n=688 to 940) Q2 2019 QPCS (n=713 to 940)
2.9 Customer Effort Score: Comparison of NSW Government Services Overall to Other Industries –Consumer
For this section, customers provided a rating for overall effort with NSW Government services in comparison to other industries/sectors Q: Thinking about all your direct dealings with each of the following Australian industries and Government services over the previous 6 months, how much effort did you personally have to put forth?
CES for NSW Government overall is relatively high when compared to other industries - consumer CES increased by 0.1 points (not statistically significant) to 6.0/10 from 5.9/10 (Q1 2019) placing NSW Government services as the second highest effort behind Telephone service providers.
When compared to the same quarter, previous year (Q2 2018), NSW Government services effort score has remained stable in the relative effort required for interactions among consumers.
Customer Effort Score benchmarked at a Whole of Government level against other industries/sectors.
Consumer
18Note: Figures in the graph are subject to rounding
*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift. Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
6.46.0 5.8
6.16.0 5.8 5.6
6.4 6.25.8
6.25.8 5.9 5.8
6.7
6.4 6.16.4 6.3 6.5
6.16.6
6.2 6.2 6.1 6.1 6.1
5.6
Telephone serviceproviders
Energy retailers Banks NSW Governmentservices
My Local Council Federal Government Airlines
Q2 2018 QPCS (n=726 to 902) Q4 2018 QPCS (n=707 to 922) Q1 2019 QPCS (n=760 to 960) Q2 2019 QPCS (n=793 to 962)
2.9 Customer Effort Score: Comparison of NSW Government Services Overall to Other Industries –Business
Compared to other industries, businesses ranked NSW Government services as the fourth highest in terms of effort, behind Telephone service providers, Energy retailers and Banks. For NSW Government services, businesses reported a decrease of 0.3 points (statistically significant) to 6.1/10 from 6.4/10 (Q1 2019). Businesses reported a decrease in effort for Airlines by 0.5 points (statistically significant) to 5.6/10 from 6.1/10 (Q1 2019) and a decrease in effort for Federal Government by 0.4 points to 6.1/10 from 6.5/10 (Q1 2019).
Business
Q: Thinking about all your direct dealings with each of the following Australian industries and Government services over the previous 6 months, how much effort did you personally have to put forth?
Customer Effort Score benchmarked at a Whole of Government level against other industries/sectors.
19Note: Figures in the graph are subject to rounding
*The lower the average Customer Effort Score, the easier customers perceive interactions to be with different services. A decrease in CES is a positive shift. Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
For this section, customers provided a rating for overall effort with NSW Government services in comparison to other industries/sectors
3. Insights on Satisfaction Drivers
20
3.1 Impact of Satisfaction Drivers – Consumer
3.2 Impact of Satisfaction Drivers – Business
3.3 Key Drivers of Satisfaction – Qualitative Research
7.8
7.5 7.7
7.4 7.7
7.4
6.8
7.4 7.4
7.0 7
.3
7.1
8.0
7.7 7.8
7.6
7.3 7
.8
7.57
.8
7.5 7.7
7.4
7.2 7
.6
7.4
7.9
7.7 7.7
7.6
7.5 7
.8
7.6
Employees were open andhonest during the process
Employees acted efficientlyand effectively to reach the
right outcomes
Communications were clear,prompt and easy to
understand
Employees acted withempathy
Employees were heldaccountable for their actions
I was provided with goodservice and outcomes I could
trust
I felt there was accountabilityfor services delivered
Q2 2018 QPCS (n=1,431 to 1,575) CSMS 2018 (n=5,690 to 6,779) Q4 2018 QPCS (n=1,146 to 1,518) Q1 2019 QPCS (n=1,155 to 1,527) Q2 2019 QPCS (n=1,146 to 1,525)
3.1 Impact of Satisfaction Drivers – Consumer (1/2)D
eriv
ed D
rive
rs
(CSM
S 2
01
8)
Drivers of Satisfaction
Comparing to Q1 2019, the employee drivers honesty and integrity of employees, employees acted with empathy and employee accountability have increased (statistically significant). Looking at the values drivers, service quality and accountability have increased (statistically significant). These attributes are positively correlated to consumer CSI indicating that overall experience with employees is having a significant impact on improving customer experience. When compared to the same time last year (Q2 2018), the employee drivers efficiency and effectiveness of employees and employees acted with empathy have increased (statistically significant).
Ave
rage
(o
ut
of
10
)
Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.
Employees Values
21
n/a
fo
r Q
1 &
Q2
20
18
–n
ew a
ttri
bu
te
intr
od
uce
d f
rom
Q4
20
18
QP
CS
Service Quality AccountabilityAccountabilityHonesty and integrity of
employees
Consumer
Efficiency and effectiveness of employees
Empathy and communication
Note: Figures in the graph are subject to rounding
n/a
fo
rC
SMS
–n
ew a
ttri
bu
te
intr
od
uce
d f
rom
Q4
20
17
QP
CS
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
7.3
7.3
8.0
7.8
7.5
6.9 7
.5
7.57.6
7.4
8.1
7.9
7.7
7.4
7.3
8.0
7.9
7.6
7.5 7.5
8.1
7.9
7.7
The process was simple and efficient Employees took initiative and madedecisions
My privacy was upheld & personalinformation was protected & respected
I understood the steps involved with theprocess
I had good access to information andcould find what I needed
Q2 2018 QPCS (n=1,435 to 1,588) CSMS 2018 (n=5,620 to 6,345) Q4 2018 QPCS (n=1,337 to 1,521) Q1 2019 QPCS (n=1,343 to 1,547) Q2 2019 QPCS (n=1,326 to 1,528)
3.1 Impact of Satisfaction Drivers – Consumer (2/2)D
eriv
ed D
rive
rs
(CSM
S 2
01
8)
Drivers of Satisfaction
Process Goals
22
n/a
–N
ot
a C
SMS
attr
ibu
te
Simplicity and Efficiency of Processes Employee Autonomy Privacy Transparency Access to information
n/a
–N
ot
a C
SMS
attr
ibu
te
Consumer
Note: Figures in the graph are subject to rounding
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
For process and goal drivers in Q2 2019, employee autonomy experienced a statistically significant increase since Q1 2019. When compared to the same time last year (Q2 2018), employee autonomy and access to information experienced a statistically significant increase.
Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.
Ave
rage
(o
ut
of
10
)
8.0
7.5 7.7
7.6 7.7
7.4
6.7
7.5
7.1 7
.4
7.3
7.0
7.9
7.7
7.3 7
.8
7.5 7.8
7.48
.0
7.8
7.5 7
.8
7.7 7.9
7.67
.9
7.7
7.4 7
.8
7.6 7.7
7.5
Employees were open andhonest during the process
Employees acted efficientlyand effectively to reach the
right outcomes
Employees were heldaccountable for their actions
Communications were clear,prompt and easy to
understand
Employees acted withempathy
I was provided with goodservice and outcomes I could
trust
I felt there was accountabilityfor services delivered
Q2 2018 QPCS (n=1,369 to 1,517) CSMS 2018 (n=1,388 to 1,519) Q4 2018 QPCS (n=1,203 to 1,503) Q1 2019 QPCS (n=1,241 to 1,426) Q2 2019 QPCS (n=1,105 to 1,344)
3.2 Impact of Satisfaction Drivers – Business (1/2)D
eriv
ed D
rive
rs
(CSM
S 2
01
8)
Drivers of Satisfaction
Business
Employees Values
23
n/a
fo
r Q
2 2
01
8 –
new
att
rib
ute
in
tro
du
ced
fro
m Q
4 2
01
8 Q
PC
S
Service Quality AccountabilityCustomer focus &
action orientedClear communicationIntegrity and high standards
n/a
for
CSM
S –
new
att
rib
ute
in
tro
du
ced
fro
m Q
4 2
01
7 Q
PC
S
Note: Figures in the graph are subject to rounding
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
In Q2 2019, there was no statistically significant changes for the drivers since Q1 2019. Similarly, when compared to the same time last year (Q2 2018) all drivers did not experience any statistically significant movement.
Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.
Ave
rage
(o
ut
of
10
)
7.4
7.4
8.2
7.9
7.7
6.8 7
.3 7.47.5
7.5
8.1
8.0
7.7
7.6
7.6 8
.1
8.0
7.8
7.5
7.5
8.1
8.0
7.7
The process was simple and efficient Employees took initiative and madedecisions
My privacy was upheld & personalinformation was protected & respected
I understood the steps involved with theprocess
I had good access to information andcould find what I needed
Q2 2018 QPCS (n=1,358 to 1,524) CSMS 2018 (n=1,405 to 1,461) Q4 2018 QPCS (n=1,350 to 1,517) Q1 2019 QPCS (n=1,327 to 1,434) Q2 2019 QPCS (n=1,224 to 1,360)
3.2 Impact of Satisfaction Drivers – Business (2/2)D
eriv
ed D
rive
rs
(CSM
S 2
01
8)
Drivers of Satisfaction
Business
Process Goals
24
n/a
–N
ot
a C
SMS
attr
ibu
te
Simplicity and Efficiency of Processes
Employee Autonomy Privacy Transparency Access to information
n/a
–N
ot
a C
SMS
attr
ibu
te
Note: Figures in the graph are subject to rounding
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
In Q2 2019, there was no statistically significant changes for process and goals related drivers since Q1 2019. Similarly, when compared to the same time last year (Q2 2018) these drivers did not experience any statistically significant movement performance of these drivers have stabilised after experience significant increases over the last year.
Q: We would now like to ask you some questions specifically about your experience with [SERVICE]. Thinking now about [its employees; its processes; service overall; the values they uphold], to what extent do you agree or disagree that each of the following statements describes [SERVICE] in NSW? 1 is ‘Strongly disagree’ and 10 is ‘Strongly agree’.
Ave
rage
(o
ut
of
10
)
3.3 Key Drivers of Satisfaction – Qualitative Research
Employee drivers support the increase in satisfaction Quality of service and accountability drive satisfaction
We were greeted nicely and the client I was assisting was
treated with the utmost respect …everything was handled
professionally, quickly, respectfully, no double
handling and I was impressed
“
Insights from Q2 Qualitative Discussion
Consumer
25
”
I was emailed with a written response confirming receipt of my written complaint with a reference number which outlined when I would
receive a call…After speaking to them I received a further email outlining what steps to take and when I would be contacted again and then
received a further phone call asking if I needed anymore help or advice
“
”
• Especially important in emergency and social services, consumers feel
that empathetic employees who are open and honest helped improve
satisfaction. This is crucial as the most important* points throughout the
process as highlighted by consumers and businesses, relate to face to
face interactions rather than online channels
• Further support is seen via negative experiences with employees who
lacked product knowledge. This contributed to dissatisfaction during an
experience, with many consumers stating this as a reason for making it
difficult to build trust with the service
• However, there are differences between consumers and businesses.
Empathy and communication as well as honesty and integrity is
commonly addressed as a key driver of satisfaction whilst, efficiency and
effectiveness is a key theme for businesses. This is due to the
importance of time and need for efficient processes within businesses
Business
• Quality of service is key to improving satisfaction. For
consumers, having simple language and being guided through
the process helps ease the interaction with government
services. For businesses, the convenience of being able to
complete multiple processes helps to decrease effort
• For consumers, accountability has also contributed to
satisfaction, with customers that interacted with accountable
staff and services having greater trust and satisfaction. Being
able to provide status updates via a preferred channel has also
been important to increasing satisfaction
• Customers showed positivity towards services that provided
status updates on time, via their preferred channel and
delivered on what was promised
Staff training so that staff can find the information they need in the system…in
timely fashion would be another driver. I think if processes are designed to be time efficient, simple and require the minimum
level of updates be an essential precursor to achieving customer satisfaction and trust
“
””*Importance is commonly defined by stages within a process which require the most effort from both respondents and government services
4. Channel Usage and Preference
4.1 Channel Usage and Preference - Consumer
4.2 Channel Usage and Preference - Business
4.3 Impact of Channel Expectations – Qualitative Research
26
36
%
20
%
21
%
11
%
9%
4%
41
%
20
% 22
%
9%
5%
2%
38
%
20
%
18
%
11
%
7%
3%
34
%
19
%
21
%
12
%
7%
4%
35
%
20
% 22
%
11
%
6%
3%
41
%
19
% 23
%
9%
5%
2%
36
%
21
%
19
%
13
%
7%
4%
35
%
21
%
19
%
14
%
7%
4%
35
%
22
%
19
%
12
%
8%
4%
39
%
20
% 23
%
10
%
6%
3%
38
%
20
%
18
%
13
%
8%
3%
40
%
21
%
18
%
11
%
7%
4%
38
%
19
%
20
%
13
%
7%
3%
0%
10%
20%
30%
40%
50%
Face to face Telephone Online Email Mail, posted letter, fax Third parties such as Australia Post
CSMS 2015 CSMS 2016 Q4 2016 Q1 2017 Q2 2017 CSMS 2017 Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018 Q1 2019 Q2 2019
39
%
15
%
28
%
10
%
3%
1%
32
%
19
%
25
%
15
%
4%
1%
40
%
17
% 21
%
13
%
2%
1%
41
%
17
%
24
%
13
%
2%
1%
38
%
17
%
24
%
14
%
3%
1%
35
%
18
%
28
%
15
%
4%
1%
38
%
18
%
19
%
15
%
3%
3%
39
%
19
% 22
%
15
%
2%
1%
40
%
17
% 22
%
13
%
3%
1%
33
%
18
%
25
%
16
%
4%
1%
41
%
18
% 20
%
14
%
2%
1%
41
%
17
% 20
%
13
%
3%
2%
40
%
16
%
23
%
14
%
3%
1%
0%
10%
20%
30%
40%
50%
Face to face Telephone Online Email Mail, posted letter, fax Third parties such as Australia Post
CSMS 2015 CSMS 2016 Q4 2016 Q1 2017 Q2 2017 CSMS 2017 Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018 Q1 2019 Q2 2019
4.1 Channel Usage and Preference - Consumer
Contact Methods Used*
Q: Which of the following contact methods have you used to carry out your direct dealings with [SERVICE] in NSW in the last 6 months? (Multi-select)
* Contact methods used is a multi response question; Figures in the graph does not include the response for “None of the above” option. Channel usage has been rebased on total number of responses for comparison to channel preference** Contact methods preferred is a single response question; Figures in the graph does not include the response for “Don’t kno w/ can’t say” and therefore, may not add up to 100%
Shar
e o
f co
nta
ct m
eth
od
use
d (%
)Sh
are
of
pre
fere
nce
(%) n=1,600
n=1,003
Consumer
27
Note: Figures in the graph are subject to rounding
Q: Which of the following contact methods do you most prefer to use when dealing directly with [SERVICE] in NSW in the last 6 months? Contact Methods Preference**
For consumers, usage of all contact methods did not experience any notable movements, with all channels increasing or decreasing by 1%-2% when compared to Q1 2019. Face to face remains the most used channel among this group and the percentage of respondents interacting face to face has increased by 3% since Q2 2018. Face to face also remains the most preferred contact channel for consumers when dealing with NSW Government services since 2015. Consumers who interact face to face have the highest expectation and satisfaction with these channels. This is also a historical trend.
27
%
24
%
20
%
15
%
10
%
4%
29
%
27
%
20
%
16
%
7%
2%
24
%
26
%
22
%
15
%
10
%
3%
29
%
22
%
19
%
15
%
9%
5%
30
%
16
%
25
%
16
%
9%
4%
27
%
25
%
19
%
20
%
7%
2%
24
%
26
%
17
%
19
%
9%
6%
32
%
20
%
19
%
16
%
9%
4%
34
%
20
%
20
%
14
%
8%
3%
28
%
24
%
19
%
19
%
6%
4%
40
%
20
%
18
%
12
%
6%
4%
30
%
22
%
18
%
18
%
8%
4%
30
%
22
%
20
%
17
%
7%
4%
0%
10%
20%
30%
40%
50%
Face to face Telephone Online Email Mail, posted letter, fax Third parties such as Australia Post
CSMS 2015 CSMS 2016 Q4 2016 Q1 2017 Q2 2017 CSMS 2017 Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018 Q1 2019 Q2 2019
38
%
20
% 24
%
13
%
3%
1%
27
%
21
%
23
%
23
%
3%
1%
26
%
24
% 27
%
16
%
3%
1%
33
%
20
%
22
%
17
%
3%
2%
35
%
24
%
16
% 20
%
3%
1%
27
%
24
%
19
% 25
%
4%
1%
33
%
24
%
19
%
11
%
7%
2%
37
%
17
%
25
%
16
%
2%
1%
37
%
17
% 22
%
18
%
3%
1%
27
%
24
%
21
%
22
%
4%
2%
41
%
16
% 21
%
14
%
3%
1%
33
%
21
%
19
%
18
%
3%
2%
34
%
21
%
22
%
18
%
2%
1%
0%
10%
20%
30%
40%
50%
Face to face Telephone Online Email Mail, posted letter, fax Third parties such as Australia Post
CSMS 2015 CSMS 2016 Q4 2016 Q1 2017 Q2 2017 CSMS 2017 Q4 2017 Q1 2018 Q2 2018 CSMS 2018 Q4 2018 Q1 2019 Q2 2019
4.2 Channel Usage and Preference - Business
Contact Methods Used*
Q: Which of the following contact methods have you used to carry out your direct dealings with [SERVICE] in NSW in the last 6 months? (Multi-select)
* Contact methods used is a multi response question; Figures in the graph does not include the response for “None of the above” option. Channel usage has been rebased on total number of responses for comparison to channel preference** Contact methods preferred is a single response question; Figures in the graph does not include the response for “Don’t kno w/ can’t say” and therefore, may not add up to 100%
Shar
e o
f co
nta
ct m
eth
od
use
d (%
)Sh
are
of
pre
fere
nce
(%)
n=1,394
n=1,009
Business
28
Note: Figures in the graph are subject to rounding1Indicative only - does not indicate statistical significance.
Q: Which of the following contact methods do you most prefer to use when dealing directly with [SERVICE] in NSW in the last 6 months?Contact Methods Preference**
Face to face is the most used channel for businesses and its usage has remained stable since Q1 2019. The use of other contact methods have remained similar to Q1 2019. Businesses prefer face to face contact and this preference has increased by 1% since Q1 2019. In Q2 2019, businesses’ preference for online has increased1 by 2% compared to Q1 2019, while preference for telephone and email have remained stable. Similar to consumer results, businesses who interact face to face or online have the highest levels of satisfaction and expectation.
4.3 Impact of Channel on Expectations – Qualitative Research
• Face to face interactions help to support satisfaction as staff can
provide a highly empathetic and personal service. Comprehensive
product knowledge was identified as key to these staff interactions
• In addition, great service quality by having the right supportive
infrastructure is also important. This creates efficiencies during the
process, such as Roads where there are efficient ticketing systems,
helping to exceed expectations
29
Face to face interactions support satisfaction via employee driversThe fact that the service assistant did as she said she would made me trust the service provider and the process in that they did what
they say they would, and therefore increased my satisfaction. It meant that I did not have to chase for an answer.
I feel like the staff were friendly (and) patient…and wanted to do all they could to get a positive outcome for me. I feel it really made
me trust the process more when I was getting calls from the same person as I feel like had a genuine interest in the process
“
“ ”
”
Insights from Q2 Qualitative Discussion
• Customers acknowledge that NSW Government online processes
have contributed to the satisfaction and expectation of government
services due to greater convenience and time efficiencies. Having
multiple channels allows them to receive status updates and conduct
dealings via their preferred channel, reducing the need to travel to a
branch or explain complications over the phone
• Businesses commonly initiate processes via an online channel and
only interact face to face if necessary. This is creates duplications and
time inefficiencies, which reduces customer satisfaction. Hence,
online channels need to be seamlessly integrated with offline
channels through constant communication
Online channels need to be integrated seamlessly to reduce effort
Just to complete one process. It makes me question whether these departments…actually communicate with one another.
“
”
I found that the NSW Service centre has a seamless process in place which all staff and customers appear to benefit from,
everything was handled professionally, quickly, respectfully, no double handling and I was impressed with it as a whole and they
could be used as a benchmark for other NSW Govt Offices / Departments.
“
”
Transitioning to an App based notification would work...such an example would be a great customer experience as it consolidates all communications within one platform, speeds up the receipt of
communications
“
”
Consumer Business
Appendix
Appendix A: Demographic Profile of Respondents
Appendix B: Background to the QPCS
Appendix C: QPCS Qualitative and Quantitative Research
Appendix D: Historical Consumer and Business Baseline Measures
Appendix E: Customer Satisfaction Index Q1 2019 QPCS (CSMS at 99% Confidence Level)
Appendix F: Terminology Definitions
30
Appendix A: Demographic Profile of Respondents - Consumer
Consumer Respondent Profile*
49% male
51% femaleGender: Age: Region:
18-3431% 35-54
33%
55-6415%
65+20%
Q2 2019 Consumers (n=1,003 respondents)
75% Metro
19% Regional
6% Rural
27% 25%
11% 10% 9%6%
3% 3% 4%
0%
10%
20%
30%
Employment Status:
17%20%
26%
13%
5% 5%
14%
0%
10%
20%
30%
Up to$30,000
$30,001 to$50,000
$50,001 to$100,000
$100,001 to$150,000
$150,001 to$180,000
Over$180,001
Prefer not tosay/Don't
know
Annual Income:
49% male
51% femaleGender: Age:
18-3431% 35-54
33%
55-6415%
65+20%
Region: 75% Metro
19% Regional
6% Rural
Employed full time
Retired Full-time domestic
duties
Unemployed Employed part time
Student Employed on a casual
basis
Other
33%
24%
11% 9% 7% 6% 5% 3% 3%
0%
10%
20%
30%
40%
Employment Status:
Employed full time
Retired Employee part time
Full-time domestic
duties
Unemployed Student Employedon a
casual basis
Self-employed /
business owner
Other
17% 16%
26%
17%
5% 5%
14%
0%
10%
20%
30%
Up to$30,000
$30,001 to$50,000
$50,001 to$100,000
$100,001 to$150,000
$150,001 to$180,000
Over$180,001
Prefer not tosay/Don't
know
Annual Income:
• Data is weighted to be representative of the NSW population (ABS) based on gender, age and region
Q1 2019 Consumers (n=1,007 respondents)
Consumer
31
Self-employed /
business owner
Note: Figures in the graph are subject to rounding and may not sum to 100%
Appendix A: Demographic Profile of Respondents - Business
Business Respondent Profile*
47% male
53% female
Gender: Region:
Q2 2019 Business (n=1,009 respondents)
75% Metro
19% Regional
6% Rural
48% male
Gender: Region: 75% Metro
19% Regional
6% Rural
52% female
Business size:
Business size:
Industry:
12%9% 9%
7% 7% 6% 5% 4% 4% 4% 3% 3% 3% 3% 2% 1% 0.8% 0.7%
15%
-2%
2%
6%
10%
14%
18%
Pro
fess
ion
al,
scie
nti
fic a
nd
tech
nic
al s
ervi
ces
Ret
ail T
rad
e
Hea
lth
car
e an
dso
cial
ass
ista
nce
Educ
atio
n an
dtr
ain
ing
Co
nst
ruct
ion
Art
s an
dre
cre
atio
n s
ervi
ces
Acc
omm
od
atio
nan
d f
oo
d se
rvic
es
Fina
nci
al a
ndin
sura
nce
ser
vice
s
Ren
tal,
hiri
ng a
nd
real
est
ate
ser
vice
s
Agr
icul
ture
,fo
rest
ry a
ndfi
shin
g
Ad
min
istr
ativ
e a
ndsu
pp
ort
ser
vice
s
Man
ufa
ctu
rin
g
Who
lesa
le T
rade
Tran
spo
rt, p
osta
lan
d w
areh
ousi
ng
Info
rmat
ion
med
iaan
dte
leco
mm
uni
cati
on
s
Pub
licad
min
istr
atio
n a
nd
safe
ty
Ele
ctri
city
, gas
,w
ater
an
d w
aste
serv
ices M
inin
g
Oth
ers
Industry:
Q1 2019 Business (n=1,003 respondents)
• Data is weighted to be representative of the NSW population (ABS) based on business size and region
20+2%
Sole Proprietor
29%2-949%
10-1920%
20+3%
Sole Proprietor
31%2-944%
10-1922%
Business
32
9%12%
10%6% 7%
4% 5%3% 3% 3% 4% 5%
3% 2%4%
1% 0.5% 0.5%
17%
-2%
2%
6%
10%
14%
18%
Pro
fess
ion
al,
scie
nti
fic a
nd
tech
nic
al s
ervi
ces
Ret
ail T
rad
e
Hea
lth
car
e an
dso
cial
ass
ista
nce
Educ
atio
n an
dtr
ain
ing
Co
nst
ruct
ion
Art
s an
dre
cre
atio
nse
rvic
es
Acc
omm
od
atio
nan
d f
oo
d se
rvic
es
Fina
nci
al a
ndin
sura
nce
serv
ices
Ren
tal,
hiri
ng a
nd
real
est
ate
serv
ices
Agr
icul
ture
,fo
rest
ry a
ndfi
shin
g
Ad
min
istr
ativ
ean
d s
up
port
serv
ices
Man
ufa
ctu
rin
g
Who
lesa
le T
rade
Tran
spo
rt, p
osta
lan
d w
areh
ousi
ng
Info
rmat
ion
me
dia
and
tele
com
mu
nica
tio
ns
Pub
licad
min
istr
atio
nan
d s
afet
y
Ele
ctri
city
, gas
,w
ater
an
d w
aste
serv
ices M
inin
g
Oth
ers
Note: Figures in the graph are subject to rounding and may not sum to 100%
Appendix B: Background - Research Scope and Approach
• The QPCS Methodology is aligned to the Annual Customer Satisfaction
Measurement Survey (CSMS) approach:
• Captures feedback across 23 different NSW Government services (described
in the customers language).
• Feedback received from customers about each of the individual services is
aggregated to provide a view of the performance of NSW Government
services overall.
• Each respondent provides feedback regarding one or two services (as a
result, the total number of responses received across services is greater
than the total number of customers who completed the survey).
• The survey was completed from 15th April 2019 to 23rd April 2019 and results are
therefore reflective of experiences with services over the six months prior i.e. from
October 2018 to April 2019.
• The Q2 2019 QPCS was completed with:
• N = 1,003 consumers, and
• N = 1,009 businesses
• As each respondent provides feedback regarding one or two services, the Q2 2019
QPCS number of responses:
• N = 1,600 for consumers, and
• N = 1,394 for businesses
• All scores reported in this document are out of 10, with the exception of the
Customer Satisfaction Index which is out of 100.
In scope services
Industry• Agricultural Advice and
Funding Services• Business Advisory Services • Water Supply• TAFE Services
Justice • Police • State Emergency Services • Prisons • Courts • Fire Brigades
Family & Community Services • Public Housing • Disability Services • Child Protection Services• Services for Older People
Transport• Public Transport • Car and Boat Registration • Major Roads
Finance, Services & Innovation
• Consumer Affairs (Fair Trading)
Planning & Environment
• Environment and Wildlife Protection
• Art Galleries and Museums
Education
• Public Schools
Health
• Public Hospitals
• Ambulance Services
Multiple clusters
• Documentation Services (including certificates for births deaths and marriages; trade licenses and certificates; and drivers licenses)
Consumer Business
33
General Considerations:
• The QPCS results do not replace the Annual CSMS results, but rather provide
a directional indication of the shift in the results.
• Although the QPCS sample characteristics are closely representative of the
NSW population, different customers have been surveyed and as such the
results are directional indicators of shifts in the Annual CSMS results only.
• The margin of error (MoE) for the QPCS needs to be considered when
interpreting the results.
Considerations for interpreting the QPCS data points:
• The QPCS results need to be interpreted in the context of the time of the year and in light of events in order to normalise seasonal trends in the data.
Therefore, overall caution should be taken when interpreting the QPCS findings until a minimum of a full year of results has been collected, so that any
seasonal impacts can be examined and adjusted accordingly.
• Contextual factors for the results (Q2 2019) have been considered and include factors such as train delays and strikes, stadium upgrades and light rail
construction. Contextual factors are further explored as part of the analysis of the summary report.
• In the slides, the results of Q2 2019 QPCS have been compared to the results of all QPCS starting from Q4 2016 and CSMS starting from 2015. Significance
testing is based on the comparison to Q2 2018 and Q1 2019 results at 95% confidence level. We have allowed the longitudinal trend analysis for CSMS to
QPCS having applied a 95% confidence level to both CSMS and QPCS datasets.
• The Annual CSMS results have been provided as additional context for the QPCS data point and should not be used as a comparison to QPCS results.
• A longitudinal dataset will need to be built over time in order to identify 'real' trends in the QPCS results and to strengthen the reliability and validity of any
conclusions drawn.
Q12017
Q22017
2017CSMS
Q42017
Q12018
Q22018
CSMS2018
Q42018
Q12019
Q22019
1. Set a benchmark with the first data point
Interpretation Plan for Tracking Study
2. Draw insights by comparing to previous quarter
3. Form a directional trend
4. Develop a lead indicator
We are hereTime
Mea
sure
ILLUSTRATION ONLY - NOT REAL DATA
Appendix B: Background - Key Considerations for Interpreting QPCS Insights
34
Appendix C: QPCS Qualitative and Quantitative Research - ApproachQ2 2019 QPCS Qualitative Research:
Qualitative research complements the QPCS survey results by providing further insights and context into the results of the quantitative survey. In Q2 2019, the qualitative research will provide insight into the Special Interest Topic – Process and status transparency.
An online discussion forum was used as the qualitative research approach in Q2 2019 QPCS. There were 2 online discussion forums, a consumer group and a business group. Participants from each group were recruited from the respondents of Q2 2019 QPCS survey with a mix of demographics. The 3-day online discussion forums ran from 13th to 15th May 2019 for the consumer group and 15th to 21st May 2019for the business group. Participants were required to answer all the pre-designed questions and encouraged to comment on other people’s posts. Moderators monitored the two forums and follow up questions were posted which prompted to participants to make sure sufficient insights were captured. The final responses came in both text and video formats which provided an in-depth understanding of baseline measures of Q2 2019 and the SIT topic.
35
The approach undertaken to assess changes between the CSMS results and the QPCS topline results is outlined below:
• In order to compare the CSMS topline results against the QPCS topline results, the confidence level of CSMS (typically 99%) has been adjusted to 95% confidence level (in line with that of the QPCS). This was undertaken in order to make the significance testing comparable between CSMS and QPCS results
• In doing this however, it should be noted that the survey methodology differs for CSMS versus QPCS in the following ways, and should be interpreted with caution:
• Respondents are asked about their direct dealings with a service reflecting on their last 12 months in CSMS but the last 6 months in the QPCS, therefore recall about their experience differs
• Sample size varies greatly i.e. 2018 CSMS sample size was n=6,733 versus 2018 Q4 QPCS sample size of n=1,545 for consumers (this can impact margin of error which directly impacts significance testing)
• 99% confidence interval is used to test significance of results for the CSMS results versus 95% confidence interval which is used to test significance of the QPCS results
Q2 2019 QPCS Quantitative Research:
Appendix C: QPCS Qualitative Research - Overview of Online Discussion Forum Approach
Approach:
• Participants were recruited from the respondents of Q2 2019 QPCS survey. Participants come from backgrounds with a mix of age, gender, location and experience interacting with NSW Government services
• The online discussion forums ran from 13th to 15th May 2019 for the consumer group and 15th to 21nd May 2019 for the business group.
• Participants were asked to provide insights into their experiences around process and status transparency.
Structure of the Discussion Guide
Introduction
Topics
Changes in experience over the past few months
Discussion on process transparency
Discussion on status transparency
Process Transparency Improvements
Storyboard of Process Transparency
Group Number of Respondents
Consumer 22
Business 16
Storyboard of Status Transparency
Best practice from real-world examples
Day 1
Day 3
Day 2
36
Activity Three: Drawing process transparency best practice from other real-world examples
Using examples of tools used to track the status of a process, we will seek to understand what specific features and attributes are important and useful for customers.
Customers can also reflect on how these aspects would improve their experience if applied to a recent interaction with a NSW Government service.
Activity Two: Storyboard of Process Transparency
Participants will recall their journey, pain points, and moments of delight from a NSW Government process they experienced. This can be a linear path or a storyboard.
Following this, participants will add in key points of status transparency
Activity One: Unpacking the quantitative insights
Using the quantitative questions and responses as a base, we will ask the participants questions around the following:
• What makes a process transparent and why?
• How are participants kept informed of their status and how does this impact their satisfaction with the service overall?
Appendix D: Overview of Historical Results
Results
at a glance
Consumers
CSMS
2015
CSMS
2016Q4 2016 Q1 2017 Q2 2017
CSMS
2017Q4 2017 Q1 2018 Q2 2018
CSMS
2018Q4 2018 Q1 2019 Q2 2019
Satisfaction 7.3 7.5 7.4 7.6 7.6 7.7 7.7 7.7 7.7 7.6 7.8 7.7 7.8
Expectation 7.7 7.8 7.8 7.9 7.8 7.9 7.8 7.9 7.9 7.9 8.0 7.8 8.0
Ideal
Service6.9 7.2 7.2 7.3 7.2 7.2 7.3 7.2 7.3 7.2 7.3 7.2 7.4
CSI 76.7 78.7 77.8 79.5 78.6 79.3 80.2 79.9 79.6 78.9 79.9 79.4 80.9
Results
at a glance
Businesses
CSMS
2015
CSMS
2016Q4 2016 Q1 2017 Q2 2017
CSMS
2017Q4 2017 Q1 2018 Q2 2018
CSMS
2018Q4 2018 Q1 2019 Q2 2019
Satisfaction 7.2 7.5 7.2 7.4 7.3 7.5 7.5 7.6 7.8 7.4 7.9 7.8 7.8
Expectation 7.6 7.9 7.6 8.1 7.5 7.7 7.6 7.7 8.1 7.7 8.0 8.0 7.9
Ideal
Service7.0 7.3 7.1 7.4 7.1 7.1 7.0 7.2 7.3 7.2 7.4 7.6 7.4
CSI 76.6 78.4 76.1 77.6 77.6 78.3 77.8 79.6 79.7 78.2 81.3 81.2 80.7
Consumer Business
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
37Note: Figures in the graph are subject to rounding
Note: CSI is out of 100; all other measures represent scores out of 10
Appendix D: Satisfaction - Consumer
Consumer – Outcome Measures
Average (out of 10)
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
10% 9% 10% 7% 7% 8% 8% 8% 9% 8% 8% 9% 8%
19% 16% 17% 16% 17% 14% 13% 14% 13% 15% 12% 15% 13%
71% 76% 73% 76% 76% 78% 78% 78% 78% 77% 80% 77% 80%
0%
20%
40%
60%
80%
100%
CSMS 2015(n=6,790)
CSMS 2016(n=7,227)
Q4 2016(n=1,652)
Q1 2017(n=1,580)
Q2 2017(n=1,647)
CSMS 2017(n=6,789)
Q4 2017(n=1,541)
Q1 2018(n=1,651)
Q2 2018(n=1,638)
CSMS 2018(n=7,000)
Q4 2018(n=1,583)
Q1 2019(n=1,611)
Q2 2019(n=1,577)
Low (1-4) Med (5-6) High (7-10)
Consumer
7.3 7.5 7.4 7.6 7.6 7.7 7.7 7.7 7.7 7.6 7.8 7.7
CSMS results
38
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Note: Figures in the graph are subject to rounding and may not sum to 100%
7.8
Appendix D: Expectation - Consumer
Consumer – Outcome Measures
Average (out of 10)
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
7% 7% 6% 5% 6% 6% 7% 6% 7% 6% 6% 8% 5%
16% 13% 13% 13% 14% 12% 14% 13% 12% 13% 11% 12% 13%
78% 80% 81% 82% 80% 82% 79% 81% 81% 81% 84% 80% 82%
0%
20%
40%
60%
80%
100%
CSMS 2015(n=6,693)
CSMS 2016(n=7,140)
Q4 2016(n=1,636)
Q1 2017(n=1,563)
Q2 2017(n=1,624)
CSMS 2017(n=6,732)
Q4 2017(n=1,537)
Q1 2018(n=1,644)
Q2 2018(n=1,642)
CSMS 2018(n=6,922)
Q4 2018(n=1,579)
Q1 2019(n=1,614)
Q2 2019(n=1,578)
Low (1-4) Med (5-6) High (7-10)
Consumer
7.7 7.8 7.8 7.9 7.8 7.9 7.8 7.9 7.9 7.9 8.0 7.8
CSMS results
39
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Note: Figures in the graph are subject to rounding and may not sum to 100%
8.0
Appendix D: Comparison to Ideal - Consumer
Consumer – Outcome Measures
Average (out of 10)
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
13% 12% 12% 9% 12% 11% 11% 12% 12% 11% 13% 13% 11%
22%18% 19% 19% 18% 19% 17% 19% 17% 20% 16% 16% 14%
65% 70% 69% 72% 69% 70% 72% 70% 71% 69% 72% 71% 75%
0%
20%
40%
60%
80%
100%
CSMS 2015(n=6,593)
CSMS 2016(n=7,015)
Q4 2016(n=1,621)
Q1 2017(n=1,548)
Q2 2017(n=1,605)
CSMS 2017(n=6,559)
Q4 2017(n=1,518)
Q1 2018(n=1,628)
Q2 2018(n=1,618)
CSMS 2018(n=6,733)
Q4 2018(n=1,545)
Q1 2019(n=1,590)
Q2 2019(n=1,564)
Low (1-4) Med (5-6) High (7-10)
Consumer
6.9 7.2 7.2 7.3 7.2 7.2 7.3 7.2 7.3 7.2 7.3 7.2
CSMS results
40
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Note: Figures in the graph are subject to rounding and may not sum to 100%
7.4
7.5
Appendix D: Satisfaction- Business
Business – Outcome Measures
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
Average (out of 10)
9% 7%15%
7% 11% 9% 8% 5% 9% 7% 5% 9% 8%
18% 18%15%
24% 17% 17% 19% 22% 11% 18% 16% 12% 14%
72% 74% 70% 69% 72% 75% 73% 73%80% 75% 79% 79% 79%
0%
20%
40%
60%
80%
100%
CSMS 2015(n=1,700)
CSMS 2016(n=1,761)
Q4 2016(n=382)
Q1 2017(n=391)
Q2 2017(n=372)
CSMS 2017(n=1,705)
Q4 2017(n=349)
Q1 2018(n=375)
Q2 2018(n=1,570)
CSMS 2018(n=1,555)
Q4 2018(n=1,553)
Q1 2019(n=1,469)
Q2 2019(n=1,387)
Low (1-4) Med (5-6) High (7-10)
Business
7.2 7.4 7.3 7.5 7.5 7.6 7.8 7.4 7.9 7.8
CSMS results
41
7.2
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Note: Figures in the graph are subject to rounding and may not sum to 100%
7.8
7.9
Appendix D: Expectation - Business
Business – Outcome Measures
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
Average (out of 10)
8% 5% 8% 3%10% 6% 6% 6% 6% 6% 5% 6% 5%
15%14% 14%
11%
16%16% 17% 16% 9% 15% 14% 11% 14%
77% 81% 79%86%
74% 77% 77% 78%84% 79% 82% 83% 81%
0%
20%
40%
60%
80%
100%
CSMS 2015(n=1,677)
CSMS 2016(n=1,738)
Q4 2016(n=381)
Q1 2017(n=390)
Q2 2017(n=370)
CSMS 2017(n=1,682)
Q4 2017(n=350)
Q1 2018(n=372)
Q2 2018(n=1,555)
CSMS 2018(n=1,531)
Q4 2018(n=1,552)
Q1 2019(n=1,465)
Q2 2019(n=1,389)
Low (1-4) Med (5-6) High (7-10)
Business
7.6 8.1 7.5 7.7 7.6 7.7 8.1 7.7 8.0 8.0
CSMS results
42
7.6
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Note: Figures in the graph are subject to rounding and may not sum to 100%
7.9
7.3
Appendix D: Comparison to Ideal - Business
Business – Outcome Measures
% D
istr
ibu
tio
n (
acro
ss lo
w,
neu
tral
an
d h
igh
sco
res)
Average (out of 10)
14% 9% 15% 9% 14% 11% 10% 7% 12% 11% 9% 8% 10%
18%20% 14%
17%18% 20% 24%
20% 15% 19% 18% 13%17%
67% 71% 71% 73% 68% 69% 66%73% 73% 71% 73% 79% 73%
0%
20%
40%
60%
80%
100%
CSMS 2015(n=1,654)
CSMS 2016(n=1,718
Q4 2016(n=375)
Q1 2017(n=380)
Q2 2017(n=365)
CSMS 2017(n=1,646)
Q4 2017(n=345)
Q1 2018(n=367)
Q2 2018(n=1,554)
CSMS 2018(n=1,506)
Q4 2018(n=1,535)
Q1 2019(n=1,455)
Q2 2019(n=1,378)
Low (1-4) Med (5-6) High (7-10)
Business
7.1 7.4 7.1 7.1 7.0 7.2 7.3 7.2 7.4 7.6
CSMS results
43
7.0
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Note: Figures in the graph are subject to rounding and may not sum to 100%
7.4
76.6
78.4 78.379.6 79.7
78.2
81.3 81.2 80.7
CSMS2015
(n=1,645)
CSMS2016
(n=1,712)
CSMS2017
(n=1,638)
Q12018
(n=365)
Q22018
(n=1,536)
CSMS2018
(n=1,494)
Q42018
(n=1,523)
Q12019
(n=1,449)
Q22019
(n=1,372)
Consumers – CSI
Businesses – CSI
Consumers – Baseline Measures
Businesses – Baseline Measures
Margin of Error (QPCS)Consumer CSI: ± 1.3
Margin of Error (QPCS)Business CSI: ± 1.2
Note - Sample size for CSI and Baseline measure may differ on the basis of “Don’t Know” option selection
Note – Significance testing between CSMS and QPCS is not possible due to differing confidence intervals
76.7
78.7 79.3 79.9 79.678.9
79.9 79.480.9
CSMS2015
(n=6,549)
CSMS2016
(n=6,971)
CSMS2017
(n=6,527)
Q12018
(n=1,610)
Q22018
(n=1,603)
CSMS2018
(n=6,701)
Q42018
(n=1,535)
Q12019
(n=1,575)
Q22019
(n=1,555)
7.3
7.57.7 7.7 7.7 7.6
7.87.7
7.87.7
7.87.9 7.9 7.9 7.9 8.0
7.88.0
6.9
7.2 7.2 7.2 7.37.2
7.3 7.2
7.4
6.6
7.1
7.6
8.1
8.6
CSMS2015
(n=6,593)
CSMS2016
(n=7,015)
CSMS2017
(n=6,559)
Q12018
(n=1,628)
Q22018
(n=1,618)
CSMS2018
(n=6,733)
Q42018
(n=1,545)
Q12019
(n=1,590)
Q22019
(n=1,564)
Satisfaction Expectation Comparison to Ideal
7.27.5 7.5
7.67.8
7.4
7.9 7.8 7.87.6
7.97.7 7.7
8.1
7.7
8.0 8.07.9
7.0
7.37.1
7.27.3 7.2
7.4
7.67.4
6.6
7.1
7.6
8.1
8.6
CSMS2015
(n=1,654)
CSMS2016
(n=1,718)
CSMS2017
(n=1,646)
Q12018
(n=367)
Q22018
(n=1,554)
CSMS2018
(n=1,506)
Q42018
(n=1,535)
Q12019
(n=1,455)
Q22019
(n=1,378)
Satisfaction Expectation Comparison to Ideal
Appendix E: Customer Satisfaction Index Q2 2019 QPCS (CSMS Sig-test at 99% significance level)Consumer Business
Statistically significant movement from same quarter previous year at 95% confidence level
Statistically significant movement from previous quarter at 95% confidence level
Statistically significant movement from the previous CSMS year at 99% confidence level
44Note: Figures in the graph are subject to rounding
45
To ensure consistency in reporting significance testing, the below table of definitions has been developed and will be used for all future reporting for CSMS and QPCS.
Appendix F: Terminology Definitions
Terminology Definition Example
Remain StableRounded numerical difference between current period result and the previous period result is less than 0.1 (in line with the chart visualisations).
Satisfaction score moved from 8.07 to 8.12 (i.e. actual difference is 0.05). In this case, rounded values are both 8.1 thus the results remained stable
Increase/DecreaseRounded numerical difference between current period outcome measures and the previous period result is larger than or equal to 0.1 (in line with the chart visualisations).
Satisfaction score moved from 8.07 to 8.21 (i.e. actual difference is 0.14). In this case, the rounded value moved from 8.1 to 8.2 thus the result increased.
Statistically significantStatistically significant movement (decrease or increase) from the previous period at 95% confidence level
Not statistically significantNo statistically significant movement (decrease or increase) detected from the previous period at 95% confidence level
Visual differenceRounding numbers up to one decimal place and updating commentary based on visible difference in charts
Satisfaction score increased from 7.66 to 7.84 (i.e. actual difference is 0.18. In this case rounded values are 7.7 and 7.8, thus difference will be stated as increase of 0.1