Northwestern Health UnitPartner Telephone Survey Report

March 2016

Table of Contents

Executive Summary ................................................................................................................. 1

Introduction .............................................................................................................................. 2

Methods .................................................................................................................................... 3

Pre-Testing ............................................................................................................................. 3

Sample ................................................................................................................................... 3

Data Collection ....................................................................................................................... 4

Analysis .................................................................................................................................. 5

Results ...................................................................................................................................... 5

Awareness of health unit programs and services .................................................................... 5

Awareness of how to contact the health unit ........................................................................... 6

Perception of the health unit ................................................................................................... 6

Partnership with the health unit .............................................................................................. 7

Visits to www.nwhu.on.ca ....................................................................................................... 8

Input into health unit priorities ................................................................................................. 8

Discussion ................................................................................................................................ 9

Limitations ...............................................................................................................................10

Conclusion ..............................................................................................................................10

Appendix 1: Partner Questionnaire ...........................................................................................11

1

Executive Summary

In 2013 the Northwestern Health Unit (NWHU) commissioned a survey to assess partner

awareness and perception of our organization. The survey was also designed to collect key

baseline information that would help us measure a few of the objectives of our 2013-2016

Strategic Plan. The initial survey of partners was carried out between May and June of 2013. A

follow-up survey was completed between November and December of 2015. The surveys were

conducted by Oraclepoll Research Limited. This report provides a summary of the results of

both surveys.

In 2013, 274 partners responded to the survey giving a response rate of 68.2%; 211

respondents participated in 2015 with a response rate of 64.7%. A partner was defined as

someone who had been in contact with or working with the health unit more than four times in

the past year. The most common fields of employment of the partners were government, not-

for-profit and health care, which together made up 81% of the samples in 2013 and 2015.

Data from the surveys was collated and analyzed using Microsoft Excel® and STATA®. Key

findings include:

In 2015, 65.4% of respondents claimed they were aware or very aware of the programs

and services offered by the NWHU. This figure had increased 14.7% from the 2013

result of 57.0% with the difference being statistically significant. This increase was

greater than the NWHU’s objective to increase awareness among partners by 10% over

baseline by 2016.

In 2015, 96.7% of respondents were aware or very aware of how to contact the NWHU,

which was higher but not significantly different than in 2013 when it was 93.4%.

Overall perception of the NWHU by partners improved between 2013 and 2015. In 2013

81.8% of respondents had a positive or very positive perception of the health unit, a

figure which increased to 90.5% in 2015. The difference is statistically significant.

Partners’ perception of their partnerships with the NWHU also increased positively

between 2013 and 2015. In 2013 72.8% of respondents viewed their partnership with the

health unit as positive or very positive; this increased to 88.6% in 2015 with the

difference being statistically significant.

In 2015, 73.9% of respondents claimed that they had been to the NWHU’s website

(www.nwhu.on.ca), which was higher than in 2013 where 58.0% had been to the

website. The increase was statistically significant.

These results provided the NWHU with valuable information about how we are perceived by our

community partners, and also help us to measure key objectives in our strategic plan. The

results from these surveys will be used to help guide further planning processes.

2

Introduction

In 2013, the Northwestern Health Unit (NWHU) hired an independent company to complete a

series of surveys to help us measure some of the objectives in our 2013-2016 Strategic Plan.

The NWHU developed two similar surveys, each with similar objectives:

A telephone survey targeted at the general public to: o Assess awareness of the NWHU and how to contact us o Assess public perception of the NWHU o Assess behavioural changes based on our healthy eating and physical activity

(HEPA) strategy o Gather input into strategic priorities

A telephone survey targeted at community partners to: o Assess awareness of the NWHU and how to contact us o Assess perception of the NWHU o Gather input to strengthen partnerships o Gather input into strategic priorities

Each survey was to be done at the beginning of the Strategic Plan in 2013 and at the end in

2016. Oraclepoll Research based out of Sudbury, ON was the vendor awarded the contract to

conduct the surveys.

The initial general population telephone survey took place in May and June 2013. The initial

partner telephone survey took place in November and December 2013. Reports for both

surveys are available at

https://www.nwhu.on.ca/MediaPressCentre/Pages/healthanddemographicreports.aspx

This report summarizes the results of the partner survey that took place in December of 2015

and provides comparisons to the results of the initial survey from 2013, with significant

differences highlighted.

3

Methods

A number of strategies were put in place in August and September of 2015 to help ensure a robust response rate and to reduce the likelihood of non-response bias:

An email was sent out to 426 contacts provided by health unit staff to let them know in advance that they would be contacted by phone in the near future to take part in the survey.

Other communication activities included posting information at www.nwhu.on.ca, a media release, and preparing staff with information and key messages to respond to any questions from the public.

The questionnaire was designed to be as short and non-invasive as possible.

Multiple contact days and times were scheduled, and multiple follow-ups were planned if the contact was unable to be reached on first attempt. The opportunity to reschedule the interview was offered, and respondents were also given the option of completing the survey electronically.

Sponsorship of the survey was identified to the respondents, and a validation source was offered if required.

Respondents were given assurances of confidentiality and that responses would not be linked to names or organizations.

Pre-Testing

To test the face validity of the survey tool initially in 2013, a thorough review was done by the

NWHU management team, Planning Officer, Epidemiologist and Continuous Quality

Improvement Officer. The group went through the questions to assess clarity, appropriateness

and if the questions would collect information that would correspond to the indicators identified

in the strategic plan.

To further test the survey tool a pilot test was completed by Oraclepoll with members of the

sample. No issues were identified with the questions, and the tool was implemented as

designed.

Sample

For the initial survey in 2013 the NWHU provided Oraclepoll with contact information for 517

partners, who were included in the list if they were in contact with or worked with NWHU staff

more than four times per year. If a partner had left the organization it was decided that the

person that had replaced them could be added to the list in their place. In 2013, a total of 274

partners completed the survey out of a possible 402 eligible participants, giving a response rate

of 68.2%.

For the 2015 follow-up survey the NWHU provided contact information for 426 partners. A total

of 211 out of 326 eligible participants completed the survey, giving a response rate of 64.7%.

4

Table 1: Survey sample by sector

Sector % of respondents

2013 (n = 274) 2015 (n = 211)

Government 34.3 40.8

Not for Profit 33.6 25.6

Health Care 11.3 17.5

Retail 8.8 2.8

Other* 4.7 8.1

Media 4.0 0.0

Education 3.3 5.2

Total 100.0 100.0

*Includes Hospitality, Manufacturing, Transportation, Social Services and Professional Services

Data Collection

Telephone calls for the initial survey took place between November 20th and December 13th

2013. The follow-up survey took place between September 14th and September 23rd 2015.

Up to 10 attempts were made to reach a contact. Calls were made between 8:30 a.m. and 6:00

p.m. There was at least one attempt to reach respondents after 6:00 p.m. and on a weekend if

there was no weekday response.

Appointments were arranged for those requesting a more convenient time to complete the

survey. Respondents were offered an option to provide an email address to fill out the survey

electronically.

The survey was conducted using computer-assisted techniques of telephone interviewing

(VOXCO CATI survey software). Twenty percent of all interviews were monitored by Oraclepoll,

and the management of Oraclepoll Research Limited supervised all of the surveys. No

respondents completed the survey electronically.

A copy of the survey questions can be found in Appendix 1.

During the initial survey in 2013, a total of 517 numbers were called to achieve 274 completed

surveys. In the 2015 survey, 426 numbers were called to achieve 211 completed surveys.

5

Table 2: Call breakdown

Result Year

2013 2015

Survey completed by an eligible* person 274 211

Ineligible 76 49

No answer 75 65

Refused by an eligible* person 58 54

Out of service/wrong number/duplicate 17 29

Prolonged absence 14 14

Fax number 3 4

Total Calls 517 426

*an eligible person is one who is reached and who meets the selection criteria

Analysis

At the end of the telephone survey stages in 2013 and 2015 the data was extracted from

VOXCO into STATA®. The data was then validated and cleaned and then coded by Oraclepoll.

Open-ended responses were content-analyzed by researchers at Oraclepoll and assigned into

categories.

Frequencies were run and cross-tabulations were prepared in an Excel report. Tests of

statistical significance were conducted using STATA®.

Results

Awareness of health unit programs and services

Partners were asked to rate their awareness of the NWHU’s programs and services on a scale

of 1 to 5 (1 being very unaware and 5 being very aware). In 2015, 65.4% of respondents

claimed they were aware or very aware of the programs and services offered. This was higher

and statistically different from the results of the 2013 survey, where 57.0% were aware.

6

Table 3: Partner awareness of NWHU programs and services

Proportion of respondents (%)

2013 2015

Very aware 17.2 21.8

Aware 39.8 43.6

Neither aware nor unaware 32.1 28.9

Unaware 10.2 4.3

Very unaware 0.7 1.4

Total 100.0 100.0

Awareness of how to contact the health unit

Partners were also asked about their awareness of how to contact the NWHU using the same scale of 1 to 5. Nearly 97% of partners were aware or very aware of how to contact the health unit in 2015, higher but not statistically different from the 93.4% in initial 2013 survey.

Table 4: Partner awareness of how to contact the NWHU

Proportion of respondents (%)

2013 2015

Very aware 69.3 76.3

Aware 24.1 20.4

Neither aware nor unaware 4.7 2.4

Unaware 1.5 0.9

Very unaware 0.4 0.0

Total 100.0 100.0

Perception of the health unit

Partners were asked about their overall perception of the NWHU, using a scale of 1 to 5 with 1

being very negative and 5 being very positive. In 2015 90.5% of respondents had a positive or

very positive perception of the health unit, which is statistically higher than the result in 2013

when it was 81.8%.

7

Table 5: Partner overall perception of the NWHU

Proportion of respondents (%)

2013 2015

Very positive 42.7 53.1

Positive 39.1 37.4

Neither positive nor negative 16.7 7.6

Negative 1.5 0.5

Very negative 0.0 0.5

Total 100.0 100.0

Partnership with the health unit

Partners were asked about how they viewed their partnership with the NWHU, using the same

positive/negative scale. In the 2015 survey, 88.6% of respondents said that their partnership

was positive or very positive. This was statistically higher than the result in the 2013 survey

where it was 72.8%.

Table 6: Partner ratings of their partnership with the NWHU

Proportion of respondents (%)

2013 2015

Very positive 46.4 56.4

Positive 31.8 32.2

Neither positive nor negative 16.0 8.5

Negative 4.4 1.4

Very negative 1.4 1.4

Total 100.0 100.0

8

Visits to www.nwhu.on.ca

Partners were asked whether they had ever been to the NWHU website. In 2015, 73.9% of

respondents said they had, which is statistically higher than in 2013 when 58.0% had been to

the website. A quarter of respondents said they hadn’t been to the website and 0.9% said they

didn’t know.

Input into health unit priorities

Respondents were asked an open-ended question about where they thought the NWHU should

be focusing its efforts. Nearly half of the respondents in the 2015 survey (45.5%) said they didn’t

know, which is similar to the 2013 survey where 32.5% said the same thing.

In 2015, about 6.6% of respondents felt that the NWHU should focus on programs for youth and

young people, 6.2% on healthy living, 5.7% on healthy eating, 4.7% on preventive care and

4.7% on public health awareness and education. These results were fairly similar to the 2013

results.

Table 7: Partner perceptions as to what the NWHU’s priorities should be

Proportion of respondents (%)

2013 2015

Don’t know 32.5 45.5

Healthy living 4.0 6.6

Programs for youth/young people 8.8 6.2

Healthy eating 2.2 5.7

Preventive care/information 6.2 4.7

Public health awareness/education 9.9 4.7

Child health/issues 0.0 3.3

Addiction programs 4.0 2.4

Sexual health 2.6 1.9

Creating/maintaining partnerships 0.4 1.9

9

Discussion

The two surveys in 2013 and 2015 were designed to measure one of the key objectives of the

NWHU’s 2013-2016 Strategic Plan: to increase the percentage of community partners who

report that they know about, and how to contact, the health unit by 10% relative to baseline by

December 2016.

The results of the 2015 survey indicated that 65.4% of community partners were aware or very

aware of the programs and services offered by the NWHU, which had increased from 57.0% in

the initial 2013 survey. This was a statistically significant increase of 14.7%. This increase was

beyond the 10% that the NWHU hoped to achieve in our strategic plan.

The results also indicated that 96.7% of partners were aware or very aware of how to contact

the NWHU, which had increased slightly from 93.4% in 2013 (the change was not statistically

significant). This change was not enough to meet the objective in our strategic plan, although it

is important to note that the 2013 results were very positive in the first place and difficult to

improve on.

Overall perception of the NWHU increased by a significant margin between 2013 and 2015. In

2015, 90.5% of respondents had either a positive or very positive perception of the health unit,

compared with 81.8% in 2013.

Perception of partnerships with the NWHU also improved significantly in this time. In 2015,

88.6% of partners had a positive or very positive perception of their partnership with the health

unit, a considerable increase from the 72.8% in 2013.

The NWHU was also able to measure the success of our new website which was launched in

2014. In 2013, 58.0% of respondents had been to our website, and this figure increased

significantly to 73.9% in 2015. This indicated that our partners are more aware of our website

and our promotional efforts have been successful so far.

One unexpected result of both surveys was the lack of input that partners had about what the

NWHU should focus on with regard to strategic priorities. Respondents were asked about this in

the form of an open-ended question, and in both surveys a large proportion said they didn’t

know (33% in 2013 and 46% in 2015).

10

Limitations

Estimates obtained from this survey rely on self-reported answers from respondents. This is a

limitation of all surveys; self-reported data will always have concerns about accuracy, as they

are dependent on individual interpretations of questions. Social desirability bias may be present

in this type of survey.

Contact lists of our partner agencies are very difficult to keep accurate and up to date, and are

only representative of a point in time. For many of the contacts the contact information was out

of date, inaccurate or missing which had an impact on both the sample size and the response

rate of the surveys.

Conclusion

The 2015 survey of partners allowed us to compare results from the baseline in 2013 to

determine if we had met key objectives in our strategic plan, and to assess partners’ awareness

and perception of the NWHU.

The results indicated that awareness of the health unit’s programs and services had increased

over the past few years, as well as the overall perception of the health unit which was very

positive amongst community partners. Results also indicated that awareness of the NWHU

website amongst partners had considerably increased.

These surveys have provided the NWHU with valuable information that will used to inform

planning processes going forward. We hope to continue to foster positive relationships and

measure our success at doing so.

11

Appendix 1: Partner Questionnaire

SURVEY – PARTNERS

PREAMBLE Good morning/afternoon/evening, my name is <$> and I am calling from the research firm Oraclepoll on behalf of the Northwestern Health Unit. May I speak to <$>?

CONTACT PROCEED

NOT AVAILABLE - ESTABLISH CALL BACK TIME

You have been identified as a partner working with the Northwestern Health Unit. We are conducting a brief survey of Northwestern Health Unit partners that will take approximately five minutes of your time and your input is very important in helping to plan and to prepare future programs in the area. Please be assured that all information gathered will be kept in strict confidence and only overall results will be reported.

REFUSED POLITELY TERMINATE

BUSY - ESTABLISH CALL BACK TIME

REFUSED TELEPHONE APPROACH – OFFER ONLINE OPTION – VERIFY EMAIL TO SEND LINK

Please keep in mind that this survey is voluntary and that you can stop it at any time or refuse to answer any questions. We are looking for your honest answers and I would like to mention once again that all individual survey responses will be kept confidential.

Q1. Using a scale from 1 – 5 with 1 being not at all aware at all and 5 being very aware, how

would you rate your awareness of the types of programs and services offered by the Northwestern

Health Unit?

1-not at all aware

2-not aware

3-neither aware nor unaware

4-aware

5-100-Don’t know

99-Refused

Q2. Using the same scale from 1 – 5 with 1 being not aware at all and 5 being very aware how would you rate your awareness of how you can contact the Northwestern Health Unit?

1-not at all aware

2-not aware

3-neither aware nor unaware

4-aware

5-very aware

100-Don’t know

99-Refused

12

Q3. Using a scale of 1-5 with 1 very negative and 5 being very positive, how would you rate

your overall perception/view of the Northwestern Health Unit?

1-very negative

2-negative

3-neither negative nor positive

4-positve

5-very positive

100-Don’t know

99-Refused

Q4. Using a scale of 1-5 with 1 being very negative and 5 being very positive, how would you rate your partnership with the Northwestern Health Unit?

1-very negative

2-negative

3-neither negative nor positive

4-positve

5-very positive

100-Don’t know

99-Refused

Not applicable option

Q5. Is there anything that the Northwestern Health Unit can do to strengthen the partnership that it has with you?

RECORD VERBATIM

97-Open

100-Don’t know

99-Refused

Q6. Have you ever been to the Northwestern Health Unit website (www.nwhu.on.ca)?

1-yes

2-no

100-Don’t know

99-Refused

Q7. The Northwestern Health Unit seeks feedback from its partners about what its priorities should be. Where do you think that the Northwestern Health Unit should focus its work to help make communities healthier? RECORD VERBATIM

97-Open

100-Don’t know

99-Refused

D1. What best describes the sector or category that your organization falls into? CODE APPROPRIATE CATEGORY

Health care Education Emergency services

Business / private sector

13

Government

Charitable organization / not for profit

Religious

OTHER RECORD