triangulate to fully measure community impact · participated in a senior only event or meetup...
TRANSCRIPT
DELIVERING INSIGHTS WITH IMPACT
Elevated Insights (EI) is a full-service market research insights agency
that provides qualitative and quantitative research, evaluation,
and data mining for private, public, and social sector organizations.
EI is differentiated from other firms by its strategic consultancy
approach and its breath of experience to harness both qualitative
and quantitative insights into smart, strategic action.
With vast Fortune 500 experience, Elevated Insights is a nationally
recognized leader in marketing research techniques,
method development, and quality assurance.
Qualitative ResearchWhen research requires an in-depth qualitative understanding, EI crafts the right combination of methods to uncover the insights behind the findings.
Industrial Modern FacilityEI’s stand-alone facility inspires creativity while delivering a wide range of amenities and secure HD/fiber technology.
Respondent RecruitingElevated Insights understands the importance of getting recruiting right. EI ensures respondents are articulate and engaged.
Quantitative ResearchEI combines engaging surveys, rigorous data quality procedures, and robust analysis techniques to produce sound insights and recommendations.
Evaluation & Data MiningEI uses hybrid / triangulation approaches to help the public and social sectors understand and meet community needs, develop /optimize programs, and measure impact / ROI.
Recent Colorado Public/Social Sector Work Includes
Revisioning Library to Meet Community Needs
How to Better Meet Survivor NeedsAge Friendly COS –Awareness, Needs, Satisfaction
• Domestic Violence Attitudes & Incidence • PSA Testing & Development
Quality of Life/Satisfaction
Professional Development Needed in the Community
Disability Employment InitiativeProgram Optimization
Employee Satisfaction
• Tracking Awareness of Branding• Resident & Business Confidence• Airport Understanding
• Tracking Awareness & Trust of Nonprofits• Community Attitudes Towards Homeless• Panhandler Immersion
QLI – Data Quality & Resident Input
Local Ballot Initiatives –City Council Pay, Marijuana, Various Mill Levies
Data TriangulationCOMMUNITY INDICATORS CONSORTIUM
Using different sources of information to increase the validity of information.
Examples: interviews, articles, documents, reports, photographs, observations, surveys, statistics, etc.
Information from different audiences: experts, residents, academia, private sector, societal trends
TriangulationCOMMUNITY INDICATORS CONSORTIUM
5QUANTITATIVE QUALITATIVE
DATA MINING
3+Sources
Utilize
In Search of Truth – Some Primary Research OptionsCOMMUNITY INDICATORS CONSORTIUM
Primary Qualitative Quantitative Analytics Audiences
QualitativeListening Sessions or
Focus GroupsIn-person, online
Needs, Gaps, Satisfaction
Quadrant Mapping
Residents/Consumers
Quantitative In-depth interviewsHabits & Practices
Correlation/Regression
Segments withinTarget Population
LongitudinalEthnography-
in-person, onlineAttitudes;
PerceptionsMax Diff/Conjoint
Stakeholders
Communities/Panels
Intercepts/Citizen on the Street
Concept/Ad Reaction
Cluster –Segmentation
Community Partners
ObservationalUsability/UX/Eye Tracking
Product/Program optimization
Factor/Importance
Experts
Point-in-time; Transactional
Bulletin Boards/ Live Chat Sessions
Tracking, equity monitoring
Multivariate modeling
Target City;Peer Cities;
Total US
In Search of Truth – Some Secondary Research OptionsCOMMUNITY INDICATORS CONSORTIUM
Secondary Big Data Analytics Data Mining
CensusSocial Media - # analysis,
influencers, scrapingCombining secondary
sources - analyzingReviewing case studies
Bureau of Labor Statistics
Analyzing reviews Integrating data Other geographies
Indicators – all typesInfo from
Internet of ThingsComparative vs.
peer cities/countiesPrior learning
Syndicated ResearchGIS Mapping/Eye Tracking
Best practices
Published studies Search engines Articles
Meta-trends Click-through rates
Why Include Primary Research?COMMUNITY INDICATORS CONSORTIUM
8
• Custom Learning, Audience, Locations
• Link the Insights to clarify findings; multiple breakouts vs. disaggregation
• Inform Effort During Development Phase
• Faster/Real-Time: Clarify ROI of Programs/ Efforts; Make Necessary Course Corrections
• Gauge importance, prioritize action to take
Additional Reasons to Include Primary ResearchCOMMUNITY INDICATORS CONSORTIUM
9
• Stakeholder & Partner Engagement
• Qualitative deep dives, context/why to indicators, touchpoint follow-ups
• Option for longitudinal read
• Regional findings can help secure grants for most needed areas
• Can recruit residents for Civic Action Committees
• Presentations come to life with videos, images, quotes
CASE STUDY:
AGE FRIENDLY COLORADO SPRINGS
CASE STUDY: Age Friendly Colorado SpringsNeeds, Awareness, & Satisfaction Benchmarking Pre-Portal Development
May 2019
Research By:Prepared for:
Funded By:
Research ObjectivesAge Friendly Colorado Springs
Inform the development of the web-based
resource portal for
seniors and caregivers
Understand marketing channels/strategies that
work best for these seniors and caregivers
Determine current awareness, usage, and
satisfaction of services and resources among seniors and caregivers of seniors
Uncover how El Paso County can make seniors’ lives better
Possible Tools to Measure Resident Impact;Changes over Time
✓ Quality of Life✓ Meaningful interactions/connections;
Frequency of feeling lonely, depressed, etc.
✓ Satisfaction
▪ With their life overall
▪ With living in [stated city]
▪ With [domains, i.e. community safety]
✓ Shifting Perceptions, Beliefs, Attitudes
✓ Participation –
Engagement in Different Activities✓ Progress Towards Life Goals
✓ Awareness & Utilization of Services / Programs
✓ Planned Length of Residence
Research MethodologyAge Friendly Colorado Springs
14
Elevated Insights conducted data mining + two phases research
21 ONLINE/PAPER SURVEYS FOCUS GROUPSN = 600 El Paso County Residents 460 Seniors 60+ years old plus140 Caregivers of Residents Aged 50+
1 group of Older Residents1 group of Caregivers
• Overall life satisfaction, Satisfaction with Colorado Springs services within Age Friendly domain areas
• Awareness, usage, and satisfaction with local resources/service providers
• Needs and desires beyond what El Paso County currently provides
• Media use & preferences
• Specific Facebook groups utilized
• Specific search terminology utilized
•More fully understanding senior and caregiver needs
•Exploring how these seniors and caregivers currently vet and select resources
•Informing the development of the web resource portal and its components through co-creation exercises
•Gauging reactions to possible website layouts and portal names
Breakouts & Geographic BoundariesAge Friendly Colorado Springs
15
Older Residents vs. Caregivers
Males vs. Females
HH Income –<$50K, $50-$100K, $100K+
Age Brackets: 60-64, 65-69, 70-74, 75+
Regions - EPC Commissioner Districts
Older Residents – Awareness & Usage of Services
Average “Aware of”: (out of 19; 37% among presented list)
Average “Used (or interacted with)”: (out of 25; 8% among presented list)
15%
16%
17%
18%
22%
24%
26%
27%
31%
36%
37%
40%
46%
47%
54%
74%
78%
84%
4%
6%
3%
2%
4%
2%
9%
8%
3%
4%
9%
2%
9%
11%
11%
24%
15%
27%
CONO Council of Neighbors & Organizations
COPPeR Cultural Office of the Pikes Peak Region
Bike COS
Rocky Mountain Health Care Services
The Independence Center
Greccio Housing
UCCS Aging Center
Area Agency on Aging (AAA)
Colorado Springs Housing Authority
Aspen Pointe
Pillar
Pikes Peak Workforce Center
Peak Vista
Trails and Open Space Coalition
Better Business Bureau
YMCA
Silver Key
AARP
Awareness Usage
Percentage Aware vs. Percentage Who Used Service in Past 12 Months(awareness & usage percentages out of total population)
Q. Which of the following are you aware of as it relates to offering resources/services to older residents in the community? (N = 457)
Q. Now which of the following have you used (or interacted with) in the past 12 months? (N = 455)
72
Older Residents – Activities
5.4 On average, older residents engaged in 5.4 of the 11 types of activities presented within the past 12 months.
3%
15%
24%
25%
29%
45%
49%
52%
65%
75%
76%
87%
None of the above
Participated in a senior only event or meetup
Attended a townhall or community meeting
Attended an in-person learning opportunity…
Participated in arts and crafts (e.g. painting,…
Participated in a spiritual or religious activity
Participated in a social activity (e.g. clubs,…
Volunteered
Participated in a cultural activity (e.g. museums,…
Participated in a physical activity (e.g. played a…
Donated to a charity
Voted in a midterm election
Participation in Past 12 Months
Q. What type of activities have you done, participated in, or attended within the past 12 months? (N = 458)
10%
20%
30%
40%
50%
60%
70%
80%
90%
<$50k $50k-$100k >$100k
Social Participation by Income
Participated in a physicalactivity
Participated in a culturalactivity
Volunteered
Participated in a socialactivity
Attended a townhall orcommunity meeting
Older Residents –Satisfaction with Quality of Life Factors
1745
5060
6465
6875757778808081818384858686878789899193
Range of Senior living situationsCommunity safety
Affordable health care/support
Participate in local government
Respect from young communityOpportunity to meet other older residents
Share opinion on community mattersLearning opportunities/classesDoctors/medical practitionersAvailability of social activities
Employment
Protect from scams/fraudsInteraction with family
Interaction with friendsRespect from family
Parks/trails/outdoors
Respect at workAvailability of local events
Arts and cultural eventsAccess/prepare healthy food
Volunteering opportunitiesInteraction with partner
Manage bills/accountsUse technology (stay connected)
Use internet (find resources)Religious/spiritual activities
Net Satisfaction of Quality of Life Factors*
Q. How satisfied or dissatisfied are you with each aspect of your life and/or living within your community? (N = 459)
Net Satisfaction is Calculated by Subtracting the % Dissatisfied (Bottom 2 box) from the % Satisfied (Top 2 Box).
-44-18
-8-1
Public transportation
Affordable housingStreets & sidewalksAvailability of jobs
Older Resident Quality of Life Factors –Satisfaction with Living in Colorado Springs vs. Derived Importance
Focus Here – High Importance, Low Satisfaction
Focus Quadrant Zoom –Satisfaction with Living in Colorado Springs vs. Derived Importance
Availability/Ease of Using Public Transportation
-25
-48
-38
-38
-69
Are there there any resources or services that you desire, but have not found in El Paso County? If so, what were they?
Availability/Ease of Using Public Transportation
Are there there any resources or services that you desire, but have not found in El Paso County? If so, what were they?
“My experience with finding transportation has been difficult. With Silver Key, you need such a wide window that it can take 2x the time. But now, with increasing dementia, that is no longer even an option.”
“Bus transportation on west side is very bad. [There is] no bus on weekends and it’s difficult to get to church downtown”
“Better and affordable public or private transportation – don't need it now, but want it to be there”
“Right now, I drive. But there doesn't seem to be any real public transportation for folks who don't drive in my area.”
Colorado Springs Age Friendly Resident Score Card Card
February 2019
49
CommunityEngagement
Pre-Portal2019
2020
P12 Month % Engagement
Voted in a midterm election 87%
Donated to a charity 76%
Physical activity 75%
Cultural activity 65%
Volunteered 52%
Social activity 49%
Religious activity 45%
Arts and crafts 29%
In-person learned opportunity 25%
Townhall/community meeting 24%
Senior only event or meetup 15%
Types of Engagement - #(Average, among 11 presented)
5.4(49%)
Used Public Transport/MMT 3%
Portal Awareness 0%
Service Awareness - # (Average, among 19 presented)
7(37%)
Pre-Portal2019
2020
% Satisfaction Very NET Very NET
Life overall 45% 88%
Living in COS 39% 82%
Mental health 51% 81%
Finances 31% 62%
Physical health 25% 61%
Outdoor Spaces & Buildings
34% 83%
Transportation 4% -23%
Housing 10% 3%
Social Participation 27% 79%
Community Health & Support
24% 67%
Respect & Social Inclusion
39% 77%
Civic Participation & Employment
26% 61%
37
Service Awareness
82
OverallSatisfaction w/
Living in COS
50
Satisfaction with Domain Areas
55
Total Resident Score Age-Friendly COS
Triangulation Approach1. Action vs. Goals2. Community Indicator Metrics3. Resident Scorecard
AARP Livability Index –El Paso County
2/1/2019
Domain Progress vs. Goals
55
Resident Scorecard
El Paso County
2/1/2019
To ensure Colorado Springs is:
1) Making progress towards its Age Friendly action plans – check action steps vs. goals.
2) Providing the infrastructure necessary to win – check Livability Index from AARP
3) For older resident awareness, engagement, and satisfaction, turn to the Older Resident Scorecard
APPENDIX:Triangulation
within Community Indicator Efforts
Evolving Community Indicator Integrated ScorecardCombines Both CI Peer Rank and Resident Input for Holistic Metric/Domain
GoalResident Scorecard
CIC Peer Rank
Status Trend
Equitable, Affordable, and Inclusive
31% 4 of 5Well Below
TargetDeclining
Strong & Authentic Neighborhoods
45% 3 of 5 Below Target Stable
Connected, Safe, and Accessible Places
27% 3 of 5 Below Target Stable
Economically Diverse and Vibrant
56% 2 of 5 Hitting Goals Improving
Environmentally Resilient 19% 5 of 5Well Below
TargetDeclining
Healthy and Active 72% 1 of 5 Above Target Improving
Education and Enrichment 51% 3 of 5 Below Target Stable
27
• Elevated Insights recognizes that a key goal of the Quality of Life effort is volunteer involvement with indicator data collection, reporting, and coordinating progress/ action against needed focus areas.
• Elevated Insights seeks to facilitate volunteer efforts, relieve the burden of weekly content curation/engagement efforts, and equip volunteers to mobilize citizen action committees. Importantly, we recommend that EI (or another professional research firm) conduct citizen benchmarking studies to ensure valid data collection, statistical analysis, and consistency over time.
CITIZEN BENCHMARK SURVEYS
Survey a census-balanced cross-section of citizens on a regular basis to monitor shifts in citizen satisfaction, needs, behaviors, attitudes, and perceptions.
Ensure methodology is consistent and sample size is adequate for desired subgroup breakouts.
Suggested frequency –every other year.
CONTENT CREATION –VISUAL INFOGRAPHICS
Using findings from benchmark study and/or the metadata volunteers assembled, Elevated Insights analysts and strategists will create 50 accurate Infographics or visual charts across a mix of domain areas for simple weekly deployment.
These 50+ options will be curated within 2 weeks of benchmark report delivery.
WEEKLY POLLING & ENGAGEMENT
Some Community Indicator Primary Research Options
HOST & EQUIP ONLINE ACTION COMMUNITIES
NATIONAL OR PEER CITY PERCEPTION TRACKING
Survey a statistically valid base of national citizens to share top of mind perceptions on COS and rate the COS area on a variety of factors, including as a place to live, to visit/vacation, to conduct business, to retire, etc. Could also be used for peer city comparisons or national QLI benchmarking.
Suggested frequency –every 3 years.
Elevated Insights can assist featured Vision Councils / Domains with ongoing engagement by posting weekly polls/surveys, open-ended questions, and video-response requests.
Our analysts and strategists can turn these into findings to be shared during the following week to ensure momentum / action.
Equip Vision Council teams to take action via citizen action groups with the aid of an online community platform.
EI will host and manage the online civic action committee activities, including recruiting passionate citizens & training Vision Council teams on how to moderate and use the tool.
EI will program & flight activities, and remind citizens to participate.
Benchmark Survey
Methodology: Census-Balanced Benchmarking Study Among Greater COS Residents
Assess and Monitor Shifting Citizen Needs, Behaviors, Satisfaction
For Example…
• Satisfaction with quality of life, both overall and within the various domains
• Awareness of different efforts or opportunities
• Behaviors – engagement in different aspects of community life
• Self-reported data – i.e. # of hours worked each week, stress level, etc.
• Attitudes on specific interest areas, i.e. marijuana, homelessness, tourism
• Relative perception for how Colorado Springs compares to peer cities or US
• Video submissions that bring resident sentiment/challenges to life
4
Elevated Insights Involvement Option
Elevated Insights would design, program and field an online survey to a census-balanced sample of local residents. The findings from this survey would:➢ Inform the engagement phase
• Which QLI aspects are the highest priority and most linked to their satisfaction?• Where are resident perceptions out of line with metadata?• How should the council advisors go about engaging citizens?
➢ Establish benchmarks to measure progress over time
Community Indicator Triangulation Opportunities
1
2
3
Resident Benchmarking
Frequent Community Engagement
Host & Equip Civic Action Committees
Now, thinking about living in the greater Colorado Springs area. How satisfied are you with the following aspects? (0-10 scale) (n=1,041)(Top 3 Box combines 8, 9 and 10 ratings on a 0-10 point scale)
66%
52%
41%44%
36%31% 29% 25% 31%
20%17%
10%
78%
54%
48%
26%
44%
30%33% 35%
11%
26% 28%
9%
2017 2018
Cost of Living/Housing
Job Market/Opportunities
Outdoor Recreation/
Activities
Public Parks/Spaces
Restaurants/Food
Biking/Walking Routes
PublicSchools
Vibrant Downtown
Area
Arts/Culture Public Transportation
Safety of Community
Entertainment Options
Quality of Life Aspects: Top 3 Box Satisfaction(% Rating 8-10 on 10-point satisfaction scale)
Benchmarking Study – Report Example31
AskCOS® 2018– Elevated Insights
Content Creation(Visual Infographics)
Methodology: Content Creation: 50 Bites of Info/Infographics from Benchmarking Data – Varied Vision Council Topics
Elevated Insights would use the results of the benchmarking study to create small infographics that could be shared with the engaged community on a weekly or monthly basis.
This
4
Elevated Insights Involvement Option
AskCOS® 2017– Elevated Insights
AskCOS® 2017– Elevated Insights
4
5/10 Men are think COS is very or extremely safe
Only 3/10 Women think COS is very or extremely safe
Average # of trips in the past 3 months:
1.90 3.71 3.03 2.95 1.88 4.47 1.70
Flying out of Denver
International Airport
Visiting Family
& Friends
Restaurants
53% 38% 35%
Shopping
34%
1. 2. 3. 4. Concerts Work Trips
(not commuting)Professional
Sports
27%20%
5. 6. 7.
25%89% of local
residents traveled to Denver for at least one of these activities in the past 3 months.
Shareable Infographic Examples
AskCOS® 2017–Elevated Insights
AskCOS® 2017– Elevated Insights
AskCOS® Example – Elevated Insights
Weekly Polling & Engagement
Weekly Polling/Engagement
Elevated Insights Involvement Option
Elevated Insights would develop, host, program, and launch an open-ended or closed-ended question for citizens to interact/engage with. The question could be hosted on the community website or distributed via email to engaged residents.
EI would then provide the results of each question in a visual manner to be shared with engaged residents the following week.
4
Week 1 Week 2 Week 3 Etc.
Question 1 is availableQuestion 2 is availableQuestion 1 results shared
Question 3 is availableQuestion 2 results shared
Aspen Citizen Survey 2019 – Elevated Insights
4
Weekly Polling/Engagement Examples
Question: Shareout (following week):
AskCOS® 2017– Elevated Insights
4
Weekly Polling/Engagement Shareout Examples
AskCOS® 2017– Elevated Insights
AskCOS® 2017– Elevated Insights
Southeastexpress.com
Host/Equip Civic Action Committees
Methodology: Vision Council Engagement Communities
Elevated Insights would equip Vision Council teams with local resident action committees ready to take on tasks or provide detailed input in needed areas.
• Host/manage online community platform
• Recruit interested citizens from QLI website; replace inactive citizens (assumes about 25 per action committee)
• Train Vision Council teams – how to use/moderate, how to download/access citizen input
• Each Vision Council can post 5 – 10 activities per year
• Program desired activities
• Manage invitations/reminders
• Vision Council teams personally moderate and interact with citizens
4
Elevated Insights Involvement Option
Methodology: Vision Council Engagement Communities – Activity Examples
Different activity types available on the Engagement Communities:
Forum: Open, topic focused discussions and picture/video uploads (influenced)
Step board: Open, topic focused discussions and picture/video uploads (uninfluenced)
Challenge: Participants can post ideas with a title, description, images, documents, or draw on an image provided to make it their own – then they can comment on others and rate their favorite submissions. They can also draw on images also to show what they like/dislike about it
Journal: Create questions/prompts for individuals to answer privately. Participants know that only the moderator can see their answers. This can be used to ask participants more personal questions that they would be more comfortable answering one-on-one
Chat: Used to speak to a select few of the community to real time chats
4
Elevated Insights Involvement Option
Activity thought-starters:Economy Forum: Identify and upload two pictures – one that describes
an area of the Colorado Springs economy that is doing well and one
that describes an area of the Colorado Springs economy that you wish
was doing better. Please provide an explanation as to why you chose
these pictures.
Community Engagement Challenge: Please post your idea of a new
event that you think Colorado Springs should host. Who is this event
for? How would this event help the citizens of Colorado Springs? After
posting your idea, please rate your favorite event ideas.
Transportation Journal:
• How would you describe the quality of Colorado Springs public transportation system? How could the transportation system improve?
• Photo Activity – Upload photos of the condition of streets and sidewalks, highlighting what areas in the city of Colorado Springs needs the most attention.
National Perception Tracking
Methodology: National Perception Tracking
4
Elevated Insights Involvement Option
Elevated Insights would design, program and field an online, mobile-friendly survey to a census-balanced nationwide sample.
Survey respondents will be asked to share top of mind perceptions of Colorado Springs and rate the COS area on a variety of factors, including as a place to live, to visit/vacation, to conduct business, to retire, etc.
Time permitting, nationwide residents could also be asked quality of life questions (rating the area they currently live) to establish national benchmarks.
Option 2 – Truly Random (Scientific) Sampling
Elevated Insights will mail postcards to randomly selected residential addresses in El Paso and Teller Counties. These postcards will provide 3 easy methods by which residents can take an online survey:
1. An easy-to-type survey link
2. A scannable QR code
3. A phone number that can be texted to have the survey link sent to their mobile device
Residents will also be given the option to request a mailed survey. This method will absorb the benefits and efficiencies associated with online surveys, while creating a truly random selection of survey participants.
Pros:
➢ Each resident has an equal opportunity to be selected for this survey (don’t have to belong to a research panel).
➢ Geographic segmentation – each postcard will have a unique code that matches with their address. This allows for more precise segmentation by resident location (as opposed to simply using zip codes).
Cons:
➢ The physical mailing of postcards leads to a more expensive project on the whole, especially when targeting larger sample sizes.
Explaining Sampling Options
44
Option 1 – Panel Sampling
Elevated Insights will source respondents through its robust panel of 10k+ Pikes Peak area residents. Emails with a survey link will be sent to randomly selected individuals who live in El Paso and Teller Counties. As responses come back, EI can shift its targeting – e.g. if many responses are coming from women, EI can send to more men to balance out the response distribution. The data will also be weighted afterwards to adjust for any deviation in response rates.
Pros:
➢ This method of collection is often much cheaper.
➢ Higher sample sizes are more attainable (especially for EI who has a robust local database and strong community relations).
➢ EI will be able to “boost” responses for specific demographics or parts of town throughout the fielding process where necessary.
Cons:
➢ Geographic segmentation can be done based on zip codes, but it’s not as exact as the segmentation possible through Option 2.
➢ Sometimes, using online panels can lead to a higher relative response rate among younger residents. This can be adjusted through weighting, but the effective sample size may suffer. However, this is less of a concern for EI since it has a significant number of older residents in our panel due to recent aging research conducted.
*Please note: To be clear, both methodologies are statistically valid and would produce accurate, probabilistic results. Option 2 is a more institutional/academic approach. Online panel sampling (Option 1), on the other hand, has become the preference in the private sector due to the speed, efficiency and sample sizes that are possible.
Appendix / EI Information
THANK YOU
525 N. Tejon StColorado Springs, CO 80903
Debbie Balch(719) 590 – 9999
Triangulate to Fully Measure Community Impact Immediate access to this Powerpoint presentation:
1 Go to www.kiwilive.com
2 Enter event keyword: Triangulate
3 Enter your contact information