research and evaluation on a dime
TRANSCRIPT
![Page 1: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/1.jpg)
RESEARCH AND EVALUATION
ON A DIME
Religion Communicators Council
2015 National Convention
Alexandria, VirginiaDouglas F. Cannon, Ph.D., APR+M, Fellow PRSA
Virginia Polytechnic Institute and State University
April 10, 2015
![Page 2: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/2.jpg)
WORKSHOP DESCRIPTION
Research is fundamental to public relations
planning and evaluation. Religion communicators
often say they don’t have the time, money or
background for research. This session looks at
simple, cheap methods—qualitative and
quantitative, primary and secondary—for
gathering information about publics and
communication effects. Communicators can use
these techniques to determine realistic
objectives, plan programs, assess results and
demonstrate effectiveness.
![Page 3: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/3.jpg)
Topics
Introduction• What’s research? What’s evaluation?• Why do we research? Why evaluate?• Research and religion communicators• 4-step public relations process
Research• Types of research• Secondary research• Research techniques• Primary qualitative, quantitative research tools
Evaluation• Things to evaluate• Basic evaluation questions• Measurement levels
![Page 4: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/4.jpg)
Introduction
![Page 5: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/5.jpg)
What’s Research?
• Gathering information (listening,
observing, monitoring)
• Testing hypotheses, strategies,
messages
• Judging results
![Page 6: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/6.jpg)
What’s Evaluation?
• Judging results against agreed-upon
objectives established during planning
• Determining value of effort
![Page 7: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/7.jpg)
Why do research?
• To learn about, understand or assess
situations
• To gather information about publics
• To gauge public opinion, track issues, monitor
industry trends
• To test ideas, plans, messages, products
• To monitor, evaluate progress or feedback
• To make decisions, prepare for planning
• To generate publicity
![Page 8: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/8.jpg)
Why Evaluate?
• To judge effects, outcomes
• To determine value of what you did
• To maintain accountability
• To document contributions to your organization
• To learn from successes, failures
• To provide information for future efforts
![Page 9: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/9.jpg)
When to research?
• Continuously: Environmental scanning
• Before a campaign• Assess problem
• Assess publics
• Assess media, social climate
• During a campaign• Implementation checking
• In-progress monitoring
• Benchmarking
• After a campaign• Evaluate outputs
• Evaluate outcomes
![Page 10: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/10.jpg)
Research & religion communicators
I conduct communication audits to identify
communication problems between the
organization and various publics.
• 2006: 1.96/5 (n=115)
• 2011: 1.98/5 (n=150)
• 2013: 1.72/5 (n=46)
![Page 11: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/11.jpg)
Research & religion communicators
I report public opinion survey results to keep
management informed of the opinions of
various publics.
• 2006: 2.96/5 (n=115)
• 2011: 2.34/5 (n=150)
• 2013: 1.67/5 (n=46)
![Page 12: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/12.jpg)
4-step
Public Relations process
• RACE
• ROPE
• R-PIE
![Page 13: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/13.jpg)
Copyright © 2013 Pearson Education, Inc. All
rights reserved.
Public Relations as a Process:
RACE
ResearchEvaluation
ActionCommunication
![Page 14: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/14.jpg)
Research
![Page 15: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/15.jpg)
Types of PR research
• Secondary: Information gathered by
others
• Primary: Information gathered firsthand
• Qualitative: impressions
• Quantitative: numbers
![Page 16: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/16.jpg)
Secondary research
• Search engines: Google Scholar, Google, Yahoo
• Research organizations
• Gallup (http://www.gallup.com)
• Pew Research Center (Religion and Public Life) (http://www.pewforum.org/)
• Institute for Public Relations (http://www.instituteforpr.org/)
• The Association of Religion Data Archives (http://www.thearda.com/)
• Academic journals
• Public Relations Journal (http://www.prsa.org/Intelligence/PRJournal/)
• Research Journal of the IPR (http://www.instituteforpr.org/research-journal/)
• Journal of Public Relations Research
• Public Relations Review
• Trade publications/blogs
• Mr. Magazine (https://mrmagazine.wordpress.com/)
• Bulldog Reporter’s Daily Dog (https://www.bulldogreporter.com/dailydog)
• CommPRO (http://www.commpro.biz)
• Ragan’s PR Daily (http://www.prdaily.com)
• RTDNA Daily (http://rtdna.org/)
![Page 17: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/17.jpg)
Other sources
• University departments, schools, colleges• Advertising
• Communication
• Journalism
• Mass Communication
• Public Relations
• University agencies• Bradley Public Relations, Brigham Young University (http://www.bradleypr.com/)
• Cardinal Communication, Ball State University (http://cardinalcomm.org/blog/)
• Center for Communication Research, Texas Tech University
(http://www.depts.ttu.edu/comc/ccr/index.php)
• Lindsey + Asp Advertising + Public Relations, University of Oklahoma
(http://www.lindseyandasp.com/)
• The Plank Center for Leadership in Public Relations, University of Alabama
(http://plankcenter.ua.edu/)
![Page 18: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/18.jpg)
research techniques
• Asking questions• Direct—to gauge knowledge
• Indirect—to probe motivations
• Observing
• Consulting sources
• Reviewing lessons learned
![Page 19: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/19.jpg)
Primary research tools
• Surveys
• Direct observation
• Interviews• One-on-one (guided, depth
or informal)
• Group (focus groups)
• Experts/panels
• Content analysis
• Copy/message testing
![Page 20: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/20.jpg)
Qualitative tools
• Interviews (formal, informal)
• Focus groups (Sunday school
classes)
• Direct observation (meetings, events)
• Expert panels (Lectionary study
groups)
• Message tests (Sunday school
classes)
• Textual/discourse analysis (reading)
![Page 21: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/21.jpg)
Quantitative tools
• Surveys• SurveyMonkey.com
• Qualtrics.com
• Content analysis
• Readability tests
![Page 22: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/22.jpg)
Survey samples
• Random
• Self-selected
• Purposive
![Page 23: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/23.jpg)
Evaluation
![Page 24: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/24.jpg)
Evaluation
• Measurement: Using metrics tied to
objectives (quantitative)
• Assessment: Judging success, failure
(qualitative)
• Evaluation: Determining value (Were
results worth the effort?) (qualitative)
![Page 25: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/25.jpg)
Things to evaluate
• Preparation (research, planning)
• Implementation (execution)
• Output (messages, efforts)
• Outcomes (immediate results compared to
objectives)
• Outgrowths (long-term or societal changes)
![Page 26: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/26.jpg)
Basic Evaluation Questions
• Adequately planned?
• Message(s) understood?
• How could strategy have been more effective?
• Audiences reached?
• Objectives achieved?
• What was unforeseen?
• Budget met?
• Future improvements?
![Page 27: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/27.jpg)
Measurement Levels
![Page 28: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/28.jpg)
Message-Exposure Measures
• Clippings/mentions
• Media impressions
(Placement x circulation/viewership/listenership)
• Internet hits, likes, follows, retweets
• Requests for information
• Systematic tracking/online analytics
(Analyze volume and content of media placements or online engagement)
• Advertising equivalency (not acceptable)
(Space/time x advertising rate)
![Page 29: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/29.jpg)
Summary
Introduction• What’s research? What’s evaluation?• Why do we research? Why evaluate?• Research and religion communicators• 4-step public relations process
Research• Types of research• Secondary research• Research techniques• Primary qualitative, quantitative research tools
Evaluation• Things to evaluate• Basic evaluation questions• Measurement levels
![Page 30: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/30.jpg)
Contact
Information
Douglas Cannon
Department of
Communication
Virginia Polytechnic
Institute and State
University
540-231-2331
106 Shanks Hall (0311)
Blacksburg, VA 24061
![Page 31: Research and Evaluation on a Dime](https://reader034.vdocuments.net/reader034/viewer/2022042502/55a622c21a28abeb278b4679/html5/thumbnails/31.jpg)