ready, set . . . evaluate your coalition - family health outcomes project · ready, set . . ....

46
Ready, Set . . . Evaluate Your Coalition CDC You Can Do It . . . We Can Help! CDC Office on Smoking and Health Evaluation Webinar Evaluation Webinar July 15, 2010 Frances D. Butterfoss, PhD Yorktown, Virginia Yorktown, Virginia

Upload: others

Post on 15-May-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Ready, Set . . . Evaluate Your Coalition

CDCYou Can Do It . . . We Can Help!

CDC Office on Smoking and Health Evaluation WebinarEvaluation Webinar

July 15, 2010

Frances D. Butterfoss, PhDYorktown, VirginiaYorktown, Virginia

Page 2: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Evaluation Mythsy

Myth 1: Evaluation is complex - we don’t have the time!

Myth 2: It’s an event to get over with and then move on!

Myth 3: Evaluation involves a new set of activities – we don’t have resources!

Myth 4: There’s a "right" way to evaluate. What if we don’t get it right?

Myth 5: Funders will reject our evaluation plan!

Myth 6: We don't need evaluation to tell if our coalition & strategies are effective!

Page 3: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Part I Get ReadyPart I Get Ready

Page 4: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Help Members Value Evaluationp

• Demystify evaluation - Integrate it into everyday work –

recognize its multiple purposes

• View evaluation as reflection: How can it improve ourView evaluation as reflection: How can it … improve our

work? Influence stakeholders? Ensure funding?

• Share impact that other evaluations have had on funding

decisions or public visibilityp y

Page 5: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Challenges Challenges Challenges Challenges

of of of of

EvaluationEvaluationEvaluationEvaluation

Page 6: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Check it Out!Check it Out!

What has gone well with your evaluation efforts?

What challenges have you faced?What challenges have you faced?

Page 7: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Evaluation Pitfalls • Evaluation results may be used for purposes other than

intendedte ded

• Evaluation becomes political vehicle to push through inappropriate agenda or conceal real problemsinappropriate agenda or conceal real problems

• Process of evaluating may upset fragile relationships

• Powerful players try to influence evaluation outcomes; all players not included or heardp y

• Evaluation results are ignored

T l P ll R i & G 1998» Taylor-Powell, Rossing & Geran, 1998

Page 8: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Are You Ready to Evaluate?

Are you committed to evaluate coalition & its work?

Are coalition’s goals realistic?

Are strategies consistent with coalition’s goals?g g

Are desired results specific & clear enough?

Are strategies well-grounded in theory/evidence?

Do you know what kinds of data will be needed?

Is data available? Can you access it?

Will l ti b d t k liti / li d i i ? Will evaluation be used to make coalition/policy decisions?

Page 9: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Are You Ready to Evaluate?

Will findings help you decide whether to expand strategies to new populations/settings?to new populations/settings?

Are adequate resources available?

Will evaluation…address key questions? … provide “lessons learned”?

Will you heed evaluation results, even if they cause your coalition to change?coalition to change?

Is evaluator/team willing to listen to coalition about how to design evaluation & interpret/report findings?

Page 10: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

PART II GET SET . . . . . .

Page 11: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Why????

When????

WhoWhat

The 4 “Ws “ of Who????????

Coalition

Evaluation

Page 12: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Why Should We Evaluate?y

1. Builds capacity within organization & community Do partners use approaches or tools for other issues?

2. Determines whether objectives are achieved Did organization achieve goals?

3. Improves strategies interventions, practices & p g , ppolicies Were strategies effective?

4 Provides accountability to stakeholders & funders4. Provides accountability to stakeholders & funders Are staff/members satisfied & contributing? Were resources used well?

Page 13: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

When Should We Evaluate?

• Change in resourcesg

• Change in membership, leadership or staff

• Change in structureChange in structure

• Expand topics, population, geographic focus

M j fli t i• Major conflict or issue

• External recognition

• Begin new project or hold important event

• Before marketing or fund-raising initiative

Page 14: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Who Should Evaluate?

• Evaluation team (Tap experts, staff or coalition members to help coordinate evaluation efforts)

• In-house evaluator but coalition makes decisions &• In-house evaluator, but coalition makes decisions & controls process

• External consultant plans/collects data & engages coalition as appropriate

• External evaluation by funder

Page 15: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

What will it Cost?

• Usually 7-10% of program budget

– Salary & benefits: Staff time & level of expertise needed

– Consultants: Special expertise & different perspectives

– Travel: Variable; depends on program & methods

– Communications: Postage & phone/fax

– Printing: Preparation of data-collection instruments, reports, & other documents

– Supplies & equipment: Computers & software

– Incentive items: Meals & gift cards

Page 16: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

What Should We Evaluate? Questions:

• What about coalition works/doesn’t work well? How can weWhat about coalition works/doesn t work well? How can we make it work better?

• Who are key members? Are we meeting their needs? Are we building their capacity?

• What resources have been most helpful or are needed? What are associated costs?

• Do strategies fit coalition’s mission & goals?

• Are strategies successful? Reaching intended groups?

• What have we accomplished?

Page 17: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Logic Modelsg

• Help describe core components of coalitionp p

• Shows connection between coalition,

strategies & expected outcomes

• Helps evaluators select appropriate measures

I l d k t t l f t• Includes key contextual factors

Page 18: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Logic Model Shows…

Strategies/ Activities

Outputs Short-term Outcomes

IntermediateOutcomes

Long-Term Outcomes

Inputs/ Resources

Actions to be taken;

action plan

Tangible results or

products of

Changes in learning,

knowledge,

Changes in behavior

o p actice

Changes in systems,

conditions or

Available resources,

fiscal &

Activities Outcomes Outcomes OutcomesResources

action plan products of activities

knowledge, skills

or practice conditions or environments

fiscal & human

Page 19: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Components of a Basic Logic ModelINPUTS

Investments or resources (e.g., time, staff,

INFLUENTIAL FACTORS

Surrounding environments in which the program exists (e.g., politics, other initiatives, socioeconomic factors, staff turnover, social norms

volunteers, money, materials)

and conditions, program history, stage of development) that can affect its success either positively or negatively

ACTIVITIES

Events or actions (e.g., workshops, curriculum

INTERMEDIATE OUTCOMES

Medium-term results (e.g.,

INITIAL OUTCOMES

Short-term effects of

OUTPUTS

Direct products of program (e.g., number of

development, training, social marketing, special events, advocacy)

behavior, normative, or policy changes)

program (e.g., knowledge, attitude, skill, and awareness changes)

people reached or sessions held)

changes)

LONG-TERM OUTCOMES

Ultimate impactGOAL

Ultimate impact (e.g., social or environmental change)

Mission or purpose of program

Page 20: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Check it Out!Check it Out!

What do you want to know about …y

How your coalition functions?y

Its partners?ts pa t e s

Its outcomes? Its outcomes?

Page 21: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

\PART III EVALUATE !\PART III EVALUATE !

Page 22: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Sources of DataExisting Coalition Documents

Data (Reports, newsletters, minutes, rosters)

Existing databases

Business records or websites

People Coalition members

Participants or participants

General public

Key informants (funders, officials)

Other Actual events, activities, practices

Before/after pictures (photovoice)

Maps, charts

Page 23: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Check it Out!Check it Out!

What methods have you used to …y

evaluate coalition effectiveness?evaluate coalition effectiveness?

l t it t t gi ?evaluate its strategies?

evaluate its outcomes?

Page 24: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

A Dozen Data Collection Methods1 S il d l t i i h1. Survey - mailed, electronic, in-person, phone

2. Interviews - in-person, phone, focus group

3 G A t f D l hi t h i N i l G P3. Group Assessment - forum, Delphi technique, Nominal Group Process

4. Observation - use trained interviewers

5 C St d lti l d t & th d5. Case Study - multiple data sources & methods

6. Formal Review - by peers, panel, experts, review committee

7 P tf li R i k l t t lit &7. Portfolio Review - work samples to rate quality & scope

8. Document Review - content analysis to assess & summarize

9 T ti i l / ti b k ti i t9. Testimonials - responses/reactions by key participants

10. Tests - standard measures of knowledge, skill, performance, physiology

11 Ph t h / id i l i & t i11. Photographs/videos - visual images & stories

12. Diaries/Journals/Logs - chronological records; personal views

Page 25: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Selecting Methodsg What data is needed to make decisions about product, program or

organization?

How much data can be collected & analyzed practically?

How accurate is data?

What methods will be used if more data is needed?

Will results be credible to decision makers?

Are methods acceptable (Will respondents engage in surveys, interviews & focus groups? Allow record access)?

Who has skills to use methods? Is training required?

How will data be analyzed?

Page 26: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Instruments & Tools

• Coalition Effectiveness• Coalition Effectiveness Inventory (CEI)

• Meeting Effectiveness Inventory (MEI)

• Teamwork surveys

• Collaboration & Inclusivity Checklists

• Member Satisfaction Surveys

• Member & Leader Interviews

Page 27: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Free/Low Cost Evaluation Tools/

• Survey Monkey: $200./yr www.surveymonkey.com

• Innovation Network: Tools for evaluation plans, logic models & more www.innonet.org/index.phpg p p

• Online Evaluation Resource Library: Create surveys, interviews observation guides: www oerl sri cominterviews, observation guides: www.oerl.sri.com

• United Way Outcome Measurement Resource Network: Downloadable or low cost (< $50. tools & instruction) http://national.unitedway.org/outcomes/resources/

Page 28: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Coalition Evaluation

•• LEVEL 1:LEVEL 1: Coalition infrastructure,

f ti

LEVEL 1LEVEL 1

process or function

•• LEVEL 2:LEVEL 2: Coalition interventions LEVEL 2

LEVEL 3LEVEL 3 H l h/ i l LEVEL 3LEVEL 3•• LEVEL 3:LEVEL 3: Health/social status or

community change (environment,

li i & ti )

LEVEL 3LEVEL 3

policies & practices)

Page 29: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Short-term Evaluation

Immediate results of coalition work

Measured more frequently – monthly or quarterly

May be influenced in 12 18 monthsMay be influenced in 12-18 months

May focus on changes in:

Coalition (budget, recruitment, meetings)

Strategies (context reach doses delivered/received)Strategies (context, reach, doses delivered/received)

Page 30: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Outcomes are more than just numbers!numbers!

• Membership: Increased # of members, not as important h t t th t & h th t ib t tas what sectors they represent & how they contribute to

coalition (e.g., taking leader role, being active in WGs)

Att d # f b tt di ti t• Attendance: # of members attending meetings not as meaningful as whether attendance is consistent (i.e., core of members who regularly attend)

• Strategies: # of people reached by activities/strategies not as important as whether they represent priority or intended population & whether strategies were of sufficient intensity to cause change

Page 31: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Intermediate evaluation

Intermediate results of coalition work

Influenced by multiple short-term outcomes

Measured semi-annually or annually Measured semi annually or annually

May be influenced in 2-3 years

May focus on changes in:

Strategies

Knowledge, attitudes, behaviors/practices

Policy, systems & environment

Page 32: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Long-term evaluationg Long-term results of coalition work

Influenced by multiple intermediate outcomes

Measured annually

May be influenced in 3-5 years

May focus on changes in: May focus on changes in:

Policy, systems & environment

Health/social status Health/social status

Statewide capacity/institutionalization

Page 33: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Check it Out!Check it Out!

What short, intermediate & long-term outcomes have you measured?term outcomes have you measured?

Page 34: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Level 1. Coalition infrastructure, function & process outcomesfunction & process outcomes

Member representation ExpectationsSkills/experience Perceived effectivenessRecruitment LeadershipP ti i ti C t /b fitParticipation Costs/benefits Role clarity Organizational structureCommitment Planning products/qualityg p q ySatisfaction Sense of ownership Decision making ConflictOrganizational culture CommunicationResources

Page 35: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Sample Short-term Outcome Measures

# f liti ti i t• # of coalition meetings in past year

• # members from priority population attending training

• # of staff members certified in communication or media training

• Level of satisfaction of coalition members• Level of satisfaction of coalition members

• # hours contributed by volunteers

Page 36: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Level 2. Coalition Intervention O

I l t ti

Outcomes

• Implementation

• Media coverageMedia coverage

• New/modified services & strategies

• Statewide actions taken

• New/modified policies & practices

Page 37: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Sample Intermediate Outcome Measures

• Increased knowledge about addictionIncreased knowledge about addiction

• Improved cultural awareness of priority population

• Increased self-efficacy about tobacco cessationIncreased self efficacy about tobacco cessation

• Improved skill in advocacy

• Improved behaviors (e.g., attending AA classes)Improved behaviors (e.g., attending AA classes)

• New policy - point of purchase tobacco sales

• New clinic referral practiceNew clinic referral practice

• Increased treatment options

Page 38: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Level 3. Health status & systems h tchange outcomes

Community capacity & competence

Organizational viabilityOrganizational viability

Health status

Availability, access & use of services

Page 39: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Sample Long-term Outcome Measures

Si t liti f d i dj t it• Sister coalition formed in adjacent community

• Long-term funding obtained by coalition

• Institutionalization of coalition intervention existing agency

• Decreased rates of substance use, decreased initiation of smoking by teens, decreased DUI rates

Page 40: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Interpreting Interpreting Interpreting Interpreting

& &

Using Using

ResultsResults

Page 41: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Interpreting & Using Results

• Evaluation team should summarize data & present to key partners/stakeholders for feedback/interpretationkey partners/stakeholders for feedback/interpretation

• Re-circulate draft report to ensure that conciseness, clarity & outstanding issues have been addressedclarity & outstanding issues have been addressed

• Final report can then be disseminated widely

• Entire coalition should develop action plan that sets responsibilities for acting on findings

Page 42: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Disseminating Results

• Coalition & partner newsletter/website articles, press conference & media releaseconference & media release

• Data summaries to legislators & foundations that might be interested in coalition effortmight be interested in coalition effort

• Social event (breakfast, reception) to thank staff, members s pporters & f ndersmembers, supporters & funders

• Written notes to key stakeholders that highlight & h k h f ib isuccesses & thanks them for contributions

Page 43: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Free/Low Cost Evaluation M l & W b iManuals & Websites

• Community Toolbox: http://www.ctb.edu

• Empowerment Evaluation Website: http://www.stanford.edu/empowermentevaluation

• CDC. Physical Activity Evaluation Handbook. (2002). US Dept of Health & Human Servicesp

• W.K. Kellogg Foundation. (1998). Evaluation Handbook. Battle Creek, MIBattle Creek, MI http://www.wkkf.org/Pubs/Tools/Evaluation/Pub770.pdf

• Coalitions Work. http://coalitionswork.comp

Page 44: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

Parting Tips

• Your questions about coalition & interventions should drive evaluation not the other way aroundevaluation – not the other way around

• Enlist partners’ help - builds buy-in & cooperation

• Start small – plan 1 or 2 evaluation activities/year

• Take advantage of data you already haveTake advantage of data you already have

• Verify partner roster to maximize response rates

• Reduce respondent burden – short, frequent surveys

• Use & adapt existing toolsp g

Page 45: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

So, So, ,,

take your mark & take your mark &

start evaluating!start evaluating!

Page 46: Ready, Set . . . Evaluate Your Coalition - Family Health Outcomes Project · Ready, Set . . . Evaluate Your Coalition You Can Do It . . . We Can Help!CDC CDC Office on Smoking and

References

• Butterfoss, FD. (2007). Coalitions and Partnerships in Community Health. San Francisco, CA: Jossey Bass.

• CDC, National Heart Disease and Stroke Prevention Program. (2007). Evaluation Guide: Fundamentals of Evaluating Partnerships.

• CDC. (1999). Framework for Program Evaluation in Public Health. MMWR 1999;48 (No. RR-11), updated 2001. Washington, DC: Superintendent of Documents, U.S. Government Printing Office.Documents, U.S. Government Printing Office. http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm

• Community Toolbox: http://www.ctb.edu

• Taylor-Powell, Rossing & Geran. (1998). Evaluating Collaboratives: Reaching the Potential. University of Wisconsin Extension. Madison, WI.