Data Collection and Management: Where the Rubber Hits the Road

Karen L. Franck, PhDExtension Specialist

UT SNAP-Ed Program

• 92 out of 95 counties funded• 2008-2009 fiscal year– 71,405 volunteer educators– 7,234,830 indirect contacts– 802,398 direct contacts

System for University Planning Evaluation & Reporting (SUPER)

• Statewide database• Multiple functions• County Agent responsible for data entry

Education & Administrative Reporting System

• Consistent reporting across states– # of participants– # contacts– Impacts– Demographics

Dueling DatabasesRacial Groups for UT SUPER• White• Black or African American• American Indian or Alaskan

Native• Asian or other Pacific

Islander

• Hispanic

Racial Groups for USDA EARS

• White• Black or African American• American Indian or Alaska

Native• Asian• Native Hawaiian or Other

Pacific Islander

• All racial groups are classified as either non-Hispanic or Hispanic

Before Collecting Data

• Data Collection PlanWHO, WHAT, WHERE, WHEN & HOWWHY should have been answered long

ago

Training for Data Collection

• Need for consistent and accurate data• Address all procedures and special

instructions• Anticipate potential issues

Lessons Learned the Hard Way

• Nutrition education classes for adults implemented in 12 counties1 county received older version of surveys 3 month follow-up right at Christmas

• SNAP-Ed programming in local school systemProgram Assistant was not aware of need

for data collection until last day of school

During Data Collection

• Ongoing training and support• Quality checks

Address major & minor issuesLook at data during collection

More Hard Lessons

• Summer cooking camp for middle graders Introduced post-collection: “Make sure your scores have

improved!”

• No participant identification numbers on mailed follow-up surveys

• Over 75% of participants didn’t fill in last question• Transposed numbers

532 out of 412 participants reported eating more fruit

After Data Collection

• Acknowledge issues and limitations resulting from data collection methods

• Learn from successes & mistakes!

Final Hard Lessons

• 5 years of data sitting in a file cabinet

• Outcome indicators changed but same survey continued to be used

Data Management Plan

• Maintain a raw data set• Backup data set• Guard confidentiality• Develop a codebook for all variables

(collected & created)

Quality Checks

For all variables (collected & created)• Run frequencies• Crosstabs• Check the range• Have a plan for outliers• Missing value analyses

Sources

Fink, A. (2003). The Survey Handbook. London: Thousand Oaks.

Schalock, R. L. (2001). Outcome-based Evaluation. New York: Kluwer Academic/Plenum Publishers.

The University of Texas-Houston Health Science Center School of Public Health and The Texas Department of Health. Practical Evaluation of Public Health Programs Workbook. Available at http://www2.cdc.gov/phtn/Pract-Eval/workbook.asp .

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. Editors. (2004). Handbook of Practical Program Evaluation. San Francisco: Jossey-Bass.

Townsend surveys and training tools: http://townsendlab.ucdavis.edu/

THANK YOU

Karen L. Franck, PhDExtension Specialist

University of Tennessee ExtensionTennessee Nutrition & Consumer

Education Program865-974-7457

kfranck@utk.edu