HeightWeightI.Q.IncomeBody mass indexAgeTestosteroneHighest grade#years collegeAmount of hair
Measures ofSelf-esteemAcceptance of OthersNarcissismTraditional sex RolesSatisfaction withlife
Qualitative measures with words Defining what makes a difference
Qualitatively and quantitativelyObserve behaviorObserve environLook at documents(diploma etc)Observe appearanceInterview spousePerson, familyKey informantsSurvey friendsGo thru trashCount Big MacWrappers in car
Look, listen andDescribe, describe describeWrite, write, writeAnalyze, analyze, analyze
• QUALITATIVE MEASURING BY USING WORDS (RIGHT HAND COLUMN) REQUIRES DIFFERENT SKILLS AND METHODS THAN USING NUMBERS
Process Evaluations and qualitative researchQuantitative answers Is there a difference? A change, effective? And How much? Are there similarites? Relationships? (correlation) And How much? Qualitative describes! Answers What is it? Why is it? How does it work? And What does it mean?
Qualitative approaches are typically used with process evaluations. . Quality assurance, CQI, TQM, program monitoring, needs assessments, formative evaluations, implementation evaluation and to some degree consumer satisfaction are all formative.
Qualitative research typically does not use measurement, numbers and statistics,. It uses words, descriptions, artifacts and observations!
PROCESS EVAL = QUALITATIVE = IMPROVEMENT OF PROGRAMOUTCOME EVAL = QUANTITATIVE = JUDGMENT ABOUT PROGRAM WORTH
Here are the things that a qualitative researcher does:• observes events and activities as they occur in the program setting.• gets directly involved with the people in the program and sometimes personally
experiences the processes used in the program.• acquires an insiders’ point of view while maintaining an analytic perspective• uses a variety of observational and interviewing techniques and social skills in
a flexible –often not pre-determined way-which allows the evaluator to follow the answers to the research questions wherever they might go.
• produces data in the form of extensive written notes, maps, diagrams and sometimes pictures in order to provide very detailed descriptions.
• watches, listens and collects data• uses artifacts of the program like records, treatment plans, staffing patterns,
absentee rates, dropout rates, ashtray use, lounge use, patterns of wear on equipment, but only to reach conclusions not to count!
• uses observations and/or several forms of interviewing.• reviews the collected data (in the form of written notes, videotapes, in-depth
written interviews, audiotapes.• in reviewing the data, begins to look for repetitive ideas, themes, common
conceptions, incongruities, strengths and weaknesses.• reviews the data again and begins to make certain hypotheses grounded solely
in the collected data!• begins to draw meanings and conclusions from the data collected.• writes a narrative about those meanings and conclusions.
• ALTHOUGH CLEAR OBJECTIVES SHOULD BE A PART OF ANY PROGRAM, THEY ARE NOT AS NECESSARY TO DO A PROCESS EVALUATION.
• HOWEVER YOU STILL NEED TO HAVE SOME SENSE OF A. WHAT YOU ARE MEASURING AND B. HOW YOU WOULD MEASURE IT IN A PROCESS EVALUATION.
• REALLY WHAT WE ARE TALKING ABOUT IS HOW TO OPERATIONALIZE THE RESEARCH QUESTION.
EXAMPLE OF A (SOMEWHAT VAGUE) PROCESS EVAL. RESEARCH QUESTION
ARE THE CLIENTS ACTIVELY ENGAGED IN THE DAY PROGRAM?
SO NOW YOU ASK YOURSEL F THESE QUESTIONS: 1. WHAT DOES ACTIVELY ENGAGED LOOK LIKE?
2. HOW DO I OPERATIONALIZE IT? (WHAT DO PEOPLE WHO ARE ACTIVELY ENGAGED DO?)
3. WHAT DO I LOOK FOR?
4.HOW DO I MEASURE IT USING WORDS?
• POSSIBLE OPERATIONAL DEFINITIONS: 1. RATE OF DAILY ATTENDENCE , ABSENTEEISM AND TARDINESS BASED ON
AVAILABLE CLIENT RECORDS 2. OBSERVATIONS BY EVALUATOR OF VERBAL INTERACTIONS AMONG
CLIENTS AND STAFF DURING ACTIVITIES. 3. OBSERVATIONS BY EVALUATOR OF CLIENTS FOLLOWING DIRECTIONS AND
COMPLETING TASKS IN ACTIVITIES. 4. OBSERVATIONS BY EVALUATOR OF CLIENT-GENERATED QUESTIONS AND
STATEMENTS THAT ARE RELEVANT TO THE ACTIVITY AT HAND. 5. OBSERVATIONS BY EVALUATOR OF NUMBER OF CLIENTS WHO LEAVE
ACTIVITIES BEFORE THEY ARE OVER OR GO OUT FOR UNSCHEDULED CIGARETTE BREAKS.
6. IN-DEPTH INTERVIEWS WITH 3 RANDOMLY SELECTED CLIENTS WHICH ASK THE QUESTIONS a. HOW IMPORTANT IS THE PROGRAM IN YOUR LIFE? b. WHY OR WHY NOT? c. WHAT DO YOU LIKE ABOUT THE PROGRAM? DON’T LIKE? THE EVALUATOR WILL LOOK FOR EVIDENCE OF IMPORTANCE AND RELEVANCE TO CLIENTS LIFE, EVIDENCE OF ENJOYMENT WHEN DISCUSSING THE PROGRAM AND THE NUMBER OF POSITIVE STATEMENTS ABOUT THE STAFF AND PROGRAM.
7. A BRIEF SURVEY OF ALL STAFF ASKING THEM TO RATE 1. OVERALL LELVEL OF CLIENT PARTICIPATION BASED ON A FIVE POINT LIKERT SCALE.
8. NUMBER OF DOCUMENTED CLIENT OUTBURSTS, NON-COMPLIANCE OR NON-PARTICIPATION, BASED ON A RANDOM AUDIT OF CLIENT CHARTS FOR ONE WEEK
NOTICE THAT BY ANSWERING THOSE FOUR QUESTIONS, YOU CREATE YOUR
1. SOURCES OF DATA
2. METHODS OF DATA COLLECTION
3. MEASURES OF “ACTIVELY ENGAGED”.
4. Coding categories (more later)
• IN QUALITATIVE RESEARCH, IT IS IMPORTANT TO MAKE SURE YOU UNDERSTAND WHAT ALL OF THE TERMS IN THE RESEARCH QUESTION LOOK LIKE!!!
• COMMONLY CONFUSED TERMS– PARTICIPATION– IMPROVEMENT– ADEQUATELY PREPARED– HOW CAN WE REACH CLIENTS– DO CLIENTS FEEL HELPED– ARE STAFF CARRYING OUT THE PROGRAM AS DESIGNED– ARE CLIENTS AND/OR STAFF “BUYING INTO” OR “INVESTED” IN THE
PROGRAM– WHAT ARE SOME OF YOURS?
IN QUALITATITIVE RESEARCH, YOU MUST BEGIN TO THINK ABOUT WHAT THESE MEAN AND WHAT TO LOOK FOR