Instrumentation,

Introduction: One of the most important components of a research design is the research instruments because they gather or collect data or information.

!  These research instruments or tools are ways of gathering data. Without them, data would be impossible to put in hand.

QUESTIONAIRE,

!  QUESTIONAIRE The most common instrument or tool of research for obtaining the data beyond the physical reach of the observer which, for ex. May be sent to human beings who are thousands of miles away or just around the corner.

" Two Forms of Questionnaire : 1.  Closed form / Closed-ended 2.  Open form / Open-ended

Advantages," Advantages Facilitates data gathering

" Is easy to test data for reliability and validity

" Is less time-consuming than interview and observation

" Preserves the anonymity and confidentiality of the respondents’ reactions and answers

Disadvantages,#  Printing and mailing are costly # Response rate maybe low # Respondents may provide only socially acceptable answers #  There is less chance to clarify ambiguous answer # Respondents must be literate and with no physical handicaps # Rate of retrieval can be low because retrieval itself is difficult

Questions to Ask Before Doing Survey Research

!  Do you have a clear hypothesis?

!  Do your questions focus on that hypothesis?

!  Will participants answers provide accurate answers to your questions?*

!  To whom will your results apply?*

Answers May Be Inaccurate Because:

!  People don’t know the causes of their own behavior

!  People’s memories are inaccurate

!  People are not good at predicting their future behavior

!  People are not telling you what they know because of: o  Social desirability bias o  Demand characteristics o  Response sets

Generalizing Your Results

!  Do you know what your population is?

!  How can you get a representative sample of that population?

!  Will nonresponse bias prevent you from generalizing your results to that population?

Planning a Survey

!  Deciding on a research question

!  Choosing the format of your questions*

!  Choosing the format of your interview--if you use an interview*

!  Editing your questions*

!  Sequencing your questions*

!  Refining your survey instrument*

!  Choosing a sampling strategy*

Choosing the Format of Your Questions

!  Fixed alternative !  Yes/No

!  Reliable !  Not powerful

!  Likert

!  Open-ended !  May not be properly answered !  May be difficult to score

Choosing the Format of Your Interview

!  Unstructured !  Interviewer bias is a serious problem !  Data may not be hard to analyze

!  Semi-structured !  Follow-up questions allowed !  Probably best for pilot studies

!  Structured !  Standardized, reducing interviewer bias

Editing Questions: Nine Mistakes to Avoid

1. Avoid leading questions

2. Avoid questions that invite the social desirability bias

3. Avoid double-barreled questions

4. Avoid long questions

5. Avoid negations

6. Avoid irrelevant questions

7. Avoid poorly worded response options

8. Avoid big words

9. Avoid ambiguous words & phrases

Sequencing Questions

!  To boost response rate, put innocuous (harmless) questions first, personal questions last

!  To avoid wasting time, qualify early

!  To increase accuracy, keep similar questions together

!  To boost response rate, put demographic questions last

Putting the Final Touches on Your Survey Instrument !  Professional appearance

!  Proof reading

!  Pilot testing

!  Practice coding responses--may lead to refining questionnaire so that it is easier to code responses

Choosing a Sampling Strategy

!  Random sampling !  Proportionate stratified random sampling

!  Convenience sampling !  Quota sampling

!  Conclusions !  Only random sampling will be representative !  Nonresponse bias may ruin your sample

Questionnaire,Design,,,,,,,,,,,,,,,,,,,,,,,,,Specify,the,Information,Needed,

Design,the,Question,to,Overcome,the,Respondent’s,Inability,and,Unwillingness,to,Answer,

,,,,,,,,,,,,,,,,Determine,the,Content,of,Individual,Questions,

,,,,,,,,,,,,,,,,,,,,,,Decide,the,Question,Structure,

, ,,,Determine,the,Question,Wording,,

,,,,,,,,,,,,Arrange,the,Questions,in,Proper,Order,

,,,,,,,,,,,,,,,,,,,,,,Reproduce,the,Questionnaire,

,,,,,, ,,,,,,,,,,Specify,the,Type,of,Interviewing,Method,

Identify,the,Form,and,Layout,

,,,,,,,,,,,,,,,,,,,,,Eliminate,Bugs,by,PreJtesting,

Questionnaire,Design,

!  Specify.Information.needed.!  It.is.the.first.step.for.questionnaire.design.as.it.helps.in.

clearly.understanding.needs..

!  Type.of.Interviewing.method.!  Mail.Questionnaire.!  Telephone.Questionnaire.!  Personal.Questionnaire.!  Electronic.Questionnaire.

(very.similar.to.mail.questionnaire).

Questionnaire,Design,(Cont’d),

!  Individual.Question.Content.!  Is.the.question.necessary?.!  Are.several.questions.needed.instead.of.one?.

!  E.g.:F.

“Do.you.think.CocaFCola.is.a.tasty.and.refreshing.soft.drink??”..

“Do.you.think.CocaFCola.is.a.tasty.soft.drink?”.

“Do.you.think.CocaFCola.is.a.refreshing.soft.drink?”..

Questionnaire,Design,(Cont’d),

!  Overcoming.inability.to.answer.!  Is.the.Respondent.informed?.e.g.:F.A.husband.may.not.be.aware.of.monthly.expenses.for.groceries.and.department.store.if.wife.make.purchase.or.vice.versa.

!  Can.the.Respondent.Remember?.!  Can.the.Respondent.articulate?.e.g.:F.If.asked.to.describe.the.atmosphere.of.the.department.store..

Questionnaire,Design,(Cont’d),

!  Overcome.Unwillingness.to.Answer.!  Effort.required.of.the.Respondents.!  Context.!  Legitimate.Purpose.!  Sensitive.information.!  Increase.the.willingness.of.Respondents.

!  Choosing.Question.Structure.!  Unstructured.Question.!  Structured.Question.

Choosing,Question,Wording,

!  define.the.issue.

!  use.ordinary.words.

!  use.unambiguous.words.

!  avoid.leading.questions.

!  avoid.implicit.alternatives.

!  avoid.implicit.assumptions.

!  avoid.generalizations.and.estimates.

!  use.positive.and.negative.statements.

Define,the,issue ,,The W's

Defining the Question

Who

The Respondent It is not clear whether this question relates to the individual respondent or the respondent's total household.

What

The Brand of Shampoo It is unclear how the respondent is to answer this question if more than one brand is used.

When

Unclear The time frame is not specified in this question. The respondent could interpret it as meaning the shampoo used this morning, this week, or over the past year.

Where

At home, at the gym, on the road?

Define,the,issue ,,

....A.question.should.clearly.define.the.issue.being.addressed.

!  Which.brand.of.shampoo.do.you.use??.

!  Which.brand.or.brands.of.shampoo.have.you.personally.used.at.home.during.the.last.month?.In.case.of.more.than.one.brand,.please.list.all.the.brands.that.apply?.

Use,ordinary,Words,

ordinary.words.should.be.used.in.questionnaire.vocabulary.level.should.match.the.respondents.vocabulary..

!  Do.you.think.the.distribution.of.soft.drink.is.adequate??.

!  Do.you.think.soft.drinks.are.readily.available.when.you.want.them.to.buy?.

Use,Unambiguous,Words,Words.in.questionnaire.should.have.a.single.meaning.

known.to.respondents.

!  In.a.typical.month.how.often.do.you.shop.in.department.store??.

Never….. .Occassional.….. .Sometimes…...

Often….. .Regularly…...

!  In.a.typical.month.how.often.do.you.shop.in.department.store?.

Less.than.once….. .1.or.2.times…. .3.or.4.times…..

More.than.4.times…...

Avoid,leading,questions,

A.leading.question.clues,.the.respondent.to.answer.in.a.certain.way.

!  Do.you.think.patriotic.Americans.should.buy.imported.automobiles.when.that.would.put.American.labor.out.of.work??.

!  Do.you.think.Americans.should.buy.imported.automobile?.

Avoid,implicit,alternatives,

Alternative.not.explicitly.expressed.is.an.implicit.alternative.

!  Do.you.like.to.fly.when.travelling.short.distances??.

!  Do.you.like.to.fly.when.travelling.short.distances.or.would.you.rather.drive?.

Avoid,implicit,assumptions,

Questions.should.not.be.worded.so.that.the.answer.is.dependent.upon.implicit.assumption.

!  Are.you.in.favor.of.a.balanced.budget??.

!  Are.you.in.favor.of.a.balanced.budget.if.it.would.result.in.an.increase.in.the.personal.income.tax?.

.

.

Avoid,Generalizations,and,Estimates,

!  Questions.should.be.specific.not.general.

!  Respondent.does.not.make.generalization.or.compute.science.

!  What.is.the.annual.per.capita.expenditure.on.groceries.in.your.household??.

!  What.is.the.monthly.expenditure.on.groceries.in.your.household.and.How.many.members.are.their.in.your.household?.

Determining,the,order,of,questions,

Opening.question.

!  Can.be.crucial.in.gaining.the.confidence,.should.be.interesting,.simple,.and.non.threatening.

Type%of%Information%

!  As.a.general.guideline,.basic.information.should.be.obtained.first,.followed.by.classification,.and,.finally,.identification.information.

Difficult.Questions.

!  Difficult.questions.which.are.sensitive,.embarrassing,.complex,.or.dull,.should.be.placed.late.in.the.sequence.

Funnel,Approach,

!  Questions.asked.in.sequence.can.influence.the.response.to.subsequent.question.

!  So.questions.asked.from.general.to.specific.

!  What.considerations.are.important.to.you.in.selecting.a.department.store??.

!  In.selecting.a.department.store.how.important.is.convenience.or.location?.

Form,and,Layout,

!  format,.spacing.and.positioning.questions.have.significant.effect.on.results.

!  should.be.numbered.serially..

!  2000.population.of.census.reveals.questions.at.the.top.received.more.importance.than.the.bottom.

!  Instructions.in.red.made.little.difference.that.they.made.the.questionnaire.appear.more.complicated.

Reproduction,of,Questionnaire,

!  Paper.quality.can.reflect.on.project.

!  Vertical.response.columns.

!  Tendency.to.crowd.should.be.avoided.

!  Instructions.to.be.placed.close.to.the.question.

!  Motto.easy.to.read.and.answer.

!  Should.be.large.and.clear..

Pilot the research instrument to:

•  identify any problems such as the wording of

questions, the length of the interview, and whether the research instrument has been compiled in a logical fashion

•  test the interviewers' / focus group moderators’ •  ability to administer the research instrument

and indicate whether further training is required.

Example,of,closedJended,questions,

1. What is your gender 1. Male 2. Female

2. What is your ethnic identification?

1. African American 2. American Indian 3. Asian/Pacific Islander 4. European American 5. Mexican American/Latino 6. Multicultural or multi-racial 7. Other ____________________

More,examples,3. What is your age?

1. 18 or under 2. 19-29 3. 30-49 4. 50 – 64 5. Over 65

4. How satisfied are you with Social Work 292?

1. Very satisfied 2. Satisfied 3. Neutral 4. Unsatisfied 5. Very Unsatisfied

INTERVIEW,

INTERVIEW It is in a sense of an oral questionnaire. Instead of writing the response, the interviewee gives the needed information orally and face-to-face. With a skillful interviewer, the interview is often superior to other data-gathering device.

The purposes of interview are : $ to verify information gathered from written sources $ to clarify points of information $ to update information and $ to collect data

Avoid to do: #  Too structured - does not allow interviewees to put

forward their own point of view and experiences #  Too unstructured - may result in too many data that are

irrelevant to the study # Complex and technical language # Multiple questions o  Ex:'What do you feel about the risks of sharing syringes

now as opposed to five years ago?‘ # Leading questions o  Ex:'Why is there so much plagiarism nowadays?' o  Ex:Should be: 'Is there any plagiarism issue in this

faculty?' or 'Tell me about plagiarism around here.'

Case,Study,Many well-known case study researchers such as Robert E. Stake (1995), Helen Simons (1980), and Robert K. Yin (1984) have written about case study research and suggested techniques for organizing and conducting the research successfully.

Steps that should be used in case study: 1)  Determine and define the research questions 2)  Select the cases and determine data gathering and analysis

techniques 3)  Prepare to collect the data 4)  Collect data in the field 5)  Evaluate and analyze the data 6)  Prepare the report

.

Step 1. Determine and Define the Research Questions

!  The.researcher.establishes.the.focus.of.the.study.by.forming.questions.about.the.situation.or.problem.to.be.studied.and.determining.a.purpose.for.the.study..

!  The.research.object.in.a.case.study.is.often.a.program,.an,entity,.a,person,.or.a,group,of,people...

!  .The.researcher.investigates.the.object.of.the.case.study.in,depth,using.a.variety.of.data.gathering.methods.to.produce.evidence.that.leads.to.understanding.of.the.case.and.answers.the.research.questions...

!  Case.study.research.generally.answers.one.or.more.questions.which.begin.with."how".or."why."..

!  To.assist.in.targeting.and.formulating.the.questions,.researchers.conduct,a,literature,review...

!  This.literature.review.establishes.what.research.has.been.previously.conducted.and.leads.to.refined,.insightful.questions.about.the.problem...

.

Step 2. Select the Cases and Determine Data

Gathering and Analysis Techniques

!  Determines.what.approaches.to.use.in.selecting.single,or,multiple,realJlife,cases.

to.examine.in,depth,and.which.instruments.and.data.gathering.approaches.to.use...

!  When.using,multiple,cases,.each.case.is.treated.as,a,single,case...!  carefully.select.cases.and.carefully.examine.the.choices.available.from.among.

many.research,tools,available,in,order,to,increase,the,validity,of.the.study...

!  Careful.discrimination.(distinction).at.the.point.of.selection.its.helps.establish,boundaries,around.the.case...

!  A.key,strength,of.the.case.study.method.involves.using,multiple,sources,and,techniques,in.the.data.gathering.process...

!  Determines.in.advance.what.evidence.to.gather.and.what.analysis.techniques.to.use.with.the.data.to.answer.the.research.questions...

!  Data.gathered.is.normally,largely,qualitative,.but.it.may.also.be.quantitative..!  .Tools.to.collect.data.can.include.surveys,,interviews,,documentation,review,,

observation,.and.even.the.collection.of.physical.artifacts...!  Entire.the.design.phase,.researchers.must.ensure.that.the.study.is.well.

constructed.to.ensure.construct.validity,.internal.validity,.external.validity,.and.reliability...

!  The.researcher.attempts.to.establish.a.chain,of,evidence,forward,and,backward...

Step 3. Prepare to Collect the Data !  Because.case.study.research.generates.a.large,amount,of,data.from.multiple.

sources,.systematic,organization,of,the,data,is.important.to.prevent.the.researcher.from.becoming.overwhelmed.by.the.amount.of.data.and.to.prevent.the.researcher.from.losing.sight.of.the.original.research.purpose.and.questions..

!  Researchers.prepare,databases,to.assist.with.categorizing,.sorting,.storing,.and.retrieving.data.for.analysis...

!  Investigators.need.to.be.good,listeners,who.can.hear.exactly.the.words.being.used.by.those.interviewed...

!  Qualifications.for.investigators.also.include.being.able.to.ask.good.questions.and.interpret.answers...

!  Good.investigators.review,documents,looking,for,facts...

!  Investigators.need.to.be,flexible,in.realF.life.situations.and.not.feel.threatened.by.unexpected.change,.missed.appointments,.or.lack.of.office.space...

!  Investigators.need.to.understand,the,purpose,of,the,study,and.grasp,the,issues...

!  Investigators.must.also.be.aware.that.they.are.going.into.the.world,of,real,human,beings,who.may.be.threatened.or.unsure.of.what.the.case.study.will.bring...

4. Collect Data in the Field !  The.researcher.must.collect,and,store,multiple,sources,of,evidence,

comprehensively,and,systematically..

!  Researchers.carefully.observe.the.object.of.the.case.study.and.identify,causal,factors,associated.with.the.observed.phenomenon...

!  Renegotiation.of.arrangements.with.the.objects.of.the.study.or.addition.of.questions.to.interviews.may.be.necessary.as.the.study.progresses...

!  Case.study.research.is.flexible,.but.when.changes.are.made,.they.are.documented,systematically...

!  Maintaining.the.relationship.between.the.issue.and.the.evidence.is.mandatory...

!  The.researcher.may.enter.some.data.into.a.database.and.physically.store.other.data,.but.the.researcher.documents,.classifies,.and.crossFreferences.all.evidence.so.that.it.can.be.efficiently.recalled.for.sorting.and.examination.over.the.course.of.the.study...

Step 5. Evaluate and Analyze the Data !  The.researcher.examines,raw,data,using,many,interpretations,in,

order,to,find,linkages,between,the,research,object,and,the,outcomes,with.reference.to.the.original.research.questions...

!  Throughout,the,evaluation,and.analysis.process,.the.researcher.remains,open,to,new,opportunities,and.insights...

!  Its.use.of.multiple,data,collection,methods,and,analysis,techniques,.provides.researchers.with.opportunities.to.triangulate.data.in.order.to.strengthen.the.research.findings.and.conclusions...

!  Researchers.categorize,.tabulate,.and.recombine.data.to.address.purpose.of.the.study,.and.conduct.crossFchecks.of.facts.and.differences.in.accounts..Focused,.short,.repeat.interviews.may.be.necessary.to.gather.additional.data.to.verify.key.observations.or.check.a.fact...

!  Researchers.use.the.quantitative.data.that.has.been.collected.to.confirm.and.support.the.qualitative.data.which.is.most.useful.for.understanding.the.rationale.or.theory.underlying.relationships...

Step 6. Prepare the report !  case.studies.report,the,data,in,a,way,that,transforms,a,

complex,issue,into,one,that,can,be,understood,.allowing.the.reader.to.question.and.examine.the.study.and.reach.an.understanding.independent.of.the.researcher...

!  Case.studies.present.data.in.very.publicly.accessible.ways.and.may.lead.the.reader.to.apply.the.experience.in.his.or.her.own.realFlife.situation...

!  Some.researchers.report.the.case.study.as.a.story...

!  The.researcher.uses.representative.audience.groups.to.review.and.comment.on.the.draft.document..Based.on.the.comments,.the.researcher.rewrites.and.makes.revisions...

!  Analyze.the.data.using:..1)  SWOT.analysis.(Strengths,.Weaknesses,.Opportunities.and.

Threats)..2)  Environmental.scan..3)  Critical.success.factors;...