Chapter 24Chapter 24

Final preparations for evaluationFinal preparations for evaluation

Completing preparationsCompleting preparations

• Geez, I thought we were all ready…

• This chapter dots the i’s and crosses the t’s I guess…

• Geez, I thought we were all ready…

• This chapter dots the i’s and crosses the t’s I guess…

Roles for evaluatorsRoles for evaluators

• If you’re the only person, you have to do everything…or, assign, I mean, delegate responsibilities

• Facilitator: “flight attendant”, looks after well being of participant “sportscaster”: data collector, keeps the participant

talking (if talk-aloud, presumably) “scientist”: ensuers data collected in scientific manner

(objective, unbiased, etc.)

• If you’re the only person, you have to do everything…or, assign, I mean, delegate responsibilities

• Facilitator: “flight attendant”, looks after well being of participant “sportscaster”: data collector, keeps the participant

talking (if talk-aloud, presumably) “scientist”: ensuers data collected in scientific manner

(objective, unbiased, etc.)

Other rolesOther roles

• How many roles are there? Lots… Note-taker: self-explanatory Equipment operator: self-explanatory Observer: good for “outsiders” (e.g.,

stakeholders) Meeter and greeter Recruiter

• Lone Evaluator: does all of the above

• How many roles are there? Lots… Note-taker: self-explanatory Equipment operator: self-explanatory Observer: good for “outsiders” (e.g.,

stakeholders) Meeter and greeter Recruiter

• Lone Evaluator: does all of the above

Use a scriptUse a script

• (Seems like I said to use a script several times already…)

• Scripts are important for several reasons: so you don’t forget to say or do something

(e.g., get consent form signed) so you don’t influence the participant, or if you

do, at least the influence is consistent (e.g., between groups)

• (Seems like I said to use a script several times already…)

• Scripts are important for several reasons: so you don’t forget to say or do something

(e.g., get consent form signed) so you don’t influence the participant, or if you

do, at least the influence is consistent (e.g., between groups)

Script exampleScript example

Welcome the participant

“Thanks for coming, please have a seat here.”

Brief the participant

“You have been invited to participate in an eye tracking study. We want to find out what you look at when navigating a web page. You don’t have to think about what you’re looking at, just act naturally.

This is an eye tracker. No, it doesn’t have any lasers.”

Obtain consent Please read and sign both copies of the consent form -- the top copy is yours to keep for your record.

Informed ConsentInformed Consent

• What’s on the Informed Consent Form? Rights of the participant

• minimal risk (no harm)• procedural info (what is going to happen)• comprehension (does user understand)• voluntariness (no coercion or pressure)• human rights (participants must be informed)

Rights of the company/school• nondisclosure (is an NDA needed?)• confidentiality (thought this was subject’s identity kept secret)• waivers (for pictures, video recordings, etc.)• legalese (use plain English)• expectations (not clear on this…)

• What’s on the Informed Consent Form? Rights of the participant

• minimal risk (no harm)• procedural info (what is going to happen)• comprehension (does user understand)• voluntariness (no coercion or pressure)• human rights (participants must be informed)

Rights of the company/school• nondisclosure (is an NDA needed?)• confidentiality (thought this was subject’s identity kept secret)• waivers (for pictures, video recordings, etc.)• legalese (use plain English)• expectations (not clear on this…)

Example of IC formExample of IC form

• See Clemson’s IRB: http://www.clemson.

edu/research/orcSite/orcIRB.htm

• See Clemson’s IRB: http://www.clemson.

edu/research/orcSite/orcIRB.htm

Pilot studyPilot study

• Before running the full experiment, run a pilot study: this will expose “minor annoyances” that you

may have missed, that are easy/quick to fix, but that may hinder collection of the “real data”

you can generally expect to throw away all data collected in the pilot

• Before running the full experiment, run a pilot study: this will expose “minor annoyances” that you

may have missed, that are easy/quick to fix, but that may hinder collection of the “real data”

you can generally expect to throw away all data collected in the pilot