Formative Research on the

Heuristic Task Analysis Process

Charles M. ReigeluthJi-Yeon Lee

Bruce PetersonMike Chavez

Indiana University

2

Domain vs. Task

Economics

Psychology

Physics

Chemistry

Domain Task

Management

Counseling

Engineering

Medicine

• Domain vs. task expertise

• Procedural vs. heuristic tasks

• Heuristic tasks: using principles and guidelines to decide what to do when

• Heuristics are often tacit knowledge

• Heuristic Task Analysis (HTA): elicits and formalizes an expert’s heuristic knowledge

What is Heuristic Task Analysis?

4

• Increasingly complex tasks in society

• Many tasks are heuristic (continuum)

• Heuristic task knowledge is often the most valuable and hardest to discover

• Heuristic task analysis methodologies are underdeveloped

Why is HTA important?

5

• Knowledge elicitation: Extraction of domain-relevant knowledge from

human experts using various techniques

• Knowledge analysis: Interpretation of the data extracted from experts

• Knowledge representation:Determined by the purpose of application (e.g. expert system or Artificial Intelligence system)

Literature on HTA

6

Literature on HTA

• Most studies deal with procedural rather than heuristic tasks

• Lack of research that provides guidelines on how to analyze and represent heuristic knowledge

• Lack of research to inform instructional design

7

The handout shows the version of the Heuristic Task Analysis Process investigated in this research.

The Initial HTA Process

8

“ What improvements can be made

to the HTA process?”

Research Question

9

• Developmental research: improves a theory and its real-world application

• Process:

– Apply a theory to an instance– Collect data to improve the application– Suggest improvements to the theory

• Iterations• Three cases

Formative Research

10

• Task: Group Counseling– Focused on personal growth group – Combination task with both procedural and

heuristic elements

• Constraints:– Due to the confidential nature of counseling,

direct observation was impossible– Dealing with interpersonal dynamics which is

hard to verbalize– Analyst unfamiliar with group counseling

Study I: Context

11

• Formative research: designed case• Data collection:

– Interview: unstructured in-person interviews– Videotapes

• Participant profile:– 3 experts experienced with personal growth group

counseling– Expertise ranged from 3 to more than 20 yrs

• Data analysis & interpretation:– Member checks with participants– Consultation with three other researchers

Study I: Methods

12

• To best elicit task experts’ heuristic process, we need to incorporate various interview and observation techniques.

• The analyst’s task expertise needs to be considerable.

• Working with multiple experts requires certain strategies.

• Providing reference material during the interview was helpful.

Study I: Results

13

Study II: Context

• Task: Tutoring on Writing– Class of problems focused on higher-

level writing concerns such as thesis, unity, and coherence

• Constraints:– Experts had little time to devote to this

project

14

• Top-down approach to identify and analyze heuristic knowledge

• Participant profile:– 2 experts experienced in tutoring college

students – had passed an evaluation of their tutoring skills

each term they had been tutoring• Data collection & analysis

– Interview: In-person interviews (60 min. each)– Iterative interview & analysis process– Index cards used to record tutor’s responses

Study II: Methods

15

Study II: Results

• Using the top-down process was effective

• Using index cards was helpful• Helping the tutor identify guidelines was

problematic• Helping the tutor access the context of the

tutoring situation was helpful• Having the tutor list, prioritize, and choose

the decisions made during the tutoring session was effective

16

• Task: – Selecting Artwork for Product Line

• Constraints:– Analyst unfamiliar with business content– Experts had little time to devote to this project

• Research focus: – Speed and effectiveness of HTA process in

business setting

Study III: Context

17

• Formative research: designed case

• HTA guidelines converted into lay questions

• 1 expert, 2 interviews, 1 1/2 hours each

• Content, process analysis after interviews

• Member checks w/ 1 content and 1 research expert

Study III: Method

18

• Speed of HTA process:– Expert was satisfied with time frame– 50% of interview was spent eliciting heuristics– 1 hour was spent on closely associated tasks– 3x5 cards helped analyst get back on track

• Effectiveness of HTA process:– Expert satisfied when heuristics were elicited– Words like feel, know, like were clues to tacit

knowledge and mental models

Study III: Results

19

• Success with the top-down approach (knowledge categories) varied in the studies

• More guidance was needed for eliciting the heuristics

• Using multiple experts presented challenges for the analyst

• Index cards were helpful for all 3 analysts• The HTA process seemed to function well in

all three fields.

Discussion

20

• Preparatory guidance was added to help focus the entire process

• Guidance was added for multiple experts

• Guidance added on how to elicit heuristics

• Bottom-up approach was added

(Revisions on handout are in italics)

Revisions to the HTA Process

Questions or Comments?

For the paper, e-mail Charles Reigeluth at reigelut@indiana.edu