Beyond Task/Technology Fit: How Information Technology

Affects PerformanceBy Transforming the Task

Dale L. Goodhue

Stefano Grazioli

Barbara D. Klein

Technology Characteristics

Individual Performance

?

This the Question We Are Interested In.

Outline

• A comparison of the previous way of conceptualizing TTF with a new way -- Different technologies present the task doer with different options for task completion processes – some of which are more “attractive” than others.

• An experiment applying these ideas to the task of accessing information from integrated or non-integrated databases

Expanding the Task/Technology Fit Perspective• Original insight from TTF: technology

improves performance when the technology “fits” the task

• Use alone is not enough!• What is “fit”, and how does it improve

performance?• How does a technology improve

performance at a task?

IndividualCharacteristics

TaskCharacteristics

Use

Individual Performance

TechnologyCharacteristics

The Technology-to-Performance Chain (Goodhue and Thompson, 1995)

Task-Technology

Fit

IndividualPerceptions/ Beliefs

A Different Perspective:Two Different Tasks?

• Organizational researchers don’t distinguish between technology and task; see task as presented to the task doer (after the application of technology)

• TTF researchers see task as existing before the application of technology

• There are two tasks! The underlying task and the task as presented to the task doer.

• Technology changes the task as presented to the task doer

Task As Underlying

Problem or Motivation

Technology

Task As Sequence

of Actions Used To Meet the

Task Need

Perrow, Fry and Slocum Actions used to transform inputs into outputs

Wood

Required acts and info cues, etc.

Jarvenpaa (89) Choose a restaurant using different choice rules.

Vessey & Galleta (91) Determine point values vs. relationships

Goodhue (95) Meet different mgmt info requirements

McGrath (1984) Task circumplex

Task as Problem Task as Solution

There are two tasks!

Task As Underlying

Problem or Motivation

Technology

1

Sequence A for Actions

To Meet the Task Need

Sequence B for Actions

To Meet the Task Need

Technology

2

Sequence C for Actions

To Meet the Task Need

Sequence D for Actions

To Meet the Task Need

Sequence E for Actions

To Meet the Task Need

Changing the technology,Changes the strategy options for task completion (the possible action sequences)

Different strategy options have different “attractiveness”

A technology that makes possible an “attractive” strategy option has high TTF

Task as Problem

Task as Solution

A Simple Example: Which Technology Will be Chosen, Which

Gives Better Performance and Why? • Task -- Decide if either of two divisions is making

excessive use of high cost shipping alternatives.

• Three Different Technologies – Paper based systems with all original documents

– Division specific accounting database systems

– Integrated accounting database system

How conceptualize and measure TTF of 3 Systems?

The Old Way: • Decide what the task requirements are. For information

access, they might be:

– right data, – right level of detail, – easy to locate, – understandable meaning, – accessiblity, – reliable systems, – training, – assistance, – accuracy, – currency, – compatibility, – Etc.

TTF the Old Way• Now, rate the three technologies on meeting task needs for these dimensions. The

best technology has highest TTF Div Spec Integr.

Paper DB DB

– right data, high high med– right level of detail, high low low– easy to locate, low high high– understandable meaning, high high high– accessiblity, low high high– reliable systems, high high high– training, high med low– assistance, high med med– accuracy, high high high– currency, high high high– compatibility, low low high– Etc.

• Examine the task process (the actions needed) when using each of the three systems to carry out the task.

• Characterize the “attractiveness” of the three ways of accomplishing the task.

• The “attractiveness” is the TTF of each technology for that task.

How conceptualize and measure TTF of 2 Systems?

A New Way:

Technologies Change the Processing Options

Presented to Task Doer 3 Different Technologies

Situation 1: Paper Documents for Each Transaction

Decide if either of two divisions is making excessive use of high cost shipping alternatives.

(Recover aggregate info for each division from records of shipping transactions)

For both divisions, manually select all shipping transactions, translate to problem categories, and consolidate.

Task Presented to Task Doer(or Technology/Processing Options)

Above, or: For each division separately: translate acctg DB system categories to problem categories, use queries to gather totals for relevant acctg categories, consolidate.

Above, or: For both divisions combined: translate acctg DB system categories to problem categories, use queries to gather totals for relevant categories, consolidate.

Situation 2: Separate Accounting DB Systems for Each Division

Situation 3: Integrated Accounting DB Systems Across Both Divisions

Underlying Task

The Big Problem

• We need to have a way of characterizing the “attractiveness” of these options.

How Characterize the “Attractiveness” of Different Processing Options?

• Narrow the focus to “intellective tasks” (McGrath 1984): solving a problem that has a correct answer (not psycho-motor, creativity, planning, etc.)

• What is it about a processing option that is changed by technology and task, and affects performance?– Task complexity (Wood 1986):

• Component complexity: How many distinct actions, information cues?• Coordinative complexity: How many precedence relationships?• Dynamic complexity: How fast is the underlying reality changing

– Difficulty (Campbell 1988): reliance upon skills, abilities, experience of individual task doer

– Task complexity is independent of the task doer, difficulty is dependent on task doer.

How Characterize the “Attractiveness” of Different Processing Options?

• Question: Can we really capture the essential differences between strategy options using task complexity and difficulty?

How Characterize the “Attractiveness” of Different Processing Options?

• Question: Can we really capture the essential differences between strategy options using task complexity and difficulty?

• Answer: Perhaps, if the strategy options are not too different.

Why Go to All this Trouble?

• Humans choose technologies on the basis of the most attractive task processing option, not the best technology characteristics

• The more they know about how the technology works, the more this is true

• Individual performance as well is a function of the technology/task process option chosen and its attractiveness

IndividualCharacteristics

TaskCharacteristics

Choice of One Processing Option (and the associated Technology)

Individual Performance

TechnologyCharacteristics

The Task Transformation Model

A Set of Processing

Options, Each With It’s

Attractiveness(Task

Complexity, Difficulty?)

IndividualPerceptions/

Beliefs

Task as problem Task as Solution

Part 2Applying These Ideas to

Integrated vs. Non Integrated Databases

Data Integration• Definition: standardization of data definitions and

structures across a collection of data sources

• Assumption: when questions require data from multiple sources, DI should reduce manual and intellectual retrieval effort

• Important part of value of ERP and DW is provision of integrated data

• No scientific assessment of how, or if assumption is true!

Non-Integrated EnvironmentDivision A Division B Comments

1. Codes for PartNumbers:Codes for 3/4"BOLT

115899 337189 Potentiallydifferent codesfor same part

2. Codes forCustomer_ID:Codes for ABC, Inc.

42765 42675,49345,47293

Potentiallydifferentstructure ofcodes

3. Codes andDefinitions forAccounts Showing

Sales and Sales Expenses

301 GROSS SALES (net of returns and allowances)302 SALES DISCOUNTS726 ADVERTISING 727 PROMOS, MAILINGS

301 GROSS SALES (net of sales discounts)401 RETURNS AND ALLOWANCES713 SML ACNT SALES EXPENSES: advert./promo.723 LRG ACNT SALES EXPENSES: advert./promo.

Potentiallydifferentdefinitionschemes

Examples of Non-Integrated Data

How Does Data Integration Change the Task Complexity of the Information Retrieval Task?

• To understand this we need a model of the information retrieval task.

• Then focus on the impact of DI on component complexity and coordinative complexity of different sub-processes in that overall task

Example Task

• Management is concerned about ratio of advertising and promotions expenses to sales revenue in two divisions

• task doer is asked to find (for each division)– year-to-date advertising and promotion

expenses – year-to-date sales (net of discounts, returns and

allowances)

Division A (65 Account Codes) Division B (65 Account Codes)..301 GROSS SALES (net of returns and allowances)302 SALES DISCOUNTS..726 ADVERTISING 727 PROMOS, MAILINGS..

.

.301 GROSS SALES (net of sales discounts)..401 RETURNS AND ALLOWANCES..713 SML ACNT SALES EXPENSES: advert./promo.723 LRG ACNT SALES EXPENSES: advert./promo.

Account Codes in A Non Integrated Data Environment

2. Semantic Specification

3. Syntactic Specification

Query Processor

5. Error Repair

4. Error Detection

Overall Result: -accuracy -time required

Process Model of Information Retrieval

1. Split Off One Subtask

7. ConsolidateSubtask Results

Problem Statement Data Environment

6. Any More Subtasks?

Error DetectedNo ErrorsDetected

No

Yes, Repeat Steps 1-6

Query Displayed Results

Hypotheses For Non Integrated Data• Subprocess 1: greater component complexity (more data items to consider)

will encourage task doers to sub-divide the task into more subtasks.• Subprocess 2: greater component complexity (more data items to consider)

makes it more likely task doer will misclassify at least one data item, in total task.

• Subprocess 3: less component complexity (fewer elements in a less complex query) makes it less likely task doer will make syntax or logic errors in any given query.

• Subprocess 3, more precedence requirements (keeping straight which database) make it more likely task doer will confuse or mis-specify the database.

Impact of Number of Sub-tasks and Error Profiles on Performance

• Time to complete will increase with the number of sub-tasks

• Time to complete will increase with the number of high feedback errors (syntax and logic) in the total set of queries used

• Likelihood of totally correct answers will decrease with the existence of one or more low feedback errors (neglecting a needed account code or included a non-needed account code)

Impact of Data Integration on Performance

Different Mix of Processing Options

Performance - accuracy - time

Choice of OptionWith Given Task Complexity

UnderlyingTask

Integrated vs. non Integrated DB

Number and Type of Errors

Size of Sub-Queries

Method• 107 student pairs

• Given: managerial questions, SQL query processor, and either integrated or non integrated database

• 4 sessions: 2 training, 2 with treatments

• Captured time to complete, accuracy, and every query submitted (1164 queries)

• LISP program determined subtasks used and error profiles of each query

• (Kappa coefficient of agreement between LISP and human coders: .93 or excellent)

Type ofError

Description Examples Frequency>= 1 error in214 Sessions

High FeedbackErrorsSyntax Violation of syntax

rules, misspellings,etc.

WEHRE acctcode = ‘301’ oracctcode = ‘302’) (Should beWHERE)

63%

Logic Misusing the logicof the WHEREclause

Where acctcode < ‘300’ and acctcode> ‘303’ (No account codes in range)

34%

Database Looking for Div Aacct code in Div Bdatabase

Where acctcode = ‘401’ (whenseeking Div A info – acctcode 401exists only on the Div B database)

9%

Low FeedbackErrorsSelectLowFeedback

Specifyingplausible butwrong field

Select acctcode, Month_to_Date(instead of “Select Year_to_Date”)

43%

UnderSpec.

Leaving out aneeded category

Where acctcode = ‘301’(when 302 or 401 is also needed)

42%

OverSpec.

Adding anincorrect category

Where acctcode = ‘301’ or acctcode =‘302’ or acctcode = ‘920’(when 920 should not be included)

38%

Interesting Aspects of Analysis• Analyzed only the first query attempt at any

subtask. (Remainder are error correction queries and much harder to predict.)

• Distribution of errors was highly skewed. Many made no errors. Inappropriate as dependent variable in regression. Used Logistic Regression for those analyses

• Used pair characteristics as additional explanatory variables: tendency to speed, tendency to accuracy

Impact of Integrated Data

Fewer Subtasks

At the Individual Query Level

At the Total Task Level

Mixed Impact on Erroneously Includingor Excluding Account Codes (Low Feedback Errors)

On UltimatePerformance

Fewer Subtasks Leads to Shorter Time to Complete

Fewer Erroneously Included or ExcludedAccount Codes (Low Feedback Errors)

Many More Logic Errors But Only a Hint of More Syntax Errors (High Feedback Errors)

More Logic Errors But No More Syntax Errors (High Feedback Errors)

Fewer Low FeedbackErrors Leads to Greater Accuracy

More Logic Errors Has No Impact On Time To Complete(More Syntax Errors Does Impact Time)

Research Implications• We can understand task doers’ choice of technologies and the impact

on individual performance by considering the “attractiveness” of the processing options provided by different technologies

• We should focus on the process of carrying out the task, not on the characteristics of the technology

• We can use “task complexity” to understand the better TTF of DI for multi-division tasks

• When we do, we see that DI does not improve performance at the query level, but allows task doers to “take larger bites” with each query, and use fewer queries, hence making fewer hard to catch errors and taking less time overall.

• In this way DI reduces time and increases accuracy for data retrieval