Diagnostic inferencein performance evaluation:

effects of cause andevent covariation and similarity*

CLIFTON BROWN University of Illinois

Abstract. This study experimentally tests some implications of psychological diagnosticinference theories and begins to extend these theories into a performance evaluationcontext. Subjects, who were required to assume the role of a manufacturing divisionmanager, were asked to estimate the likelihoods of four potential causes of an assemblydepartment's labor efficiency variance. The subjects were asked to reevaluate their causallikelihoods following: 1) evidence concerning the magnitude of the labor variance and thedeviations of the four potential causes from their normal levels (similarity evidence) and 2)evidence concerning the covariation of a potential cause, and labor efficiency variancesover the past five years (covariation evidence). The results generally confirmed a set ofhypotheses predicting the effects of cause/event similarity and covariation upon individ-uals' causal inferences. Implications of these results for accounting information systemsdesign are discussed in a concluding section.

Risumi. Cette 6tude teste, au moyen d'une experience, quelques-unes des conclusions desthdories d'inference diagnostique psychologique et amorce l'application de ces theoriesdans un contexte d'6valuation de la performance. Les sujets, qui devaient assumer le role dedirecteur d'une division de fabrication, ont 6te invitds a estimer les probabilit^s de quatrecauses possibles de l'&art de rendement sur main-d'oeuvre d'un atelier d'assemblage. Lessujets ftirent invites i rfi^valuer leurs probabilites causales selon: 1) des informationsrelatives k l'ampleur de l'^cart sur main-d'oeuvre et h. la dispersion des quatre causespotentielles par rapport ^ leur niveau normal (preuve de similitude) et 2) des informationsrelatives i la co-variation d'une cause potentielle, et aux hearts de rendement sur main-d'oeuvre des cinq demi^res anndes (preuve de co-variation). De fagon g^n6rale, les resultatsconfirment l'ensemble des hypotheses prdvoyant les effets de similitude et de co-variationcause/6v6nement sur les inferences causales des individus. Les consequences de cesr^sultats sur la conception des systemes d'information comptable sont examin6es dans laconclusion.

IntroductionMost managers' judgments involve predicting future events and determining the

* This paper was funded, in part, by a Coopers & Lybrand research grant administered by theDepartment of Accountancy at the University of Illinois at Urbana-Champaign and by the Depart-ment of Accounting at the University of Arizona.

I would like to acknowledge comments by the participants in a research forum at the Universityof Arizona and by an anonymous reviewer.

Contemporary Accounting Research Vol. 4 No. 1 pp 111—126

112 C. Brown

causes of past events. Judgments concerned with the performance of subsystemsunder a manager's responsibility also involve these cognitive processes of predic-tion and diagnosis. The planning and structuring of subsystem performance forfuture periods involve forecasting a variety of events and circumstances (e.g.,sales volume given specific economic circumstances). The operation and controlof these subsystems overtime involves understanding the causes of any significantdifferences between planned performance and actual performance (e.g., thecauses of a particular variance from a sales budget).

Diagnostic inference is a central component for understanding one's experiencewith the world. Individuals identify relationships within experienced events andobjects as a result of analyzing specific instances of those events and objects (cf.,Kuhn's (1977 and 1970, pp. 174-210) discussion of the role of exemplars andparadigms in the learning of scientific knowledge). One's theory of the world isinferred through the repeated diagnosis of experience. The importance of diagnos-tic inference can be seen in terms of its effect on prediction of events and on choiceof action. Prediction depends upon the individual's understanding (inferredtheory) ofthe underlying process that generates outcomes (Einhom and Hogarth,1982). The actions taken will depend, at least in part, on beliefs concerning thecircumstances that caused (or will cause) the event or situation (Hogarth, 1981).For example, a manager's understanding of a sales budget variance will dependupon his inferred theory of the processes that generate sales budget variances.Different performance evaluations and control actions will result if a sales budgetvariance is believed to have been caused by a given circumstance (e .g., a decline inproduct demand), than if the cause is believed to be a different circumstance (e.g.,inadequate production).

The objectives of this paper are to experimentally test some theoretical implica-tions concerned with diagnostic inference and to begin the extension of such theoryinto the context of performance evaluation. This study, building upon andextending that of Brown (1985), employed standard labor efficiency variances asaccounting reports of operating performances requiring diagnosis prior to formu-lation of performance evaluations and choice of actions.' Subjects were asked toassume the role of a manufacturing division manager. They were given anexhaustive set of potential causes and were asked to estimate the likelihoods thatthe potential causes were actual causes of labor efficiency variances for assemblydepartments within their division. The subjects were asked to reestimate theircausal likelihoods after they were given evidence that related to the occurrence ofthe potential causes during the period of the variance, and again after they were

1 A standard labor efficiency variance is defined to be the difference between the labor hours in-curred for a particular level of production (actual hours) and the labor hours that should have beenincurred for that level of production (standard hours allowed). Although this study employs aproduction definition of an operation, this is done for purposes of maximizing the structure ofaccounting information available within the experimental context. The concepts used in this studyapply equally when an operation is defined to be the accomplishment of any business task.

Inferences in Performance Evaluation 113

given the frequencies with which the potential cause (chosen by the subject) andlabor efficiency variances had jointly occurred in the past.

The following section of this paper develops a conceptual framework for therole of causal judgments in diagnostic inference and formulates hypotheses basedupon the conceptual framework. A description of the research design, theexperiment, results and discussion, and conclusions are contained in the remainingsections.

Conceptual frameworkEinhom and Hogarth (1986) have developed a theory of diagnostic inference inwhich causal perceptions are affected by three types of information: (1) theassumed causal background (e.g., the technology employed in the productiveprocess); (2) the number and strength of specific altemative causes (e.g., inade-quate production management, poor raw material quality, out-of-date laborefficiency standards); and (3) the perceived strength of potential cues-to-causalitywithin the circumstances being evaluated (e.g., the covariation, similarity,temporal order, and contiguity between a potential cause and the reportedvariance). Within this study the first two types of information will be held constantand the third type of information, the cues-to-causality, will be explored.

A number of researchers have proposed that individuals use certain cues-to-causality in judging the causal strength of a potential explanation (Einhom andHogarth (1986); Kelley and Michela (1980); Mackie (1980, pp. 29-36)). Thecues-to-causality include such factors as temporal order, contiguity, covariation,and similarity of cause and effect. Brown (1985) examined individuals' use of twoof these cues-to-causality: temporal order and covariation. This study builds onand expands that of Brown (1985) by exploring individuals' use of cause and effectsimilarity and covariation cues-to-causality.

Similarity of cause and effectAttribution theory employs the rule of similarity by which "properties of the causeare assumed to be similar to properties of the observed effect..., so that the lattercan be used to infer the former" (Kelley and Michela (1980, p. 466)).^ Tversky(1977) proposed a model of perceived similarity in which objects are representedas collections of features, and similarity judgments result from a feature matchingprocess in which common and distinctive features are combined linearly. Speci-fication of common and distinctive features is required to generalize Tversky'smodel from objects to events. Einhom and Hogarth (1986), citing Nisbett andRoss (1980), discuss several long-standing, popular notions of cause and eventfeatures for similarity, including the notion of congruent lengths and strengths ofcause and effect. For example, given a labor efficiency variance that was 30

2 This definition of similarity is closely related to the concept in normative logic of a priori neces-sity: knowledge of the effect tells us that the event was produced by the cause (Mackie, 1980,pp'. 11-13).

114 C. Brown

percent greater than the standard hours allowed (an undesirable event), which ofthe following potential causes would be perceived as having greater similarity tothe labor variance: Potential Cause A (e.g., raw material quality) that was twopercent below the level of A used to set the labor efficiency standard, or PotentialCause B (e.g., production worker training) that was 28 percent below the level ofB used to set the labor efficiency standard? Although a specific answer to thisquestion would at least partially depend upon the judge's theory ofthe underlyingproduction process, the congruity of cause and effect suggests that Potential CauseB would be perceived as being more similar to the labor variance than wouldPotential Cause A. Einhom and Hogarth's (1986) diagnostic inference modelviews similarity as both a compensatory and a noncompensatory cue-to-causalityused by individuals in assessing causal strength. That is, there is some minimumlevel of perceived similarity required for the potential cause to be given anylikelihood, but above this threshold, low levels of perceived similarity may becompensated for by higher levels of other cues-to-causality.

When individuals receive evidence about the occurrence of a potential causeduring the period of a labor efficiency variance, the evidence will indicate themagnitude ofthe potential cause's occurrence. This study employed the deviationsof the potential causes from their normal levels relative to the magnitude of thelabor efficiency variance itself, as cues to the strength of cause and effect. Basedupon the congruity of cause and effect:

HI Judged causal likelihoods will be monotonically increasing with causes whosedeviations from normal levels are more similar to the magnitude of thereported labor efficiency variance.

CovariationAttribution theory employs the principle of covariation by which the effect isattributed to that factor with which it is perceived to covary (cf., Kelley andMichela (1980)). Einhom and Hogarth (1986) view covariation as a compensatorycue-to-causality that individuals use in assessing causal strength. Previous researchhas found that individuals have difficulties evaluating the extent of covariationpresent in evidence concerning covariation of potential causes and effects (Crocker(1981); Nisbett and Ross (1980, p. 10)). The findings indicate that when theevaluations are data based, individuals tend to underestimate the objective extentof covariation, and when the evaluations are theory based, individuals tend tooverestimate the objective extent of covariation (cf., Jennings, Amabile and Ross(1982)). Modeling covariation in a 2 X 2 contingency table where both potentialcause and effect are considered to be dichotomous variables (either occurring ornot occurring), Einhom and Hogarth (1986) view covariation judgments as linearcombinations of the subjectively weighted contingency table cell frequencies.Einhom (1980) and Einhom and Hogarth (1978) discuss difficulties of leamingcovariation from experience due to the inability to observe all events associated

Inferences in Perfomiance Evaluation 115

with the contingency table cells (in particular, when a variance does not occur,managers would rarely know if a potential cause did or did not occur).

When individuals select a specific potential cause as the most likely cause of anevent, they form an expectation that the covariation between that cause and theevent is greater than or equal to some minimum level. Brown (1985) demonstratedthat if subsequent to an evaluation of causal likelihood the individual receivesevidence indicating the covariation is actually zero (i.e., below an expectedminimum), the individual will reduce the likelihood of that potential cause (as wellas increase the likelihood of an alternative cause). Brown (1985) also demonstratesthat if the subsequent evidence indicates a stronger positive covariation (i .e., at orabove an expected minimum), the individual will maintain or increase thelikelihood of that potential cause (as well as not increase the likelihood of analternative cause).

H2 Individuals will maintain or increase causal likelihoods assigned to potentialcauses that have a stronger positive covariation with the labor efficiencyvariances, but will decrease causal likelihoods assigned to potential causesthat have a weaker covariation.

Given the research findings that individuals have difficulties assessing theobjective extent of covariation, a summary statistic that describes the objectiveextent of covariation (e.g., a correlation coefficient) should aid individuals inmaking covariation evaluations closer to the objective covariation. Assumingsubjects' evaluations are largely theory based (i.e., based on expectations),estimates of covariation would be less overestimated when a summary statistic isprovided, and lower causal likelihoods would be assessed. On the other hand, ifsubjects' evaluations are largely data based (i.e., based upon inspection of con-tingency table infonnation), estimates of covariation would be less underestimatedwhen a summary statistic is provided, and higher causal likelihoods would beassessed.

H3 Individuals will assign different causal likelihoods when the potential causeand effect covariation relationship is described by a summary statistic.

Research design

Experimental environmentThe subjects were asked to assume the role of a department manager in amanufacturing company. One of the manager's responsibilities was to analyzestandard cost variance reports on his department. Information available for thesubjects to meet this responsibility came from three sources. The first source wasinfonnation from past experience and was presented to the subjects in the form of abackground information pamphlet. This pamphlet described the role the subject

116 C. Brown

was being asked to assume, the company and its manufacturing processes, theaccounting control system, and the subject's task and objectives within theexperiment. The intent of this pamphlet was to give the subjects a commonknowledge with respect to the experimental task, partially controlling the subjects'causal backgrounds. Thus, posterior evaluations should not be affected bydivergent prior expectations.

The second source was the variance report. This report consisted of a laborefficiency variance and a list of the four potential causes of such variances. For allsubjects, the labor efficiency variance reported was unacceptable. An unaccept-able labor efficiency variance was one in which the difference between the hoursworked and the hours allowed for achieved output was greater than 15 percent ofthe hours allowed. The subjects were told that out of an exhaustive set of fourpotential causes (workload schedules, raw material quality, worker training, anddepartment manager's efforts), past experience had shown that unacceptable laborefficiency variances were generally produced by two of the potential causesoccurring at the same time. Information available with this report included theprior probability of an unacceptable labor efficiency variance and the priorprobabilities of each of the four potential causes.

The third source of information was subsequent evidence received by thesubjects. The evidence pertained to the four potential causes and was of two types:similarity evidence and covariation evidence. Similarity evidence indicated boththe magnitude of the labor efficiency and the deviations of each of the potentialcauses from normal levels during the previous eight weeks (including the week ofthe variance). Covariation evidence indicated the estimated frequencies of occur-rence over the past five years of a potential cause chosen by the subject and laborefficiency variances.

Experimental designThe overall experimental design, presented in Figure 1, was a 2^ x 4 x 3 repeatedmeasures design with three between-subjects variables, each at two levels, andtwo within-subjects variables. The between-subjects variables were covariationstrength (either weak or medium), the covariation summary statistic (eitherprovided or not provided to the subject), and the similarity of cause and effect(either the deviation of raw material quality or of worker training from normallevels had the greatest similarity to the magnitude of the variance). One within-subjects variable, at four levels, was the potential cause. The other within-subjects variable, at three levels, was the repeated evaluations: first prior toreceiving evidence, second after receiving the similarity evidence only, and thirdafter receiving both the covariation and the similarity evidence.^

3 The presentation of the two types of evidence (similarity and covariation) was not counter-balanced. The reasons for this were that counterbalancing would have doubled the required numberof subjects and Brown (1985) found no evidence of an order effect in the presentation of temporalorder and covariation evidence.

(4-1 ^

• i i

U

cc

u <

u

a

u

«

U <

•!=•§>. •a % 3> % 3 t5 x 2 x 3

•a

1

I

I

118 C. Brown

Operationalization of variablesCausal likelihood was elicited from subjects using ten-point scales where causallikelihood ranged from most unlikely (-10) to equally likely as unlikely (0), andfrom equally likely as unlikely to most likely (10).

The unacceptable labor efficiency variances had prior probabilities equal to 25percent. One potential cause (workload schedule) had a prior probability equal to80 percent. The other three potential causes had prior probabilities equal to 30percent (raw material quality), 25 percent (worker training), and 20 percent(departmental manager efforts). Since prior probability was not an experimentalvariable, but rather controlled to be common for all subjects, the intent was toestablish a set of prior probabilities that appeared valid to the subjects.•*

The covariation evidence was presented in the form of a 2 x 2 contingency tableof the frequencies over the past five years of a chosen potential cause occurring ornot and of labor efficiency variances being unacceptable or acceptable. Thespecific covariation evidence given to subjects depended on the cause they were"investigating," as well as their assigned level of the covariation condition. Foreach relevant potential cause, however, the medium correlation coefficient wasfrom 1.64 to 1.87 times larger than the weak correlation coefficient.^ Subjectsassigned to the condition in which the covariation summary statistic was providedwere given the correlation coefficient with their covariation evidence (togetherwith an explanation that defined the coefficient as a strength of association indexthat always lies between 0 [no association] and 1 [complete association]).

The magnitude of the reported labor efficiency variance was 21 percent of thestandard labor hours allowed for the work achieved. For one level of the cause andeffect similarity variable, worker training was 19 percent below its normal level(and was the cause with the greatest similarity to the magnitude of the laborvariance) and raw material quality was three percent below its normal level. Forthe other level of the cause and effect similarity variable, raw material quality was19 percent below its normal level (and was the cause with the greatest similarity tothe magnitude of the labor variance) and worker training was three percent belowits normal level.*

4 The causal pnors were made approximately equal except for one cause's prior (workloadschedules) which was set substantially higher as a check on subjects' use of pnor probabilities.Results indicated that subjects consistently assessed this cause's likelihood to be the lowest of thefour possible causes, which supports the proposition that subjects utilized the pnor probabilityinformation.

5 A medium covariation level was selected over a strong covariation level (e.g., 0.80 to 0.90) due toconsiderations of extemal validity. Most situations in actual business, reflecting greater complex-ity, would involve medium and low covanations, rather than strong correlations. The correlationcoefficients implied by the covariation evidence for the raw material quality, worker training,and department manager's effort causes, respectively, were as follows: a) medium covariationlevel: 0.53, 0.59, and 0.51 and b) weak covariation level: 0 30, 0.32, 0.31. The coefficients im-plied by the covanation evidence for the workload schedule cause was the same over both co-variation levels: 0.02. The potential causes could not have the same coefficients within eachcovariation level due to the fact that the causes had different marginal frequencies (base rates).

6 For both similarity levels, the workload schedule and the department manager's efforts were tenpercent and 13 percent below their normal levels, respectively.

Inferences in Performance Evaluation 119

The experiment

SubjectsThe subjects were undergraduate students enrolled in junior/senior level mana-gerial (cost) accounting courses in the business school of a large state university. Afixed payment of $5.00 was offered for participating in the experiment, providinga total of 56 volunteer subjects.'

The subjects were randomly assigned to the between-subjects treatment condi-tions with the restriction that the cell sizes remained equal. Upon assignment to atreatment condition, each subject received the background information pamphlet.Limitations that result from the use of students as subjects are discussed in a latersection of this paper.

ProceduresThe experiment was conducted in two phases: an experiment phase immediatelyfollowing a training phase. Both phases were conducted in group sessions rangingfrom three to ten in size.

The training phase. Training within all treatment conditions consisted ofadditional written instructions, a period of time in which subjects could askquestions, and a practice labor efficiency variance case.

The experiment phase. The experimental phase consisted of obtaining thesubjects' responses to a second labor efficiency variance case. Based only upon thebackground information booklet and the labor efficiency variance report, thesubjects were asked to estimate how likely they believed each of the four potentialcauses were to have been one of the actual causes of the department's reportedlabor efficiency variance.

The subjects were given a report of the deviations of the four potential causesfrom their normal levels and the magnitude of the labor efficiency variance duringthe previous eight weeks (including the week of the reported variance), and werethen asked to reestimate their four causal likelihoods. Finally, the subjects weregiven a report of covariation between the potential cause of their choice and laborefficiency variances, and were asked to again reestimate their four causal likelihoods.

Results and discussion

Cause and event similarityThe hypothesis concerning cause and event similarity was tested using a 2 X 2mixed-design ANOVA that examined changes in subjects' causal likelihoodbetween the first two evaluations. The independent variables were the similarity

7 The subjects included 35 males and 22 females, all of whom had taken at least an introductorystatistics course and an introductory managerial accounting course. A total of 65 subjects volun-teered to participate, but only 57 completed the experiment. One subject was randomly droppedfrom the analysis to maintain equal cell sizes in the between-subjects' conditions.

120 C. Brown

variable (between-subjects) and the two potential causes involved in operationaliz-ing the similarity variable (within-subjects).

Prior to receiving similarity evidence, the subjects should evaluate the threemost likely potential causes as equally likely.^ After receiving similarity evidence,the subjects should evaluate as more likely the potential causes with the greatestdeviations from their normal levels (i.e., those most similar to the variance), andshould evaluate as less likely the potential causes with the least deviations fromtheir normal levels (i.e., those least similar to the variance). Since the specificpotential causes which have the greatest and least deviations from their normallevels differ between the levels of the cause and event similarity variable.Hypothesis One predicts a significant interaction between the similarity variableand the potential causes variable. This interaction was significant (the F = 36.957with one and 54 d.f.; p < 0.001; M^ = 0.336 and the model R^ = 0.347), is in theform predicted and is presented in Table 1.

These results support the hypothesis that perceived similarity of cause and event,when defined as relative deviations from normal levels, affects individuals' evalu-ations of causal likelihood. Given a large labor efficiency variance (21 percent ofstandard where 15 percent of standard was the unacceptability threshold), subjectssignificantly increased their estimates of causal likelihood for potential causes thathad large deviations from normal levels (from 2.32 to 6.96 when deviation in rawmaterial quality was most similar, and from 2.25 to 6.54 when deviation in workertraining was most similar to the labor variance). Further, subjects significantlydecreased their causal likelihoods for causes that had small deviations from normallevels (from 3.04 to 0.82 when deviation in raw material quality was least similarto the labor variance, and from 2.68 to 1.07 when deviation in worker training wasleast similar). The magnitudes of the changes in causal likelihood did not differbetween levels of the similarity variable (i.e., changing the nature of the mostsimilar potential cause (worker training or raw material quality) did not modify theeffect of similarity on causal likelihoods).

Although not hypothesized, when deviation in raw material quality was mostsimilar to the labor efficiency variance, the mean likelihood assigned to rawmaterial quality was significantly greater than that assigned to the departmentmanager's efforts (6.96 for raw material quality versus 5.61 for departmentmanager's efforts; t = 4.0319, 54d.f., p < 0.001). On the other hand, whendeviation in worker training was most similar to the labor efficiency variance, themean likelihood assigned to worker training was not significantly greater than thatassigned to the department manager's efforts (6.50 for raw material quality versus6.21 for department manager's efforts; t = 0.7867, 54d.f., p < 0.45). Anunderlying difference in the nature of raw material quality and of worker trainingas potential causes of labor efficiency variances is controllability by the depart-ment manager: worker training generally has greater controllability than does raw

8 These were the potential causes with approximately equal prior probabilities: raw material quality,worker training and department manager's efforts.

2 c

ii

>.g

1•a

•c

I-2•c 2cd

II

II

^S

•c

II

II

S >•

c I

CiCJ

T

00 —.

>O <N

I

^8o «->

;?3

IS

o -^

•S i

It

z

122 C. Brown

material quality. The subjects may have interpreted the situation in which rawmaterial quality had the greatest similarity to the labor variance as being lesscontrollable by the department manager (thus, assigning greater causal likelihoodto raw material quality than to department manager's efforts). They also may haveinterpreted the situation in which worker training had the greatest similarity to thelabor variance as being more controllable by the department manager (thus,assigning approximately equal causal likelihood to worker training and thedepartment manager's efforts). Since these results are post hoc, the inferencesdrawn from them must be qualified as tentative and subject to future empiricaltesting. Altemative hypotheses could account for these results.

Cause and event covariationThe hypothesis concerning cause and event covariation was tested using a 2 x 2 X2 mixed-design ANOVA that examined changes in subjects' causal likelihoodsbetween the second and third evaluations. The independent variables were thecovariation and similarity variables (both between-subjects), and the two potentialcauses for each subject that had the highest assessed likelihoods in the secondevaluation (creating a within-subjects potential cause rank variable).

Prior to receiving covariation evidence (but after receiving the similarityevidence), the subjects' causal likelihoods should be unaffected by the yet to bemanipulated covariation variable. After receiving covariation evidence, subjectswithin the medium covariation treatment should either maintain or increase thelikelihood assigned to the potential cause they had considered to be most likelyprior to the evidence. Conversely, subjects within the weak covariation treatmentshould substantially decrease the likelihood they had assigned to their most likelyprior cause. Since the potential causes are an exhaustive set, subjects within theweak covariation treatment should, at the same time, increase the likelihoodassigned to the potential cause they had considered to be the second most likelyprior to the covariation evidence (see Brown, 1985). The potential cause consid-ered to be the most likely prior to the covariation evidence should differ betweenthe levels of the potential cause rank variable. This reasoning implies thatHypothesis Two would predict a significant interaction involving the covariationand the potential cause rank variables. This interaction was significant (the F —229.1 with 1 and 52 d.f.; p < 0.001; w^ = 0.327 and the model R^ = 0.877), is inthe direction predicted and is presented in Table 2.

These results support the hypothesis that cause and event covariation effects per-ceived causal likelihoods. Prior to receiving covariation evidence, the subjects'causal likelihoods were unaffected by the yet to be manipulated covariation vari-able. Given evidence indicating medium covariation between their most likelypotential cause and past labor efficiency variances, the subjects did not signifi-cantly change the causal likelihoods they had assigned to either their most likely orsecond most likely potential causes (from a mean of 6.64 to 6.71 for their mostlikely cause and from a mean of 5.89 to 6.07 for their second most likely cause).Given evidence indicating weak covariation between their most likely potential

T3

1•o

O

I

u

d —

VIf jC

"oQ

•2o

Q

_ > 1

O

f- > H

"H . 2 -y,

III s

oT3

5 f

i2

ffi

124 C. Brown

cause and past labor efficiency variances, the subjects significantly reduced thelikelihood they had assigned to the most likely potential cause (from a mean of7.07 to 0.36). At the same time, the subjects receiving weak covariation evidencesignificantly increased the likelihood they had assigned to their second most likelypotential cause (from a mean of 5.82 to 6.89).

Summary covariation statisticThe hypothesis conceming the summary covariation statistic was tested using a 2X 2 X 2 mixed design ANOVA that examined subjects' causal likelihoods in thethird evaluation. The independent variables were the covariation and summarystatistic variables (both between-subjects), and the two potential causes for eachsubject that had the highest assigned likelihoods in the second evaluation (againcreating the within-subjects potential cause rank variable).

Depending upon whether their assessment of covariation is theory based or databased, subjects who receive a descriptive summary statistic with their covariationevidence should either perceive a lower or higher covariation, and should assign acorresponding lower or higher causal likelihood, than those subjects who do notreceive a summary statistic. Although the effect ofthe summary statistic variablewas not significant (the F = 2.380 with 1 and 52 d.f.; /? < 0.125), it was involvedin a significant interaction with the covariation variable (the F = 4.231 with 1 and52 d.f.; p < 0.042) that was negligible in size (w^ = 0.004 with the model's R^ =0.877). Subjects who received weak covariation evidence assessed lower causallikelihoods when a summary statistic was also provided (a mean of 3.57 versus amean of 3.68). Subjects who received medium covariation evidence, however,assessed higher causal likelihoods when a summary statistic was also provided (amean of 6.75 versus a mean of 6.00). These results are mixed with respect to thedirections predicted by the theory based/data based concepts, with the weakcovariation condition consistent with theory based evaluations and the mediumcovariation condition consistent with data based evaluations.

LimitationsPotential limitations exist that could affect conclusions drawn from this experi-ment. With respect to the objective of testing diagnostic inference theory, a lack ofsubject understanding for the descriptive covariation summary statistic may haveresulted in its negligible, mixed effect on subjects' causal likelihoods.

Other potential limitations exist regarding the objective of an initial expansionof diagnostic inference theory into the context of performance evaluation. A majorlimitation to the generalizability of results would be the use of student subjectswithin a hypothetical performance evaluation environment. Because these sub-jects lacked prior training and experience with performance evaluation and controlin business contexts, their causal backgrounds are different from those of actualmanagers. Further, the reduction of possible causes of labor efficiency variances tofour in number and the assumption that these possible causes were exhaustive is anadditional limitation on the generalizability of results. Within an actual perfor-

Inferences in Performance Evaluation 125

mance evaluation/control situation the number of possible causes could be largerand would not be explicitly stated.

ConclusionsA major impact of managerial accountants within businesses is on the design andoperation of management information systems and on the training of individuals toutilize these systems. Knowledge of subjective diagnostic processes is essential toaccomplish the objectives of effective system design and adequate individualtraining. Diagnostic inference within the context of labor efficiency variances cantake the following form. "A significant, negative labor efficiency variance hasoccurred in Department A. How likely was inadequate worker training, rather thaninadequate raw material quality, to have been the cause?"

This study, extending Brown (1985), presents some evidence that a manager'sanswer to this question can be affected by the perceived strength of particularcues-to-causality within the circumstances being evaluated. Two cues-to-causalitywere manipulated in this study: similarity and covariation. Supporting the simi-larity hypothesis, the subjects' causal likelihoods were significantly greater forpotential causes that had deviations from their normal levels similar to themagnitude of the reported variance. Supporting the covariation hypothesis, thesubjects decreased the likelihoods they had assigned to causes that proved to haveweak covariation, but did not decrease the likelihoods they had assigned to causesthat proved to have medium covariation.

A potential implication for accounting information systems design is thepossibility of incorporating causal evidence into the systems. Current accountingsystems, such as standard cost systems, primarily signal potential deviations fromnormal, standard, or expected performance. Little information concerning poss-ible causes of such deviations is simultaneously available with the signals them-selves. This information is available on an exception basis (such as varianceinvestigation) or is considered to be part of the "knowledge base" of the managersinvolved (which assumes experienced managers who are efficient and effectivelearners). Incorporation of causal evidence into accounting information systemscould involve tracking the major potential causes of standard cost variances in amanner similar to that used for the standard cost variances themselves, deviationsfrom standard or normal levels (thereby providing similarity, contiguity, and/ortemporal order evidence). Additionally, accounting information systems couldprovide cumulative historical associations, for relatively stable production sys-tems, between major potential causes of standard cost variances and the standardcost variances themselves (thereby providing covariation evidence). A caveat,however, is that costs and benefits associated with incorporating causal evidence,which probably differ across technologies, should be studied prior to generalizingto actual accounting information systems.

In addition to cost/benefit issues, future research should extend this study aswell as address the study's limitations discussed above. Of particular importancewould be the use of natural subjects within experimental environments based upon

126 C. Brown

their natural environments. Additional avenues of future research would be thestudy of other potential cues-to-causality and their interactions, the use of morediscretionary performance situations (e.g., research and development depart-ments), and the manipulation of information system variables such as the validityof evidence sources, conflicting evidence, as well as the information (evidence)report format, frequency, and level of aggregation. A long term research objectiveshould be to stmcture performance evaluation and control mechanisms and to trainindividuals who utilize these mechanisms in a manner that will facilitate individualperformance within businesses.

ReferencesBrown, C , "Causal Reasoning in Performance Assessment: Effects of Cause and Effect

Temporal Order and Covariation," Accounting, Organizations and Society Vol. 10,No. 3 (1985) pp. 255-266.

Crocker, J., "Judgment of Covariation by Social Perceivers," Personality and SocialPsychology Bulletin (September 1981) pp. 272-292.

Einhom, H.J., "Leaming from Experience and Suboptimal Rules in Decision Making," inT.S. Wallsten (ed.) Cognitive Processes in Choice and Decision Behavior (LawrenceErlbaum Associates: Hillsdale, NJ: 1980) pp. 1-20.

, and R.M. Hogarth, "Judging Probable Cause," Psychological Bulletin (January1986) pp. 3-19.

-, "Prediction, Diagnosis, and Causal Thinking in Forecasting," Journalof Forecasting (January-March 1982) pp. 23-36.

-, "Confidence in Judgment: Persistence of the Illusion of Validity,"Psychological Review (September 1978) pp. 395-416.

Hogarth, R.M., "Beyond Discrete Biases: Functional and Dysfunctional Aspects ofJudgmental Heuristics," Psychological Bulletin (September 1981) pp. 197-217.

Jennings, D.L., T.M. Amabile and L. Ross, "Informal Covariation Assessment: Data-Based versus Theory-Based Judgments," Judgment Under Uncertainty: Heuristics andBiases, in D. Kahneman, P. Slovic, and A. Tversky (eds.) (Cambridge UniversityPress: Cambridge, England: 1982) pp. 211-230.

Kelly, H.H. and J.L. Michela, "Attribution Theory and Research," Annual Review ofPsychology, Vol. 31 (1980) pp. 457-501.

Kuhn, T.S., "Second Thoughts on Paradigms," in F. Suppe (ed.) The Structure of Scien-tific Theories (University of Illinois Press: Urbana, IL: 1977) pp. 459-517.

, The Structures of Scientific Revolutions, 2nd Edition (University of ChicagoPress: Chicago: 1970).

Mackie, J.L., The Cement ofthe Universe: A Study of Causation (Oxford at the ClarendonPress: Oxford, England: 1980).

Nisbett, R. and L. Ross, Human Inferences: Strategies and Shortcomings of SocialJudgment (Prentice-Hall: Englewood Cliffs, NJ: 1980).

Tversky, A., "Features of Similarity," Psychological Review (July 1977) pp. 327-352.