validity of questionnaire and tat measure s of …vktonay/migrated/psyc166/tatmetanalysis.pdf ·...

Psychological Bulletin1992. Vol. 112, No. 1,140-154 Copyright 1992 by the American Psychological Association, Inc.

0033-2909/92/S3.00

Validity of Questionnaire and TAT Measures of Need for Achievement:Two Meta-Analyses

William D. SpanglerSchool of Management

State University of New \brk at Binghamton

Proponents of the Thematic Apperception Test (TAT), most notably McClelland, have argued thatthe TAT and questionnaires are valid measures of different aspects of achievement motivation.Critics of the TAT have argued that questionnaires but not the TAT are valid measures of the needfor achievement. Two meta-analyses of 105 randomly selected empirical research articles foundthat correlations between TAT measures of need for achievement and outcomes were on averagepositive; that these correlations were particularly large for outcomes such as career success mea-sured in the presence of intrinsic, or task-related, achievement incentives; that questionnaire mea-sures of need for achievement were also positively correlated with outcomes, particularly in thepresence of external or social achievement incentives; and that on average TAT-based correlationswere larger than questionnaire-based correlations. The theoretical implications of these findingsare discussed.

Over the course of 4 decades, McClelland, Atkinson andtheir associates have studied the motivational bases of humanbehavior. Much of their work has focused on the sources andeffects of achievement motivation. This work has ranged fromlaboratory studies of the effects of need for achievement onperformance (Atkinson & Litwin, 1960), studies of perfor-mance and success of people such as entrepreneurs in voca-tional settings (McClelland & Winter, 1969), training effortsaimed to increase the need for achievement of individuals(McClelland, 1965), as well as studies linking the achievementmotive to the economic growth and decline of civilizations(McClelland, 1961). During this period a number of theories ofmotivation have been developed (e.g., Atkinson, 1957; McClel-land, 1985).

At the same time, McClelland, Atkinson, and their col-leagues have devoted much research to the issue of measuringthe need for achievement in individuals. The focus of this workhas been the Thematic Apperception Test (TAT; Atkinson,1982; McClelland, 1972, 1980, 1985; McClelland, Atkinson,Clark, & Lowell, 1958; McClelland, Clark, Roby, & Atkinson,1958). TAT presents the subject with a set of pictures, generalinstructions to be creative, and a set of four questions to guidethe respondent in writing stories. The respondent writes a shortstory interpreting each picture, and the stories are then codedfor the presence of various types of achievement imagery.

The TAT method of measuring the achievement motive hasinspired substantial criticism as well as defense. Critics havecharged that TAT measures of the achievement motive demon-strate poor test-retest and internal consistency reliability (Ent-wisle, 1972; Fineman, 1977; Weinstein, 1969) and have low andinconsistent correlations with actual achievement-oriented be-

Correspondence concerning this article should be addressed to Wil-liam D. Spangler, School of Management, State University of NewYork, P.O. Box 6000, Binghamton, New York 13902-6000.

havior (Entwisle, 1972; Fineman, 1977; Klinger, 1966; Scott &Johnson, 1972; Weinstein, 1969). Critics have further arguedthat questionnaire measures demonstrate adequate reliabilityand greater predictive validity than TAT measures of theachievement motive (Fineman, 1977; Mischel, 1972; Scott &Johnson, 1972). Finally, critics have pointed out that question-naire and TAT measures of achievement are virtually uncorre-lated (Fineman, 1977; Weinstein, 1969), providing evidence ofthe poor convergent validity (Campbell & Fiske, 1959) of theTAT measure.

McClelland, Atkinson and their associates have respondedto these criticisms (Atkinson, 1982; deCharms, Morrison, Reit-man, & McClelland, 1955; McClelland, 1972,1980,1985). Inparticular, McClelland and his associates have argued thatwhen the TAT is properly administered, derived motive scoreshave adequate test-retest reliability (McClelland, 1980;McClelland, Koestner, & Weinberger, 1989; Winter & Stewart,1977). They have further asserted that the TAT predicts long-term "real-world" behavior better than do questionnaire mea-sures, and McClelland has argued that TAT and questionnairemeasures of the achievement motive are virtually uncorrelatedbecause they are measures of distinct aspects of personality andtherefore should not be correlated.

This dispute over the relative merits of TAT versus those ofquestionnaire measures of achievement motivation has beenvaluable in the sense that the published criticisms and defensesof TAT and questionnaire measures provide the basis for pre-cise testable hypotheses. However, previous reviewers of theliterature have faced a number of difficulties in testing hypothe-ses. First, thousands of empirical studies on achievement moti-vation are available, and it is virtually impossible to analyzemore than a small fraction of them. It is not clear that thestudies chosen for analysis from the mass of available researchwere randomly selected. Second, there have been no statisticaltests of hypotheses. Third, empirical research on achievementmotivation has used many statistics in addition to correlations

140

MEASURES OF NEED FOR ACHIEVEMENT 141

coefficients such as chi-squared statistics and contingency ta-bles, F and t tests, p values, and tables of means and standarddeviations, but prior reviews have not made use of these statis-tics. Fourth, studies vary in sample size, and previous reviewshave not weighted their statistics by sample size.

Over the last decade, a set of analytical and statistical tech-niques has been developed that allow researchers to addressthese four difficulties. These techniques collectively are re-ferred to as meta-analytic techniques (Hedges & Olkin, 1985;Hunter, Schmidt, & Jackson, 1982; Rosenthal, 1984; Wolf,1986). Using these newly developed procedures, it is possible tosystematically analyze a large body of published research or arandomly selected subset of it, statistically test hypotheses,convert various measures of association to a common metricsuch as correlations (Rosenthal, 1984; Wolf, 1986), and weightindividual statistics by their respective sample sizes.

The present investigation's purpose is to test the competingclaims of McClelland and colleagues against those of their crit-ics by using newly developed meta-analytic procedures to ana-lyze previously published empirical research.

Theoretical BackgroundTo understand the nature of the controversy between advo-

cates of TAT measurement and proponents of questionnairemeasurement of motives, it is necessary to first discuss sometheoretical concepts advanced by McClelland and his col-leagues.

McClelland has argued that the dispositions measured by theTAT are implicit motives. Implicit motives are dispositions thathave traditionally been labeled needs, for example, need forachievement (nAch), need for power (nPower), and need for affil-iation (nAff). McClelland considers these TAT-measured dis-positions to be motives because they "drive behavior (i.e., ener-gize it), direct behavior (i.e., focus attention on relevant activity),and select behavior (i.e., produce better learning or perfor-mance)" (McClelland et al., 1989, p. 696). These motives arelabeled implicit motives because the person writing stories to aset of pictures is not explicitly describing him- or herself.Rather, stories written reflect nonconscious motives of the au-thor. McClelland et al. (1989) speculate that these implicit mo-tives are built on affective experiences with natural incentivesearly in life before the development of language in the person.There is some evidence that implicit motives drive, direct, andselect behavior through physiologically based reinforcementprocesses. For example, McClelland et al. (1989) discuss a studyin which the presentation of a romantic film was associatedwith the increased release of dopamine for high nAff subjects.

McClelland has defined a second category of personality dis-positions, which are currently called self-attributed motives,such as self-attributed achievement (sanAch), self-attributedpower (sanPower), and self-attributed affiliation (sanAff). Pre-viously, these self-attributed motives had been referred to asvalues, but this terminology evidently led to some confusionwith values defined as "normative beliefs about desirable goalsand modes of conduct" (McClelland et al., 1989, p. 690). Thesedispositions represent the value or worth to individuals of spe-cific achievement-, affiliation-, or power-related activities.

According to McClelland and his associates, these self-attrib-

uted motives differ from implicit motives in at least four funda-mental ways. First, they are uncorrelated with measures of im-plicit motives. Second, they correlate with different types ofoutcomes. Implicit motives tend to predict long-term spontane-ous behavioral trends over time such as entrepreneurial success(nAch) or success in management (nPower), whereas self-attrib-uted motives predict responses to immediate and specific situa-tions and choice behavior. Third, self-attributed motives arerelatively conscious perceptions of what is important to the indi-vidual and of what is valued by the individual's culture. Theseself-attributed motives are part of the individual's self-concept.Fourth, self-attributed and implicit motives have different de-velopmental histories. Implicit motives develop early in life as aresult of experiences with various incentives and do not requirethe presence of language for their development. Self-attributedmotives develop somewhat later in life, require the presence oflanguage, and come from the individual's understanding of so-cial incentives and demands made verbally by others in theenvironment. As a result of these distinct developmental histo-ries, implicit motives are related to physiological processes suchas release of norepinephrine and dopamine; self-attributed mo-tives evidently are not related to such physiological processes.

From the perspective of McClelland (McClelland, 1980,1985; McClelland et al., 1989), individual differences in im-plicit and self-attributed motives do not by themselves predictindividual differences in behavior. Motives predict behavioronly in the presence of appropriate incentives. If there are noachievement incentives in a given research or work situation,there is no reason to believe that achievement-oriented individ-uals will behave any differently than those low in achievementmotive. That is, successful performance of some specified activ-ity will not result in outcomes that are reinforcing to the per-former. In achievement situations, on the other hand, thereshould be a positive correlation between strength of the achieve-ment motive and achievement-related behavior because theperson's activity may lead to reinforcing outcomes. In otherwords, achievement behavior is an interactive effect of implicitand self-attributed motives for achievement and environmentalachievement incentives.

Recently, McClelland et al. (1989) made a distinction be-tween social incentives and activity incentives. Social incentivesare characteristics of situations such as rewards, prompts, ex-pectations, demands, and norms that come from outside thetask itself. These incentives may be provided by a boss, by anexperimenter, by co-workers, or by a group. Social achievementincentives include challenging goals set by an experimenter(McClelland, Atkinson, Clark, & Lowell, 1958), achievement-oriented instructions in an experiment (French, 1955; McClel-land, Clark, Roby, & Atkinson, 1958), achievement worknorms, and pretreatment experimental manipulations. Activityincentives are characteristics of the task itself. The individualhigh in some implicit motive is reinforced by performing thetask itself. Activity achievement incentives include moderatetask risk (Atkinson, 1957; Atkinson & Feather, 1966; Atkinson& Litwin, 1960; Weinstein, 1969), task contingency (Raynor,1969, 1970), achievement work content (McClelland, Atkin-son, Clark, & Lowell, 1958), time pressure, and a high objectiverelationship between performance and some achievement-re-lated outcome in the immediate situation.

142 WILLIAM D. SPANGLER

The relevance to the present investigation of this distinctionbetween social and activity incentives is the proposition madeby McClelland et al. (1989) that specific classes of incentivesinteract with specific types of motives. Specifically, social in-centives interact with self-attributed motives but not implicitmotives, and activity incentives interact with implicit motivesbut not self-attributed motives. A number of studies haveshown that social incentives but not activity incentives interactwith self-attributed motives to predict behavior (deCharms etal., 1955; McClelland et al., 1989; Patten & White, 1977). Theo-retically, if self-attributed motives arise as a result of exposure todemands, goals, incentives and values expressed by others, thenit follows that these self-attributed motives should, later in life,respond to external or social incentives. Activity incentives butnot social incentives will interact with implicit motives to pro-duce behavior. The theoretical reason for this matching of im-plicit motives with activity incentives is the possibility that im-plicit motives are based on incentives involved in doing or expe-riencing certain things early in life. McClelland et al. (1989)have summarized a number of studies in which TAT-based im-plicit motives and activity incentives, but not social incentives,together predicted behavior.

To understand the controversy between proponents and op-ponents of TAT measurement, it is also necessary to examineMcClelland's use of the terms operant and respondent. Operantbehavior is behavior that the subject generates spontaneously.At least it is not possible to specify or control the stimulus thatelicits the behavior, and such behavior appears to be freely emit-ted. Respondent behavior is behavior that is controlled by char-acteristics of the subject's environment. That is, it is behaviorelicited by known stimuli in the environment. Behavior is moreseriously and overtly constrained by the environment in re-spondent situations than in operant situations. Of course, behav-ior is not either entirely operant or entirely respondent. Somedegree of environmental control is assumed even in supposedlyoperant situations, and even in respondent situations the sub-ject has some degree of freedom in responding.

According to McClelland (1980), it is possible to arrange be-havior along a continuum from behavior emitted under ex-treme environmental control to behavior that is relatively freefrom explicit environmental control. Relatively operant out-comes include income, job level attained in an organization,professional rank, publications, participation and leadership incommunity organizations, and social behavior occurring undernatural conditions. These are examples of operant behavior be-cause it is not possible to specify the exact stimuli that lead tothe behavior in question. Respondent outcomes include schoolgrades, intelligence and achievement test scores, results of per-sonality inventories, and opinion surveys. According toMcClelland (1980), these are examples of respondent behaviorbecause the expression of the behavior is elicited and con-strained by environmental stimuli. For example, a class exammay specify the stimulus items, response options, instructions,time allotted, and interactions with other students and the ex-aminer. In an opinion survey, respondents may be limited toagreeing or disagreeing with pre-selected items. Behavior orperformance typically measured in laboratory experimentsfalls somewhere between these extremes and may be labeled

semioperant behavior (McClelland, 1972; McClelland et al.,1989).

With this theoretical background in mind, it is possible toderive the seven hypotheses that were tested in the present in-vestigation.

Seven HypothesesCritics of TAT measurement have argued that questionnaires

predict outcomes better than does the TAT. Fineman (1977)summarized correlations and other measures of association be-tween questionnaire measures of need for achievement and cri-teria such as school grades, job success, and laboratory perfor-mance. Twenty-two of 30 measures were significant, and Fine-man noted that the criterion validity of these measures wasbetter than the criterion validity of the TAT measure nAch.Scott and Johnson (1972) directly compared the predictive va-lidity of TAT measures of power, achievement, and affiliationwith questionnaire measures and found correlations of ques-tionnaire measures with criteria to be higher than correlationsof TAT measures with outcomes. Likewise, Entwisle (1972)found that correlations between questionnaire measures andoutcomes were generally higher than correlations between TATmeasures and outcomes.

McClelland has argued that in general it is inappropriate tocompare the predictability of TAT and questionnaire measuresof the achievement motive regardless of the type of outcomepredicted. The TAT generally is a better predictor of long-termoperant behavioral trends, and questionnaires tend to be betterpredictors of choices and attitudes. However, McClelland(1972,1980) has agreed with critics of TAT measurement thatunder many circumstances questionnaire measures of theachievement motive will generate superior correlations. Thesecircumstances include situations in which questionnaires andbehavior are assessed within a short time of each other, situa-tions in which respondents infer their need for achievementfrom their perceptions of their behavior, and occasions inwhich the questionnaire and the behavioral measures shareitems.

These considerations led to the first hypothesis tested in thepresent research:

Hypothesis 1. The motive-outcome correlation is the resultof a type-of-motive-measure main effect. That is, the correla-tion between TAT measures of the achievement motive andoutcomes will be smaller than the correlation between ques-tionnaire measures of achievement and outcomes.

McClelland (1980; McClelland et al., 1989) has argued thatthe magnitude of the correlation between a measure of theachievement motive and an outcome depends both on the typeof motive measure and the number of achievement incentivesin the situation. Motives per se do not predict behavior, activityincentives interact with implicit motives, and the TAT measuresimplicit motives. Therefore, the correlation between a TATmeasure of nAch and outcomes in a particular situation de-pends on the number of activity achievement incentives in thesituation. Furthermore, questionnaires measure self-attributedmotives and interact with social incentives rather than activityincentives. Therefore, the correlations between questionnaire


measures of self-attributed achievement and outcomes shouldnot vary substantially with variations in the number of activityincentives in the situation. From these arguments Hypothesis 2was derived:

Hypothesis 2. The motive-outcome correlation is the resultof a Type of Motive Measure X Number of Activity Incentivesinteraction. Specifically, the effect of number of activity incen-tives on the magnitude of the motive-outcome correlation willbe greater for TAT measures of the achievement motive as com-pared with questionnaire measures of the achievement motive.

According to McClelland et al. (1989), questionnaires mea-sure the self-attributed need for achievement, now labeled san-Ach. SanAch interacts with social incentives in the environ-ment, and therefore, questionnaire-based motive-outcomecorrelations will increase with the number of social achieve-ment incentives in the situation. Furthermore, according to thearguments of McClelland et al., the number of social incentivesin the experimental or work situation should not substantiallyaffect the predictability of TAT measures of achievement be-cause the TAT measures nAch, which interacts primarily withactivity incentives in the environment.

Hypothesis 3. The motive-outcome correlation is the resultof a Type of Motive Measure X Number of Social Incentivesinteraction. The effect of number of social incentives on themagnitude of the motive-outcome correlation will be less forTAT measures of the achievement motive as compared withquestionnaire measures of the achievement motive.

Critics of TAT measurement argue that in general question-naires demonstrate superior predictability. According toMcClelland (1980) and McClelland et al. (1989), the superiorityof questionnaires versus the TAT depends on the type of out-come measured, specifically on the operant or respondent char-acter of the outcomes.

The average correlation between TAT achievement and oper-ant, or real-life, outcomes should be larger than the averagecorrelation between TAT measures and respondent outcomes(McClelland, 1980,1985; McClelland et al., 1989). McClellandet al. (1989, p. 695) provide one explanation for this expectedrelationship. Social incentives found in the environment inter-act with the self-attributed need for achievement (sanAch);achievement incentives in the activity itself interact with theimplicit motive for achievement (nAch). McClelland et al.(1989) suggest that social incentives are likely to be associatedwith respondent outcomes, whereas activity incentives arelikely to be associated with operant outcomes. Therefore, theexpected correlation between TAT achievement and operantoutcomes will be positive because of the interaction betweenactivity achievement incentives and need for achievement.

McClelland et al. (1989) imply, however, that correlations be-tween TAT achievement and operant outcomes should be largerthan correlations between TAT achievement and respondentoutcomes for a second reason. In the case of TAT measurementand operant outcomes, environmental stimuli do not stronglycontrol either TAT responses or operant outcome responses, soit is possible for variations in TAT implicit need for achieve-ment to produce variations in behavioral responses. On theother hand, in the case of TAT measurement and respondentoutcomes, environmental stimuli will elicit and constrain re-

spondent behavior but not TAT responses, thereby reducingany association between TAT implicit need for achievementand respondent outcomes.

McClelland (1980, 1985) and McClelland et al. (1989) havecontended that questionnaire measures may predict respon-dent behavior such as school grades or attitudes on pencil-and-paper inventories better than operant behavior such as careersuccess. Three reasons account for this superiority of question-naire measures in predicting respondent outcomes versus oper-ant outcomes.

First, respondent outcomes are very likely associated withsocial incentives, and according to McClelland et al. (1989),social incentives in the environment interact with the self-attrib-uted need for achievement measured by questionnaires to pro-duce achievement-related respondent behavior. On the otherhand, operant achievement behavior arises from the interac-tion of nAch interacting with activity incentives. In respondentsituations with questionnaire measures, there are quite likelyfew activity incentives, and questionnaires measure self-attrib-uted need for achievement, so operant behavior will not be wellpredicted by questionnaire measures of achievement.

Second, questionnaire measures of achievement motivationare respondent measures, that is, the responses are elicited andshaped by the instrument and situation. Likewise, respondentbehavior is behavior elicited and shaped by stimuli. If the stim-uli eliciting responses to the instrument are the same as thoseeliciting the respondent behavior, then there should be a strongcorrelation between the two sets of measures. In the extremecase, a questionnaire measure of achievement and achievementbehavior may be two measures of the same thing (McClelland,1972). For example, a questionnaire measure of achievementmotivation relies on a subject's conscious opinions about him-or herself. This questionnaire is "validated" by friends' orteachers' observations of the respondent, but the observersmerely report what they have heard the respondent tell of him-self or herself. McClelland (1980) further suggested that ob-served correlations between questionnaire measures of achieve-ment motivation and outcomes may be inflated by such con-taminants as single-source response bias.

Third, according to McClelland (1980,1985), questionnairesmeasure the conscious value people place on achievement, apart of the self-concept, whereas the TAT measures the under-lying and nonconscious motive for achievement, nAch. There-fore, questionnaires predict those aspects of behavior that arelinked to conscious values and processes such as choice andsusceptibility to expert opinion (deCharms et al., 1955), re-sponses to formal instruments such as personality inventories,and performance on exams. However, according to McClelland(1980,1985), much of operant real-life behavior is determinedby nonconscious motives that drive, direct, and select behavior.

From this reasoning, Hypothesis 4 was derived and tested inthe present study.

Hypothesis 4. The motive-outcome correlation depends ona Type of Motive Measure X Outcome Operant Level interac-tion. The degree of increase in the motive-outcome correlationas a function of type of outcome (going from respondent tooperant outcomes) will be greater for TAT measures of the


achievement motive as compared with questionnaire measuresof the achievement motive.

McClelland and his associates have explored 2 two-way inter-actions between type of motive measure and number of achieve-ment activity incentives (Hypothesis 2) and between type ofmotive measure and outcome operant level (Hypothesis 4).However, McClelland and his associates have not explored thepossible three-way interaction of type of motive measure, num-ber of achievement activity incentives in the environment, andoutcome operant level. If there are no achievement incentives inthe situations that generate either operant or respondent out-comes, then high need-for-achievement individuals would notbe motivated to perform well in either situation and the averagecorrelation for operant outcomes should not be much largerthan the average correlation for respondent outcomes. How-ever, in situations characterized by many achievement incen-tives, operant outcome measures should produce substantiallyhigher correlations with TAT measures of need for achievementthan respondent outcome measures. High-need-for-achieve-ment persons performing respondent tasks may find them-selves constrained by the nature of the task or situation or notreinforced by successful completion of the task, in which casethe presence of incentives may have a small effect on behavior.Motive-outcome correlations that are based on questionnairemeasures will not be substantially affected by number of activ-ity incentives in the environment in either operant or respon-dent situations because activity incentives do not interact sub-stantially with self-attributed motives measured by question-naires.

Hypothesis 5. The motive-outcome correlation depends ona Type of Motive Measure X Number of Activity Incentives XOutcome Operant Level interaction. The greater effect of activ-ity incentives on the motive-outcome correlation for TAT mo-tive measures as compared with questionnaire motive measureswill increase as outcomes become more operant.

Although McClelland and his colleagues have investigatedthe possible two-way interactions between type of motive mea-sure and number of social incentives (Hypothesis 3) and be-tween type of motive measure and outcome operant level (Hy-pothesis 4), they have not considered the possible three-wayinteraction of type of motive measure, number of achievementsocial incentives in the environment, and outcome operantlevel. Social incentives interact with the self-attributed need forachievement. Because this self-attributed need is measured byquestionnaires, there should be a positive relationship betweenquestionnaire-based motive-outcome correlations and numberof social incentives in the environment. But operant outcomesmay allow greater opportunity for the value of achievement asmeasured by questionnaires to be expressed as compared withrespondent measures of outcomes, which restrict or constrainthe expression of questionnaire-measured achievement moti-vation. Neither the nature of the outcome, operant versus re-spondent, nor the number of social incentives in the situationshould have much effect on TAT-based motive-outcome corre-lations because the TAT primarily measures implicit motives,which interact with activity incentives rather than with socialincentives in the environment.

Hypothesis 6. The motive-outcome correlation is a functionof a Type of Motive Measure X Number of Social Incentives X

Outcome Operant Level interaction. The effect of social incen-tives on the motive-outcome correlation for TAT motive mea-sures as compared with questionnaire motive measures willbecome increasingly negative as outcomes become moreoperant.

A further aspect of the controversy between McClelland andhis defenders and critics of TAT measurement centers on corre-lations between TAT and questionnaire measures of needs.Klinger (1966), Entwisle (1972), Fineman (1977) and others re-ported that correlations between questionnaire and TAT mea-sures of achievement are low. Proponents of TAT measurement(e.g., Atkinson & Litwin, 1960; McClelland, 1980, 1985) haveconcurred. However, there are at least two interpretations ofthis pattern of results. One interpretation is that TAT and ques-tionnaire measures are different measures of a single constructsuch as need for achievement, that they should be highly corre-lated, and that the failure of TAT measures to correlate highlywith questionnaire measures indicates a lack of convergent va-lidity on the part of the TAT measure. McClelland (1980,1985;McClelland et al., 1989) has argued that the two techniquesmeasure different constructs. Questionnaires measure self-at-tributed motives; the TAT measures more physiological andnonconscious implicit motives. In the present study a positivebut small correlation between questionnaire and TAT measuresof achievement was expected, either because the two type ofmeasures measure the same construct or because the two typesof measures might be subject to common cues or response biasof measurement.

Hypothesis 7. The average correlation between TAT andquestionnaire measures of achievement will be positive andsignificant.

Summary

To test the competing claims of McClelland and associatesand those of critics of TAT-based measures of achievement mo-tivation, 105 empirical studies using either TAT or question-naire measures of achievement motivation or both were contentanalyzed to provide data for two separate meta-analyses. In thefirst meta-analysis, the dependent variable was the achieve-ment motive-outcome correlation. Independent variables inthe first meta-analysis included type of motive measure (ques-tionnaire, TAT), the number of activity incentives in the situa-tion, and the number of social achievement incentives in thesituation. Outcomes were classified as respondent (e.g., atti-tudes, opinions, school outcomes, ability and achievementtests), semioperant (laboratory measures of performance), oroperant (e.g., income earned, occupational success, naturallyoccurring social behavior) on the basis of the presumed degreeof environmental control over measured behavior. The first sixhypotheses discussed above were tested by regressing the de-pendent variable, namely the motive-outcome correlation, onthe four independent variables and on interaction termsformed from these four independent variables. The seventhhypothesis was tested in a second meta-analysis that was basedon correlations between TAT and questionnaire measures ofthe achievement motive.


Method

Selection of ArticlesTo test the seven hypotheses developed in the present study, a com-

prehensive meta-analysis of original empirical research was under-taken. The first step was to identify available studies. All original re-search articles cited in four major literature reviews (Entwisle, 1972;Klinger, 1966; McClelland, 1980; Weinstein, 1969) were listed. A com-puter search for additional articles was undertaken. These two stepsproduced a list of 286 potentially useful articles. Articles randomlyselected from this list were coded until a total of 139 articles had beenselected, from which 105 articles were coded. At this point I decidedthat further data collection would not materially affect the results ofthe three meta-analyses, because the available 105 articles provided490 correlations for analysis. Of the 34 articles selected but not used, 11had no dependent variables or achievement was the dependent vari-able (articles with achievement as a dependent variable were elimi-nated because the focus of the present study was the predictive validityof measures of achievement motivation), 7 were not relevant to themeta-analysis at hand (e.g., need for affiliation rather than need forachievement was related to performance), and 16 articles containedinsufficient data to be used.

A list of the 105 articles used in the present study may be found inthe Appendix.

Coding of ArticlesThe first step in coding the selected 105 articles for subsequent analy-

sis was to develop a coding form. Next I prepared a coding manual,which explained each item on the form and gave examples of howmaterial from the articles was to be coded. A graduate research assis-tant and I independently coded the research articles. However, I re-viewed the articles coded by the research assistant, and we discussedand resolved discrepancies. All data were completely coded and inputbefore any tests of the seven hypotheses developed in this researchwere undertaken.

Table 1 summarizes the coding of four articles used in the presentthree meta-analyses.

Unit of ObservationAn article may report results from more than one independent study

or may provide statistics for a number of independent groups within agiven study. Furthermore, a given article may test relationships be-tween TAT-based measures of achievement motivation and perfor-mance on the one hand and relationships between questionnaire mea-sures of achievement and performance on the other. An article mayreport one correlation on the basis of an operant measure of perfor-mance and another correlation on the basis of a respondent measure ofperformance. One correlation was perhaps based on a situation withachievement incentives, and another correlation from the same inves-tigation was based perhaps on a no-achievement-incentives situation.

In these circumstances, it is not legitimate to collapse distinct corre-lations into some average statistic representing all research findingsreported in an article. For example, to average a correlation that isbased on an operant measure of performance with one from the samestudy that was based on a respondent measure would make it impossi-ble to test any hypothesis involving outcome operant level, specificallyHypotheses 4, 5, and 6.

In other cases however, a number of statistics may be found for asingle group of people on the basis of a single situation with a givennumber of achievement incentives, in which all outcome measures areeither operant or respondent, and in which only one type of achieve-ment measure, for example, TAT, was used to measure performance.

There are multiple statistics perhaps because several similar measuresof performance were used or because the measure of achievement wasindividually correlated with items from a personality test. In thesecases, it is not legitimate to analyze these statistics separately for tworeasons. First, these statistics are not independent because they comefrom the same sample. Second, including several similar correlationsfrom a single group of individuals artificially inflates the total samplesize used in the analysis. That is, a group of subjects is counted morethan once.

In the present investigation, the unit of observation was defined to bea single statistic that was based on one type of achievement measure-ment (TAT or questionnaire), the same outcome operant level (respon-dent, semioperant, or operant), and one set of achievement incentivesfor a single group of individuals. In cases in which several similarcorrelations were available on a single group of people in a study, thecorrelations were averaged to produce one correlation for subsequentanalysis.

This definition of the unit of observation produced 490 correlationsfor subsequent analysis. Of these, 190 were correlations of TAT achieve-ment motivation with outcomes that were based on a total of 12,961subjects; 193 were correlations between questionnaire measures ofachievement and outcomes that were based on a sample size of 15,328;and 36 were correlations between TAT and questionnaire measures ofachievement motivation that were based on a sample size of 2,785. Theremaining 71 correlations were either correlations between projectivebut non-TAT measures and outcomes or overall study correlations thatduplicated subgroup correlations used in the three meta-analyses.

Treatment of Estimated DataOf the 490 correlations available for analysis, 45 were based on one

or more estimated statistics such as F < 1 or r not significant. In thesecases, the source article did not report the actual value of the statistic.If these nonsignificant statistics had been eliminated from the presentstudy, the calculated average correlations might have been substan-tially higher than their population values. Therefore, I decided to esti-mate statistics reported as not significant. F not significant was con-verted into an equivalent correlation of 0, F< 1 was set to F= .5, p <. 1was set at .05, correlations not significant were set to 0, / not significantwas set to .5 where there was some evidence of a positive but nonsigni-ficant relationship, for example, group means. In cases in which anonsignificant t was reported together with means that were approxi-mately the same, t was set to 0.

Determination of Sign DirectionEach statistic was examined for sign direction according to available

theories of achievement motivation. For example, Atkinson (1957)proposed that high-need-for-achievement individuals would exhibit agreater preference for moderate risk than individuals low on need forachievement. However, a specific researcher might test this hypothesisby correlating TAT nAch with an index of preference for extreme risk.A negative correlation or a negative / statistic would confirm Atkin-son's (1957) hypothesis, but to add this confirmatory negative correla-tion to other correlations would reduce the magnitude of the averagecorrelation between TAT achievement and outcomes. So in these casesit was necessary to change the sign of the correlation or ( statistic sothat a positive sign always indicated a finding consistent with theoryand a negative sign indicated a finding inconsistent with achievementtheory.

A further difficulty emerged in coding the data. Much of the rele-vant research used statistics other than correlations, for example, ta-bles of means and standard deviations, F statistics, and chi-squaredstatistics. Because these statistics do not have a sign associated with

146

Table 1Illustrative Coding of Articles

WILLIAM D. SPANGLER

Incentives

Source

Miyamoto(1981)

Miyamoto(1981)

Miyamoto(198!)

Miyamoto(1981)

Atkinson andLitwin(1960)



Singh (1979)

Singh (1979)

Outcome Group and size

Measured with

Res 135 students,Grades 1-7

Res 1 35 students,Grades 1-7

Res 135 students,Grades 1-7

Res 135 students

Res 45 collegestudents

Res 44 collegestudents

Sem 45 collegestudents

Op 200 farmers

OP 100 entre-preneurs

Result r

Thematic Apperception Test (TAT)

Grades .37

IQ .26

Rated .49activity

Attitudes, .41anxiety . 1 5

Final exam .28"persistence

Final grade .32*

Ringtoss .25game

Farm output .68

Industrial .48output

Activity

Tast contingency,achievementwork content;objectiverelationship*

Time pressure,taskcontingency,achievementwork content

Taskcontingency,achievementwork content

Taskcontingency,achievementwork content

Time pressure,taskcontingency,achievementwork content;objectiverelationship

Time pressure,taskcontingency,achievementwork content;Objective

Moderate risk,achievementwork content;Objectiverelationship

Taskcontingency,achievementwork content;objectiverelationship


Social

Achievementinstructions


—

—




—

—

—

Measured with questionnaire

Hickson andDriskill(1970)


Res 68 students

Res 68 students

GPA .42

GPA .36






Table 1 (continued)

Incentives

Source





Atkinson andLitwin(I960)


Singh (1979)

Singh (1979)

Singh (1979)


Miyamoto(1981)

Outcome

Res

Res

Res

Res

Res

Sem

Op

Op.

Op

—

—

Group and size

Measured68 students

68 students

68 students

44 collegestudents

44 collegestudents

43 collegestudents

200 farmers

200 farmers

100 entre-preneurs

Correlation47 college

students135 students,

Grades 1-7

Result Activity Social

with questionnaire (continued)Honors

program

Sensationseeking

Sensationseeking

Final exampersistence

Final grade

Ringtossgame

Farm output

Farm output

Industrialoutput

.34

.14

.11

-.13"

.18"

-.26"

-.01

-.03

.00

Achievementwork content;objectiverelationship

—

—



Moderate risk,achievementwork content;objectiverelationship




Achievementnorms

—

—




—

—

—

between TAT and questionnaire

—

-.05

.41 — —

Note. All studies measured achievement. Res = respondent, Sem = semioperant, Op = operant, GPA = grade point average. Objective relation-ship = high probability that individual action leads to achievement-related outcomes.* Correlation coefficient derived from a contingency table.


them or the sign is always positive, it was necessary to assign a sign,positive or negative, to each of these statistics to indicate whether theachievement-related hypothesis under investigation was supported.

Transformation of Raw DataThe raw data found on the coding forms were transformed to pro-

vide measures of effect size, type of motive measure, number of activ-ity incentives, number of social incentives, and outcome operant level.

Effect size. The articles coded in the present investigation reporteda number of statistics in addition to correlations. Specifically, contin-gency tables, chi-squared statistics, /-"tests,; tests, p values, and tablesof means and standard deviations were used. I converted all of thesestatistics into correlations using formulas provided by Rosenthal(1984, pp. 24-26) and Wolf (1986, pp. 35-36). A given study mightreport a number of related statistics. For example, a study might reportfour correlations or F statistics relating TAT achievement to a numberof similar outcome measures for a single group of participants. For thepresent investigation, these multiple measures were first convertedinto correlations where necessary. The multiple correlations constitut-ing a single observation were then transformed using Fisher's r-to-ztransformation, a weighted average of the transformed correlationswas calculated, and the average was transformed back into a correla-tion (Rosenthal, 1984, p. 27). In cases in which similar measures ofassociation were averaged to form one final correlation, a simple arith-metic average of the component sample sizes was the sample size forthe final average correlation.

Type of motive measure. I used a dummy variable (0 = question-naire; 1 = TAT) to classify motive-outcome correlations.

Number of activity incentives. Each observation was coded for thepresence or absence (absence = 0; presence = 1) of the following 5activity achievement incentives: moderate task risk, task contingency,achievement work content, time pressure, and a high objective rela-tionship between performance and some achievement-related out-come in the immediate situation. An overall index of activity incen-tives was created for each observation by adding up the 5 scores. In thepresent set of data, number of activity incentives ranged from 0 to 5 perobservation.

Number of social incentives. Each observation was coded for thepresence or absence (absence = 0; presence = 1) of the following socialachievement incentives: challenging goals set by an experimenter,achievement-oriented instructions in an experiment, achievementwork norms, and pretreatment experimental manipulations. Thesescores were added up to provide a total social achievement-incentiveindex for each observation that in principle could range from no socialachievement incentives in the work or experimental situation (0) to 3social incentives in the work or experimental situation, 5 activity incen-tives in the pretreatment situation, and 3 social incentives in the prelreat-menl situation (11). Pretreatment activity incentives were coded as so-cial incentives because they were extrinsic to the task or situation thatgenerated the motive-outcome correlations analyzed. The actual num-ber of social achievement incentives recorded from the 105 articlesused in the present research ranged from 0 to 4.

Outcome operant level. School outcomes, ability and achievementtests, and measures of attitudes, opinions, and personality were classi-fied as respondent outcomes and coded 1. Measures of performance inlaboratory settings were classified as semioperant measures and coded2. Income, job level attained in an organization, professional rank,publications, participation and leadership in community organiza-tions, and social behavior occurring under natural conditions wereclassified as operant outcomes and coded 3.

Interrater ReliabilityOnce the coding of the articles had been completed, I undertook an

analysis of interrater reliability. I hired and trained three students

enrolled in a PhD psychology program. I randomly selected, on thebasis of a table of random numbers, 20 articles from those which hadbeen used in the basic analysis. The three assistants read 2 backgroundarticles on the work of McClelland (McClelland, 1985; McClelland etal., 1989), studied the coding instructions used in the original coding,discussed with me basic aspects of the work of McClelland, and thenpractice coded the first 10 of the randomly selected articles. Theircodes were compared with the original codes for the same articles anddiscrepancies were discussed. Subsequently, the three coders indepen-dently coded the remaining 10 articles from the set. These 10 articlesproduced a total of 33 distinct observations. Although the three re-search assistants were familiar with the basic theoretical propositionsof McClelland and associates from their training, they were not awareof the seven specific hypotheses tested in the present research.

Data from the receding of the second set of 10 articles were corre-lated with the original data for the same 10 articles to obtain 3 in-terrater reliability coefficients for motive-outcome correlation, typeof motive measure, outcome operant level, number of activity incen-tives, and number of social incentives. Excluding 1 interrater reliabilitycoefficient of .33 for social incentives, the coefficients ranged from .67for number of social incentives to 1.00 for outcome operant level. Themedian interrater reliability coefficient for these 14 coefficients was.89. The low coefficient of .33 may be accounted for in large part by thefailure of one coder to code correctly achievement-oriented group ororganization norms in 6 of 33 cases.

Statistical Procedures

To test the first six hypotheses developed in this article, motive-out-come correlations were regressed on type of motive measure, numberof activity incentives, number of social incentives, outcome operantlevel, and seven interaction terms calculated from these independentvariables. In this regression, z transformations of the original correla-tions were used, and each observation was weighted by n - 3, that is, byits observed sample size - 3 (Hedges & Olkin, 1985, pp. 237-239).Each independent variable was "centered," that is, the mean of thevariable was subtracted from each observation. Interaction termsspeci-fied by Hypotheses 2-6 were created by multiplying together specifiedcentered variables. In addition to the five interaction terms specifiedby Hypotheses 2-6, terms representing Number of Activity Incen-tives X Outcome Operant Level interaction and the Number of SocialIncentives X Outcome Operant Level interaction were included in theregression. These terms were included because valid tests of higherorder interactions such as those predicted by Hypotheses 5 and 6 re-quired the inclusion in the regression equation of all component lowerorder interaction terms (Cohen & Cohen, 1983, pp. 345-346). Centeredvariables rather than the original variables were used in this analysis toreduce multicollinearity among independent variables and interactionterms calculated from these independent variables (Cohen & Cohen,1983, p. 325).

Hypothesis 7 was tested by first calculating the weighted average zscores zw of all 36 available z-transformed correlations between TATand questionnaire measures of achievement motivation and then usingthe statistic ̂ (N— 3fc)!/2, where k = number of correlations, to test thenull hypothesis that the population correlation was 0 against the one-tailed alternative that it was greater than 0 (Hedges & Olkin, 1985,p. 231).

Results

Table 2 contains descriptive statistics for motive-outcomecorrelations arranged by outcome operant level. Results fromthe regression used to test Hypotheses 1-6 are summarized in


Table 2Motive-Outcome Correlations by Type of Motive Measureand Outcome Operant Level

Statistic

QuestionnaireMSDnP

TATMSDnP

Respondent

.15

.1789

.0001

.19

.18108

.0001

Outcome type

Semioperant

.15

.2492

.0001

.22

.3045

.0001

Operant

.13

.1312

.006

.22

.2237

.0001

Note. Respondent outcomes = measures of attitudes, opinions, per-sonality, school outcomes such as grade point average, and achieve-ment and ability tests scores. Semioperant outcomes = measures ofperformance and social behavior in laboratory situations. Operant out-comes = income earned, occupational success, participation in andleadership of community organizations, sales success, job perfor-mance, social behavior occurring in natural settings. TAT = ThematicApperception Test.

Table 3. All results are expressed in terms of the original corre-lations rather than z-transformed correlations.

Hypothesis 1 predicted that the average motive-outcomecorrelation would be larger for questionnaire than TAT mea-sures of achievement. Contrary to expectation, TAT-basedcorrelations were significantly larger than questionnaire-basedmotive-outcome correlations. Hypothesis 3 was not con-firmed. Hypotheses 2, 4, 5, and 6 were confirmed. Hypothesis2 predicted that the effect of activity incentives on the motive-outcome correlation would be larger for TAT measures than forquestionnaire measures of the achievement motive. Accordingto Hypothesis 4, the motive-outcome correlation would de-pend on a Type of Motive Measure X Outcome Operant Levelinteraction. The degree of increase in the motive-outcomecorrelation as a function of type of outcome (going from re-spondent to operant outcomes) would be greater for TAT mea-sures of the achievement motive as compared with question-naire measures of the achievement motive. Hypothesis 5 pre-dicted a Type of Motive Measure X Number of ActivityIncentives X Outcome Operant Level interaction. The greatereffect of activity incentives on the motive-outcome correlationfor TAT motive measures as compared with questionnaire mo-tive measures would increase as outcomes became more oper-ant. According to Hypothesis 6, the motive-outcome correla-tion would be a function of a Type of Motive Measure X Num-ber of Social Incentives X Outcome Operant Level interaction.The effect of social incentives on the motive-outcome correla-tion for TAT motive measures as compared with questionnairemotive measures would become increasingly negative as out-comes become more operant.

Hypothesis 7 predicted that the average correlation betweenTAT nAch and questionnaire measures of achievement motiva-tion would be positive and significant. From the content analy-sis of 105 articles used in the present research, 36 correlationsbetween TAT and questionnaire measures of achievement were

found with an average correlation of .088. The z statistic testingthe hypothesis of no significance against the alternative that thecorrelation was greater than 0 was 4.548, p< .001. Hypothesis7 was therefore accepted.

Discussion

These findings have a number of implications for the relativemerits of TAT and questionnaire measures of achievement mo-tives, the predictability of achievement behavior, the distinc-tion between implicit motives and self-attributed motives, thedistinction between social incentives and activity incentives,the possible suppressing effects of incentives on behavior, andthe utility of laboratory studies of personality.

Relative Merits of TAT and Questionnaire Measuresof Achievement Motives

Opponents of the TAT have maintained that TAT measuresof achievement motivation are unreliable and invalid and thattherefore the average correlation between questionnaireachievement measures and outcomes should be larger than theaverage correlation between TAT measures of achievement andoutcomes. From Table 2 it is evident that in the present study

Table 3Tests of Hypotheses 1 Through 6 From the Regression ofMotive-Outcome Correlations on Independent Variablesand Interaction Terms

Regression termPredicted

sign Coefficient

Hypothesis 1InterceptTATOutcome operant levelNumber of activity incentivesNumber of social incentives

Hypothesis 2TAT X No. Activity Incentives

Hypothesis 3TAT X No. Social Incentives

Hypothesis 4TAT X Outcome Operant LevelNo. Activity Incentives X

Outcome Operant LevelNo. Social Incentives X

Outcome Operant LevelHypothesis 5

TAT X No. Activity Incentives XOutcome Operant Level

Hypothesis 6TAT X No. Social Incentives X

Outcome Operant Level

.165ftt

.072ft

.040}

.032ftt-.046ft

.032*

.027

.085*

.025

.031

.102**

-.118*

Note. Thematic Apperception Test (TAT) was coded 0 for question-naire measures of achievement motivation and 1 for TAT measures ofachievement motivation. Outcome operant level was coded 1 for re-spondent outcomes such as school outcomes, ability and achievementtests, and measures of attitudes, opinions, and personality, 2 forsemi-operant outcomes such as performance in laboratory settings, and 3 foroperant outcomes such as job level, professional rank, participationand leadership in community organizations, and social behavior occur-ring in natural settings. Coefficients in this table are unstandardizedregression coefficients.*p < .05, one-tailed. **p < .001, one-tailed, f p < .05, two-tailed, ff p < .01, two-tailed, fff p < .001, two-tailed.


the average correlation between TAT achievement and out-comes was higher than the average correlation between ques-tionnaire achievement and outcomes for respondent, semio-perant, and operant outcomes. Furthermore, from Table 3 it isevident that motive-outcome correlations that were based onthe TAT were significantly larger than correlations based onquestionnaire measures by .072 correlation points.

Predictability of Achievement BehaviorAlthough five of seven hypotheses were statistically sup-

ported in the present investigation and an unexpected positiveeffect for TAT measures was found, a question remains: DoTATs and questionnaires predict behavior at a nontrivial level?That is, are the results practically significant as well as statisti-cally significant? The results presented in Tables 2 and 3 do notappear to be impressive, even though they are highly significantstatistically. In Table 2, the average correlation between achieve-ment measures and outcomes ranges from. 13 to .22. That is, atmost 5% of the variance in outcomes is predicted by either TATor questionnaire measures of the achievement motive. Like-wise, the coefficients in Table 3, although several are statisti-cally significant, appear to be modest.

To understand the significance of the findings reported inTables 2 and 3,1 calculated several expected motive-outcomecorrelations by inserting various values of independent vari-ables and interaction terms into the regression equation summa-rized in Table 3 and by solving for the motive-outcome correla-tion. Certain combinations of type of motive measure, outcomeoperant levels, and levels of social and activity incentives pro-duce low expected correlations. For example, the expectedcorrelation between questionnaire achievement and respon-dent outcomes in the absence of activity and social achievementincentives is. 13, rather modest. Similarly, the expected correla-tion between TAT achievement and respondent outcomes inthe absence of social and activity achievement incentives is. 13.On the other hand, the expected correlation between question-naire achievement and real-world (i.e., operant) behavior in thepresence of, for example, four social achievement incentivesand no activity incentives is .35. The expected correlation be-tween TAT achievement and operant outcomes in the presenceof four activity incentives and no social incentives is .66.

In short, neither questionnaires nor the TAT predict achieve-ment behavior well in respondent situations in the absence ofappropriate incentives, but the TAT in the presence of activityincentives predicts operant behavior extraordinarily well, andquestionnaire measures in the presence of social incentivesstrongly predict operant behavior. Of course, it was preciselyunder conditions of activity achievement incentives and oper-ant outcomes that TAT measures of achievement were expectedto predict behavior (Hypothesis 5), and it was precisely underconditions of operant outcomes and social incentives that ques-tionnaire measures of achievement were expected to predictoutcomes (Hypothesis 6).

Implicit Motives and Self-Attributed MotivesFor some time, McClelland has contended that the TAT mea-

sures nonconscious needs for achievement and other motives,whereas questionnaires measure more conscious values ofachievement, affiliation, and power. Opponents of TAT mea-surement have not generally accepted this distinction. Theyappear to be! ieve there is one set of motives with two alternative

methods of measurement, TAT versus questionnaire. Ques-tionnaires are reliable and valid, and TAT measures are neitherreliable nor valid in their view.

Results of the present investigation support the distinctionmade by McClelland and his associates. TATs and question-naires appear to be measuring different aspects of personality.First, TAT nAch and questionnaire measures of achievementwere found to have a low average correlation of .088. Second,the argument that the TAT is an invalid measure of need forachievement does not appear to be correct, given the findingsthat TAT achievement strongly predicted operant behavior inthe presence of activity achievement incentives. Third, one indi-cation that two measures are measures of different underlyingconstructs is their differential interaction with environmentalstimuli. In the present study, type of motive measure, numberof activity incentives, and outcome operant level interacted topositively affect motive-outcome correlations (Hypothesis 5),but type of motive measure, social incentives, and outcomeoperant level interacted to negatively affect motive-outcomecorrelations (Hypothesis 6).

Social Achievement Incentives and Activity AchievementIncentives

For some time, McClelland (e.g., 1980, p. 59) has maintainedthat it is unreasonable to expect TAT nAch to predict behaviorin situations with no achievement incentives. Recently, McClel-land et al. (1989) divided incentives into two categories: socialincentives and activity incentives. According to McClelland etal. (1989), activity incentives interact with implicit motives, andsocial incentives interact with self-attributed motives. Results ofthe present meta-analysis strongly suggest that environmentalincentives may be subdivided into activity and social incentives.In Table 3 it may be seen that activity incentives interact withthe TAT for operant outcomes (Hypothesis 5), and social incen-tives interact with questionnaires for operant outcomes (Hy-pothesis 6).

An Unexpected Finding: The Suppressing Effectsof Incentives

Hypotheses 5 and 6 were based on the proposition advancedby McClelland et al. (1989) that activity incentives would inter-act with implicit motives as measured by the TAT and thatsocial incentives would interact with self-attributed motives asmeasured by questionnaires. Both hypotheses were confirmed,but an inspection of the data suggested that activity incentivespossibly interacted with self-attributed motives and that socialincentives possibly interacted with implicit motives. In bothcases, these Motive X Incentive interactions apparently acted toreduce the motive-outcome correlation, particularly for moreoperant outcomes.

To test this observation, I regressed motive-outcome correla-tions on activity incentives, social incentives, the Social Incen-tives X Outcome Operant Level interaction term, and the Activ-ity Incentives X Outcome Operant Level interaction term twice,once for questionnaire-based motive-outcome correlations andonce for TAT-based motive-outcome correlations. In the TAT-based regression, the Activity Incentives X Outcome OperantLevel interaction term significantly predicted the motive-out-come correlation (j3 = .082, / = 4.64, p < .001, one-tailed) as


expected, but the Social Incentives X Outcome Operant Levelinteraction term was negatively related to the motive-outcomecorrelation (0 = -. 106, t = -3.24, p < .005, two-tailed). In thequestionnaire-based regression, the Social Incentives X Out-come Operant Level interaction term was significant as ex-pected (j8 = .093, / = 2.74, p < .005, one-tailed), but the ActivityIncentives X Outcome Operant Level interaction term was nega-tively although not significantly related to the motive-outcomecorrelation (ft = -.027, t = -1.36, p = . 175).

A tentative explanation for this phenomenon may be ad-vanced. It is possible that just as a high level of a motive and anappropriate incentive creates a pleasurable effect that rein-forces future similar behavior, a high level of some motive com-bined with an inappropriate incentive may create an aversiveeffect that temporarily suppresses or extinguishes behavior.

Utility of Laboratory Studies of PersonalityA characteristic of much laboratory research is the coercive

nature of the research setting. Researchers want to control theirexperimental variables, measure reliably and validly their de-pendent variables, eliminate competing explanations of theirfindings, and provide enough structure to make their workreplicable by other researchers. An unintended consequence ofthe scientific method may be to minimize the expression ofindividual differences and the interaction of individual differ-ences and environmental characteristics.

The present study provides some evidence of this effect. Asoutcomes become more operant, motive-outcome correlationsbecome larger for TAT-based correlations compared with ques-tionnaire-based measures (Hypothesis 4). The impact of activ-ity incentives on motive-outcome correlations measured by theTAT becomes larger as outcomes become more operant (Hy-pothesis 5), and the impact of social incentives on motive-out-come correlations that are based on questionnaire measuresincreases as outcomes become more operant (Hypothesis 6).

In summary, the relative merits of questionnaire versus TATmeasures of achievement motivation have been debated in theliterature for decades. The present study suggests that both TATand questionnaire measures of motives have an important rolein understanding and predicting human behavior.

ReferencesAtkinson, J. W (1957). Motivational determinants of risk-taking behav-

ior. Psychological Review, 64, 359-372.Atkinson, J. W (1982). Motivational determinants of thematic apper-

ception. In A. Stewart (Ed.), Motivation and society (pp. 3-40). SanFrancisco: Jossey-Bass.

Atkinson, J. W, & Feather, N. T. (1966). A theory of achievement motiva-tion. New York: Wiley.

Atkinson, J. W, & Litwin, G. H. (1960). Achievement motive and testanxiety conceived as motive to approach success and motive to avoidfailure. Journal of Abnormal and Social Psychology, 60, 52-63.

Campbell, D. T., & Fiske, D. W (1959). Convergent and discriminantvalidation by the multi-trait-multimethod matrix. PsychologicalBulletin, 56, 81-105.

Cohen, C, & Cohen, P. (1983). Applied multiple regression/correlationanalysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

deCharms, R., Morrison, H. W, Reitman, W, & McClelland, D. C.

(1955). Behavioral correlates of directly and indirectly measuredachievement motivation. In D. C. McClelland (Ed.), Studies in moti-vation (pp. 414-423). Englewood Cliffs, NJ: Prentice-Hall.

Entwisle, D. R. (1972). To dispel fantasies about fantasy-based mea-sures of achievement motivation. Psychological Bulletin, 77, 377-391.

Fineman, S. (1977). The achievement motive and its measurement:Where are we now? British Journal of Psychology, 68,1-22.

French, E. G. (1955). Some characteristics of achievement motivation.Journal of Experimental Psychology, 50, 232-236.

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis.San Diego, CA: Academic Press.

Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis:Cumulating research findings across studies. Beverly Hills, CA: Sage.

Klinger, E. (1966). Fantasy need achievement. Psychological Bulletin,66,291-306.

McClelland, D. C. (1961). The achieving society. Princeton, NJ: VanNostrand.

McClelland, D. C. (1965). Toward a theory of motive acquisition. Amer-ican Psychologist, 20, 321-333.

McClelland, D. C. (1972). Opinions predict opinions: So what else isnewt Journal of Consulting and Clinical Psychology, 38, 325-326.

McClelland, D. C. (1980). Motive dispositions: The merits of operantand respondent measures. In L. Wheeler (Ed.), Review of personalityand social psychology (Vol. 1, pp. 10-41). Beverly Hills, CA: Sage.

McClelland, D. C. (1985). How motives, skills, and values determinewhat people do. American Psychologist, 40, 812-825.

McClelland, D. C, Atkinson, J. W, Clark, R. A., & Lowell, E. L. (1958).A scoring manual for the achievement motive. In J. W Atkinson(Ed.), Motives in fantasy, action, and society (pp. 179-204). Prince-ton, NJ: Van Nostrand.

McClelland, D. C., Clark, R. A., Roby, T. B., & Atkinson, J. W (1958).The effect of the need for achievement on thematic apperception. InJ. W Atkinson (Ed.), Motives in fantasy, action, and society (pp. 64-82). Princeton, NJ: Van Nostrand.

McClelland, D. C, Koestner, R., & Weinberger, J. (1989). How do self-attributed and implicit motives differ? Psychological Review, 96,690-702.

McClelland, D. C, & Winter, D. (1969). Motivating economic achieve-ment. New York: Free Press.

Mischel, W (1972). Direct versus indirect personality assessment: Evi-dence and implications. Journal of Consulting and Clinical Psychol-ogy, 38, 319-324.

Patten, R. L., & White, L. A. (1977). Independent effects of achieve-ment motivation and overt attribution on achievement behavior.Motivation and Emotion, I, 39-59.

Raynor, J. O. (1969). Future orientation and motivation of immediateactivity: An elaboration of the theory of achievement motivation.Psychological Review, 76, 606-610.

Raynor, J. O. (1970). Relationships between achievement-related mo-tives, future orientation, and academic performance. Journal of Per-sonality and Social Psychology, 15, 28-33.

Rosenthal, R. (1984). Meta-analylic procedures for social research. Bev-erly Hills, CA: Sage.

Scott, W A., & Johnson, R. C. (1972). Comparative validities of directand indirect personality tests. Journal ofConsulting and'Clinical Psy-chology, 38, 301-318.

Weinstein, M. S. (1969). Achievement motivation and risk preference.Journal of Personality and Social Psychology, 13,153-172.

Winter, D. G., & Stewart, A. J. (1977). Power motive reliability as afunction of retest instructions. Journal of Consulting and ClinicalPsychology, 45, 436-440.

Wolf, F. M. (1986). Meta-analysis: Quantitative methods for researchsynthesis. Newbury Park, CA: Sage.

(Appendix follows on next page)


Appendix

Articles Used in the Three Meta-Analyses

Adams, A. J., & Stone, T. H. (1977). Satisfaction of need for achieve-ment in work and leisure time activities. Journal a/Vocational Behav-ior, 11, 174-181.

Andrews, J. D. W. (1967). The achievement motive and advancement intwo typesof organizations. Journal of Personally and Social Psychol-ogy, 6, 163-168.

Atkinson, J. W, & Litwin, G. H. (1960). Achievement motive and testanxiety conceived as motive to approach success and motive to avoidfailure. Journal of Abnormal and Social Psychology, 60, 52-63.

Atkinson, J. W, & Raphelson, A. C. (1956). Individual differences inmotivation and behavior in particular situations. Journal of Personal-ity, 24, 349-363.

Bailey, R. C., & Mettetal, G. W (1977). Sex differences in the con-gruency of perceived intelligence. Journal of Genetic Psychology,131, 29-36.

Bendig, A. W (1958). Predictive and postdictive validity of needachievement scores. Journal of Educational Research, 52, 119-120.

Catalano, J. F, & Berger, D. F. (1970). The effect of need for achieve-ment and normative information on rotary pursuit performance.Psychonomic Science, 18, 69-71.

Ceranski, D. S., Teevan, R., & Kalle, R. J. (1979). A comparison of threemeasures of the motive to avoid failure: Hostile press, test anxiety,and resultant achievement motivation. Motivation and Emotion, 3,395-404.

Cooper, W H. (1983). An achievement motivation nomological net-work. Journal of Personality and Social Psychology, 44, 841 -861.

Covington, M V, & Omelich, C. L. (1979). Are causal attributionscausal? A path analysis of the cognitive model of achievement moti-vation. Journal of Personality and Social Psychology, 37,1487-1504.

Crockett, H. J., Jr. (1962). The achievement motive and differentialoccupational mobility in the United States. American SociologicalReview. 27, 191-204.

deCharms, R., & Dave, P. N. (1955). Hope of success, fear of failure,subjective probability, and risk-taking behavior. Journal of Personal-ity and Social Psychology, 1, 558-568.

Durand, D., & Shea, D. (1974). Entrepreneurial activity as a function ofachievement motivation and reinforcement control. Journal of Psy-chology, 88, 57-63.

Elder, G. H., & Maclnnis, D. J. (1983). Achievement imagery inwomen's lives from adolescence to adulthood. Journal of Personalityand Social Psychology, 45, 394-404.

Epps, E. G., Katz, I., Perry, A., & Runyon, E. (1971). Effect of race ofcomparison referent and motives on Negro competitive perfor-mance. Journal of Educational Psychology, 62, 201-208.

Erdwins, C. J., Tyer, Z. E., & Mellinger, J. C. (1980). Personality traits ofmature women in student versus homemaker roles. Journal of Psy-chology, 105, 189-195.

Farley, F. H., & Mealiea, W L. (1973). Motivation and recall of com-pleted and uncompleted achievement items. Journal of EducationalResearch, 66, 302-306.

Feather, N. T. (1965). The relationship of expectation of success to needachievement and test anxiety. Journal of Personality and Social Psy-chology, 1, 118-126.

Ferguson, E. D. (1971). Relationship between ego-involvement instruc-tions and various motivation and performance measures. Psychologi-cal Reports, 29, 547-556.

Forward, J., Wells, K., Canter, R., & Waggoner, M. (1975). Teachercontrol strategies and choice of educational objectives among col-lege students. Journal of Educational Psychology, 67, 757-763.

French. E. G. (1955). Some characteristics of achievement motivation.Journal of Experimental Psychology, 50, 232-236.

Gould, S. (1980). Need for achievement, career mobility, and the Mexi-can-American college graduate. Journal ofVocational Behavior, 16,73-82.

Greene, D. L., & Winter, D. G. (1971). Motives, involvements, and lead-ership among Black college students. Journal of Personality, 39,319-332.

Hall, L. H. (1972). Personality variables of achieving and non-achiev-ing Mexican-American and other community college freshmen.Journal of Educational Research, 65, 224-228.

Hancock, J. G, & Teevan, R. C. (1964). Fear of failure and risk-takingbehavior. Journal of Personality, 32, 200-209.

Harrell, A. M., & Stahl, M. J. (1981). A behavioral decision theoryapproach for measuring needs. Journal of Applied Psychology, 66,242-247.

Heise, D. R. (1966). Sensitization of verbal response-dispositions by Naffiliation and N achievement. Journal of Verbal Learning and VerbalBehavior, 5, 522-525.

Helmreich, R. L., Spence, J. X, Beane, W E., Lucker, G. W, & Matth-ews, K. A. (1980). Making it in academic psychology: Demographicand personality correlates of attainment. Journal of Personality andSocial Psychology, 39, 896-908.

Helmreich, R. L., Swain, L. L., & Carsrud. (1986). The honeymooneffect in job performance: Temporal increases in the predictivepower of achievement motivation. Journal of Applied Psychology, 71,185-188.

Henschen, K. P., Edwards, S. W, & Mathinos, L. (1982). Achievementmotivation and sex-role orientation of high school female track andfield athletes versus nonathletes. Perceptual and Motor Skills, 55,183-187.

Hickson, R. H., & Driskill, J. C. (1970). Needs for achievement: Differ-ences between honors and non-honors students. Journal of Experi-mental Education, 38, 37-38.

Holmes, D. S., & Tyler, J. D. (1968). Direct versus projective measure-ment of achievement motivation. Journal of Consulting and ClinicalPsychology, 32, 712-717.

Honor, S. H., & Vane, J. R. (1972). Comparison of Thematic Appercep-tion Test and questionnaire methods to obtain achievement atti-tudes of high-school boys. Journal of Clinical Psychology, 28, 81 -83.

Hurley, J. R. (1957). Achievement imagery and motivational instruc-tions as determinants of verbal learning. Journal of Personality, 25,274-282.

Inkson, J. H. (1971). Achievement motivation and occupationalchoice. Australian Journal of Psychology, 23, 225-234.

Jardine, E., & Winefield, A. H. (1981). Achievement motivation, psy-chological reactance, and learned helplessness. Motivation andEmotion, 5, 99-113.

Johnson, D. M. (1973). Relationships between selected cognitive andnoncognitive variables and practical nursing achievement. NursingResearch, 22, 148-153.

Johnston, R. A. (1955). The effects of achievement imagery on maze-learning performance. Journal of Personality, 23,145-152.

Johnston, R. A. (1957). A methodological analysis of several revisedforms of the Iowa Picture Interpretation Test. Journal of Personality,25, 283-293.

Karabenick, S. A. (1972). Valence of success and failure as a function ofachievement motives and locus of control. Journal of Personality andSocial Psychology, 21, 101-110.

Karabenick, S. A., & Youssef, Z. I. (1968). Performance as a function ofachievement motive level and perceived difficulty. Journal of Person-ality and Social Psychology, JO, 4] 4-419.

Kelly, R. T, Harve, E., & Rodger, L. (1973). Interaction effects of


achievement need and situational press on performance. Journal ofSocial Psychology, 89,141-145.

Kukla, A. (1972). Attributional determinants of achievement-relatedbehavior. Journal of Personality and Social Psychology, 21,166-174.

Lao, R. C. (1980). Differential factors affecting male and female aca-demic performance in high school. Journal of Psychology, 104,119-127.

Latta, R. M. (1978a). Hope of success and fear of failure components ofMehrabian's scales of resultant achievement motivation. Journal ofResearch in Personality, 12,151.

Latta, R. M. (1978b). Interactive effects of initial achievement orienta-tion and prior success feedback on the mastery of subsequent diffi-cult and easy tasks. American Educational Research Journal, /5,17-24.

Lifshitz, M. (1974). Achievement motivation and coping behavior ofnormal and problematic preadolescent kibbutz children. Journal ofPersonality Assessment, 38, 138-143.

Lindgren, H. C., Moritsch, B., & Thulin, E. K. (1986). Studies of threemeasures of achievement motivation. Psychological Reports, 59,123-136.

Lowe, C. A., & Medway, F. J. (1976). Effects of achievement motive ondivergent self-other attributions for success and failure. Perceptualand Motor Skills, 42,127-132.

Lowell, E. L. (1952). The effect of need for achievement on learningand speed of performance. Journal of Psychology, 33, 31 -40.

Lueptow, L. B. (1973). Need for achievement and continuation in col-lege. Psychological Reports, 33, 455-458.

McAdams, D. P., & Vaillant, G. E. (1982). Intimacy motivation andpsychosocial adjustment: A longitudinal study. Journal of Personal-ity Assessment, 46, 586-593.

McClelland, D. C, & Liberman, A. M. (1949). The effect of need forachievement on recognition of need-related words. Journal of Person-ality, 75,236-251.

McFall, R. M., & Schenkein, D. (1970). Experimenter expectancy ef-fects, need for achievement, and field dependence. Journal of Experi-mental Research in Personality, 4, 122-128.

McKeachie, W J. (1961). Motivation, teaching methods, and collegelearning. In M. R. Jones (Ed.), Nebraska symposium on motivation:1961 (pp. 111-142). Lincoln: University of Nebraska Press.

McKeachie, W J., Isaacson, R. L, & Milholland, J. E. (1968). Studentachievement motives, achievement cues, and academic achieve-ment. Journal of Consulting and Clinical Psychology, 32, 26-29.

Meade, R. D. (1966). Achievement motivation, achievement, and psy-chological time. Journal of Personality and Social Psychology, 4,577-580.

Melnick, M. (1974). Need achievement, need affiliation, and percep-tion of laterally in other people. Perceptual and Motor Skills, 38,467-470.

Mitchell,! V, Jr. (1961). An analysis of the factorial dimensions of theachievement motivation construct. Journal of Educational Psychol-ogy, 52,179-187.

Miyamoto, M. (1981). Instrumental activity in achievement motiva-tion. Japanese Psychological Research, 23, 79-87.

Morgan, H. H. (1952). A psychometric comparison of achieving andnonachieving college students of high ability. Journal of ConsultingPsychology, 16, 292-298.

Morgan, H. H. (1953). Measuring achievement motivation with "pic-ture motivations." Journal of Consulting Psychology, 17, 289-292.

Morris, J. L., & Fargher, K. (1974). Achievement drive and creativity ascorrelates of success in small business. Australian Journal of Psychol-ogy, 26,217-222.

Moulton, R. W (1965). Effects of success and failure on level of aspira-tion as related to achievement motives. Journal of Personality andSocial Psychology, 1, 399-406.

Nygard, R. (1982). Achievement motives and individual differences in

situational specificity of behavior. Journal of Personality and SocialPsychology, 43, 319-327.

O'Connell, A. N. (1980). Effects of manipulated status on perfor-mance, goal setting, achievement motivation, anxiety, and fear ofsuccess. Journal of Social Psychology, 112, 75-89.

Orso, D. P. (1969). Comparison of achievement and affiliation arousalon nAch. Journal of Protective Techniques and Personality Assess-ment, 33, 230-233.

Patrick, A. W, & Zuckerman, M. (1977). An application of the state-trait concept to the need for achievement. Journal of Research inPersonality, 11, 459-465.

Patten, R. L., & White, L. A. (1977). Independent effects of achieve-ment motivation and overt attribution on achievement behavior.Motivation and Emotion, 1, 39-59.

Ray, J. J. (1980). Effects of individual differences on productivityamong farmers in India. Journal of Social Psychology, 112, 11-17.

Raynor, J. O. (1970). Relationship between achievement-related mo-tives, future orientation, and academic performance. Journal of Per-sonality and Social Psychology, 15, 28-33.

Raynor, J. O., & Smith, C. P. (1966). Achievement-related motives andrisk-taking in games of skill and chance. Journal of Personality, 34,176-198.

Reiter, H. H. (1964). Prediction of college success from measures ofanxiety, ach ievement motivation, and scholastic aptitude. Psychologi-cal Reports, 15, 23-26.

Rodgers, C. W (1973). Relationship of projective to direct expression ofselected needs for nonpsychotic Ss. Perceptual and Motor Skills, 36,571-578.

Rosen, B. C. (1955). The achievement syndrome. American Sociologi-cal Review, 21, 203-211.

Sachs, P. R. (1982). Avoidance of diagnostic information in self-evalua-tion of ability. Personality and Social Psychology Bulletin, 82, 242-246.

Sampson, E. E. (1963). Achievement in conflict. Journal of Personality,31, 510-516.

Schroth, M. L. (1985). The effect of differing measuring methods onthe relationship of motives. Journal of Psychology, 119, 213-218.

Schultz, C. B., & Pomerantz, M. (1976). Achievement motivation, locusof control, and academic achievement behavior. Journal ofPersonal-ity, 44, 38-51.

Scodel, A., Ratoosh, P., & Minas, J. S. (1959). Some personality corre-lates of decision making under conditions of risk. BehavioralScience, 4, 19-28.

Scott, W A., & Johnson, R. C. (1972). Comparative validities of directand indirect personality tests. Journal of Consulting and Clinical Psy-chology, 38, 301-318.

Sedney, M. A., & Turner, B. F. (1975). A test of causal sequences in twomodels for development of career-orientation in women. Journal ofVocational Behavior, 6, 281-291.

Sherwood, J. J. (1966). Self-report and projective measures of achieve-ment and affiliation. Journal of Consulting Psychology, 30, 329-337.

Singh, N. P. (1969). rcAch among successful-unsuccessful and tradi-tional progressive agricultural entrepreneurs of Delhi. Journal ofSocial Psychology, 79, 271-272.

Singh, N. P. (1970). n/Ach among agricultural and business entrepren-eurs of Delhi. Journal of Social Psychology, 81, 145-149.

Singh, S. (1979). Relationships among projective and direct verbalmeasures of achievement motivation. Journal of Personality Assess-ment, 43, 45-49.

Skolnick, A. (1966). Motivational imagery and behavior over twentyyears. Journal of Consulting Psychology, 30, 63-478.

Smith, C. P. (1963). Achievement-related motives and goal settingunder different conditions. Journal of Personality, 31, 124-140.

Sorrentino, R. M., & Field, N. (1986). Emergent leadership over time:The functional value of positive motivation. Journal of Personalityand Social Psychology, 50,1091-1099.


Steers, R. M. (1975). Task-goal attributes, n achievement, and supervi-sory performance. Organizational Behavior and Human Perfor-mance, 13, 392-403.

Tasaki, T., & Seki, F. (1975). Effects of affiliation motive, achievementmotive, and pressure sources on conformity. Japanese PsychologicalResearch, 17, 174-181.

Teevan, R. C, & Smith, B. D. (1975). Relationship of fear-of-failure andneed achievement motivation to a confirming-interval measure ofaspirational levels. Psychological Reports, 36, 967-976.

Teglasi, H. (1978). Sex-role orientation, achievement motivation, andcausal attributions of college females. Sex Roles, 4, 381-397.

Trope, Y. (1975). Seeking information about one's ability as a determi-nant of choice among tasks. Journal of Personality and Social Psychol-ogy, 32, m*-\Q\?>.

Weiner, B., & Kukla, A. (1970). An attributional analysis of achieve-ment motivation. Journal of Personality and Social Psychology, 15,1-20.

Weiner, B., & Sierad, J. (1975). Misattribution for failure and enhance-ment of achievement strivings. Journal of Personality and Social Psy-chology, 31,415-421.

Weinstein, M. S. (1969). Achievement motivation and risk preference.Journal of Personality and Social Psychology, 13,153-172.

Weitzenkorn, S. D. (1974). An adjusted measure of achievement moti-vation for males and females and effects of future orientation onlevel of performance. Journal of Research in Personality, 8, 361-377.

Williams, J. M., Hoepner, B. J., Moody, D. L., & Ogilvie, B. C. (1970).Personality traits of champion female fencers. Research Quarterly,41, 446-453.

Winefield, A. H., & Jardine, E. (1982). Effects of differences in achieve-ment motivation and amount of exposure on responses to uncontrol-lable rewards. Motivation and Emotion, 6, 245-257.

Winefield, A. H., & Norris, P. J. (1981). Effects of exposure to uncon-trollable events as a function of achievement motivation and initialexpectation of success. Motivation and Emotion, 5, 235-248.

Winter, D. G., & Wiecking, F. A. (1971). The new Puritans: Achieve-ment and power motives of New Left radicals. Behavioral Science,16, 523-530.

Wolk, S., & Du Cette, J. (1971). Locus of control and achievementmotivation: Theoretical overlap and methodological divergence.Psychological Reports, 29, 755-758.

Yamauchi, H., & Doi, K. (1977). Factorial study of several motives.Psychological Reports, 41, 795-801.

Zander, A., & Forward, J. (1968). Position in group, achievement moti-vation, and group aspirations. Journal of Personality and Social Psy-chology, 8, 282-288.

Received October 25,1989Revision received August 3,1991

Accepted August 12,1991

validity of questionnaire and tat measure s of …vktonay/migrated/psyc166/tatmetanalysis.pdf ·...

Documents