aerj operational definitions

7/29/2019 AERJ Operational Definitions

http://slidepdf.com/reader/full/aerj-operational-definitions 1/20

American Educational Research Association

Operational DefinitionsAuthor(s): Robert H. EnnisSource: American Educational Research Journal, Vol. 1, No. 3 (May, 1964), pp. 183-201Published by: American Educational Research AssociationStable URL: http://www.jstor.org/stable/1162219

Accessed: 02/06/2009 19:48

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at

http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless

you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at

http://www.jstor.org/action/showPublisher?publisherCode=aera.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the

scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that

promotes the discovery and use of these resources. For more information about JSTOR, please contact [email protected].

American Educational Research Associationis collaborating with JSTOR to digitize, preserve and extend

access to American Educational Research Journal.

http://www.jstor.org

http://www.jstor.org/stable/1162219?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp

http://www.jstor.org/action/showPublisher?publisherCode=aera

http://www.jstor.org/action/showPublisher?publisherCode=aera

http://www.jstor.org/page/info/about/policies/terms.jsp

http://www.jstor.org/stable/1162219?origin=JSTOR-pdf



OPERATIONAL DEFINITIONS*

ROBERT H. ENNIS

CornellUniversity

INTRODUCTION

On all sides we are warned that our resultsdepend

on theinstrumentsand procedures used, and we are admonished to define our terms in a

manner that takes account of these instruments and procedures (oftenthis admonition specifies that the definition should be operational); yetwe want to express our conclusions in terms that are not limited to the

particular instruments and procedures. That sets my problem: Howcan we give operational definitions without unduly restricting the mean-

ing of the terms in which we state our conclusions?

In this paper I shall examine various forms that operational defi-

nitions might take and shall develop and defend a set of guides formaking these definitions. These guides will enable us to connect ourabstract terms to our instruments and procedures without completelylimiting the meaning of the terms to these instruments and procedures.

THE SPIRIT OF OPERATIONISM

An early expression of what has come to be called "operationism"is found in P. W. Bridgman's The Logic of Modern Physics: "The con-

cept of length involves as much as and nothing more than the set of

operations by which length is determined. In general, we mean by anyconcept nothing more than a set of operations; the concept is synony-mouswith the correspondinget of operations"(Bridgman,1927;p. 5).Although Bridgman, who is regarded as the father of operationism, goestoo far in this statement, the focus on instruments and procedures, whichis the essence of operationism, comes through clearly. This focus may beviewed as one of the empiricist and pragmatic trends of recent years,as A. C. Benjamin has shown in his interesting summary and appraisalof

Bridgman'sideas and their

development under criticism (Benjamin,1955).

* The preparationof this paper was supported through the Cooperative ResearchProgram of the Office of Education, United States Department of Health, Educa-tion, and Welfare. An earlier version was presented to the Philosophy of EducationSociety in San Francisco on April 8, 1963. I have profited from the criticisms ofProfessorsH. Broudy, H. Burns, J. Canfield,D. B. Gowin, J. Millman, F. Neff, andN. Champlin,and a number of my studentsat CornellUniversity.

183



184 ROBERTH. ENNIS

As a thesis, the spirit of operationism can be loosely put as follows:

There s an importantrelationshipbetweenthe meaningof a termand the

instrumentsand procedures hat one would use to see whetherthe termappliesto a particular ituationand, if so, how.

FORMS IN WHICH THIS SPIRIT HAS BEEN EXPRESSED

In the literature on operationism one finds four basic approaches to

operational definitions: 1) giving examples; 2) giving a set of operationsas the meaning of a concept; 3) equating a phrase or sentence containingthe term in question with a phrase or sentence about a combination of

operations and observations; and 4) providing implication relationshipsamong operations, observations, and the concept in question. The pro-

posed guides fit under the fourth approach, the description of which has

been vague because of the several variations possible.Since the defense of these guides rests heavily on showing the diffi-

culties of the first three approaches, difficulties that are avoided if one

follows the guides, we must look carefully at all four approaches.

1. GivingExamples

Although Bridgman had something more rigorous in mind, the use of

examples of abstract concepts sometimes indicates the instruments and

procedures involved when using the concepts and, at an unsophisticated

level, does provide an empirical interpretation. G. A. Lundberg, the

sociologist, indicates endorsement of the example approach in the followingstatement: "The simplest form of an operational definition of a wordis to point to its referent while enunciating the word. Thus we define theword 'cat' to a child by pointing to a certain kind of animal or a succession

of animals denoted by the word in our language" (Lundberg, 1942a; p.730).

Examples are very useful in clarifying terms because they connectthem to the concrete world-concreteness is one of the virtues of opera-tional definitions. But in giving an example, one does not necessarilyspecify a manipulation by an investigator, and thus exemplification missessome of the spirit of operationism. The "cat" example above does not

specify a manipulation by an investigator; instead it specifies particularcats.

Nevertheless, it would not be a serious error to treat examples as

operational definitions-provided that we have distinct names for whatwould then be two different kinds of operational definitions. Obvious,though rather wordy, names are "example-type operational definitions"and "manipulative operational definitions." For verbal economy I preferto mark the distinction with the terms "example" and "operational defi-

nition," both of which have established usages. Accordingly, exampleswould not be operational definitions.



OPERATIONAL DEFINITIONS 185

2. Giving a Set of Operations as the Meaning of a Concept

(Form: Concept = Operations)

This approach is suggested by a strict interpretation of Bridgman'sstatement quoted earlier. According to this approach, length means a set

of operations, such as putting down a ruler end over end and (presumably)

counting the number of times this is done. Length means what you do; it is

not a property of the thing you do it to. It does not mean what you find

out; it means what you do when finding out.

If we apply this approach strictly to a concept often used in social-

science research, IQ might mean administering and (presumably) scor-

ing the California Test of Mental Maturity* (henceforth referred to asCTMM). An IQ would not then be a quality of a child; the child's IQwould mean what you had done to him.

Another feature of this approach is that it implies as many concepts of

length (IQ, etc.) as there are ways of measuring it. According to Bridg-

man, the concept length in length of a city lot is different from the concept

length in length of a large piece of land if measuring sticks alone are used

for determining the first and measuring sticks plus triangulation for the

second, because the sets of operations are different (Bridgman, 1927; p.14). Similarly, there would be as many concepts of IQ as there are tests

for IQ and ways of administering them. If meaning is identical with

a set of operations, different operations imply different meanings. Bridg-man says, "If we have more than one set of operations, we have more

than one concept, and strictly there should be a separate name to cor-

respond to each different set of operations" (Bridgman, 1927; p. 10).

Thus, there are these two distinguishing features of this, the second,

approach to operationism: (a) the meaning of a concept is limited to the

operations, and (b) different operations imply different concepts. I shallcriticize each feature.

(a) To treat the meaning of a concept (or term) as limited to set of

operations seems odd, to say the least. When I say that the length of a

bench is five feet, I intend to be talking about the bench-not about what

I did. When I say that Johnny has a low IQ, I intend to be talking about

Johnny-not about what I did. If one interprets Bridgman's originalformula strictly, there seems no way to include the observations that one

makes, observations that reveal the qualities measured. One cannot talk

about the things measured; one can talk only about what the experimenterdoes-not about what he perceives.

It may be held that this reading of Bridgman's statements is unfair-that he did not mean them as I have presented them. Perhaps so. I do notsee how anyone could really mean them that way although some soci-

"* o simplify the presentation,I have not given the form, edition, or level of thetests discussed.



186 ROBERT H. ENNIS

ologists and psychologists may appear to have so taken them.* An ex-

amination of The Logic of Modern Physics and the works of these social

scientists reveals that it is hard to be sure just what they do mean. How-ever, my purpose here is not to criticize them but to clarify the ad-

vantages and disadvantages of various expressions of the operationist

spirit. Strictly interpreted, this, the second, approach to operationism

neglects the results one gets after doing the operations.

(b) There is no doubt that people (including Bridgman in The Logic ofModern Physics and most social scientists that I have read on the sub-

ject) have taken seriously the second feature of the second approachto operationism (that different operations imply different concepts), and

there has been considerable dispute on this point. I shall not go into it

thoroughly here because to do so would require systematic treatment of

the nature and content of scientific theories, about which there is a vast

literature. I shall only comment on the serious difficulties of this view.

These comments are carefully elaborated, however, because they con-

tribute substantially to refuting the third (and most popular) approachto operationism.

The first point to clarify is how to tell if two operations are actuallydifferent. Is

myadministration of the CTMM at 9:00 a.m. on

Monday,April 6, 1964, in Fall Creek School a different operation from my admin-

istration of the CTMM at 10:00 a.m. on Monday, April 6, 1964, in Belle

Sherman School? The only differences are the time and place of admin-

istration. To my knowledge, no satisfactory criterion, operational or

otherwise, has been provided by operationists of the concept different

operation. This is a significant weakness of the thesis that different opera-tions require different concepts. We do not know how different, or in what

respects different, the operations have to be to require different concepts.

Let us, however, assume that different times and places do not re-quire a judgment of different operations. But presumably the use of

measuring sticks and that of measuring sticks plus triangulation would

be different operations, as would the use of the CTMM and that of the

Lorge-Thorndike Intelligence Tests. Similarly, using the mercury ther-

mometer, the alcohol thermometer, and the resistance thermometer would

be three different operations.But why should there be two concepts of length, two concepts of IQ,

and three concepts of temperature to correspond to the various opera-

tions mentioned in the previous paragraph? Why not one concept of

length and two ways of measuring it? One concept of IQ and two waysof estimating it? One concept of temperature and three ways of deter-

mining it?

Let us consider the concept temperature. (Analogous things could be

said about length, but these things would be considerably more difficult.)

"*or examples, see Lundberg (1942b, p. 89), Stevens (1951, p. 28), and Marx(1951,p. 11).




What would be the consequences of having three concepts of temperature,

say M-temperature for the mercury thermometer, A-temperature for the

alcohol thermometer, and R-temperature for the resistance thermometer?Since there are many more ways of measuring temperature, there would,of course, have to be many more concepts.

One difficulty is that people would misunderstand someone who heldthis view since people think of temperature as one quality, which ismeasured in various ways. But this is not an insuperable objection. One

might stipulate three senses for the word "temperature" that represent

qualities highly correlated with one another and hold that these sensesshould be distinguished for scientists and sloppily merged for laymen.

This is a plausible answer; scientists are not bound by ordinary usage intheir research.

A second difficulty, however, is that physics itself would become im-

mensely more complicated. The law of thermal expansion would becomethree laws, one for each concept. (And since there are many more ways of

measuring temperature, there would have to be many more laws, where

previously we managed with only one.) Furthermore, the law based on

M-temperature would extend over a different range from that of the lawbased on R-temperature since the two thermometers have different

ranges. Not only convenience would be sacrificed, but also simplicity and

elegance.The attempt to explain differences in temperature would run into

difficulty. The model provided by the kinetic theory of heat explains them

by means of differences in the mean kinetic energy of molecules. Thereis just this one phenomenon, the mean kinetic energy of molecules, to

explain differences in temperature. Minor aberrations and inconsistenciesbetween instruments at extreme points have auxilary explanations, but

the fact remains that there is one underlying phenomenon associated withtemperature in physical theory, which implies that temperature is justone thing. (This discussion assumes ordinary contexts; later on, we shallconsider a special context with somewhat different results.)

Of course, we could conceivably elevate what I have called "auxiliaryexplanations" to the status of central explanations, in which case wecould have a different central explanation for the phenomenon associatedwith each instrument, but to do so would be inconvenient. The simplest ex-

planation, and the one that suffices in most cases, is the one that refers

only to the mean kinetic energy of the molecules.What about fruitfulness-the ability of theory to generate new pre-

dictions and to suggest new ways of looking at other fields? It is difficultto see how the complex structure that is implicit in the requirement thatdifferent operations imply different concepts could be a provoker of newideas or an aid to seeing new applications; it would be so hard to see thestructure as a whole, to see it intuitively.

While there is ordinarily only one concept of temperature functioning



188 ROBERT H. ENNIS

within our normal range of experience, the concept does become elusive

in extreme situations. When we try to measure temperature at points suc-

cessively farther from the earth's surface, different operations tend to callfor different concepts because different operations produce radically dif-

ferent readings at the same point and because our interest in hotness and

coldness out in space calls for great attention to sources of radiant heat.

The reading on a thermometer depends on whether the thermometer is

exposed to radiant heat and, if so, on the direction in which the thermom-

eter is aimed. This is something like the dependence of a reading on

whether a thermometer is in the sun or in the shade. There are needed,so to speak, a concept, shade-temperature, and a concept, directional-

radiant-temperature. A thermometer suspended in a reflecting spherewould indicate shade-temperature (to the extent that the sphere success-

fully reflects radiant energy). A thermometer in a reflecting tube openat one end (thus admitting radiant energy from one direction) would

indicate directional-radiant-temperature. Thus, under conditions that ob-

tain as we leave the earth, it becomes convenient to have two concepts of

temperature since different operations give answers that at times are

radically different and since our fundamental concern, hotness and cold-

ness,is

radicallyaffected

byradiant heat.

In summary, once we are willing to bypass the difficulties attendant

upon violating the conventions of everyday speech and ordinary technical

language, the difficulty with having a different concept for each operationof temperature measurement is the inconvenience. But under certain

conditions, given our interests, we have to accept this inconvenience be-

cause the results from two different methods of measurement disagree so

greatly. The conclusion is that different operations do not by themselves

imply different concepts although, under certain circumstances for cer-

tain purposes, they may call for different concepts.Now, do the same considerations apply to IQ? In ordinary use (among

teachers and others who use the concept but who are not scientists), IQis held to be one thing, estimated with varying degrees of validity by the

various intelligence tests. But this in itself is not sufficient reason for edu-

cational scientists to adopt a unitary concept. They might hold that the

matter is not that simple-the measures we have differ so much that, al-

though they contain common elements, they are not enough alike to war-

rant calling them measures of the same thing. At best, IQ tests correlate

around .8. Thus they have much in common, but not everything in com-

mon, and we know from analyzing them that they emphasize different

abilities, such as spatial reasoning, verbal facility, and numerical skill.

Furthermore, there is not yet an explanatory theory for IQ comparableto the kinetic theory of heat. In view of these facts, it would be safestto develop a number of concepts of IQ, one for each test. (This, in effect,is part of what people are doing who say that "intelligence" means what

is measured by a given intelligence test.)




But there still are the arguments for a unitary concept that depend on

simplicity, intelligibility, and fruitfulness. In view of these considera-

tions, I treat IQ as a single concept for research purposes in the CornellCritical Thinking Project, with which I am associated. IQ in this sense is

measured with some accuracy by the various tests and, in my experience,is well-enough related to the intellectual performance of people who are

similarly motivated and whose backgrounds are roughly the same.

This is a matter about which reasonable men can differ. One may

argue that the verbal score yielded by the Lorge-Thorndike IntelligenceTests emphasizes verbal ability more than does the score yielded by the

CTMM and therefore we should treat these scores as connected to differ-

ent concepts, perhaps verbal IQ and general IQ. If these scores correlatequite differently with the variable in which we are interested at this

time-the ability to think critically, we have evidence for treating them

as being connected to different concepts. The main point is that this is

a question that must be settled by appealing to the circumstances and

purposes involved. Since it makes sense to argue about the question even

after it is agreed that the operations are different, it follows that different

operations do not by themselves imply different concepts of IQ.

Again a difference in operationsby

itself does notimply

different con-

cepts, although different concepts are sometimes called for by different

operations in conjunction with certain circumstances. In spite of the fact

that the second approach to operational definitions is unsatisfactory on

the basis of the first feature alone, detailed examination of the second

feature was worth while because rejection of the claim that different

operations imply different concepts is a central part of the argument

against the third approach to operational definitions.

3.Equating

a Phrase or SentenceContaining

the Term With a Phrase

or Sentence About a Combination of Operations and Observations

This appears to be the approach of the radical behaviorists; for ex-

ample, B. F. Skinner (1953, p. 585). It is May Brodbeck's approach in

N. L. Gage's Handbook of Research on Teaching (Brodbeck, 1963; p.50) and may be the approach actually intended by Bridgman in The

Logic of Modern Physics. A first statement of this form follows:

3.1a. "X has the property T"meansthe same as "A given operationwas

performednda

givenobservationwas

made,"where T is, or contains, the term being defined. Example 1 below is an

operational definition of IQ according to Form 3.1a.

Example 1: "X has an IQ of n" means the same as "X was given theCTMMandreceiveda score of n."

Form 3.1a is limited to cases where T is in a sentence that is equated to

another sentence. I should like to generalize the form so that T can ap-



190 ROBERT H. ENNIS

pear in either a phrase or a sentence that is equated to another phrase or

sentence:

3.lb. "Tx"meansthe sameas "OPxandOBSx,"

where Tx is a phrase or sentence containing the term T to be defined;

OPx is a phrase or sentence about the performance of operations; and

OBSx is a phrase or sentence about observations. OPx and OBSx maybe merged into one phrase or sentence. Example 1 fits Form 3.1b, as does

example 2 below.

Example2: "A person's Q"means the sameas "a person'sscoreresulting

from the administration f the CTMM"(or, more simply, "a person'sscoreonthe CTMM").

The term (or concept) IQ is tied down to a particular intelligence test

in this, the third, approach to operational definitions; but, in contrast to

the second approach, an IQ is a characteristic of a person, not of the

operations performed, though it is still dependent on those operations.An immediate objection presents itself. In example 1, let us assume

that X actually has an IQ of, say, 120. If this is not known because he has

not yet taken the test, one is obliged to say that it is false that he has anIQ of 120 since the statement alleged to be identical in meaning is false;that is, it is false that X was given the CTMM, so it is false that he was

given it and got a score of 120. Hence, by this interpretation, it must be

false that he has an IQ of 120. This contradicts our assumption that he

has an IQ of 120. No matter what IQ we assume X to have, a contradic-

tion develops. Since he must have some IQ, the definition is faulty.A similar problem exists for the phrase approach. According to ex-

ample 2, a person who does not have a score on the CTMM does not have

an IQ at all since "score on the CTMM" and IQ mean the same thing. Ifthat were so, it would make no sense for a principal to ask his guidancecounselor to obtain IQ's for new students who had never before been

tested since, by this interpretation, they have no IQ's (because they have

no scores on the CTMM).A way out of this difficulty is to use between OPx and OBSx an im-

plication relationship that is expressed conditionally and can be put in

the subjunctive mood. The revised form is:

3.2. "Tx" means the same as "If OPx, then OBSx." Our sentence ex-amplebecomes:

Example3: "X has an IQ of n" meansthe same as "If X is (were) giventhe CTMM,he will (would)get a scoreof n."

Ourphrase example becomes:

Example 4: "A person's IQ" means the same as "the score he will

(would)get if giventhe CTMM."




There are problems remaining, but the use of implications that canbe put in the subjunctive* solves this first one. If a person has never

taken the CTMM-even if he is never going to take it, he neverthelesshas an IQ, perhaps never to be known. And if he has taken it and has a

score, there still is no inconsistency, for that does not preclude his being

given it again.In future examples, reference to the subjunctive will be omitted for

simplicity's sake. But it should be understood that each example is in-

tended to be convertible to the subjunctive-if need be. Furthermore,

phrase-type examples will no longer be given; with appropriate modifi-

cations, the general points to be made apply also to them.

Let us now consider another difficulty with the third approach. Itlimits the meaning of a concept to a particular set of operations (in the

case discussed above, the administration of the CTMM) together withthe results of this set of operations. Thus, like the second approach, it

implies that different operations require different concepts. As indicated

earlier, this view is generally false-although, under certain circumstancesand purposes, different operations do require different concepts.

What is needed is a format that will not commit us to having different

operations requiredifferent

conceptsin all

cases but will permit someleeway. We must replace the relationship, means the same as, with onethat does not limit the meaning to a particular set of operations. An im-

plication relationship that does not claim equivalence of meaning willmeet this requirement. This idea leads us to the fourth approach.

4. Providing Implication Relationships Among Operations, Observations,and the Concept

We can avoid thedifficulty

causedby

therelationship, means thesame as, by replacing it with the relationship, if and only if.t The

amended form follows:

4.1. Tx; if andonly if; if OPx,then OBSx.

The use of "if and only if" permits us to have two, or more, different

* See Chisholm (1949), Goodman (1952), von Wright (1957), and Will (1947) forinteresting discussions of counter-factualconditionals. This problem need not bother

us here unless we try to reduce all logical relationshipsto conjunction and negation(or somethingsimilar)-an unwise course,in my opinion.t This approach is along the lines recommended by Carl Hempel (1952, 1961),

who has applied Rudolph Carnap's (1953) notion of the reduction sentence to theformulation of operational definitions. My approachdiffers from theirs primarily inthe interpretation of the if-then relationship, theirs being a truth-functional inter-pretation and mine being an ordinary-language nterpretation.See P. F. Strawson'sIntroduction to Logical Theory (1952) for a discussion of the difference. If thereaderwho is unacquainted with this differencein approachsimply interpretsif-thensentences in the way to which he is accustomed, he will be interpreting these sen-tences as they are here intended.



192 ROBERT H. ENNIS

kinds of operations to measure the same thing. The following examplescan both be operational definitions of the same concept, temperature:

Example 5: X has a temperatureof t; if and only if; if a mercurythermometers inserted n X, the thermometerwillreadt.

Example 6: X has a temperatureof t; if and only if; if an alcohol

thermometers inserted n X, the thermometerwill readt.

If "means the same as" appeared in these definitions instead of "if and

only if," we should be committed to saying that "mercury thermometer"

means the same as "alcohol thermometer." In effect, the if-and-only-ifformulation allows several accurate ways of measuring the same thing.

If we apply this approach to the measurement of IQ, another prob-lem becomes prominent. There is much less agreement among IQ tests

than there is among thermometers. Even the if-and-only-if formulation

seems too rigid. Consider these two possible definitions of the concept

IQ:

Example7: X has an IQ of n; if and only if; if the CTMM is admin-isteredto X, X willget a scoreof n.

Example8: X has an IQ of n; if and only if; if the Lorge-Thorndike

IntelligenceTests are administeredo X, X will get a score of n.

Together these definitions commit us to saying that a person will getthe same score on the CTMM as on the Lorge-Thorndike. For at least

two reasons we do not want to be fully committed to this.

First, the conditions of administration may not be standard for both

tests in any given pair of situations. If one test is administered under

standard conditions and the other is not, we can hardly expect the same

scores. This difficulty can be handled by adding some such phrase as

"under standard conditions." It is desirable that a list of standard condi-tions be available, perhaps in the test manual. Incidentally, this same

qualification holds for thermometers when precision is necessary.

Second, each instrument has its idiosyncrasies, which inevitablyinterfere with measurement at some level of precision. If these are large

enough to make a practical difference, some word like "approximately"should be added to the operational definition. Since the idiosyncracies of

IQ tests do make a practical difference at the level of precision at which

they are used, "approximately" should probably be added to operational

definitions of IQ. I should not ordinarily add it to operational definitionsof temperature since the idiosyncracies of thermometers do not make a

significant difference in the contexts with which I am familiar. However,for certain purposes and situations, some qualifying word should beadded for thermometers also.

To remind us of the qualifications, we can add "(WQ)," which standsfor "with qualifications," to Form 4.1:

4.2. Tx; if and only if: if OPx,then OBSx (WQ).



OPERATIONAL EFINITIONS 193

An operational definition of IQ might look like this:

Example9: X has an IQ of approximatelyn; if andonly if; if the CTMM

is administeredo X understandardconditions,X willget a scoreof n.

In this example, "approximately" was inserted in the clause containingthe concept being defined, IQ. This is the proper place for the qualifierwhen we are trying to judge what someone's IQ is; the method of deter-

mination gives an approximation. On the other hand, when we are reason-

ing from an assumption about what someone's IQ is to a predicted

score, "approximately" should appear in the clause about the score:

Example10: X has an IQ of n; if and only if; if the CTMMis adminis-

tered to X understandardconditions,X will get a scoreof approximatelyn.

Thus, the location of the qualifiers, as well as the decision about whether

to make them explicit, depends to some extent on the situation. Hence-

forth I shall use "(WQ)" in the examples, leaving placement of the

qualifiers to be determined by the context.

People who are not adept at dealing with complicated if-then rela-

tionships tend to find Form 4.2 hard to understand. Some read into it

the suggestion that a person who has not been given the test does not

have an IQ. They also are puzzled by the three occurrences of the word"if" so close together. A more understandable formulation follows:

4.3. If OPx; then Tx,if andonlyif, OBSx(WQ).

Example 11: If the CTMM is administered o X under standardcondi-

tions; then X has an IQ of n, if and only if, he gets a scoreof n (WQ).

Although this new formulation is easier to understand, it does not

say quite the same thing as Form 4.2. To see this, consider the situation

in which we are trying to make a judgment about an individual's IQ onthe basis of his score on the CTMM. Using 4.3, we can conclude that X

probably has an IQ of approximately n, where n is the score that we

have. However, using 4.2, we cannot draw that conclusion without an

auxiliary assumption: our conclusion that X has an IQ of approximatelyn depends on the generalization that n is the score he gets whenever he

takes the test; it does not rest simply on his getting the score of n this

time. To make this generalization, one would probably assume that thescore we have is typical for him, an assumption we often make in dealing

with test scores. Form 4.3 has this assumption built in, so we should notuse 4.3 unless we are fairly confident that the scores on which we are

basing our judgments are typical.Neither formulation allows us to escape the problem of typicality.

The problem is faced in applying 4.2 and in adopting 4.3. Since we shallwant to adopt 4.3 without being completely committed to the belief thatall the scores are typical, the word "probably" should be included amongthe qualifiers referred to by "(WQ)."



194 ROBERT H. ENNIS

Forms 4.2 and 4.3 suffice for the majority of cases calling for an

operational definition of a concept (or term). Why do they not suffice

for all cases? The trouble is that they commit us to giving the sameconditions as both necessary and sufficient for the use of the phrase

containing the concept. For example, given the administration of the

CTMM under standard conditions, a student's getting a score of n is

roughly a necessary and sufficient condition for saying that probably his

IQ is approximately n. That we sometimes want to avoid such a com-

mitment can be seen in a situation that faced the staff of the Cornell

Critical Thinking Project. The discussion will use 4.3 as a springboard,but with appropriate modifications I could say the same things starting

from 4.2.We built a test called The Cornell Conditional Reasoning Test, Form

X. Among its seventy-two items are six that supposedly test for masteryof the principle that denial of the consequent implies denial of the

antecedent; in other words, knowledge that the following form is valid:

p implies q (p is the antecedent; q the consequent)

q is false (denial of consequent)

therefore, p is false (conclusion: denial of antecedent)Here is one of the six items:

29. Supposeyou know that

If the bicycle n the garage s Bob's,thenit is red.

Thebicycle n the garage s not red.

Then wouldthisbe true?

Thebicycle n the garage s not Bob's.

A. YES It mustbe true.B. NO It can't be true.

C. MAYBE It may be true or it may not be true. You weren'ttold enough to be certain whether the answer is

YES or NO.

The correct answer is YES.

The other five items embodying this principle vary in several ways:the valid conclusion is denied (making the answer NO); the content is

abstract; the premises are reversed and the antecedent negated; the

truth status of the conclusion is obviously different from the validity

status of the argument. The six items are numbered 8, 16, 22, 29, 35,and 39.

Our problem, somewhat simplified, was how to give an operationaldefinition of "mastery of the principle that denial of the consequent

implies denial of the antecedent." One approach is to change the concept.As stated, it refers to a dichotomous variable, mastery, whereas we

could shift to a continuous variable, degree of mastery. (IQ and tempera-

ture are continuous variables.) The revised question would be: "What is




an operational definition of 'degree of mastery of the principle thatdenial of the consequent implies denial of the antecedent'?" We can

answer that question with the following operational definition:

Example 12: If X is given The Cornell ConditionalReasoning Test,Form X; then X has masteredto the degreek the principlethat denial ofthe consequent mplies denial of the antecedent,if and only if, he gets ascoreof k righton the following tems: 8, 16, 22, 29, 35, and 39 (WQ).

Using this definition, we obtain a score that can vary from zero tosix and that indicates degree of mastery of the principle. But this scorehas limitations. It means little to people who are not well acquainted

with the test or with the scores of other individuals whose degree ofmastery of this principle is known. Yet the audience for our results is

likely to consist largely of such people.Another drawback is that one of our interests is the determination

of the per cent of students of a given description who have mastered the

principle. This interest calls for the judgment that a particular student

has or has not done so; it does not call for a judgment about his degreeof knowledge. We need a definition that will fit our attempts to judgewith some assurance whether a student has mastered this

principle.One might suppose that these difficulties could be handled by a defi-nition that gave a certain minimum score as a necessary and sufficientcondition for having mastered the principle. For example,

Example13: If X is given The CornellConditionalReasoningTest, Form

X; then X has masteredthe principlethat denial of the consequent mpliesdenial of the antecedent, f and only if, X answerscorrectlyat least four oftheseitems: 8, 16, 22,29, 35,and39 (WQ).

In thisexample, getting at least four items right is a rough necessary-and-sufficient condition for a person's knowing the principle (we assume

that he takes the test). However, we do not want to be committed to

any one minimum score as both necessary and sufficient; we do not wantto draw that sharp a line between mastery and nonmastery. What weshould like to say is something to the effect that getting at least five rightis a probable sufficient condition and getting at least four right is a

probable necessary condition. That is, we should like to say of a studentwho gets at least five right that he probably has mastered the principle

and of a student who gets fewer than four right that he probably has notmastered it. About the students who get exactly four right, we are notsure what to say. They are borderline cases-and we should like a defi-nition form that will allow us to leave them that way.

The following form permits us to present the sufficient condition:

4.4. If OPx; then, if OBSx, then Tx (WQ).** The sufficient-conditionform correspondingto 4.2 is: Tx; if; if OPx, then

OBSx (WQ). This form, with the adjacent "if's,"is harderto understand.



196 ROBERT H. ENNIS

The operational definition corresponding to this sufficient-condition

form follows:

Example 14: If X is given The Cornell ConditionalReasoning Test,FormX; then, if X answerscorrectlyat least five of items 8, 16, 22, 29, 35,and 39, X has mastered the principlethat denial of the consequent mpliesdenialof the antecedent(WQ).

The following form permits us to present the necessary condition:

4.5. If OPx; then, Tx,only if OBSx(WQ).*

The operational definition corresponding to this necessary-conditionform follows:

Example 15: If X is given The Cornell ConditionalReasoning Test,Form X; then X has masteredthe principlethat denial of the consequentimpliesdenialof the antecedent,only if X answerscorrectlyat least four oftheseitems: 8, 16,22,29, 35,and 39 (WQ).

To give an operational interpretation of the concept, knowledge that

denial of the consequent implies denial of the antecedent, we supply both

operational definitions, each of which provides a partial interpretation.

Combined, they still do not provide a complete interpretation of the

concept, but they give a basis on which to work and reason.It is not necessary to agree with our specific decisions in the case

above to see the need for Forms 4.4 and 4.5. That is, without contradict-

ing my basic thesis, one can hold that answering correctly a minimum of

one certain number of the items should be considered both necessary and

sufficient, or one can hold that the number of items in the necessarycondition should differ from that in the sufficient condition but that fourand five are not the proper numbers. It is necessary only to see that these

formsmay

sometimes be needed.

In summary, I recommend that operational definitions start with an

if-clause that specifies an operation or a set of operations and that this

clause be followed by an implication relationship between a phrase or

sentence containing the concept (or term) to be defined and a phrase or

sentence specifying an observation or a set of observations. Appropriatequalifications should be implicit or explicit.

OPERATIONAL INTERPRETATION VERSUS OPERATIONAL DEFINITION

Now that all these qualifications have been introduced, one maywonder if the result is a definition at all. It does not exhaust the meaningof a concept; it is loose; and a pair of operational definitions of the same

concept sometimes implies an empirical fact. For example, the two defi-nitions of IQ using the CTMM and the Lorge-Thorndike imply a highcorrelation between the tests; this is certainly an empirical matter. Some

"*The necessary-conditionform corresponding to 4.2, which again is harder tounderstand, s: Tx; only if; if OPx, then OBSx (WQ).




people regard the implication of empirical facts as a serious defect, for

presumably definitions should give the meaning of concepts, not give

facts.Fortunately, the utility of the guides that I am proposing does not

depend on the resolution of this difficult question. If one boggles at callingthe examples I have given "operational definitions," call them "opera-tional interpretations." In either case they help indicate the meaning ofa concept, and they do so by focusing on concrete things, especially the

manipulations of investigators. They help both to delimit and to fill inthe meaning of a concept. What harm is there in calling them "defini-tions" so long as we remember that they differ from the classical type

of definition, which provides expressions that are equivalent in meaning?Since their purpose and function is to indicate meaning, the most reason-able term for them, it seems to me, is "operational definitions." But Ido not insist on this, for not much turns on the terminology.*

THENECESSITYORDELIBERATEANIPULATION

Carl Hempel, among others, has suggested that a deliberate manipula-tion by an investigator is not really necessary for an operational defini-

nition-that all we need is some condition, whether it be a deliberatemanipulation or not. He points out that although emphasis on deliberate

operations "is of great interest for the practice of scientific research,...it is inessential in securing experimental import for the defined term"

(Hempel, 1961; p. 59). It is true that a deliberate manipulation is not a

necessary condition for experimental import, but there is an importantdistinction in discussions of the methodology of scientific research be-tween those definitions that specify manipulations and those that specifyother conditions.

If we were proceeding on the assumption that all terms must be

operationally defined, Hempel's advice should be heeded because notall terms require manipulation by an experimenter as part of their

interpretation (unless such activities as looking are regarded as manip-ulations, in which case operationism reduces to empiricism). PerhapsHempel offered his advice in the context of an assumed recommendationthat all terms be operationally definable. If one does not make that

recommendation, one can preserve an independent meaning for theterm

"operationism,"a

meaning that emphasizes the manipulations ofan investigator. In this case, a recommendation that a term be defined

* The philosopher would say that the sharp distinction between analytic andsynthetic statements is blurred by using the term "definitions." The question is adifficult and subtle one, but I might note that the sharpness of this distinction inthe area of empirical science has been questioned recently even in the writings ofCarl Hempel, who says (1961, p. 66), "It . . appears doubtful whether the dis-tinction between analytic and synthetic sentences can be effectively maintained ina formal model of the language of empirical science." Keith Donnellan (1962) hasprovideda valuable discussion of this question.



198 ROBERTH. ENNIS

operationally would imply that the definition should specify some bona

fide manipulations by the research worker.

As with the case in which it was possible (but not preferable) to callexamples "operational definitions," I recommend that we preserve the

independent meaning of "operational definition" and that some other

term, perhaps "conditional definition," be used to cover both operationaldefinitions and definitions that are similar in form but contain a non-

manipulative condition in the first if-clause. This approach will preservethe necessary distinctions without violating the original spirit of opera-tionism as expressed by Bridgman and recognized for its value by many

empirical scientists.

SUMMARY

In this paper I have examined various forms for the operational defi-

nition of concepts, or terms, and have formulated the following set of

guides:

A. Operational definitions should

1. start with an if-clause specifying the nature of the operation

performable bythe investigator.

2. contain an implication relationship that holds when a given

operation has been performed. This relationship can be

necessary (but not sufficient), sufficient (but not necessary),or both necessary and sufficient.

3. be convertible to the subjunctive mood if they are not

already in the subjunctive.4. not be taken to require a separate concept for each opera-

tional definition. Some concepts will have many operational

definitions.5. contain, either explicitly or implicitly, qualifying words or

phrases like "approximately," "probably," and "under stand-

ard conditions."

B. Three useful forms for operational definitions follow:

Let: Tx represent a phrase or sentence containing the term

(or concept) T being defined. For example, "X has

an IQ of n," in which IQ is the concept.

OPx represent a phrase or sentence about the perform-ance of an operation or a set of operations. For ex-

ample, "X is (were) given the CTMM."

"*Rudolph Carnap (1953) has suggested "reduction sentence," a term in wide-spread use among philosophers. Because this term is closely associated with truth-functional logic and suggests that abstract concepts are reduced to concrete termswithout loss (this is not Carnap's intent), I prefer "conditional definition." Thisterm is free from these connotations and indicates the conditional aspect of thedefinition.



OPERATIONAL EFINITIONS 199

OBSx represent a phrase or sentence about an observa-

tion or a set of observations. For example, "X's

score is (will be, would be) n."WQ indicate that certain qualifications like "approx-

imately" and "probably" should be included in the

definition.

Form 1. In which OBSx is a necessary and sufficient condition,

given OPx:

If OPx; then Tx, if and only if, OBSx (WQ). (4.3)

Example: If X is given the CTMM; then X has an IQ of n, if

and only if, X's score is n (WQ).

Form 2. In which OBSx is a sufficient condition, given OPx:

If OPx; then, if OBSx, then Tx (WQ). (4.4)

Form 3. In which OBSx is a necessary condition, given OPx:

If OPx; then Tx, only if OBSx (WQ). (4.5)

It was not claimed that all definitions in the empirical sciences should

be operational. It was assumed that it is often a good idea to define

concepts (or terms) operationally because the specific connections al-leged between the concrete world and an abstract concept are importantand because an especially important set of these connections involves

the particular instruments and procedures used by the investigator.

REFERENCES

BENJAMIN, A. CORNELIUS. Operationism. Springfield, Ill.: Charles C.

Thomas, 1955. 154 pp.

BERGMANN, GUSTAV. "Sense and Nonsense in Operationism." The Vali-dation of Scientific Theories. (Edited by Philipp G. Frank.) New

York: Collier Books, 1961, pp. 46-56.

BERGMANN, GUSTAV, and SPENCE, KENNETH W. "Operationismand

Theory Construction." Psychological Theory. (Edited by Melvin H.

Marx.) New York: Macmillan Co., 1951, pp. 54-66.

BRIDGMAN, PERCY W. The Logic of Modern Physics. New York: Mac-

millan Co., 1927. 228 pp.

BRIDGMAN, PERCY W. "The Present State of Operationalism." The Vali-

dation of Scientific Theories. (Edited by Philipp G. Frank.) New York:Collier Books, 1961, pp. 75-80.

BRODBECK,MAY."Logic and Scientific Method in Research on Teaching."Handbook of Research on Teaching. (Edited by Nathaniel L. Gage.)

Chicago: Rand McNally and Co., 1963, pp. 44-93.

CARNAP, RUDOLPH. "Testability and Meaning." Readings in the Philos-

ophy of Science. (Edited by Herbert Feigl and May Brodbeck.) New

York: Appleton-Century-Crofts, 1953, pp. 47-92.



200 ROBERT H. ENNIS

CHAPIN, F. STUART. Contemporary American Institutions. New York:

Harper and Brothers, 1935. 423 pp.

CHAPIN, F. STUART. "Definition of Definitions of Concepts." Social Forces18: 153-60; December 1939.

CHISHOLM, RODERICK M. "The Contrary-to-Fact Conditional." Readingsin Philosophical Analysis. (Edited by Herbert Feigl and Wilfrid

Sellars.) New York: Appleton-Century-Crofts, 1949, pp. 482-97.

Coombs, Clyde H. "Theory and Methods of Measurement." Research

Methods in the Behavioral Sciences. (Edited by Leon Festinger and

Daniel Katz.) New York: Dryden Press, 1953, pp. 471-535.

DODD, STUART C. "Operational Definitions Operationally Defined." Amer-

ican Journal of Sociology 48: 482-89; January 1943.DODD,STUARTC. "A System of Operationally Defined Concepts for

Sociology." American Sociological Review 4: 619-34; October 1939.

DONNELLAN, KEITH S. "Necessity and Criteria." Journal of Philosophy59: 647-58; October 25, 1962.

FEIGL, HERBERT. "Operationism and Scientific Method." Readings in

Philosophical Analysis. (Edited by Herbert Feigl and Wilfrid Sellars.)New York: Appleton-Century-Crofts, 1949, pp. 498-509.

FRANK,PHILIPP

G.,editor. The Validation

of ScientificTheories. New

York: Collier Books, 1961. 220 pp.

GOODMAN, NELSON. "The Problem of Counterfactual Conditionals." Se-

mantics and the Philosophy of Language. (Edited by Leonard Linsky.)

Urbana, Ill.: University of Illinois Press, 1952, pp. 231-46.

GRUiNBAUM, ADOLF. "Operationism and Relativity." The Validation of

Scientific Theories. (Edited by Philipp G. Frank.) New York: Collier

Books, 1961, pp. 83-92.

HEMPEL, CARL G. Fundamentals of Concept Formation in Empirical

Science. International Encyclopedia of Unified Science, Vol. 2, No. 7.Chicago: University of Chicago Press, 1952. 93 pp.

HEMPEL, CARL G. "A Logical Appraisal of Operationism." The Validation

of Scientific Theories. (Edited by Philipp G. Frank.) New York:

Collier Books, 1961, pp. 56-69.

KOCH, SIGMUND,editor. Psychology: A Study of a Science, Vol. 3. NewYork: McGraw-Hill Book Co., 1959. 837 pp.

LANGFELD, HERBERT S., editor. "Symposiumon Operationism."Psycho-

logical Review 52: 241-94; September 1945.LAZARSFELD, PAUL F. "Latent Structure Analysis." Psychology: A Study

of a Science, Vol. 3. (Edited by Sigmund Koch.) New York: McGraw-Hill Book Co., 1959, pp. 476-543.

LUNDBERG, GEORGE A. "Operational Definitions in the Social Sciences."American Journal of Sociology 47: 727-43; March 1942a.

LUNDBERG, GEORGE A. Social Research. Second edition. London: Long-

mans, Green and Co., 1942b. 426 pp.




AIARGENAU,HENRY. "Interpretations and Misinterpretations of Opera-

tionalism." The Validation of Scientific Theories. (Edited by Philipp

G. Frank.) New York: Collier Books, 1961, pp. 45-46.MARX, MELVIN H. "The General Nature of Theory Construction." Psy-

chological Theory. (Edited by Melvin H. Marx.) New York: Mac-

millan Co., 1951, pp. 4-19.

SCHLESINGER, G. "P. W. Bridgman's Operational Analysis: The Differ-

ential Aspect." British Journal for the Philosophy of Science 9: 299-

306; February 1959.

SJOBERG, GIDEON. "Operationalism and Social Research." Symposium on

Sociological Theory. (Edited by Llewellyn Gross.) Evanston, Ill.:

Row Peterson and Co., 1959, pp. 603-27.SKINNER, BURRHUS F. "The Operational Analysis of Psychological

Terms." Readings in the Philosophy of Science. (Edited by Herbert

Feigl and May Brodbeck.) New York: Appleton-Century-Crofts, 1953,

pp. 585-95.

STEVENS, S. SMITH. "Psychology and the Science of Science." Psycho-

logical Theory. (Edited by Melvin H. Marx.) New York: Macmillan

Co., 1951. pp. 21-54.

STRAWSON, PETER F. Introduction to Logical Theory. London: Methuen

and Co., 1952. 266 pp.

VALOIS, A. JOHN. A Study of Operationism and Its Implications for Edu-

cational Psychology. Washington, D. C.: Catholic University Press,1960: 162 pp.

WILL, FREDERICK. "The Contrary-to-Fact Conditional." Mind 56:

236-49; July 1947.

WRIGHT, GEORG H. VON. Logical Studies. New York: Humanities Press,1957. 195 pp.

aerj operational definitions

Documents