chapter title: nonreactive methods - university of … readings...nonreactive methods 6 tivity...

Nonreactive methods 1

First draft Fritsche, I., & Linneweber, V. (2004). Nonreactive (unobtrusive) methods. In M. Eid & E. Diener

(Eds.), Handbook of Psychological Measurement - A Multimethod Perspective (Theo-retical Foundations, Methodological Approaches, Methods, Applications). Washington, D.C.: American Psychological Association (APA).

Chapter title: NONREACTIVE METHODS

Nonreactive Methods

Immo Fritsche

(Friedrich-Schiller-Universität Jena)

Volker Linneweber

(Otto-von-Guericke-Universität Magdeburg)


NONREACTIVE METHODS

1 The Phenomenon of Reactivity

Participants in social science research usually think. That is, they interpret the

actions of a researcher and relate it to their own beliefs, emotions and inten-

tions. Accordingly, their behavior exhibited during investigations is controlled by

what they think is appropriate depending on their interpretation of the study

situation and their motivation to comply with these suspected requirements.

Hence, then what is often measured is not a “natural state” but the participant’s

intentional presentation. Assume inviting participants to a study on helping be-

havior in which you are interested in the individual inclination to help others in

need. You ask the subjects to indicate whether they would be willing to donate

blood for a charitable organization. However, in many cases the answer might

not say a lot about the subject’s actual behavior and neither about correspond-

ing intentions, but rather about proper understanding of the study’s demand

characteristics.

However, even if we adopt the more optimistic view that research partici-

pants do not think (even not unconsciously), the validity of our data obtained in

investigations specifically designed to record the variable of interest, might still

be doubtful. Since, at least from the researcher we must expect thinking, she1

will have arranged the research setting in a way that allows measuring a spe-

cific construct conveniently. Unfortunately, even the non-thinking participant’s

willingness to donate blood does not have to say a lot about his daily conduct. It

can be supposed that in daily life, your subject will never get in contact with

1 If nothing else is indicated then gender specifying expressions refer to both sexes,.


helping-appeals like the one you were kind enough to present in your investiga-

tion.

Both kinds of participant behavior can be called “reactive”, as they repre-

sent the subject’s reaction to a specific situation, intentionally created for re-

search by a researcher. Here, bias occurs not only as the fault of participants

(subject bias) but also due to the influence of the researcher (experimenter or

observer bias). Reactive measurement restricts what Brunswik (1947) called the

data’s “ecological validity”. Although not yet ultimately defined, high ecological

validity indicates that results from an investigation may predict the item’s behav-

ior in its ordinary context to a great extent.

Violations of the demand for ecological validity due to reactive measure-

ment have been described by various authors. One of the most popular exam-

ples is the comprehensive monograph in which Rosenthal (1976) systematically

elaborates experimenter effects in psychology. Rosenthal’s intention is to dem-

onstrate that psychology at least occasionally is unaware of – what have later

been called – the “Rosenthal effect” in psychological experimentation and the

“Pygmalion effect” in education (McNatt, 2000).

Psychology has tried to cope with the shortcomings of reactive methods

by introducing various measures directed at the prevention of subject and ex-

perimenter biases. Probably the most radical approach has been the proposal

of “nonreactive” or “unobstrusive” measures, first stated in the ground-breaking

work of Webb, Campbell, Schwartz & Sechrest (1966). They systematically

start discussing what has been called “reactive effect of measurement” and “re-

active arrangement bias” before (p. 13).The content of this book’s chapters indi-


cates what was of interest, when psychology started discussing alternatives to

traditional measurement: As physical traces, natural erosion and accretion

measures are discussed. Also archives, like various public and private records

are suggested to be utilized. Finally, the potential of simple and contrived ob-

servation methods for unobtrusive measures are elaborated. More than ten

years later, in a reader edited by Lee Sechrest (1979), Bochner (1979) adds

some considerations concerning unobtrusive field experiments in social psy-

chology. This indicates that particularly in topics sensible for social desirability

as well as in applied research, further developments occurred.

2 Approximation Towards Nonreactivity

Now, as we have introduced the phenomenon of reactivity and have briefly

mentioned the development of nonreactive techniques, we will elaborate in

more detail on the nature of nonreactive measurement. Following this, we give

an overview of classical as well as recently developed techniques and sketch

out the boundaries of nonreactive measurement.

Nonreactive Measures as Distinct Techniques

In the literature, the term “nonreactive measurement” is used in at least

two senses, in a dichotomic as well as in a continuous sense. Textbooks often

think of nonreactive measures as representing a distinct set of procedures,

sharply different from reactive methods. There is a comparatively stable core of

measures which are commonly subsumed under this heading. For instance, in

social psychology, one of the most cited and hence most prototypical non-


reactive method is the so called “lost letter technique” (Milgram, Mann, & Harter,

1965) used as an indirect measure of attitudes. Here, stamped letters are dis-

tributed in specified residential areas, appearing to be lost by someone. Their

inscriptions indicate the address of different organizations, representing the atti-

tude objects. For example, if the researcher is interested in the relative approval

of religious groups like Christians and Moslem, not biased by social desirability

concerns, she could “loose” letters addressed either to a church or to a mosque.

Actually, the letters are addressed to the researcher via post office. The amount

of letters she receives is taken as an indicator of the attitude against the respec-

tive group in a specified area. Although, the interpretation of this method might

be flawed with some problems (see below), this technique guarantees non-

reactive measurement in so far as the participants are not aware of their partici-

pation in an attitude test. This is the most important criterion of nonreactivity,

identified by Webb, Campbell, Schwartz, and Sechrest (1966) in their classic

work on unobstrusive measures. This term is applied to those measures in

which the studied individual “is not aware of being tested and there is little dan-

ger that the act of measurement will itself serve as a force for change or elicit

role-playing that confounds the data” (p. 175). Unobtrusiveness and nonreactiv-

ity are often used as synonyms. However, the term of nonreactivity should be

differentiated from unobtrusiveness in two ways. At first, nonreactive measure-

ment should be defined in a more comprehensive way than it has been done for

unobstrusive techniques (compare Folger & Belew, 1985), not only focusing on

whether and how research subjects perceive the act of measurement but also

having in mind the researcher and her potential contribution to instrument reac-


tivity (experimenter or observer effects, e.g. Rosenthal, 1976). That is, ideally,

also the researcher should not be aware of the measurement when it occurs. As

a second criterion of a definition, it should be made clear, that nonreactivity re-

fers to the ultimate outcome of measurement and not to the means by which

this is achieved (e.g. that subjects are not aware of being tested). Hence, we

define nonreactive measures simply as those measures in that participant’s be-

havior is not influenced by social interaction with the researcher, since both di-

rections of this interaction are perceived as potential sources of distortion. How-

ever, although measures like the lost-letter-technique are commonly perceived

as nonreactive, in many of these measures an interaction of researcher and

research subject is present and might influence participant’s behavior in an indi-

rect fashion. With regard to our example, the letters are not really lost but are

precisely placed by researchers. Time and place of the letter distribution as well

as the way of inscription follow intentional action by the researcher that might

subtly influence the participant’s behavior in the sense of “cuing” (Folger &

Belew, 1985). One might be able to counteract this potential distortions by bal-

ancing time, place and research assistant according to assumed confounded

variables like the economic background of the sample, social status or political

attitudes of the assistants. However, although those additional steps would be

likely to reduce reactivity, they are not a unique part of the technique itself. Fur-

thermore, those additional techniques which reduce measurements’ reactivity

are incorporated in most psychological methods of data acquisition. Hence, in-

stead of thinking of reactive and non-reactive measures in a dichotomous way,

a continuous definition of nonreactivity might be more suitable.


Nonreactivity as a Continuous Concept

A continuous concept assumes nonreactivity being an ideal state, rather

than a feature inherent in certain techniques of measurement. Here, different

measures come close to this ideal to a different degree. According to this no-

tion, reactivity of a measure increases with the extent to which both research

subject and researcher are involved in the act of measurement.

Insert Table 1 about here

Type 5. Data generated without any initial intention of measurement on

the side of the researcher and hence also without any awareness of such an

intention on the side of the participant, represent the most nonreactive and un-

obstrusive method that can be thought of (table 1, type 5). Examples for such

measures are the various natural accretion and erosion measures (Webb et al.,

1966) referring to material and settings, not placed or designed for research

purposes. Those measures use physical „traces“ of behavior that might mani-

fest as remnants like garbage, indicating certain life styles or can be found in

erosion phenomena like wild path’s in the snow showing preferred routes. An-

other example of type 5 measures (table 1) is the content analysis of archival

material. It may be used as nonreactive measure if it is guaranteed that the re-

cords represent natural behavior or if at least the conditions and original aims of

potentially reactive measurement are known to the researcher and are assumed


not having distorted measurement in a way that could influence the results of

present research.

Accordingly as data base for the measurement of individual health for in-

stance, using archival self-report data on individual absenteeism should be less

reliable than analyzing individual costs statistics of a health insurance. Of

course, such analyses are not immune against interpretational bias exhibited by

the researcher, but the mere generation of the data is indeed free of researcher

bias.

Type 4. Table 1 specifies further types of measurement that exhibit dif-

ferent lower degrees of nonreactivity. Type 4 measures do not make use of

naturally occurring records but rather measure behavior in a deliberately se-

lected setting or even create opportunities for subjects to behave in a certain

way. This behavior represents a top-down operationalization of variables the

researcher is interested in and has assumptions about. However, in this type of

measures, participants are not aware of being object of measurement. Exam-

ples for those techniques are hidden observation, many of the controlled accre-

tion and erosion measures (Webb et al., 1966), as well as the lost letter tech-

nique mentioned above. In the latter method a behavior is recorded that would

not have occurred naturally and without the intervention of the researchers,

simply because the letter to the mosque would not have layed in front of the

post box. Above, we have discussed possible experimenter effects that might

come into play when applying the lost letter technique. Of course, type 4 meas-

ures also have important advantages over the use of already existing records. If

for example a researcher is interested in inferences of causal relationships be-


tween pre-defined variables, she must be able to manipulate specific features of

the subject’s environment in order to ascribe differences in the dependent

measure to the work of specific experimental conditions (see also chapter [16:

Methods of Experimental Psychology]). Take for instance the director of an arts

museum who has the assumption that children are more attracted by colored

pictures than adults and that adults opposed to children are more interested in

looking at pictures of high unconventionality. In order to test both hypotheses in

a 2 (picture colored vs. black and white) x 2 (picture of high and low unconven-

tionality) x 2 (adults vs. children) factorial design the director could firstly equip

one exhibition room with pictures representing combinations of the respective

conditions (colored and high unconventionality, colored and low unconventional-

ity, black/white and high unconventionality, black/white and low unconventional-

ity). In a second step, he had to measure the attention each picture receives

and also if it is spent by children or adults. (Webb et al., 1966) would suggest to

measure carpet erosion in front of each picture to gain an estimate of relative

attractiveness. However, because of the need to differentiate between adult and

children visitors the underground material should be sensitive enough to depict

the size of shoes. Hence, the director decides for high and soft flooring. He in-

structs an assistant to hoover the room every 30 minutes and to count the dif-

ferently sized foot-prints, before. It is obvious from this example that controlled

measures open ways to more purposeful and controlled measurement which is

mostly more efficient than the usage of already existing records as it is some-

times the only way to answer specific research questions in a nearly nonreac-

tive manner.


Type 3. In measures of type 3, not only the research setting is prepared

by the researcher, but the research context is also not intended to be unobstru-

sive for the participants. Hence, the participants are very likely to be aware of

participating in a study. However, it is important to note that the general knowl-

edge about the research context does not necessarily imply knowledge about

the research aims. In fact, type 3 participants are not informed about or are

even actively hindered from elaborating assumptions about the research ques-

tion. As people are usually very interested in knowing about the aim of the re-

search they are participating in, the latter measure is necessary in most type 3

methods. Social psychological experiments, for example often involve active

deception of research participants to avoid distortions that can often be rooted

back to social desirability concerns (for an overview of different respective

methods see e.g. Aronson & Carlsmith, 1990). For instance, in a study on the

impact of account-giving on the violation of pro-environmental norms, Fritsche

(2003) had the aim to manipulate the accessibility of valid accounts prior to the

measurement of norm violating behavior. As it can be expected that people who

are informed about the research topic would be able and willing to influence the

dependent measure in the direction of (or just contrary to) own hypotheses

about the relationship of account-giving and socially appropriate behavior. In

order to prevent participants from generating hypotheses concerning the re-

search topic, the study was introduced as an investigation on communication in

the internet. The following instructions and procedures made this plausible by

asking people to interview an anonymous chat-partner about apparently ran-

domized pairs of topics, including “environmental protection” and “bad con-


science”. Shielded by this cover-story, the confederate chat-partner was able to

present standardized accounts in the course of “natural communication” without

revealing the actual research question. The participants were invited by the ex-

perimenter, to order a drink that they would receive later during the experiment.

After the manipulation of specific account’s accessibility and validity a secretary

brought the announced drinks, all packed in cans. A previous study with a com-

parable sample had shown that drinking from cans was perceived as prototypi-

cally harmful to the environment. Since nearly all of the participants who dis-

posed of valid accounts for drinking from a can actually took the can, only 64 %

of the participants without compelling justifications or excuses did so. Asked to

indicate the “true” research question, none of the participants could specify the

topic correctly. In addition to the technique of designing a plausible cover story,

dependent variables are often measured on an implicit or even physiological

level. Those measures are assumed to be immune against intentional distortion

by the participant. In this textbook, both methods are discussed in separate

chapters [references to chapters 11 & 12 ?].

Type 2. In type 2 measures, participants are informed about the general

topic of an investigation but particular hypotheses are hidden. This kind of tech-

niques might be the measurement most often used in psychology. Examples

can be found in most questionnaire techniques measuring state or trait person-

ality, situated cognitions or emotions. Here, although people most often know

that they are actually asked about environmental attitudes, the degree of their

introversion or actual self-esteem, as a rule, they do not know about specific

hypotheses like item assignment to different scale dimensions or assumptions


about relationships between variables or the expected results. However, not

only participants in the narrower sense should be blind to specific hypotheses.

In order to minimize possible experimenter effects beyond those associated

with the mere research setting, assistants of the experimenter should be in-

volved who are also not informed about the hypotheses (type 2) or even about

the whole research question (type 3). Those research assistants, involved in the

so-called double-blind studies are also participants in the wider sense, as they

are ideally not identical with the researcher. Being blind to particular hypotheses

implies that it is not possible for participants to counteract the primary research

goal systematically. Nevertheless, participants who know about the field of re-

search they are contributing to, might be motivated to extend their contribution

to the part of the researcher. They often generate own hypotheses that they

want to proof immediately by arranging their responses in a respective manner.

Hence, type 2 measurement should be flawed with increased unsystematic

measurement error.

Type 1. In some studies it can not be avoided that the participants get to

know the hypotheses of the researcher. However, there are also investigations

where hypotheses are deliberately disclosed to participants. This might be the

case for ethical as well as for feasibility reasons. Despite revealing hypotheses

is generally assumed to open the door to systematic distortion, this tendency

can be counteracted by lowering or prevent the awareness of a measure’s ma-

nipulability. This is tried in nonreactive measures of type 1. Although different

measures can incorporated that might reduce perceived manipulability (e.g. an-

nouncement of non-manipulability, high complexity of materials), the most


prominent and best investigated example of type 1 measures is the bogus pipe-

line technique (Jones & Sigall, 1971). Here participants are made to believe that

it is possible to pump the their psyche directly by an apparatus, apparently re-

cording physiological signals. This technique has been found to reduce reactiv-

ity in attitude assessment that is rooted in social desirability concerns of the par-

ticipants, considerably (for a review see Roese & Jamieson, 1993).

Type 0. Now, we have enlarged on the range of different types of meas-

urement techniques that approximate to nonreactivity to different degrees. For

being complete, table 1 also includes type 0 techniques, that definitely do not

fulfill the criterion of nonreactivity. In those measures, the interaction of re-

searcher and participants is designed and perceived as serving the investiga-

tion of particular hypotheses and the participant is fully aware of being able to

manipulate the results. One might assume that an atmosphere of cooperation

between researcher and participant, fueled by full disclosure of all hypotheses

can minimize participant’s possible tendency of sabotage or might even moti-

vate participants to give their best. For partial purposes this effect might have

some benefits, for example when there is high dependency on the information

of single participants, like it is the case for witnesses of rare events. However,

the negative implications of fully informed participants for measurement quality

prevail possible benefits. Research on demand characteristics of psychological

studies (e.g. Orne, 1962) have often described the tendency of cooperatively

motivated subjects to distort their behavior or statements into the direction of

the assumed hypothesis. This fundamental danger (as well as the further pitfalls


of reactivity) concerns all forms of collaborative and consensual research tech-

niques (e.g. Page, 2000).

In the following, we will describe and discuss some important nonreactive

measures in more detail. First, we deal with a few classical measures, followed

by a description of recent trends and techniques in nonreactive measurement,

that have in part developed due to socio-technological changes of human be-

havior.

3 Classical Measures and Techniques

Looking back to several decades of concern about reactivity in psychological

research as well as neighboring fields (eg. school education), we meanwhile

may talk of classical methods for avoiding or reducing reactivity. In various con-

tributions, we find classifications closely related to the one presented by Webb

et al. (1966). Shaughnessy & Zechmeister (1990) as well as Schweigert (1998)

differentiate physical traces (use traces and products) and archival data (analy-

ses of communications and trends and assessing the effects of natural treat-

ments). Bloom & Fischer (1982) differentiate the latter in public vs. private and

consider simple observations as additional category. In the following section, we

will briefly describe those classical methods discussed in the literature. As de-

scribed above and in accordance with the position of Bungard & Lück (1997),

we generally consider methods as more or less (non)reactive instead of differ-

entiating reactive vs. nonreactive.

Physical Traces


Our physical environment provides various sources for recognizing behavior

without observing it directly and hence without being in danger of influencing

what is under investigation. Traces may be utilized to acknowledge preferences

and interpret these in various situations: Magazines available for reading in a

doctor’s waiting room indicate the relative rate of being read after some time,

the abrasion of floors in museums indicates the amount of visitors being at-

tracted by the respective exhibit (see above). Even temporary traces like finger

prints on glass doors may indicate the age of people using the door since these

vary in altitude.

Analyses of traces are particularly interesting for environmental design

and other fields concerning the interface between people and their physical sur-

roundings. Trace may be analyzed in order to recognize sub-optimal fittings be-

tween behavior, needs and arrangements conceptualized to meet these. Barker

(1968) called this “synomorphy”, which intends to indicate a kind of fitting be-

tween persons (regularly occurring behavior) and settings. Traces may indicate

activities to enhance synomorphy. In offices and on other workplaces where this

is possible, e.g. people re-arrange furniture and other items in order to work

more effectively and/or comfortably. We may also read deficiencies of environ-

mental design. In restrooms of German trains eg, traces indicate that signs for

activating the flush, water tubs and dryer have been confused with switches for

initiating these functions. For remodeling, traces have been correctly under-

stood.

Nature is even more indicative for human activities. In public parks for

exampe, trails indicate where people regularly walk or drive. Since more solid


surfaces are less sensible, the natural emergence of traces on lawn or uncov-

ered soil has been utilized to arrange paths not until the park has been used for

some time. Not only the route but also the width of alleys could be varied follow-

ing this unobtrusive method. Since user needs analyses (Linneweber, 1993;

Sommer, 1983) became an applied field for psychology increasingly, it is impor-

tant to learn for environmental design in method and theory. The role of psy-

chologists is reading and – even more demanding – interpreting traces and ad-

vising designers.

Well known is nonreactive research of litter. As traces of consumption,

packing material, tins, empty bottles and other types of garbage indicate what

those people use and/or prefer which have thrown away the respective items.

This may be highly important with respect to specific target individuals or

groups, e.g. adolescents but also for market research.

Traces are also informative for basic research. While answers to re-

search questions directly addressed suffer from social desirability at least in

societally relevant areas, analyses of traces reduce this bias. Consider a

household being asked to indicate the relative amount of fast or canned food

consumed per week. Since it may be desirable to underestimate this amount in

favor of fresh products, an analysis of waste is much more representative. In

economic psychology, contents of shopping carts have been analyzed in order

to figure out consumption patterns and/or preferences. These may be related to

further variables like economic situation, family constellation and other variables

relevant for typologies. Although here process of measuring is not at all unob-

strusive, the behavior in question (buying the specific item) is not affected.


Archival Data

While traces may be considered as indications for behavior “simply there”, ar-

chives and other types of records are arranged intentionally. This may be re-

lated to research questions, e.g. in observation studies. With respect to our

topic, primarily those archives are of interest which exist independent of scien-

tific investigations. Data files in administration but also sales records may be

subsumed under this category. As archival data, also material written by per-

sons being the target of research is of interest. Descriptions of accidents, com-

plains about nuisances but also private material like letters and other forms of

written communications may be interesting psychologically. Basically, as soon

as the genesis of the material is not affected by the research process, the

method is highly nonreactive as it is likely to represent techniques of type 5.

We must bear in mind, however, that archival data may be generated

under circumstances to be considered as highly reactive. Actors may very well

know that important data are documented, and they may be motivated to influ-

ence these. With respect to our above considerations on nonreactive methods,

two solutions concerning this problem are offered. First, the process and result

of intentionally influencing archival data may be target of research. This is pos-

sible as soon as different perspectives are available like in records of conflict,

where e.g. mutual accusations directed to involved actors or third parties are of

interest. Second, if we have reasons to assume that biases during the creation

of archival data are independent of those which are relevant for interpreting

these in the investigation, we may cope with the effect.


Compared with physical traces, archival data may be more suitable and

effective for research purposes, but they are more sensible for undesirable ef-

fects at the same time. Traces may be less simple to read and to interpret, but

usually they are less sensible to those effects we intend to minimize.

Simple Observations

Observations for scientific purposes may significantly vary with respect to their

obtrusiveness and hence (non)reactivity. When social sciences started develop-

ing the arsenal of methods, alternatives of – at least at that time – highly obtru-

sive procedures have been discussed. In the beginning of the debate, “Partici-

patory observation” has been – and still is – called a type of research with fuzzy

definitions of the relation between investigator and target (Couto, 1987). The

investigator is not unobtrusive at all since she acts in the field under investiga-

tion. Her specific impact on processes to be observed however, is considered

as being low since her position as a researcher is masked by her activity in the

field being less extra-ordinary than the position of a scientific observer requires.

Presently, the concept is of only marginal importance in culture-specific and/or

some areas of clinical and family research.

“Simple observations” are considered as having some potential towards

nonreactivity. Bloom & Fischer (1982) discuss four types:

- observation of physical and body signs (jewelry, changes in hair style,

clothing and makeup) being potentially indicative for attitudes or behav-

iors. This is also addressed by studies on symbolic self completion

(Wicklund & Gollwitzer, 1981).


- analysis of expressive movements (smiles, frowns, gestures) indicative

for attitudes and feelings

- physical location analysis indicating attitudes (e.g. seating patterns as in-

dex for interracial relations)

- analysis of language behavior (tape recorded for various research goals)

The process of observation may not be unobtrusive in its literal sense at all

since persons or equipment are present. However, as long as “the observer has

no control over the behavior or sign in question, and plays an unobserved, pas-

sive and unobtrusive role in the research situation” (Webb, 1966, p. 112), reac-

tivity is considered as being minimized.

4 New Approaches

Socio-technological conditions often determine the development of psychologi-

cal assessment methods in two ways. On the one hand, the content of research

is often oriented towards questions relevant to society and suitable methods

have to be developed or adapted. For instance, the need to do research on ag-

gression in car traffic made it necessary to find valid measures of driver behav-

ior (e.g. Boyce & Geller, 2002). On the other hand, the research instruments

itself depend heavily on tools or aids that are available in a particular socio-

technological context. For instance, increasing technological possibilities of

monitoring brain activity have led not only to increased and sometimes inflation-

ary use of imaging methods but also to the differentiation of several techniques

of interpreting and applying those methods (see chapter [Brain imaging]). In the

following we will show how particularly those nonreactive measures building on


data that is not originally recorded or rather not perceived as being recorded for

research purposes (type 5 and 4 measures, table 1) have developed with

changing socio-technological conditions. Here, we exemplarily focus on nonre-

active online research in the internet.

Nonreactive Research Using the Internet

The rapid development of computer mediated communication in global

network structures during the last decade has led to an increased interest in

research on “online behavior”. At the same time the internet emerges to be-

come a powerful research tool that can be efficiently used for the collection of

“real” world data (see chapter [web based methods]). Besides interesting oppor-

tunities for web-based survey as well as experimental studies (e.g. Reips &

Bosnjak, 2001), particularly nonreactive measurement of type 5 and 4 can

benefit from the technological properties of the internet environment.

Analyzing Written Material in the Internet. One kind of nonreactive inter-

net data is written material, people produce in and for the internet, for instance

in virtual discussion groups or e-mail lists. These texts are usually written for a

potentially unrestricted public and the researcher has the opportunity to save

and analyze such data. Interestingly, communication in the internet is often at

least two-sided, so that it is possible to analyze interaction between individuals

and also within higher social aggregates. Bordia (1996), for example, describes

the use of online discussion group archives in rumor transmission research.

Here it is possible to take a processual perspective on occasions of naturally

occurring rumor transmission that can be found in the internet comparatively


easily. Bordia (1996) analyzed relevant episodes by quantitative content analy-

sis, using statement categories like “interrogatory statements” or “prudent

statements” referring to tentativeness or hesitancy in discussing a rumor. The

quantity of these statements could not only be compared across the whole dis-

course but also across time. Using the latter perspective, Bordia (1996) found

that while the analyzed discussions progressed, the frequency of prudent

statements related to the rumor decreased. The author highlights that although

this phenomenon has been mentioned in the literature before, it was the first

time this could be shown in a natural context. Here, it might be asked, whether

communication in the internet can legitimately be called “natural”. However, al-

though several differences beween computer mediated communication and

face to face interaction have been identified (e.g. Kiesler, Siegel, & McGuire,

1984), wide-spread accessibility and increasing competence in using the tech-

nological environment has made online communication an integral part of be-

havior in the well-industrialized countries. Hence, the specific nature of internet

behavior does not make it less natural.

Log File Analysis. Analyzing text available in the internet, represents the

classical method of using archival data, although the interactional, dynamic and

mostly well documented structure of internet content increases analyses’ po-

tency. However, there is also another, possibly even more important way in

which online research might enhance the capacities of nonreactive measure-

ment. This is rooted in the fact that internet behavior is continuously and auto-

matically recorded without the explicit awareness of users. Interestingly, those

records can be assigned to the behavior of single individuals or at least to single


machines. Analyzing those log-files generated on internet server machines and

optionally also on client computers making hidden protocolls accessible to

broad and complex research activities, is a highly nonreactive technique not yet

frequently used in psychological research, however. A simple form of log-file

analysis is previously applied in advertisement contexts, indicating the attrac-

tiveness of certain web pages and the success of particular advertising links, by

following users’ navigation through the net (e.g. Wiedmann & Buxel, 2001). In

similar fashion, log-file analysis can also be used in descriptive research. Berker

(2002), for example, reports a nonreactive study on internet behavior, exhibited

by the users of an account at the internet server of a big German university.

Analyzing proxy log-files for a time of two weeks in 1998 revealed an interesting

preference order of web-pages chosen by the users. 24 % of the requests were

directed to nudity pages, followed by multi-purpose pages like probably preset

internet providers (22 %) as well as by pages offering technical support and by

search engines (both 9 %). Using also log-file information like time and duration

of access led to additional results concerning content specific user habits. A

more controlled and theory-driven study of type 4 is reported by Kulikowich &

Young (2001), who assessed problem solving behavior of individuals using log-

file data of an online learning tool. Different problem solving activities were rep-

resented by accessing particular web pages. The sequential order and duration

of access was analyzed with regard to the students’ interaction with their learn-

ing environment.

Like in conventional accretion measures, the traces of individual behavior

can be followed through the internet or specified online environments. Further


development of this method might include also analyzing navigation behavior

more directly and should not only restricted to behavior, relevant only with ref-

erence to the virtual environment. In the context of environmental planning for

instance, it might be a useful strategy to 3D-visualize architectural alternatives

and analyze how long the respective models are visited, what places are en-

tered and what perspectives are selected. This may become a new type of “so-

cial design” (Sommer, 1983). Readers who are interested in conducting log-file-

analysis may retrieve one of various free software offers from the internet (e.g.

Analog, 2003). For a compilation of log file analysis tools see Janetzko (2003).

One major problem in conducting log-file analysis is that accessing an

internet page is not always recorded in the same way at the same place and

that not every access to a particular page is actually documented by a log-file.

The reduction of the first problem might imply standardization of log file proto-

cols’ content and location for research purposes (type 4 measurement). The

latter problem that internet actions are not always recorded comes up, when

pages are retrieved from cache memories without accessing a server machine

or when proxy servers are involved that do not always inform the original server

machine about access to one of its pages. Conversely, if records are analyzed

that reside on the user machine, this method might suffer from the non-

acceptance of so-called cookies, set by many users. In order to deal with these

problems, particularly in the commercial sector, efforts are made of standardiz-

ing the feedback of proxy servers (Werner, 2002). Further, potential self-

selection of the sample covered by log-file analysis should be kept in mind, de-

pending on the activation of proxies, cookies and cache-use in the user ma-


chine. Ideally, possible confounding with variables relevant to the research sub-

ject should be ruled out in a pre-study, comparing for instance those users who

have activated proxies in the preferences menu of their web-browser with those

who have not. Concerning patterns of internet use, results by Berker (2002) in-

dicate only little differences between both groups.

Another problem of nonreactive internet research might be the identifica-

tion of single person behavior. Although, log-files usually identify single access-

ing machines, it is both not clear whether multiple individuals use this machine

at one time and whether pages are accessed automatically without the purpose

or even the awareness of the user. To reduce these interpretational weak-

nesses, it can sometimes be recommendable to set a duration of inactivity that

if exceeded mark the beginning of another session by a different user (e.g. ex-

ceeding average time of inactivity by 1,5 standard deviations as suggested by

Catledge & Pitkow, 1995). Further, depending on the research question one

should exclude those web addresses from analysis, usually contacted auto-

matically (e.g. pictures or homepages of browser software, Berker, 2002). As

individual behavior can be identified with satisfying reliability without the knowl-

edge or agreement of subjects, anonymity of analysis is an important demand.

Berker (2002) for example eliminated all information from the log-files that could

have facilitated identification of individual machines before starting his analysis.

This brief overview of two different pitfalls of nonreactive online research

points to the role, technological properties of the medium as well as of the re-

cording of information play in the use and interpretation of behavioral traces not

only in the internet. Therefore, it is recommended to get well acquainted with


up-to-date knowledge about the technological opportunities and constraints, raw

data – e.g. from the internet – set for interpretation.

Socio-Technological Changes – Chance and Risk for Nonreactive Measure-

ment

One reason why we have described nonreactive online research in more

detail is that this field has recently led and will soon lead to a multitude of new

nonreactive research strategies offering analyses of unique potency with regard

to availability of information and efficiency of analysis. The other reason laid in

the exemplary character, this research has for the influence, technological de-

velopments may exert on the development of nonreactive measurement. How-

ever, access to information has also increased due to rather societal than tech-

nological developments. In Wim Wender’s science fiction movie “Until the End

of the World” from 1991, the protagonist can be found by his pursuers on his

voyage around the world only by having access to his card banking data. Now,

more than 12 years later, the inhabitants of industrialized societies are used to a

broad variety of relatively new technological equipment including satellite navi-

gation for private cars, health insurance data cards or globally working mobile

phones, producing additional traces of behavior that can be gathered and ana-

lyzed fairly well by “researchers” of different provenience. Beyond technological

developments, a strong need for security in many industrialized societies seems

to further increase the recording of individual data (e.g. body search at school,

security cameras observing public places). Although, systematic access to this

kind of data is often restricted, it can be both a valuable source for nonreactive


research (e.g. Brizer, Crowner, Convit, & Volavka, 1988) as well as a serious

danger for individual freedom by data abuse. A related question is, whether an

awareness of increased data recording or even surveillance will change public

and semi-public behavior of individuals. Although, people will not become

chronically self-presenting inhabitants of a reality TV show, they might behave

in a more self-focused (Carver & Scheier, 1981) and socially desirable way

when they enter settings where surveillance is salient or even feared, for exam-

ple at a cash machine or in a non-smoking subway station.

5 The Limits of Nonreactive Measurement

In the introduction, we attested participants of social scientific research those

attributes also present in most naturally occurring everyday situations: aware-

ness and capacities for information processing. In a reader edited more than 20

years ago (Bungard, 1980) however, various authors ironically discussed why in

social psychology “good subjects” don’t think. What is the origin for this discrep-

ancy? We have learned that occasionally hiding goals and/or process of data

collection may be desirable for social scientific research in order not to “spoil”

what is under investigation, i.e. behavior. In other words, we have to prevent

subjects from behaving extra-ordinarily.

Ethical Limits

In highly nonreactive research, “subjects” or “participants” are not aware

ob being – which sounds less polite – objects of research. They are not aware

of person(s) or installations recording data related to themselves. They may not

be aware that others in their proximity or able to observe themselves by using


technical equipment are in the positions of scientist, experimenter or confeder-

ate. Consequently, they are unable to refuse participating. This fact has been

discussed with respect to ethical considerations, and we will discuss these re-

ferring to the ethical principles of psychologists and code of conduct of the APA

(American Psychological Association) (APA, 2002). In other national scientific

societies we find formulations more or less similar. Since the APA-standards

have been revised only recently, we will concentrate on these.

At a first glance, unobtrusive research contradicts to principle 3.10 “In-

formed Consent” which says: “When psychologists conduct research ... in per-

son or via electronic transmission or other forms of communication, they obtain

the informed consent of the individual or individuals ... to that person or per-

sons.” (p. 1065). Also principle 4.03 “Recording” seems to be violated: “Before

recording the voices or images of individuals to whom they provide services,

psychologists obtain permission from all such persons or their legal representa-

tives”. However, principle 3.10 continues “except when conducting such activi-

ties without consent is mandated by law or governmental regulation or as oth-

erwise provided in this Ethics Code.” Referring to formulations in Part 8 (Re-

search and Publication of the ethical principles), the term “otherwise” hence le-

gitimizes research activities without consent.

Since formulations 8.03, 8.05 and 8.07 are of particular interest with re-

spect to unobtrusive research, they are cited in full length:

“8.03 Informed Consent for Recording Voices and Images in Research.

Psychologists obtain informed consent from research participants prior to re-

cording their voices or images for data collection unless (1) the research con-


sists solely of naturalistic observations in public places, and it is not anticipated

that the recording will be used in a manner that could cause personal identifica-

tion or harm, or (2) the research design includes deception, and consent for the

use of the recording is obtained during debriefing.”

“8.05 Dispensing With Informed Consent for Research. Psychologists

may dispense with informed consent only (1) where research would not rea-

sonably be assumed to create distress or harm and involves (a) the study of

normal educational practices, curricula, or classroom management methods

conducted in educational settings; (b) only anonymous questionnaires, natural-

istic observations, or archival research for which disclosure of responses would

not place participants at risk of criminal or civil liability or damage their financial

standing, employability, or reputation, and confidentiality is protected; or (c) the

study of factors related to job or organization effectiveness conducted in organ-

izational settings for which there is no risk to participants’ employability, and

confidentiality is protected or (2) where otherwise permitted by law or federal or

institutional regulations.”

“8.07 Deception in Research. (a) Psychologists do not conduct a study

involving deception unless they have determined that the use of deceptive

techniques is justified by the study’s significant prospective scientific, educa-

tional, or applied value and that effective nondeceptive alternative procedures

are not feasible. (b) Psychologists do not deceive prospective participants about

research that is reasonably expected to cause physical pain or severe emo-

tional distress. (c) Psychologists explain any deception that is an integral fea-

ture of the design and conduct of an experiment to participants as early as is


feasible, preferably at the conclusion of their participation, but no later than at

the conclusion of the data collection, and permit participants to withdraw their

data.”

“8.08 Debriefing. (a) Psychologists provide a prompt opportunity for par-

ticipants to obtain appropriate information about the nature, results, and conclu-

sions of the research, and they take reasonable steps to correct any

misconceptions that participants may have of which the psychologists are

aware. (b) If scientific or humane values justify delaying or withholding this

information, psychologists take reasonable measures to reduce the risk of

harm. (c) When psychologists become aware that research procedures have

harmed a participant, they take reasonable steps to minimize the harm.”

(p. 1069-1070)

As we can see, the formulations are rather comprehensive. We find an

explicit reference to naturalistic observations which is not yet the case in ethical

standards of other National Psychological Associations. The principles allow

nonreactive research of type 4 and 5 even without debriefing (8.08) where some

criteria are met, e.g. preventing personal identification or harm (8.03 (1)) and

the risk of criminal or civil liability or damage participants’ financial standing,

employability, or reputation, and confidentiality (8.05 (b)). Deception, that might

be included in nonreactive methods from type 1 to 4 is accepted as an excep-

tion only when explicitly justified (8.07 (a)) and when consent for the use of the

recording is obtained during debriefing (8.03 (2) and 8.07 (c)).

With respect to our discussion of various nonreactive measures we realize that

the scientific community developed principles for highly responsible conduct in


observations, in laboratory settings as well as in field experiments. This includes

reports to ethic commissions as well as consulting peers. Ethic standards are

continuously reviewed. Facing various developments particularly with respect to

unobtrusive methods, the necessity to continue this effort is evident.

The possible future of unobtrusive research

Summarizing our view on nonreactive measurement, we have suggested

a continuous concept of nonreactivity, identifying five types of more or less non-

reactive research rather than defining nonreactive methods in a dichotomous

way. Measures of high nonreactivity are usually marked by a high degree of

unobtrusiveness with regard to the subject’s actual or perceived participation in

a research study. Specifically those kinds of nonreactive measures (type 4 and

5) may profit or at least change with current and future socio-technological de-

velopments enabling us to apply new techniques and media explicitly relevant

for unobtrusive measures. The term “traces of behavior” meanwhile appears in

new light since electronic ways of collecting, saving and processing data have

significantly improved within the last decades. There are at least two areas with

rapid developments still going on: Consumer behavior in various fields and indi-

vidual behavior in “intelligent” technological environments.

Our credit card companies realize unusual expenditures on the basis of

our previous behavior. Of course “only” – or at least primarily – in order to pro-

tect us from misuse of our card. Our cell phones “know” where we are. Our cars

already determine service intervals on the basis of our driving behavior. Vehi-

cles soon will start communicating so that they can form clusters for defined


parts of routes (“knowing” our destination and computing the optimal routes)

enabling us to do more important things than steering – of course these activi-

ties are recorded for later on improving our own comfort.

In household technology, the revolution is still to come: When approach-

ing our home, the TV-set will be switched to our favorite station, the room tem-

perature and illumination will be regulated based on analyses of our prefer-

ences and our refrigerators will order items on the basis of continuously

monitoring our consumption patterns.

Although these immense amounts of data may in fact be protected and at

least not related to those individuals they refer to. Basically however, for “new

forms” of psychological research, the data basis for unobtrusive research

seems to explode. Presently, we may anticipate one development with a high

degree of certainty: the responsibility in collecting new and processing available

data will become a highly important topic for basic as well as applied research.

This is particularly true for information we can access or collect in unobtrusive

ways.

References

Analog. (2003). Retrieved May 2, 2003, from http://www.analog.cx APA. (2002). Ethical Principles of Psychologists and Code Conduct. American

Psychologist, 57(12), 1060-1073. Aronson, E., & Carlsmith, J. M. (1990). Methods of research in social psychol-

ogy. New York: McGraw-Hill. Barker, R. G. (1968). Ecological Psychology. Stanford, CA: Stanford University

Press. Berker, T. (2002). World Wide Web use at a German University - computers,

sex and importt names. Results of logfile analysis. In B. Batinic, U.-D. Reips & M. Bosnjak (Eds.), Online Social Science. Seattle: Hogrefe & Huber Publishers.

http://www.analog.cx/


Bloom, M., & Fischer, J. (1982). Reactivity and the use of unobstrusive meas-ures. In M. Bloom & J. Fischer (Eds.), Evaluating Practice: Guidelines for the Accountable Professional (pp. 200-217). Englewood Cliffs/N.J.: Pren-tice-Hall.

Bochner, S. (1979). Designing unobstrusive field experiments in social psychol-ogy. In L. Sechrest (Ed.), Unobtrusive measurement today (pp. 33-45). San Francisco: Jossey-Bass.

Bordia, P. (1996). Studying verbal interaction on the internet: The case of rumor transmission research. Behavior Research Methods, Instruments and Computers, 28(2), 149-151.

Boyce, T. E., & Geller, E. S. (2002). An instrumented vehicle assessment of problem behavior and driving style: Do younger males really take more risks? Accident Analysis and Prevention, 34(1), 51-64.

Brizer, D. A., Crowner, M. L., Convit, A., & Volavka, J. (1988). Videotape re-cording of inpatient assaults: A pilot study. American Journal of Psychia-try, 145(6), 751-752.

Brunswik, E. (1947). Systematic and representative design of psychological ex-periments. Berkeley, CA: University of California Press.

Bungard, W. (Ed.). (1980). Die gute Versuchsperson denkt nicht: Arfefakte in der Sozialpsychologie. München: Urban und Schwarzenberg.

Bungard, W., & Lück, H. E. (1997). Nichtreaktive Meßverfahren. In J. L. Patry (Ed.), Feldforschung: Methoden und Probleme sozialwissenschaftlicher Forschung unter natürlichen Bedingungen. Bonn: Verlag Hans Huber.

Carver, C. S., & Scheier, M. F. (1981). The self-attention-induced feedback loop and social facilitation. Journal of Experimental Social Psychology, 17(6), 545-568.

Catledge, L., & Pitkow, J. (1995). Characterizing browsing strategies in the world-wide-web. Paper presented at the Third International World-Wide Web Conference. Retrieved March 6, 2003, from http://www.igd.fhg.de/archive/1995_www95/papers/80/userpatterns/UserPatterns.Paper4.formatted.html

Couto, R. A. (1987). Participatory research: Methodology and critique. Clinical Sociology Review, 5, 83-90.

Folger, R., & Belew, J. (1985). Nonreactive measurement: A focus for research on absenteeism and occupational stress. Research in Organizational Behavior, 7, 129-170.

Fritsche, I. (2003). Prebehavioural account-giving and its impacts on behav-ior.Unpublished manuscript.

Janetzko, D. (2003). Statistical applications via internet - collecting, analyzing and presenting data in network environments. The technical side. Re-trieved March 7, 2003, from http://cogweb.iig.uni-freiburg.de/SAI/TECHNICS/

Jones, E. E., & Sigall, H. (1971). The bogus pipeline: A new paradigm for measuring affect and attitude. Psychological Bulletin, 76(5), 349-364.

Kiesler, S., Siegel, J., & McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39(10), 1123-1134.

http://www.igd.fhg.de/archive/1995_www95/papers/80/userpatterns/UserPatterns.Paper4.formatted.html

http://www.igd.fhg.de/archive/1995_www95/papers/80/userpatterns/UserPatterns.Paper4.formatted.html

http://cogweb.iig.uni-freiburg.de/SAI/TECHNICS/

http://cogweb.iig.uni-freiburg.de/SAI/TECHNICS/


Kulikowich, J. M., & Young, M. F. (2001). Locating an ecological psychology methodology for situated action. Journal of the Learning Sciences, 10(1), 165-202.

Linneweber, V. (1993). Wer sind die Experten? "User needs analysis" (UNA), "post occupancy evaluation" (POE) und Städtebau aus sozial- und um-weltpsychologischer Perspektive. In H. J. Harloff (Ed.), Psychologie des Wohnungs- und Siedlungsbaus: Psychologie im Dienste von Architektur und Stadtplanung (pp. 75-85). Göttingen; Stuttgart: Verlag für Angewandte Psychologie.

McNatt, D. B. (2000). Ancient Pygmalion joins contemporary management: A meta-analysis of the result. Journal of Applied Psychology, 85(2), 314-322.

Milgram, S., Mann, L., & Harter, S. (1965). The lost-letter technique: A tool of social science research. Public Opinion Quarterly, 29, 437-438.

Orne, M. T. (1962). On the social psychology of the psychological experiment: with particular reference to demand characteristics and their implications. American Psychologist, 17, 776-783.

Page, S. (2000). Community research: The lost of unobstrusive methods. Jour-nal of Applied Social Psychology, 30(10), 2126-2136.

Reips, U.-D., & Bosnjak, M. (2001). Dimensions of internet science. Lengerich, Germany: Pabst.

Roese, N. J., & Jamieson, D. W. (1993). Twenty years of bogus pipeline re-search: A critical review and meta- analysis. Psychological Bulletin, 114(2), 363-376.

Rosenthal, R. (1976). Experimenter effects in behavioral research. New York: Irvington Publ.

Schweigert, W. A. (1998). Physical trades and archival data: Two nonreactive measurement techniques. In W. A. Schweigert (Ed.), Research methods in psychology : a handbook (pp. 174-182). Pacific Grove, CA: Brooks/Cole.

Sechrest, L. (Ed.). (1979). Unobtrusive measurement today. San Francisco: Jossey-Bass.

Shaughnessy, J. J., & Zechmeister, E. B. (1990). Unobstrusive measures of behavior. In J. J. Shaughnessy & E. B. Zechmeister (Eds.), Research methods in psychology (4th ed., pp. 159-186). New York: McGraw- Hill.

Sommer, R. (1983). Social design. Creating buildings with people in mind. Englewood Cliffs, NJ: Prentice-Hall.

Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. (1966). Unobstru-sive measures: nonreactive research in the social sciences. Chicago: Rand McNally.

Werner, A. (2002). Contact measurement in the WWW. In B. Batinic, U.-D. Reips & M. Bosnjak (Eds.), Online social science. Seattle: Hogrefe & Huber Publishers.

Wicklund, R. A., & Gollwitzer, P. M. (1981). Symbolic self-completion, attempted influence, and self-deprecation. Basic and Applied Social Psychology, 2(2), 89-114.

Wiedmann, K.-P., & Buxel, H. (2001). Using non-reactive observation methods in online research. Planung & Analyse, Special English Edition, 38-42.


Table 1: Continuum of nonreactivity in measurement.

level of nonreactivity

low high

setting initially designed or selected for research yes yes yes yes yes no

Participants are aware of the research setting likely likely likely likely no no

Participants are aware of the research question likely likely likely no no no

Participants are aware of the research hypothesis likely likely no no no no

Participants are aware of the measures’ manipulability likely no no no no no

Type 0 Type 1 Type 2 Type 3 Type 4 Type 5

chapter title: nonreactive methods - university of … readings...nonreactive methods 6 tivity...

Documents