Lecture 2What is a Theoretical Contribution?

RESEARCH METHODSDOCTORAL PROGRAM, NATIONAL RESEARCH

UNIVERSITYHIGHER SCHOOL OF ECONOMICS

DR C S LEONARD

JUNE 2011

RESEARCH METHODS

2 OUTLINE

What is a theoretical contribution?

What is a concept?

A variable is a concept

Validity

External, Internal8/6/2011

RESEARCH METHODS 3

THEORY

Publishing

(Evaluative)

Criteria

8/6/2011

RESEARCH METHODS

4PRACTICAL

GUIDELINES

Choose a hypothesis important in literature but for which no systematic study exists—decide in favour or against the hypothesis

Choose a theory you suspect is false and investigate if it is, indeed, and what alternate might replace it

Design research to illuminate unquestioned assumptions in the literature

Argue that a topic of importance has been overlooked

Show relation of theory from one literature to separate problem from another

8/6/2011

RESEARCH METHODS

5REVISION OF FORMAL

THEORIES?

More than just adding a variable

Showing how it changes perception of the problem

Producing data that are inconsistent with a theory

8/6/2011

RESEARCH METHODS

6BORROW FROM

OTHER DISCIPLINES?

Shed new light on old theories

Investigate qualitative boundaries of a theory, not just quantitative ones

Show why a theory will not work in a new application, not just that it does not

Focus on multiple elements of a theory, not just one inconsistency

8/6/2011

RESEARCH METHODS

7EVIDENCE REQUIRED

FOR REVISING A THEORY

Marshal compelling evidence.

This evidence can be logical (e.g., the theory is not internally consistent),

empirical (its predictions are inconsistent with the data accumulated from several studies), or

epistemological (its assumptions are invalid-given information from another field).

8/6/2011

RESEARCH METHODS

8PROVIDE REMEDIES,

ALTERNATIVES

Is the original actually inferior,

Or simply the best we can do?

8/6/2011

RESEARCH METHODS

9IS IT

PUBLISHABLE?

How radically new is the idea?

Is it linked to evidence, will it change practice?

Why so? Are the underlying logic and sup-porting evidence compelling? Are the author's assumptions explicit? Are the author's views be-lievable?

Well rounded, broad, deep understanding?

At professional standards?

Who cares? Why now?

8/6/2011

RESEARCH METHODS

10WHAT THEORY IS NOT:

(1) REFERENCES

For example:

"This pattern is consistent with findings that industrialization produces growth (Marx 1880, Solow 1956)

This sentence lists publications that contain

conceptual arguments (and some findings).

But there is no theory because no logic is presented

to explain why industrialization produces growth

8/6/2011

RESEARCH METHODS

11WHAT THEORY IS NOT:

(2) DATA

Introduced to show what patterns are there

Not why they are there

This approach relies on brute empiricism,

where hypotheses are motivated by prior data

And not the relevance of theory

8/6/2011

RESEARCH METHODS

12(3) LISTS OF VARIABLES

AND DEFINITIONS

Lists of concepts, similarly, are not theory

Need connections between variables explained

8/6/2011

RESEARCH METHODS

13(4) DIAGRAMS

ALONE

What is needed is causal inference

Time horizon and change over time

Diagrams as Stage props, not the performance

Logic needs to be spelled out, verbally

8/6/2011

RESEARCH METHODS

14(5) HYPOTHESES: WHAT IS EXPECTED TO OCCUR

Well-crafted conceptual argument includes hypotheses

They serve as crucial bridges between theory and data

Making explicit how the variables and relationships that follow from a logical argument will be operationalized.

These are statements about what is expected to occur, not why it is expected to

8/6/2011

RESEARCH METHODS

15IDEAS AND CONCEPTS

A theory generally stems from a small set of research ideas, not a list of testable hypotheses

8/6/2011

RESEARCH METHODS

16QUALITATIVE/QUANTITATIVE

DISTINCTIONS

Qualitative: Research Questions

Quantitative: Set of hypotheses (nul or directional)

8/6/2011

RESEARCH METHODS

17EVIDENCE FOR YOUR

HYPOTHESES/TESTING THEORY

When theories are particularly interesting or important, empirical support can be partial: a small set of interviews, a demonstration, experiment, a pilot survey, archival data

8/6/2011

RESEARCH METHODS

18EVIDENCE FOR YOUR

HYPOTHESES

May point to why a particular process

might be true.

Subsequent research indicated

May show whether the theoretical

statements hold up, or whether they

can repeatedly be falsified 8/6/2011

RESEARCH METHODS 19

EXTERNAL VALIDITY

Economics

8/6/2011

RESEARCH METHODS

20

TWO CONCEPTS OF EXTERNAL VALIDITY,

AS APPLIED TO THEORY

1: Replicability, that the study would hold for other persons, settings, times, or places.

Only one form of validity when the objective of research is to test theory

8/6/2011

RESEARCH METHODS

21EXTERNAL

VALIDITY

2: Applicability

It arises primarily through severe and rigorous tests of theory

rather than by attempts to incorporate "real world' variables into individual studies designed to test theory.

Such variables only become important in the context of evaluating interventions based on theory.

8/6/2011

RESEARCH METHODS

22 CONSTRUCT VALIDITY

One needs to make a distinction between

(1) the construct validity of a concept, as

reflected in the convergence (and

discrimination) of some particular set of

operationalizations of it, and

(2) the construct validity of a relation between

two concepts, as reflected in the "fit" of that

relation within some nomological network.

The fit is linked to considerations of external

validity.

8/6/2011

RESEARCH METHODS

23SAMPLING IS CRITICAL

FOR VALIDITY

Four major sampling strategies that might be

adopted vis a vis any one aspect or facet of the

events under study

1. Sampling homogeneously over the entire study

2. Sampling several subsets, each homogeneous

within subset on the facet but differing on it

between subsets, so that all the subsets

together span the whole range

8/6/2011

RESEARCH METHODS

24 SAMPLING

3. Sampling heterogeneously. but in a

way that yields an overall distribution of

the facet among the cases within the

study that reflects (is representative of)

the distribution of the facet among cases

"in nature”

4. Sampling heterogeneously on the

facet but without regard to

representativeness.

8/6/2011

RESEARCH METHODS

25SAMPLING THREATS

TO VALIDITY

These four strategies offer different opportunities for—and pose different threats to—the exploration of the external validity of any given set of findings with respect to the facet in question.

Define external validity:

the deliberate and systematic search, on a number of facets, for both the scope and the limits over which that given set of findings does and does not hold.

8/6/2011

RESEARCH METHODS

26 REPORTING

Validity as robustnesss

Where are the boundaries?

Can findings be replicated?

8/6/2011

RESEARCH METHODS

27 TESTS FOR VALIDITY

STAGE 1 STAGE 2

Design:

Instrument validity instrument use validity

Comparison validity execution validity

Hypothesis:

Experiment:

Construct validity Operational

Nomalogical validity Predictive

8/6/2011

RESEARCH METHODS

28CHECKING

VALIDITY

Observation:

State validity Attribute

Pattern validity Process

8/6/2011

RESEARCH METHODS 29

THEORY

What is a good

theory?

8/6/2011

RESEARCH METHODS

30 A THEORY DOES NOT COME FIRST IN RESEARCH

Theory:

A reasoned and precise speculation about the answer to a research question, including a statement about why the proposed answer is correct

Implies several specific descriptive or causal hypotheses

Is consistent with prior evidence about a research question

8/6/2011

RESEARCH METHODS

31CHOOSING A

THEORY

Possibly, it will be wrong

It must be falsifiable-what evidence would convince us?

Requires observable implications (many)

Observe the principle of parsimony (a judgment about the nature of the world, which is assumed to be simple)

These rules assume you have not yet collected the data, which can be used afterwards, to modify your theory

8/6/2011

RESEARCH METHODS

32AFTER YOUR

RESEARCH

What to do with a theory

Expand it, dropping a condition or variable, to, say, all countries, all regions

Make it embrace a larger range of phenomena

Do not do the opposite If the theory does not work for some of your

observations, do not squeeze new theory out of a revised and qualified research base

Ie, make it less restrictive, but not more restrictive—unless you go back and collect more data and observations beginning with the new hypothesis

8/6/2011

RESEARCH METHODS

33 DATA

Ensure reliability

Maximize leverage

Explain as much as possible with as little as possible Increase numbers of observable

implications with confirmation

Improve the theory, improve the data

From the beginning, list all the possible implications of your hypotheses—outcomes, responses, interviews, at all levels

8/6/2011

RESEARCH METHODS

34 SCEPTICISM

Report negative findings

Report alternative hypotheses

8/6/2011

RESEARCH METHODS 35

THEORY CONCEPT

How to build a

concept

8/6/2011

RESEARCH METHODS

36CONCEPT

FORMATION

Rigorous approach to concept formation (Osigweh) Defining meaning

boundaries (what the concepts do not include)

Minimizing concept misuse, confusion

8/6/2011

RESEARCH METHODS

37CONCEPT

PRECISION

General, yet precise Moving a concept from low

to high levels of abstraction

Economic oncepts must span several contexts

Yet they must provide a precise guide to what is not included

8/6/2011

RESEARCH METHODS

38CONCEPT

IMPRECISION

Useless: new contexts too easily fit

New topics can be suspect, vulnerable to herding by scholars (CR impact on performance)

8/6/2011

RESEARCH METHODS

39 CONCEPTS

Can be decomposed taxonomically

8/6/2011

RESEARCH METHODS

40 VARIABLES

A variable is a special

kind of concept.

It is a classification into two or more mutually exclusive and totally inclusive

categories (e.g., Hage, 1972; Smith, 1975).

8/6/2011

RESEARCH METHODS 41

CONCEPTS

Stretching and

Travelling

8/6/2011

RESEARCH METHODS

42CONCEPT TRAVELLING:

POSITIVE CONTRIBUTION

Concept travelling, in this sense, means that the concept is precise enough to allow researchers to define it in the same way, and so to test it in a wide range of situations—that is, that the concept is a universal.

8/6/2011

RESEARCH METHODS

43 EXAMPLE

Puzzle -- Each use creates essentially the same image anywhere:

(a) a maze of activities that cannot be correctly solved outside settings or mental frameworks that prescribe following certain paths or specific combination of paths;(jigsaw?Rubik’s cube)?

(b) an exercise of figures, numbers, or behaviours that cannot be engaged in or successfully disengaged in, except through a specific order or steps, procedures, or systems of thought processes;

(c) activities that when completed cannot fail to yield exact answers, known solutions, or ultimates that could be exactly predicted beforehand.

8/6/2011

RESEARCH METHODS

44CONCEPT STRETCHING

(TO BE AVOIDED)

Multi level governance

when refracted through the lens of

lived political experiences-- notions of

policy transfer and of the complex

politics of scale of interventions in

time, place and space--needed

8/6/2011

RESEARCH METHODS

45TOO VAGUE

(STRETCHING)

Decentralization

Would be misapplied in different societies

Too broad

8/6/2011

RESEARCH METHODS

46

TOO NARROW, AS WELL AS VAGUE: OBSERVATIONAL

CONCEPTS

Can be moved up or down the abstract ladder

Communication puzzle, group, decision, problem, conflict, and participation.

8/6/2011

RESEARCH METHODS

47

8/6/2011

Connotation (Depth, Intension) I HAL Low Middle High

DENOTATION

(EXTENSION,

BREADTH)

High abstract level • Traveling concept

•Stretched concept

domain domain

— Broad coverage, of — Broad coverage

distinct classes of — Too many classes

things of things lumped

— Few, but determinate together (with little

attributes of attention to the

distinct classes precision of their

— Precise at a many attributes)

universal level — Packed or too allembracing

RESEARCH METHODS

48

8/6/2011

Connotation (Depth, Intension)

I I I I I I I I I I I I

Low Middle High

DENOTATION (EXTENSION,

BREADTH)

MAL

Mid abstract level • Generalizable

nonuniversal concept domain

— Breadth balanced

— Medium range concepts

— Similarities

RESEARCH METHODS

49

Connotation (Depth, Intension) I I I I I I I I I I I I HAL Low Middle High

DENOTATION (EXTENSION, BREADTH)

MAL LAL • Configurative • Taxonomic domain

Low abstract level

situational concept doman

— Narrow coverage

— Narrow coverage

— Many attributes

— Too few attributes for each class

— Configurative/ — Distinctive typologies

Imprecise taxonomies or precise taxonomies

specific generalities precise at a specific level

taxonomies — Precise at a specific level

8/6/2011

RESEARCH METHODS

50MOVING TO THE

UPPER LEFT CORNER

Move up the ladder by negating

Concepts with negation are precise

Those without are indeterminant

8/6/2011

RESEARCH METHODS

51EXAMPLE: STUDIES OF

“PROBLEM”

The concept of problem is studies without its meaning boundaries being distinctively delineated.

Generated by concrete, intuitive, ad hoc conceptualizations.

Studies of structured problems fall widely apart from a focus on conflict resolution to one on negotiation.

8/6/2011

RESEARCH METHODS

52THEORETICAL

CONCEPTS

Theoretical or universal concepts are defined by their systemic meaning in the sense that each meaning derives from the part that the concept plays in the theory.

Open concepts reflect the availability of different operational criteria for application to different contexts.

As a result, their meanings are not fully defined by reference to observable things and their characteristics.

8/6/2011

RESEARCH METHODS

53 EXAMPLES

Efficacy (Jones, 1986; Osigweh, 1983)

Synergy, feedback, adaptation (Osigweh 1985b, pp. 153- 157),

Homeostasis and isomorphism

8/6/2011

RESEARCH METHODS

54EMPIRICAL CONCEPTS

Observational

Can be moved up or down

8/6/2011

RESEARCH METHODS

55 EXAMPLES

communication,

puzzle,

group,

decision,

conflict

8/6/2011

RESEARCH METHODS

56MOVING UP THE

LADDER

Requires that you keep it clear and not just extend meaning to other kinds of observations

8/6/2011

RESEARCH METHODS

57 ADVICE

Define a concept by saying what it is not;

Give it precise boundaries

Make it clear

8/6/2011

RESEARCH METHODS 58

THEORETICAL CONTRIBUTION

Definitions and

Approaches

8/6/2011

RESEARCH METHODS

59 COMPONENTS

Which factors (variables, constructs, concepts) logically should be considered as part of the explanation

Two criteria exist for judging the extent to which we have included the "right" factors: comprehensiveness (i. e., are all relevant

factors included?) and

parsimony (i.e., should some factors be deleted because they add little additional value to our under-standing?).

8/6/2011

RESEARCH METHODS

60 RELATIONSHIPS

Operationally this involves using "arrows" to connect the "boxes."

Such a step adds order to the conceptualization by explicitly delineating patterns.

It typically introduces causality.

8/6/2011

RESEARCH METHODS

61 WHY IMPORTANT?

What justifies the selection of factors and the proposed causal relationships?

This rationale constitutes the theory's assumptions

It welds the model together.

Why should colleagues give credence to this particular representation of the phenomena?

The answer lies in the logic underlying the model.

The soundness of fundamental views of economics or processes

8/6/2011

RESEARCH METHODS

62FROM THE

EDITOR:

“The mission of a theory-development journal is to challenge and extend existing knowledge, not simply to rewrite it. Therefore, authors should push back the boundaries of our knowledge by providing compelling and logical justifications for altered views.”

8/6/2011

RESEARCH METHODS

63 SET CONTEXTUAL

LIMITS

These temporal and contextual factors set the boundaries of generalizability, and as such constitute the range of the theory.

Do theoretical effects vary over time, either because other time-dependent variables are theoretically important or because the theoretical effect is unstable for some reason.

8/6/2011

RESEARCH METHODS

64 SUMMARY

For purposes of publishing:

Clarity in main ideas

Currency in quality

8/6/2011

RESEARCH METHODS

65PUBLISHING IN

ECONOMICS

Papers are assumed to vary along two quality dimensions

The former is interpreted as the importance of a paper’s main ideas and

the latter as other aspects of quality. Observed trends are thought of as reflecting increases in this

8/6/2011

THE END