Doing EmpiricalPolitical Research

James TVI. CarlsonProvidence College

Mark S. HydeProvidence College

Houghton IVIifFlin Company Boston New York

Contents

Preface xix

Part 1 Science and Behavior 1

Chapter 1 How Do We Know What's True? 2LEARNING O B J E C T I V E S 2

Asking and Answering Questions About Politics 2Investment 3Authority 5Logic 6Faith 8Science 9

The Boundaries and Limits of Science 10Tentative Truth 10Fact-Based Versus Value-Based Questions 11Recursive Behavior 12Assessing Objective Reality 13Free Will Versus Determinism 14

Conclusion 15Summary of the Main Points 16Terms Introduced 16Selected Readings 16Activities 16

Activity 1.1 Knowledge and Attitudes 16Activity 1.2 Empirically Based Scientific Knowledge 17

Chapter 2 Using the Scientific lVlethod in Political Science 18L E A R N I N G O B J E C T I V E S 18

We Are All Scientists 18

Characteristics and Assumptions of the Scientific Approach toUnderstanding Politics 20

Characteristics of a Useful Social Science 20Assumptions of Social Science 25

V

Contents

The Wheel of Science Describes the Stages in the Research Process 28The Wheel of Science 28Stages in the Research Process 34

Conclusion 37Summary of the Main Points 38Terms Introduced 38Selected Readings 38Activities 38

Activity 2.1 Identifying Distortions in News Reportsand Commentaries 38

Activity 2.2 Description and Explanation 39Activity 2.3 Deductive and Inductive Inference 41

Part 2 Preparing for Research 43

Chapter 3 Formulating Problems and Hypotheses 44L E A R N I N G O B J E C T I V E S 44

How to Develop a Political Research Question 44

Sources of Research Topics 45

Criteria for Evaluating Potential Political Research Topics 47

Ethical Issues in Political Science Research 50Some Specific Ethical Dilemmas 51

Risk of Harm to Subjects 51Voluntary Participation, New Inequalities, and Coercion 52Covert Research, Invasion of Privacy, and Deception 53

Protecting the People We Study 54Institutional Review Boards 54Informed Consent and Debriefing 55Anonymity and Confidentiality 55Ethical Guidelines Set Forth by Professional Organizations 56

The Freedom to Conduct Research and the Rightsof People Under Study 56

Transforming Research Topics into Researchable Questions:Narrowing the Focus 57

Useful Hypotheses: Definition and Functions 59Characteristics of Useful Hypotheses 59

Elements in Hypotheses: Concepts, Variables, and Units of Analysis 66

Contents

Conclusion 68Summary of the Main Points 68Terms Introduced 69Selected Readings 69Activities 69

Activity 3.1 Identifying Examples of Ethics Violationsin Social and Political Research 69

Activity 3.2 Converting a General Topic to Hypotheses 70Activity 3.3 Identifying Hypotheses 70Activity 3.4 Reformulating Value Statements 71Activity 3.5 Reformulating Hypotheses 72Activity 3.6 Identifying Components in Hypotheses 72

Chapter 4 Determining What Is Known:Building a Bibliography 73

L E A R N I N G O B J E C T I V E S 73

Serious Political Science Research Begins (But DoesNot End) with the Library 73

Developing a Strategy for Finding Sources and Keeping a Record 76

Finding Resources 78Using the Library Catalog to Identify Books 79Reference Works: Dictionaries, Encyclopedias, Almanacs,

and Yearbooks 81Specialized Dictionaries 81Encyclopedias 82Almanacs and Yearbooks 82

Finding Articles in Periodicals 84Online Databases 85Locating Unpublished Professional Papers 85Locating Material on the World Wide Web 86Letting Others Help with the Work: Bibliographical References 87

Determining Whether Sources Are Relevant 88Books 88Scholarly Articles 89Web Sites 90Creating a Bibliography: A Matter of Form 90

Conclusion 92Summary of the Main Points 92

Contents

Terms Introduced 92Selected Readings 92Activities 93

Activity 4.1 Using Search Engines 93Activity 4.2 Comparing Resources Revealed by

Academic and Nonacademic Databases 94Activity 4.3 Building a Bibliography 94

Chapter 5 Reviewing Previous Research 96L E A R N I N G O B J E C T I V E S 96

Reading and Evaluating Empirical Political Research 96

Reading and Dissecting an Article Reporting Research 98

"Gender and Citizen Participation: Is There a Different Voice?" 101Abstract 101Introduction 101Review of Literature 102Conceptual Hypotheses 104Methods 104Initial Findings 105Findings 108Conclusion 116Appendix 118References 118

Writing a Review Comparing Research Reports 121

Conclusion 123Summary of the Main Points 123Selected Readings 123Activities 124

Activity 5.1 Writing a Summary of an ArticleThat Reports Empirical Political Research 124

Activity 5.2 Identifying Parts of an Article Publishedin a Social Science Journal 124

Activity 5.3 Writing a Comparative Literature Review 124

Chapter 6 Assessing Relationships:Association or Causality? 125

LEARNING OBJECTIVES 125

Contents

Looking for Explanations 125Independent and Dependent Variables:

The Direction and Strength of Relationships 126Association Versus Causation 128Criteria for Causality 129

Research Design 130Null Hypothesis 131Control Variables and Causality 132Elaborating a Causal Hypothesis 135

Conclusion 139Summary of the Main Points 139Terms Introduced 140Selected Readings 140Activities 140

Activity 6.1 Hypotheses, Variables, and Causality 140Activity 6.2 Causal Chains 141

Chapter 7 Conceptualizing, Operationalizing,and Measuring Variables 143

L E A R N I N G O B J E C T I V E S 143

From Abstract Concept to Concrete Measurement 143

Concepts and Variables 147

Operationalization and Measurement 149Precision: Levels of Measurement 151Accuracy of Measurement 155Validity 158Reliability 160Maximizing Validity and Reliability 162

From Conceptual to Operational Hypotheses 164

Conclusion 165Summary of the Main Points 166Terms Introduced 166Selected Readings 167Activities 167

Activity 7.1 Inferring Concepts from Measurements 167Activity 7.2 Identifying Levels of Measurement 168Activity 7.3 Operationalizing Concepts and Hypotheses 168

Contents

Part 3 Data Collection 171

Chapter 8 Organizing and Managing Data 172L E A R N I N G O B J E C T I V E S 172

Mounds of Data 172

Data Analysis Software 174The Data Matrix 174

Codebooks 180Creating Your Own Codebook and SPSS Data File 184

Running a Frequency Distribution to Describe Your Data 187

Conclusion 189Summary of the Main Points 190Terms Introduced 190Selected Readings 190Activities 190

Activity 8.1 Running a Frequency Distribution in SPSS 190Activity 8.2 More Frequency Distributions Using SPSS 191

Chapter 9 How to Achieve Maximum Representativeness:Sampling 192

L E A R N I N G O B J E C T I V E S 192

Choosing Representative Units of Analysis 192

The Concept and Terminology of Sampling 193

Types of Samples 196Probability Sampling 198

Simple Random Sampling 198Systematic Samples 199Stratified Sampling 200Cluster Samples 202Telephone Samples 205

Nonprobability Sampling 206Convenience Sampling 207Judgmental Sampling 207Quota Sampling 208Snowball Sampling 210

Contents

Sample Error and Sample Size 211Sampling Error 211Sample Size 214

Conclusion 216Summary of the Main Points 217Terms Introduced 217Selected Readings 218Activities 218

Activity 9.1 Evaluating Samples Reported in Articles inProfessional Journals 218

Activity 9.2 Evaluating Samples Reported Along with Poll Resultsby the Popular Press 220

Activity 9.3 Using SPSS to Select a Random Sample 220Activity 9.4 Selecting Student Samples 221

Chapter 10 Collecting Data Using Surveys 222LEARNING OBJECTIVES 222

Acquiring Survey Data 222

Developing Questions 224Question Form 224

Open-Ended Questions 224Closed-Ended Questions 225Filter and Contingency Questions 229

Focus on Content: Appropriate Question Wording 230Biased Questions 230Ambiguous Questions 232Double-Barreled Questions 232Negative Questions 232Questions That Encourage Socially Desirable Responses 232Questions That Assume Respondent Knowledge 233Questions of Excessive Length 234

Using Questions Developed by Others 235

Assembling the Survey Instrument 236Introduction and Instructions 236Question Order 237Format 239Pretesting 240

Contents

Administering the Survey 240Face-to-Face Interviews 240Telephone Interviews 241Self-Administered Questionnaires 242

Secondary Analysis of Survey Data 244

Conclusion 245Summary of the Main Points 245Terms Introduced 246Selected Readings 246Activities 247

Activity 10.1 Constructing a Questionnaire 247Activity 10.2 Accessing Data for Secondary Analysis 247

Chapter 11 Collecting and OrganizingData from Published Sources 249

L E A R N I N G O B J E C T I V E S 249

Previously Collected Data 249

Published Data 250Locating Published Data 250Evaluating the Usefulness of Published Data 251Collecting and Organizing Published Data 253

Published Data on Geographic Regions or Organizations 255Sources of Published Data That Describe Collectivities 258

Characteristics of Governments and Event Data for Nations 258Data on the United States and Its Geographical Subdivisions 259Data on Political Organizations 259

Cautions About Using Published Data Describing Collectivities 260Missing Data 260Ecological Fallaq' 261

Published Data on People 262Sources of Published Data That Describe Political Elites 262

Political Candidates 262Members of Congress 262Members of the Judiciary 263Presidents and Members of the Executive Branch 263Elites Outside the United States 263

Contents

Media Messages: Content Analysis 264Three Examples of Content Analysis 265

Elite Discourse During the Cold War 265Cops, Suspects, and Race on "Reality" Television Programs 265Characteristics of Countries with Elaborate

Parliamentary Web Sites 266The Process of Content Analysis 267

Specification of the Research Question and the Population 267Taking a Sample of Units of Observation and Recording Units 267Defining Variables and Categories of Content 268Developing an Instrument for Recording Data 269

Cautions Concerning Reliability and Validity 271

Conclusion 272Summary of the Main Points 272Terms Introduced 273Selected Readings 273Activities 274

Activity 11.1 Location of Data from Published Sources 274Activity 11.2 Preparing to Collect and Analyze Data 274Activity 11.3 A Content Analysis Project 275

Chapter 12 Studying Only a Pew Cases:Intensive Approaches 276

LEARNING OBJECTIVES 276

Extensive and Intensive Approaches 276

Case Studies 278

Experimentation 279Logic of Experiments 280Internal Versus External Validity 283Other Types of Experiments 283Quasi-Experiments 289Reminders About Ethics in Experiments 290

Q-Technique 291Overview 292

Q-samples 292P-samples 293The Sorting Operation and Conditions of Instruction 293Distinguishing Points of View 294

Contents

Using Q-Technique to Distinguish Conceptionsof Representation 294

Disadvantages and Advantages 296

Focus Groups 297

Conclusion 298Summary of the Main Points 299Terms Introduced 299Selected Readings 300Activities 300

Activity 12.1 Designing a Classic Experiment 300Activity 12.2 Designing an Experiment for Evaluation 300Activity 12.3 Designing an Experiment for a Hypothetical Situation 300Activity 12.4 Points of View Emerging from a Q-Sort 301

Part 4 Analyzing and Reporting Results 303

Chapter 13 How to Describe and Summarizea Single Variable 304

LEARNING OBJECTIVES 304

Why Statistics? 304

How Many Variables at What Level of Measurement? 306

Variables Measured at the Nominal and Ordinal Levels 307Frequency Distribution 308Measures of Central Tendency and Dispersion 311Mode, Median, Range, and Percentile 313

Variables Measured at the Interval and Ratio Levels 315The Mean 315The Standard Deviation 317The Standard Deviation and the Normal Curve 320The Standard Deviation and Probability Sampling 321Standard or Z-scores 323

Conclusion 324Summary of the Main Points 324Terms Introduced 325Selected Readings 32,5Activities 326

Activity 13.1 Calculating the Mean and Standard Deviation 326Activity 13.2 Using SPSS for Describing Frequency Distributions 326

Contents

Chapter 14 Constructing and Interpreting Bivariate Tables 328LEARNING O B J E C T I V E S 328

Tables Tell Us a Lot 328

Characteristics and Construction of Bivariate Tables 329Evaluating the Direction and Strength of Relationships 332Measures of Association 334

Lambda 336Gamma 337

Statistical Significance 338

Alternative Means for Organizing Percentage Tables 339

Conclusion 341Summary of the Main Points 341Terms Introduced 342Selected Readings 342Activities 342

Activity 14.1 Understanding the Information in a Table 342Activity 14.2 Percentage Tables and Hypotheses 343Activity 14.3 Creating a Bivariate Percentage Table with

Nominal Level Variables, Using SPSS 344Activity 14.4 Creating a Bivariate Percentage Table with

Ordinal Level Variables, Using SPSS 345

Chapter 15 Graphing and Describing LinearBivariate Relationships 346

L E A R N I N G O B J E C T I V E S 346

Relationships Between Two Interval/Ratio Variables 346The Scatterplot 347Outliers 349

Interpreting a Scatterplot by Using a Regression Line 352Some Concerns About Regression Analysis 354Correlation—-A Measure of Association 356Regression Analysis Results in SPSS 358

Conclusion 359Summary of the Main Points 360Terms Introduced 360Selected Readings 361

XVI Contents

Activities 361Activity 15.1 Drawing a Regression Line in SPSS 361

Activity 15.2 Calculating a Regression Coefficient, Beta Weight,

and Correlation Coefficient in SPSS 361

Chapter 16 Analyzing More Than Two Variables 363L E A R N I N G O B J E C T I V E S 363

Sorting Out Multiple Influences 363

Nominal- and Ordinal-Level Data 365Adding a Control Variable to the Test of a Bivariate Relationship 365

Effects of Control Variables on Original Relationships 368Direct Effect of the Control Variable on

the Dependent Variable 370Combined Effect of Control and Independent

Variables on the Dependent Variable 372Condensing Tables to Simplify Interpretation 372

Interval- and Ratio-Level Data 375An Example of Multiple Linear Regression 377Measure of Association—Multiple Correlation Coefficient 381

Conclusion 382Summary of the Main Points 382Terms Introduced 383Selected Readings 383Activities 383

Activity 16.1 Reading a Multivariate Percentage Table 383

Activity 16.2 Constructing a Condensed Table 384

Activity 16.3 Creating and Editing a Multivariate Percentage Table

Using SPSS 384

Activity 16.4 Multiple Regression Using SPSS 386

Chapter 17 Determining the StatisticalSignificance of Results 387

L E A R N I N G O B J E C T I V E S 387

Sample Versus Population Relationships 387

The Framework of Statistical Significance 388Type I Versus Type II Errors 389

Contents

Statistical Significance Versus Substantive Significance 389Accepted Benchmarks for Statistical Significance 390

Tests of Statistical Significance 391Chi Square 392t-test and F-test 396Statistical Significance for Regression Analysis 401

Conclusion 402Summary of the Main Points 403Terms Introduced 403Selected Readings 404Activities 404

Activity 17.1 Computing and Interpreting a Chi Square 404Activity 17.2 Computing Chi Square Using SPSS 405Activity 17.3 Using SPSS to Compare Two Means with a t-test 405Activity 17.4 Using SPSS to Compare Three Means

with an F-test (ANOVA) 406

Chapter 18 Reporting the Results of EmpiricalPolitical Research: Pulling It All Together 407

L E A R N I N G O B J E C T I V E S 407

The Work Is Not Finished Until You Communicate Your Results 407

Forms of Reporting Empirical Political Research 408

Organization and Presentation of the Elementsof a Research Report 410

Title 411Abstract 411Introduction 412Review of Literature 412Statement of Hypotheses 414Method 414Findings 415Discussion and Interpretation of Findings 416Summary and Conclusions 416Appendix and Substantive Notes 416References 416

Writing: Style and Form 417

Contents

Presenting Quantitative Results 419Creating Effective Tables 420

Title 420Structure and Identifiers 420Analyzed Data in the Body of the Table 423Supplementary Information at the Bottom of the Table 423

Creating Effective Graphs and VisualRepresentations of Findings 424

Bar Charts and Histograms 425Pie Charts 426Line Graphs 427Mapping Data 428

Ethical Considerations in Reporting Research 430

Conclusion 432Summary of the Main Points 432Terms Introduced 433Selected Readings 433Activities 434

Activity 18.1 Media Reports of Social Research 434Activity 18.2 Writing Abstracts of Research Reports 434Activity 18.3 Creating a Simple Bar Chart Using SPSS 434Activity 18.4 Creating a Simple Pie Chart Using SPSS 435Activity 18.5 Editing Tables in SPSS and Exporting Them to

Word Processing Documents 435

Appendix 1 A-1Avoiding Plagiarism A-1

Appendix 2 A-6Table A: Random Numbers A-6Table B: Critical Values of Chi Square A-8Table C: Critical Values of t A-9Table D: Critical Values of F A-10

Notes N-1

Glossary G-1

Index 1-1