i/o chapter 2 by jason manaois
DESCRIPTION
I/O chapter 2 by Jason ManaoisTRANSCRIPT
Research in Psychology
A Scientific Endeavor
Objectives Describe the goals of psychological research
Learn the process of doing research
Understand the different research methods
Explore the advantages and disadvantages of doing research and its different methodologies
Apply ethical consideration in doing research
Definition RESEARCH A formal process by which knowledge is
produced and understood
GENERALIZABILITY The extent to which conclusions drawn from
one research study spread or apply to a larger population.
Goals of Psychological Research
Description of social behavior Are people who grow up in warm climates different
from those in cold climates?
Establish a relationship between cause & effect Does heat cause higher amounts of aggression?
Develop theories about why people behave the way that they do We dislike Duke students to feel better about
ourselves
Application Creating effective therapeutic treatments, more
successful negotiation tactics, and greater understanding amongst groups of people
Empirical Research Cycle
Empirical ResearchEmpirical Knowledge based on direct
observation
Theory Set of ideas which try to explain what
we observe Theoretical diversity A statement that proposes to explain
relationship among phenomena of interest.
Statement of a ProblemInductive Method A research process in which conclusions are
drawn about a general class of objects or people based on knowledge of a specific member of the class under investigation.
DATA THEORY
Deductive Method A research process in which conclusions are
drawn about a specific member of a class of objects or people based on knowledge of the general class under investigation.
THEORY Collects DATA
Design of the Research StudyResearch Design A plan for conducting scientific research for the purpose of
learning about the phenomenon of interest
Internal Validity The degree to w/c the relationship evidenced among the
variables in a particular research study are accurate or true.
External Validity The degree to w/c the relationship evidenced among the
variables in a particular research study are generalizable or accurate in other contexts.
Concerns
Naturalness of the Research Setting Laboratory vs. Field
Degree of Control Are you able to control or manage the
conduct of the research?
Primary Research
Laboratory ExperimentQuasi-experimentQuestionnaireObservation
META-analysis
Secondary Research
Qualitative Research
Ethnography A research method that utilizes field
observation to study a society’s culture EMIC – insider’s view ETIC – external view
Measurement of Variables
Variables
Quantitative Variables E.g. Age, weight
Categorical Variables E.g. Gender, race
Variables
Independent Variables A variable that can be manipulated to
influence the values of the dependent variable. (the one that you manipulate)
Dependent Variables A variable whose values are influenced by
the independent variables. (the one that you measure)
Examples
Leadership Style
Employee Performance
Employee Performance
Employee Trainability
Variable in I/O
Predictor Variable A variable used to predict or forecast
a criterion variable
Criterion Variable A variable that is a primary object of
a research study; it is forecasted by a predictor variable
Examples
Personality LeadershipStyle
Employee Performance
Employee Morale
Analysis of Data
Descriptive Statistics Mean Median Mode
Variability Range SD Correlation
Conclusions from Research
After collection and analyzing data, the researcher draws conclusions. Answers research hypothesis or research problem.Generalizability of research findings???
Ethical Issues
Right to informed consentRight to privacyRight to confidentialityRight to protection from deceptionRight to debriefing
Another Perspective
The Process of Doing Research
First, select a topic Good theory:
Has predictive power Is simple & straightforward
Then, search the literature Find out what others have
done that may be applicable to your area of interest
The Process of Doing Research
Next, formulate hypotheses Hypothesis: specific statement
of expectation derived from theory State the relationship between
two variables
Variable: can be any event, characteristic, condition, or behavior
The Process of Doing Research
Then pick your research method Experimental vs. correlational (DesignDesign) Field vs. laboratory (SettingSetting)
Finally, collect & analyze yourdata
Let’s take a closer look . . .at variables
Dependent variable (outcome variable) Dependent on the influence of other factor(s) How do we operationalize?
Independent variable (predictor variable) Factor(s) that change the outcome variable How do we operationalize & manipulate? Control group
Let’s take a closer look . . . at research methodsExperimental vs. correlational designs Correlational: observe the relationship
between two variables Describe patterns of behavior
Types include Naturalistic observation Case studies Surveys
Correlational researchAdvantages Sometimes manipulation of variables
is impossible or unethical Efficient – look at lots of data
Disadvantages CANNOT DETERMINE CAUSATION Could be a lurking variable
Experimental ResearchResearcher manipulates one variable (IV) to see effect on other variable (DV) Try to hold everything else constant
True experiments have Random sampling: selecting Ps
randomly from population Random assignment: chance
assignment to condition
Ethics in ResearchShould the study be done? Value vs. potential cost APA guidelines, colleagues
How do we protect Ps? Informed consent Confidentiality & anonymity Debriefing
Organizational Research Methods:
CAUSALITY
What Do We Mean By Causality?
Relationship between two events where one is a consequence of the otherDeterminism: A (cause) leads to B (effect)“In the strict formulation of the law of causality—if we know the present, we can calculate the future—it is not the conclusion that is wrong but the premise”.On an implication of the uncertainty principle. Werner Heisenberg
Heisenberg & Uncertainty Principle
Certain properties of subatomic particles are linked so the more accurately you know one, the less accurately you know the other We can compute probabilities not certainties Argues against determinism
“Physics should only describe the correlation of observations; there is no real world with causality”Heisenberg, 1927, Zeitschrift für Physik
Psychology, like quantum physics, is probabalistic
Cause Versus Effect
Effect of a Cause (Description) What follows a cause?
Cause of an Effect (Explanation) Why did the effect happen?
Holland, P. W. (1988). Causal inference, path analysis, and recursive structural equations models. Sociological Methodology, 18, 449-484.
Three Elements of Causal Case
Cause and effect are relatedCause preceded effectNo plausible alternative explanations
John Stuart Mill
Experiment
Vary something to discover effects Shows association Shows time sequence Can rule out only some alternatives
Confounds Boundary conditions (generalizability)
Good for causal description not explanationNatural science control through precise measurement Sterile test tubes, electronic instruments
Studying and Performance
Students randomly assigned to study amountTest scores as DVDid studying lead to test results? Encouragement led to test results Impact on studying unclear Effect of studying unclear
What was cause of test results?
Holland, P. W. (1988). Causal inference, path analysis, and recursive structural equations models. Sociological Methodology, 18, 449-484.
Nonexperimental Research Strategy
1. Determine covariation2. Test for time sequence
• Longitudinal design• Quasi-experiment
3. Rule out plausible alternatives• Based on data/theory• Logical
Can Job Satisfaction Cause Gender?
Correlation of Gender and satisfaction = group mean differencesSatisfaction can’t cause someone’s genderSatisfaction can be the cause of gender distribution of a sampleSuppose Females have higher satisfaction than MalesMultiple reasons
Alternative Gender-Job Satisfaction Model
Females more likely to quit dissatisfying jobsDissatisfaction causes gender distributionGender moderates relation of satisfaction with quitting
Sati
sfact
ion
MalesFemales
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
….
.
.
.
More Alternatives1. Women less likely to take
dissatisfying job (better job decisions)2. Women less likely to be hired into
dissatisfying jobs (protected)3. Women less likely to be
bullied/mistreated4. Women given more realistic previews
(lower expectations)5. Women more socially skilled at
getting what they want at work
How To Use Controls
Controls great devices to test hypotheses/theoryRule in/out plausible alternativesBest based on theorySequence of tests
Control Strategy1. Test that A and B are related
• Salary relates to job satisfaction
2. Confirm/disconfirm control variable• Gender relates to both
3. Generate/test alternative explanations for control variable
• Differential expectations• Differential hiring rate• Differential job experience• Differential turnover rate
Validity and Threats To Validity
Validity Interpretation of constructs/results Inference based on purpose
Hypothesized causal connections among constructs
Nature of constructs Population of interest
People Settings
Four Types of Design Validity
Statistical conclusion Appropriate statistical method to make
desired inference
Internal validity Causal conclusions reasonable based on
design
Construct validity Interpretation of measures
External validity Generalizeability to population of interest
Threats to ValidityStatistical Conclusion Statistics used incorrectly Low power Poor measurement
Internal Validity Confounds of IV with other events,
variables Group differences (pre-existing or
attrition) Lack of temporal order Instrument changes
Threats To Validity 2Construct Validity Inadequate specification of theoretical
construct Unreliable measurement Biases Poor content validity
External Validity Inadequate specification of population Poor sampling of population
Subjects Settings
Qualitative Methods
What are qualitative methods Collection/analysis of written/spoken text Direct observation of behavior
Participant observationCase studyInterviewWritten materials Existing documents Open-ended questions
Qualitative Research 2
Accept subjectivity of science Is this an excuse?
Less driven by hypothesisAssumption that reality a social construction If no one knows I’ve been shot, am I really dead?
Interested in subject’s viewpointMore open-endedMore interested in contextLess interested in general principlesFocus more on interpretation than quantification
Analysis
Content Analysis Interviews Written materials Open-ended questions Audio or video recordings Quantifying
Counts of behaviors/events Categorization of incidents Multiple raters with high agreement
Nonquantitative Analysis of case Narrative description
The Value of the Qualitative Approach
What is the value/use of this approach?Is this science?Must everything be quantified?
Qualitative Organizational Research: Job Stress
Quantitative survey dominatesRole ambiguity and conflict dominated in 1980s & 1990s (Katz & Kahn)Dominated by Rizzo et al. weak scales
Keenan & Newton’s SIR
Stress Incident Record Describe event in prior 2 weeks Aroused negative emotion
Top stressful events for engineers Time/effort wasted Interpersonal conflict Work underload Work overload Conditions of employment
Subsequent SIR Research
Comparison of occupations Clerical: Work overload, lack of control Faculty: Interpersonal conflict, time
wasters Sales clerks: Interpersonal conflict, time
wasters
Informed subsequent quantitative studies Focus on more common stressors
Interpersonal conflict Organizational constraints
Cross-Cultural SIR Research
Comparison of university support staffIndia vs. U.S.
Stressor India US
Overload 0% 25.6%
Lack of control 0% 22.6%
Lack of structure 26.5% 0%
Constraints (Equipment) 15.4% 0%
Conflict 16.5% 12.3%
Research As Craft
Scholarly research as expertise not bag of tricksLogical caseGo beyond sheer technique Research not just formulaic/trends Not just using right design, measures,
stats
Developing the CraftExperienceTrying different things Constructs Designs/methods Problems Statistics
ReadingReviewingTeachingThinking/discussingCourses necessary but not sufficientLifelong learning—you are never done
Developing the CraftField values novelty and rigorDon’t be afraid of exploratory research Not much contribution if answer known in
advance
Look for surprisesDon’t be afraid to follow intuitionAsk interesting question without a clear answerFocus on interesting variablesGood papers tell stories Variables are characters Relationships among variables
Construct & External ValidityandMethod Variance
Constructs
Theoretical level Conceptual definitions of variables Basic building blocks of theories
Measurement level Operationalizations Based on theory of construct
What We Do With Constructs
DefineOperationalize/MeasureEstablish relations with other constructs Covariation Causation
External Validity: Population
Link between sample and theoretical populationDefine theoretical populationIdentify critical characteristicsCompare sample to population Employed individuals Do students qualify?
External Validity: SettingLink between current setting and other settings Organization Occupation
Identify critical characteristics of settingsCompare setting to others Lab to field
External Validity: Treatment/IV
Link between current treatment/IV and othersCompare treatment/IV Distance learning vs. traditional
External Validity: Outcome/DV
Link between current outcome/DV and othersWill results in study work similarly in nonresearch condition?Will different operationalizations of outcome have same result? Supervisor rating of performance vs.
objective Safety behavior versus accidents/injuries
When Politics Attack Science
EvolutionIQ and performanceDifferential validity of IQ testsOthers?
Quasi-Experimental Design
What is an experiment? Random assignment Creation of Conditions? Naturally occurring experiment
Quasi-experiment
Design without random assignmentComparison of conditionsResearcher created or existingCan characteristics of people be an IV? Gender Personality
Is a survey a quasi-experiment?
SettingsLaboratory vs. fieldLaboratory Setting in which phenomenon doesn’t
naturally occur
Field Setting in which phenomenon naturally
occurs
Classroom field for educational psychologistClassroom lab for us
Lab vs. Field Strengths/Weaknesses
Lab High level of control Easy to do experiments Limits to what can be studied Limited external validity of
population/setting
Field Limited control Difficult to do experiments Wide range of what can be studied High reliance on self-report High external validity
Lab in I/O Research
What’s the role of lab in I/O research?Stone suggests lab is as generalizeable as field. Do you agree?Stone says I/O field biased against lab. Is it?When should we do lab vs. field studies?
Challenges To Field Research
Access to organizations/subjectsLack of control Distal contact with subjects (surveys) Who participates Contaminating conditions
Participants discussing study
Lack of full cooperationOrganizational resistance to change
Survey Methods & Constructs
Survey methodsSamplingCross-cultural challenges Measurement equivalence/invariance
Survey SettingsWithin employer organizationWithin other organization University Professional association Community group Club
General population Phone book Door-to-door
MethodsQuestionnaire Paper-and-pencil E-mail Web
Interview Face-to-face Phone Video-phone E-mail Instant Message
PopulationSingle organizationMultiple organizations Within industry/section
Single occupationMultiple occupationsGeneral population Employed students
Sample Versus PopulationSurvey everyone in population vs. sample Single organization or unit of organization
Often survey goes to everyone Multiple organizations
Kessler: All psychology faculty Other organization
Professional association Often survey everyone
General population
Sampling DefinitionsPopulation – Aggregate of cases meeting specification All humans All working people All accountants Not always directly measurable
Sampling frame – List of all members of a population to be sampled List of all USF support personnel
Sampling Definitions cont.Stratum – Segment of a populationDivided by a characteristic Demographics
Male vs. female Job level
Manager vs. nonmanager Job title Occupation Department/division of organization
Instrument IssuesLinguistic meaning Translation – Back-translation
Calibration Numerical equivalence Cultural response tendencies
Asian modesty Latin expansiveness
Measurement equivalence Construct validity Factor Structure
What Is A Theory?Bernstein Set of propositions that account for
predict and control phenomena
Muchinsky Statement that explains relationships
among phenomena
Webster General or abstract principles of
science Explanation of phenomena
Types of Theories
Inductive Starts with data Theory explains observations
Deductive Starts with theory Data used to support/refute theory
Common Usage of Theory
Conjecture, opinion, speculation or hypothesis Wikipedia
Advantages
Integrates and summarizes large amounts of dataCan help predictGuides researchHelps frame good research questions
Disadvantages
Biases researchers“Theory, like mist of eyeglasses, obscures facts” (Charlie Chan in Muchinsky)“Facts are the enemy of truth” (Levine’s boss)A distraction as research does not require theory (Skinner)
Hypothesis
Statement of expected relationships among variablesTentativeMore limited than a theoryDoesn’t deal with process or explanation
Model
Representation of a phenomenonDescription of a complex entity or process Webster
Boxes and arrows showing causal flow
Theoretical Construct
Abstract representation of a characteristic of people, situation, or thingBuilding blocks of theories
Paradigm
Accepted scientific practiceRules and standards for scientific practiceLaw, theory, application and instrumentation that provide models for research. Thomas Kuhn
What Are Our Paradigms?
Behaviorism?Environment-perception-outcome approachSurveys
Ethics In Research
Ethical Practices
Conducting Research Treatment of human subjects Treatment of organizational subjects
Data Analysis/InterpretationDisseminating Results Publication
Peer reviewing
Ethical Codes
Appropriate moral behavior/practiceAccepted practicesBasic Principle: Do no harmProtect dignity, health, rights, well-beingCodes APA??
American Psychological Association Code
Largely Practice orientedFive principles Beneficence and Nonmaleficence [Do no
harm] Fidelity and Responsibility Integrity Justice Respect for People’s Rights and Dignity
Standards and practicesApplies to APA membershttp://www.apa.org/ethics/
PreamblePsychologists are committed to increasing scientific and
professional knowledge of behavior and people's understanding of themselves and others and to the use of such knowledge to improve the condition of individuals, organizations, and society. Psychologists respect and protect civil and human rights and the central importance of freedom of inquiry and expression in research, teaching, and publication. They strive to help the public in developing informed judgments and choices concerning human behavior. In doing so, they perform many roles, such as researcher, educator, diagnostician, therapist, supervisor, consultant, administrator, social interventionist, and expert witness.
APA Conflict Between Profession and Ethical Principles
Restriction of Advertising Violation of the law
Maximization of income for membersTolerance of torture Convoluted statements
Other associations manage to avoid such conflicts
Academy of Management CodeLargely academically orientedThree Principles
Responsibility Integrity Respect for people’s rights and dignity
Responsibility to Students Advancement of managerial knowledge AOM and larger profession Managers and practice of management All people in the worldhttp://www.aomonline.org/aom.asp?ID=&page_ID=239
Professional PrinciplesOur professional goals are to
enhance the learning of students and colleagues and the effectiveness of organizations through our teaching, research, and practice of management.
Principles Vs. Practice
Principles clear in theoryEthical line not always clearEthical dilemmas Harm can be done no matter what is
done Conflicting interests between parties
Employee versus organization Whose rights take priority?
Example: Exploitive Relationships
Principle Psychologists do not exploit persons over
whom they have supervisory, evaluative, or other authority
What does it mean to exploit?
Conducting Research
PrivacyInformed consentSafetyDebriefingInducements
Privacy
Anonymity: Best protection Procedures to match data without
identities
Confidentiality Security of identified data
Locked computer/cabinet/lab Encoding data Code numbers cross-referenced to names
Removing names and identifying information
Informed ConsentSubject must know what is involved Purpose Disclosure of risk Benefits of research
Researcher/society Subject
Privacy/confidentiality Who has access to data Who has access to identity
Right to withdraw Consequences of withdrawal
Safety
Minimize exposure to risk Workplace safety study: Control
group
Physical and psychological risk
Debriefing
Subject right to knowEducational experience for studentsWritten documentPresentationSurveys: Provide contact for follow-upProvide results in future upon request
Inducements
Pure Volunteer – no inducementCourse requirement Is this coercion?
Extra creditFinancial payment Is payment coercion?
Successful Research Career
Conducting good research Lead don’t follow
Visibility Good journals Conferences Other outlets Quantity
First authored publications Important more early in career
ImpactGrants
Programmatic
Program of research More conclusive Multiple tests Boundary conditions More impact through visibility Helps getting jobs Helps with tenure/promotion Can have more than one focus
Conducting Successful Research
Develop an interesting question Based on theory Based on literature Based on observation Based on organization need
Link question to literature Theoretical perspective Place in context of what’s been done Multiple types of evidence Consider other disciplines
Conducting Successful Research 2
Design one or more research strategies Lab vs. field Data collection technique
Survey, interview, observation, etc. Design
Experimental, quasi-experimental or observational
Cross-sectional or longitudinal Single-source or multisource
Instrumentation Existing or ad hoc
Conducting Successful Research 3
AnalysisHierarchy of methods simple to complex Descriptives Bi-variable relationships Test for controls Complex relationships
Multiple regression Factor analysis HLM SEM
Conducting Successful Research 4
Conclusions What’s reasonable based on data Alternative explanations Speculation Theoretical development Suggestions for future
ImpactEffect of work on field/worldCitations Sources
ISI Thomson Harzing’s Publish or Perish Others
Self-citation Citation studies
Individuals (e.g., Podsakoff et al. Journal of Management 2008)
Programs (e.g., Oliver et al. TIP, 2005)
Being attacked