![Page 1: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/1.jpg)
USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND
PREDICTORS OF POST-TRAUMATIC STRESS SYMPTOMATOLOGY
IN A SAMPLE OF FEMALE RAPE VICTIMS
Amy M. Cohn, James Grice, Brett Hagman, and Liz Schlimgen
November 17, 2012ABCT
![Page 2: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/2.jpg)
Aim of Presentation
Test a mediation model with daily diary data
Compare multi-level modeling (MM) to Observation Orientated Modeling (OOM)
Daily post-traumatic
stress symptoms
Daily negative
affect
Daily alcohol involvement
![Page 3: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/3.jpg)
Assumptions of MM have limitations
Homogeneity between individuals Within-person fluctuations in behavior represented
as aggregated, over-time association (slope) Linear monotonic changes in behavior over time
Associations between variables are additive, not (necessarily) dynamic
Random sampling Normal population distribution (for differences)
Abstract population parameters that have little (or no) empirical basis
The population is completely theoretical
![Page 4: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/4.jpg)
What’s wrong with the p-value? Not a new argument Relies on population statistics that may
not represent the data The sample is different, the way you collect
the data is different, the questions you ask are different….
Creates a “false belief” in the validity and generalizability of findings Many study results cannot be replicated
![Page 5: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/5.jpg)
Observation Orientated Modeling
“Why is it that the patterns of phenomena are the way they are?” (Harre, 1986)
“Fundamentally incompatible with prevailing research tradition in Psychology”(Grice, 2012)
![Page 6: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/6.jpg)
OOM Incompatible with MM
Variable-based approach (such as MM) is linear, causal, and based on aggregate statistics such as betas and variances
OOM approach is integrative and focused at the level of the individual
Independent variable
Dependent variable
![Page 7: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/7.jpg)
OOM
Non-parametric, idiographic Examines qualitative pattern in the data Rooted in Aristotle’s notion that most things in
nature are not produced by people The researcher does not control everything in a
study Eschews null hypothesis significance testing
(NHST) Results based on probabilities found within the
data, not comparison to population distribution Variables not described in a cause-effect format
OOM describes how the effect conforms to the cause
![Page 8: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/8.jpg)
OOM
To Repeat……. EFFECTS SHOULD CONFORM TO THEIR
CAUSES What the @#*&$?
We do not always know why participants do what they do
Effects are never truly “causal” Unmeasured pieces of “error” or “garbage” in
the data collection process With OOM, patterns of observations reveal
what are in the data – The EFFECTS
![Page 9: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/9.jpg)
Study 1 Hypotheses
NA will be greater on days characterized by greater PTSD
Craving and consumption will be greater on days characterized by more intense PTSD and NA
Daily PTSD
symptoms
Daily NAa
Alcohol Involvem
ent
b
c (c’)
![Page 10: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/10.jpg)
Sample characteristics (n = 54)
Characteristic Statistic
Age 26 (SD = 9.08)
Some college or post high school education
70% (n = 38)
Employed (full or part-time) 25% ( n = 15)
Single 60% (n = 34)
Caucasian 70% ( n = 38)
Income (Median) $9,000 (SD = $20,032)
54 untreated female rape victims who completed at least one day of daily interactive voice response (IVR) monitoring
![Page 11: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/11.jpg)
IVR Assessment
1x a day in the evening (6pm to 12am) Alcohol use, negative affect intensity,
craving intensity, and PTSD symptoms (presence/absence)
Since previous phone call 93% compliance rate
13/14 calls were completed
![Page 12: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/12.jpg)
HLM Analysis DV’s = Number of drinks consumed and
intensity of craving (850 observations) Controlled for day of week Poisson distribution with log link function
for drinking Examined relationship of one variable
EACH DAY to the outcome variable ON THAT SAME DAY
![Page 13: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/13.jpg)
Figure 1. Mediation of NA on the PTSD-alcohol link.
Daily NA
Daily PTSD symptoms
Number of standard
drinks/day
0.13*** 0.42***
-0.14
*** p < .001
(-0.02)
In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)
![Page 14: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/14.jpg)
Figure 2. Mediation of NA of the PTSD-craving link.
Daily NA
Daily PTSD symptoms
Daily craving intensity
0.13*** 0.39***
-0.10
Note. Covariates included day of the week, baseline PTSD symptom severity, baseline alcohol use. *** p < .001
(-0.12)
In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)
![Page 15: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/15.jpg)
OOM Analysis: Mediation Steps
Daily PTSD symptoms
Daily NA
Number of standard
drinks/day
Step 1: Because the effect conforms to the cause, we first examine the probability that number of standard drinks consumed each day conforms to daily ratings of NA intensity
![Page 16: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/16.jpg)
OOM Results
Accuracy rate: % observations correctly classified out of total number of observations Missing data is not a problem
Randomization test Out of1000 trials of randomized versions of the
same observations, what number of instances do we obtain a result high or higher than percent correct classification?
Binomial p-value or chance value should be small (less than .01) Indicates pattern is unique
Results for individual and group-level patterns
![Page 17: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/17.jpg)
Perfect Ordinal Matches for 14 Occasions
Proportion of Matches = 1.00; Binomial p-value = .00012
![Page 18: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/18.jpg)
Weak Ordinal Matches for 14 Occasions
Proportion of Matches = .15; p-value = .99
![Page 19: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/19.jpg)
Overall Results (n = 54 women) :
Number of Matches : 123 Number of Observations : 399 Proportion of Matches : 0.31
Randomization Results :
Observed Proportion of Matches : 0.31
Number of Randomized Trials : 5000.00 Minimum Random Proportion of Matches : 0.24 Maximum Random Proportion of Matches : 0.36 Values >= Observed Proportion : 1758.00 Matching c-value : 0.35
Proportion of matches is unimpressive at .31
C-value of the Randomization Test indicates that .31 is not an unusual aggregate outcome compared to randomized versions of the same observations
Aggregate Results for all 54 Women
![Page 20: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/20.jpg)
Aggregate Results for all 54 Women 1.Proportion of Matches > .50 for only 9
women(5 of these women had 7 or fewer data points)
2.Fourteen women (26%) showed no variability in their drinking across the 14 days
3.An additional 6 women drank on only one day
![Page 21: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/21.jpg)
Conclusions
Women showing no variability in drinking and those who did not drink across 14 days are “swept” into HLM aggregates Should this disturb us?
OOM recognizes women with no variability in their drinking Since OOM not based on means and
variances, impact of these women does not adversely effect the overall percent matches
![Page 22: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/22.jpg)
Conclusions
OOM “effect sizes” are proportions of matches that are readily interpretable and linkable to individual women No need for interpretations- such as Cohen’s effect
sizes Idealized p-values are primary in HLM, even over
effect sizes Even if effect is small, if p < .05 we say “YES”!
Proportions of matches consistent with causal hypotheses are primary in OOM Distribution free p-values (from binomial and
randomization tests) are secondary
![Page 23: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/23.jpg)
Erroneously enticed to posit a mediation mechanism that operates successfully for every woman with HLM OOM treats the women and their individual observations as primary Does not rely on p-values, means, or
variances estimated from a theoretical population
OOM develops integrated models More accurately explains patterns of
observations
Summary
![Page 24: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/24.jpg)
Acknowledgments
Participants who dedicated their time and effort
Research assistants: Jessica Mitchell, Stephanie Bramm, Sarah Ehlke, Ruschelle Leone, Joanne Wang
Grants: NIDA P30DA028807; USF 582000 / MHBCSG
![Page 25: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/25.jpg)
Thank you!
Questions?
Dr. James GriceDepartment of PsychologyOklahoma State University
Stillwater, OK [email protected]
![Page 26: USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND PREDICTORS OF POST- TRAUMATIC STRESS SYMPTOMATOLOGY IN A SAMPLE OF FEMALE RAPE VICTIMS](https://reader036.vdocuments.net/reader036/viewer/2022070410/56649efd5503460f94c11157/html5/thumbnails/26.jpg)
Deep Structure Transformation
M F
0 10 11 01 0
0 1 2 3 4 5 6 7 8 9 10
0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 1 00 0 1 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 1 00 0 1 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0
0 10 11 01 0
0 10 11 01 0
ConformedEffectObservations
EffectObservations
CauseObservations
EffectObservations
3. Accuracy is our central judgment (not statistical significance) and shows how many observations were correctly classified by the algorithm, or how many observations match the pattern.
1. Observations are transformed into their “deep structure”
2. Rotate deep structure effect observations into “conformity” with deep structure observations