the effects of raking and cell phone integration on brfss outcome s
DESCRIPTION
The Effects of Raking and Cell Phone Integration on BRFSS Outcome s. Machell Town, M.S. Carol Pierannunzi, Ph.D. . Division of Behavioral Surveillance. Office of Surveillance, Epidemiology, and Laboratory Services. Division of Behavioral Surveillance. Brief Agenda. Weighting procedures - PowerPoint PPT PresentationTRANSCRIPT
The Effects of Raking and Cell Phone Integration on BRFSS Outcome s
Machell Town, M.S. Carol Pierannunzi, Ph.D.
Division of Behavioral Surveillance
Office of Surveillance, Epidemiology, and Laboratory ServicesDivision of Behavioral Surveillance
Brief Agenda Weighting procedures
Design weights Post stratification Iterative proportional fitting
Why change weighting procedures now? Cell phone Computer capacity
Impact of changes on estimation BRFSS Examples of small and large impact Changes when cell phones are incorporated
Conclusions Brief look at state level phone use data
(preliminary)
WEIGHTING PROCEDURES
Design and GeoStrata Weighting Takes into account the geographic
region/strata of the sample. Design weight uses number of adults in
household and number of phones in household for landline sample.
BRFSS landline sample is drawn using low/high density strata within each of the regions (usually smaller than states)
Stratum weight (_STRWT) = NRECSTR/ NRECSEL
Calculating the Design Weight Design Weight = _STRWT* (1/NUMPHON2) *
NUMADULT NUMPHON2= number of phones within the household NUMADULT = number of adults eligible for the survey
within the household Questions for the design weights are asked
in screening questions and in demographic sections of the survey
WEIGHTINGPost -Stratification
Data Weighting Data weights take the design weighting and
incorporate characteristics of the population and the sample
Final Weights (_FINALWT) = Design Weight * some form of data weighting In past BRFSS used post stratification In 2008 Iterative Proportional Fitting was first used In 2011 Iterative Proportional Fitting will be only method
of data weighting for BRFSS
Where We Have Been---Post Stratification
Post Stratification is based on known demographics of the population. For BRFSS Post stratification included:
· Regions within states· Race/ Ethnicity (in detailed categories)· Gender· Age (in 7 categories)
Post-stratification forces the sum of the weighted frequencies to equal the population estimates for the region or state by race, age ,and gender.
Post stratification weights are applied to the responses, allowing for estimates of how groups of non-respondents would have answered survey questions.
Post-stratification Post-stratification Adjustment Factor is
calculated for each race/ethnicity, gender, and age group combination.
_POSTSTR = Population/Design weight within the weighting class cell.
Weight Trimming Sometimes post-
stratification resulted in very small or disproportionately large weights within age/race/gender/region categories.
Weight trimming or category collapsing would be done if categories were disproportionately large or too small (< 50 responses).
WEIGHTINGIterative Proportional Fitting (Raking)
Iterative Proportional FittingGender by
race/ethnicity
Age by gender
Age by race/ethnici
ty
Renter/owner
Education level
Marital status
Detailed race/ethnicity
Regions within states
Phone source
Rather than adjusting weights to categories, IPF adjusts for each dimension separately in an iterative process.The process will continue up to 75 times, or until data converges to Census estimates.
New Variables Introduced as Controls With IPF
Education Marital status Home ownership/renter Telephone source (cell phone
or landline)
Post Stratification vs. Iterative Proportional Fitting
Post Stratificatio
n
Iterative Proportional
Fitting
Operates with less computer time
Allows for incorporation of new variables.Allows for incorporation of cell phone data.Seems to more accurately represent population data (reduces bias).
Why Incorporate IPF Now? Computer capacity has increased. Cell phones are becoming larger percentage
of the total number of calls. Noncoverage with declining response rates
makes weighting more important than ever.
Raking – Iteration 1
16
First Control Variable
Output Weight Sum of
WeightsTarget Total
Sum of Weights
Difference
% of Output
WeightsTarget % of
WeightsDifference
in %Age 18-24,Male 87122.60 95468 -8345.40 6.533 7.159 -0.626Age 18-24,Female 77180.40 90249 -13068.60 5.788 6.768 -0.980Age 25-34,Male 109419.36 118670 -9250.64 8.206 8.899 -0.694Age 25-34,Female 114395.17 112007 2388.17 8.579 8.400 0.179Age 35-44,Male 121328.71 117184 4144.71 9.099 8.788 0.311Age 35-44,Female 115609.98 113779 1830.98 8.670 8.533 0.137Age 45-54,Male 138658.26 127077 11581.26 10.398 9.530 0.869Age 45-54,Female 136904.33 127439 9465.33 10.267 9.557 0.710Age 55-64,Male 90338.77 95032 -4693.23 6.775 7.127 -0.352Age 55-64,Female 91693.43 97422 -5728.57 6.876 7.306 -0.430Age 65-74,Male 57475.54 54171 3304.54 4.310 4.062 0.248Age 65-74,Female 62709.50 61828 881.50 4.703 4.637 0.066Age 75+,Male 49772.58 46515 3257.58 3.733 3.488 0.244Age 75+,Female 80867.37 76635 4232.37 6.064 5.747 0.317
Raking – Iteration 1
17
Second Control Variable
Output Weight Sum of Weights
Target Total
Sum of Weights
Difference
% of Output
Weights
Target % of
WeightsDifference
in %WH NH 1151321.16 1156947 -5625.84 86.340 86.762 -0.422
OT NH 15305.42 12036 3269.42 1.148 0.903 0.245
HISP 85300.51 84230 1070.51 6.397 6.317 0.080
BL NH,AS NH,AI NH 81548.91 80263 1285.91 6.116 6.019 0.096
Raking - Iteration 1
Third Control Variable
Input Weight Sum of Weights
Target Total
Sum of Weights
Difference
% of Input
WeightsTarget %
of WeightsDifference
in %Less than HS 89962.05 143928 -53966.35 6.746 10.793 -4.047
HS Grad 412857.99 414505 -1646.81 30.961 31.085 -0.123
Some College 388163.96 448218 -60054.20 29.109 33.613 -4.504
College Grad 442492.00 326825 115667.37 33.183 24.509 8.674
18
Raking – Iteration 1
19
Fourth Control Variable
Output Weight Sum of
WeightsTarget Total
Sum of Weights
Difference
% of Output
WeightsTarget % of
WeightsDifference
in %Married 816399.38 792326 24073.29 61.223 59.418 1.805
Never married, member unmarried couple
277180.73 300111 -22930.01 20.786 22.506 -1.720
Divorced, Widowed, Separated 239895.88 241039 -1143.29 17.990 18.076 -0.086
Fifth Control Variable
Output Weight Sum of Weights
Target Total
Sum of Weights
Difference
% of Output
WeightsTarget % of
WeightsDifference
in %Phone interruption 78558.62 82944 -4385.49 5.891 6.220 -0.329
No Phone Interruption 1254917.38 1250532 4385.49 94.109 93.780 0.329
Raking – Iteration 1
20
Sixth Control Variable
Output Weight Sum of
WeightsTarget Total
Sum of Weights
Difference
% of Output
WeightsTarget % of
WeightsDifference
in %Male, WH NH 553107.34 552171 936.34 41.479 41.408 0.070
Male, BL NH,AS NH,AI NH,OT NH,HISP
101008.49 101946 -937.51 7.575 7.645 -0.070
Female, WH NH 598213.82 604776 -6562.18 44.861 45.353 -0.492
Female, HISP 38304.69 32837 5467.69 2.873 2.463 0.410
Female, BL NH,AS NH,AI NH,OT NH
42841.66 41746 1095.66 3.213 3.131 0.082
Raking – Iteration 1
21
Seventh Control Variable
Output Weight Sum of
WeightsTarget Total
Sum of Weights
Difference
% of Output
WeightsTarget % of
WeightsDifference
in %18-34, WH NH 308020.95 332809 -24788.05 23.099 24.958 -1.859
18-34, BL NH,AS NH,AI NH,OT NH,HISP
80096.58 83585 -3488.42 6.007 6.268 -0.262
35-54, WH NH 442299.71 421539 20760.71 33.169 31.612 1.557
35-54, BL NH,AS NH,AI NH,OT NH,HISP
70201.57 63940 6261.57 5.265 4.795 0.470
55+, WH NH 401000.50 402599 -1598.50 30.072 30.192 -0.120
55+, BL NH,AS NH,AI NH,OT NH,HISP
31856.70 29004 2852.70 2.389 2.175 0.214
Raking – Iteration 1
22
Eighth Control Variable
Output Weight Sum of
WeightsTarget Total
Sum of Weights
Difference
% of Output
WeightsTarget % of
WeightsDifference
in %Cell Phone Only 210390.11 197088 13302.35 15.778 14.780 0.998
Landline Only 270206.34 280297 -10090.31 20.263 21.020 -0.757
Landline and Cell Phone 852879.55 856092 -3212.04 63.959 64.200 -0.241
Raking – Iteration 2
23
First Control Variable
Output Weight Sum of Weights
Target Total
% of Output
WeightsTarget %
of Weights
Difference in % from
Iteration1Difference
in %Age 18-24,Male 94727.80 95468 7.104 7.159 -0.626 -0.056Age 18-24,Female 87222.36 90249 6.541 6.768 -0.980 -0.227Age 25-34,Male 116312.81 118670 8.723 8.899 -0.694 -0.177Age 25-34,Female 110348.83 112007 8.275 8.400 0.179 -0.124Age 35-44,Male 118670.65 117184 8.899 8.788 0.311 0.111Age 35-44,Female 113723.15 113779 8.528 8.533 0.137 -0.004Age 45-54,Male 130207.90 127077 9.765 9.530 0.869 0.235Age 45-54,Female 130419.01 127439 9.780 9.557 0.710 0.223Age 55-64,Male 93001.49 95032 6.974 7.127 -0.352 -0.152Age 55-64,Female 96092.37 97422 7.206 7.306 -0.430 -0.100Age 65-74,Male 54156.67 54171 4.061 4.062 0.248 -0.001Age 65-74,Female 62303.45 61828 4.672 4.637 0.066 0.036Age 75+,Male 47039.67 46515 3.528 3.488 0.244 0.039Age 75+,Female 79249.83 76635 5.943 5.747 0.317 0.196
Raking - Iteration 7
First Control Variable
Output Weight Sum of
WeightsTarget Total
% of Output
Weights
Target % of
Weights
Difference in % from
Iteration1Difference
in %Age 18-24,Male 95491.87 95468 7.161 7.159 -0.626 0.002Age 18-24,Female 90265.83 90249 6.769 6.768 -0.980 0.001Age 25-34,Male 118621.93 118670 8.896 8.899 -0.694 -0.004Age 25-34,Female 111985.21 112007 8.398 8.400 0.179 -0.002Age 35-44,Male 117205.13 117184 8.789 8.788 0.311 0.002Age 35-44,Female 113769.71 113779 8.532 8.533 0.137 -0.001Age 45-54,Male 127088.93 127077 9.531 9.530 0.869 0.001Age 45-54,Female 127437.46 127439 9.557 9.557 0.710 -0.000Age 55-64,Male 95037.18 95032 7.127 7.127 -0.352 0.000Age 55-64,Female 97426.08 97422 7.306 7.306 -0.430 0.000Age 65-74,Male 54168.73 54171 4.062 4.062 0.248 -0.000Age 65-74,Female 61831.76 61828 4.637 4.637 0.066 0.000Age 75+,Male 46503.23 46515 3.487 3.488 0.244 -0.001Age 75+,Female 76642.96 76635 5.748 5.747 0.317 0.001
24
Raking - Iteration 7
Eighth Control Variable
Output Weight Sum of Weights
Target Total
% of Output
Weights
Target % of
Weights
Difference in % at
Iteration 1
Difference in %
Cell Phone Only 197101.32 197088 14.781 14.780 0.998 0.001
Landline Only 280285.25 280297 21.019 21.020 -0.757 -0.001
Landline and Cell Phone
856089.43 856092 64.200 64.200 -0.241 -0.000
25
**** Program terminated at iteration 7 because all current percents differ from target percents by less than 0.025*****
IMPACT OF CHANGING TO RAKING (IPV) ON THE BRFSS
BRFSS 2010 Combined States a DataDifference In Weighted Percentages
A Excludes AK, DC, TN, SD
incom
e cate
gory1
incom
e cate
gory2
incom
e cate
gory
3
educa
tion c
ategor
y 1
educa
tion c
ategor
y 2
educa
tion c
ategor
y 3
marr
ied
age gr
oup 1
age gr
oup 2
age gr
oup 3 whit
e
black/
AA
Hispani
c0
10
20
30
40
50
60
70
80
LL Post stratified LL Raking LLCP Raking
Marginal Changes Weighted Percentages for Demographic Characteristics, BRFSS 2010
Lowest
Incom
e cate
gory
Middle
incom
e cate
gory
Highest
incom
e cate
gory
Lowest
educa
tion l
evel
Middle
educa
tion l
evel
Highest
educa
tion l
evel
Married
Youn
gest
age g
roup
Middle
age g
roup
Oldest
age g
roup
White
Black/
African
American
Hispan
ic
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
BRFSS 2010 Combined States DataDifference In Weighted Percentages of Health
Outcomes
ASTHMA
STROKE
HEART A
TTACK
CORONARY HEA
RT DISEA
SE
DIABETES
FAIR/PO
OR HEALTH
NO PHYS
ICAL ACTIV
ITY
OBESITY
HEAVY D
RINKING
CURRENT S
MOKER
NO HEALTH
INSURANCE
0
5
10
15
20
25
30
35
LL Post stratified LL Raking LLCP Raking
A Excludes AK, DC, TN, SD
Marginal Changes for in Weighted Percentage s Health Outcomes, BRFSS
2010
Asthm
aStr
oke
Heart A
ttack
Corona
ry Hea
rt Dise
ase
Diabete
s
Fair/P
oor H
ealth
No Phy
sical
Activ
ity
Obesit
y
Heavy
Drinkin
g
Curren
t Smok
er
No Hea
lth In
suran
ce0
0.20.40.60.8
11.2
Weighted Prevalence Estimates for Current Smoker by Year, Weighting
Method
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100
5
10
15
20
25
Landline Post Stratification Landline Raking WeightingLandline/ Cell Phone Raking Weighting
Year
Prev
alen
ce E
stim
ate
NOTE: All US states and territories except SD and TN
STATE LEVEL OUTCOMES
In Some Cases, Small Changes(Landline Only)
Table 1State-level Responses to Question:
“Has a doctor, nurse or other healthcare provider ever told you that you have diabetes?”
By Type Of Weighting Procedure for Landline DataResponse Landline
Weighted frequency with Post-Stratification
Landline PercentWith Post-Stratification
Landline Weighted frequency with Raking
Landline Percent With Raking
Differences in Landline Percentages(Post-Stratification-Raking)
Yes 434,858 12.26 440,694 12.43 -0.17Yes, but only during pregnancy
26,306 0.74 26,262 0.74 0.00
No 3,031,681 85.44 3,029,545 85.42 0.02No, Pre-diabetes/ borderline diabetes
55,454 1.56 50,196 1.42 0.15
In Some Cases, Larger Differences– But Not Consistent Differences
(Landline Only)Table 2
State-level Responses to Question:“Would you say that in general your health is excellent, very good, good, fair or poor?”
By Type Of Weighting Procedure for Landline DataResponse Landline Weighted
Frequency With Post-Stratification
Landline PercentWith Post-Stratification
Landline Weighted Frequency With Raking
Landline Percent With Raking
Differences In Landline Percentages(Post-Stratification - Raking)
Excellent 631,742 17.83 575,541 16.27 1.56Very Good 1,037,345 29.27 963,330 27.23 2.04Good 1,107,272 31.26 1,111,484 31.42 -0.16Fair 519,248 14.65 591,716 16.73 -2.07Poor 247,424 6.98 295,425 8.35 -1.37
In Some Cases, Consistent Differences(Landline Only)
Table 3State-level Responses to Question:
“During the past month, other than your regular job, did you participate in any physical activities or exercises such as running, calisthenics, golf, gardening, or walking for
exercise?”By Type Of Weighting Procedure for Landline Data
Response Landline Weighted Frequency With Post-Stratification
Landline PercentWith Post-Stratification
Landline Weighted Frequency With Raking
Landline Percent With Raking
Differences In Landline Percentages(Post-Stratification - Raking)
Yes 2,448,288 68.97 2,342,381
65.98 2.99
No 1,101,378 31.03 1,207,643
34.02 -2.99
But Differences Go Away Sometimes When Cell Phones Are Added
Table 4State-level Responses to Question:
“During the past month, other than your regular job, did you participate in any physical activities or exercises such as running, calisthenics, golf, gardening, or walking for exercise?”
By Type Of Weighting Procedure for Landline and Cell Phone Data
Response
Landline Weighted Frequency With Post-Stratification
Landline PercentWith Post-Stratification
Landline Weighted Frequency With Raking
Landline Percent With Raking
Differences In Landline Percentages(Post-Stratification - Raking)
Landline And Cell Phone Weighted Frequency With Raking
Landline And Cell Phone Percent
Landline And Cell Phone Differences In Percentages (Post-Stratification - Raking)
Yes 2,448,288
68.97 2,342,381
65.98 2.99 2,447,823
68.96 0.02
No 1,101,378
31.03 1,207,643
34.02 -2.99 1,102,053
31.04 -0.02
Persistent Differences May Exist Even When Adding Cell Phone Responses
Table 5State-level Responses to Question:
“Do you smoke cigarettes every day, some days or not at all?”By Type Of Weighting Procedure for Landline and Cell Phone Data
Response Landline Weighted Frequency With Post-Stratification
Landline PercentWith Post-Stratification
Landline Weighted Frequency With Raking
Landline Percent With Raking
Differences In Landline Percentages(Post-Stratification - Raking)
Landline And Cell Phone Weighted Frequency With Raking
Landline And Cell Phone Percent
Landline And Cell Phone Differences In Percentages (Post-Stratification - Raking)
Every day
581,967 36.32 704,831
40.95 -4.63 676,129
40.40 -4.08
Some Days
213,724 13.34 248,782
14.45 -1.12 199,278
11.91 1.43
Not At All
806,827 50.35 767,708
44.60 5.75 798,181
47.69 2.65
CONCLUSIONS
Conclusions (1) New weighting procedures are needed to
keep pace with changes in personal communications.
The inclusion of new variables and more complex weighting procedures for large scale survey data are now feasible, because of improvements in computer capacity.
There will be some differences in estimates when weighting procedures change and when new variables for weighting are introduced.
Examples shown here are only depictions of potential outcomes of changes at the BRFSS.
Conclusions (2) Good news: demographic characteristics
adjusted to more closely match Census data.
Most health outcomes indicate increases in risk behaviors (especially when state data are aggregated).
Some increases in chronic conditions, but uneven across states.
Thank You
For more information please contact Centers for Disease Control and Prevention
1600 Clifton Road NE, Atlanta, GA 30333Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: [email protected] Web: http://www.cdc.gov
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Office of Surveillance, Epidemiology, and Laboratory ServicesDivision of Behavioral Surveillance