quinte longitudinal study : purpose, principles and methodological lessons learned dr. rob williams...
TRANSCRIPT
QUINTE LONGITUDINAL STUDY :
PURPOSE, PRINCIPLES AND METHODOLOGICAL LESSONS LEARNED
Dr. Rob WilliamsProfessor, Faculty of Health Sciences &Coordinator, Alberta Gaming Research
InstituteUniversity of Lethbridge
Lethbridge, Alberta, Canada
AGRI ANNUAL CONFERENCEBANFF, ALBERTAAPRIL 10, 2010
Acknowledgements
Robert Hann (Principal Investigator)Patricia McLaughlin (Project Manager)Bev West (Research Analyst)Nick White (Research Assistant)Kate King (Research Assistant)Don Schopflocher (Co-Investigator)Rob Wood (Co-Investigator)Earl Grinols (Consultant)Trevor Flexhaug (Technical Support)
Funder: Ontario Problem Gambling Research Centre
Purpose and Value of Longitudinal Research
To establish an etiological model to aid in prevention and treatment (risk, protective, and recovery factors) retrospective reports are unreliable cross-sectional correlates of problem gambling (PG) tells you
nothing about their chronology and causal sequence
To establish incidence of PG
To establish stability of PG
To establish predictive validity of PG instruments
Prior Longitudinal Studies
Have been many (see el-Guebaly et al., 2008 & Slutske, 2007 for a review)
Furthermore, they have provided interesting and useful findings
However, they all have limitations either because they have: An insufficient number of variables Small sample sizes and/or insufficient numbers of ‘at risk’ gamblers Too circumscribed an age range (e.g., just adolescents) A short time span (e.g., just a few years) A limited number of collection periods (e.g, 2 or 3) Poor retention rates
Alberta Longitudinal Study
Leisure, Lifestyle, Lifecycle Project (LLLP)
el-Guebaly, Hodgins, Smith, Williams, Schopflocher, Wood, Casey
$2.3 million; funded by Alberta Gaming Research Institute
Feb 2006 – April 2011
1808 Albertans (524 ‘at risk’)
5 initial age ranges: 13-15; 18-20; 23-25; 43-45; 63-65
4 comprehensive assessments (2 – 4 hrs each; primarily self-administered) with 20 month intervals between the start of each assessment
73.1% retention rate after 3 assessments
Quinte Longitudinal Study
Also known as the QERI or Belleville Study
Quinte Region in southeastern Ontario
Williams, Hann, Schopflocher, Wood, Grinols
$3.1 million; funded by Ontario Problem Gambling Research Centre
March 2006 – April 2011 ++ ?
4123 residents from southeastern Ontario (1216 ‘at risk’)
Initial age range: 17 – 90
5+ comprehensive assessments (1.5 – 2.5 hrs each; all self-administered) with 12 months between start of each assessment
94.0% retention rate after 4 assessments
Principles of Longitudinal Studies and Methodological
Lessons Learned
Oversample High Risk Groups
Need enough people who will become Problem Gamblers (PGs) during the course of the study Several hundred if wish to study/understand PG
subgroups
This problem is compounded with declining PG rates and lower retention of PGs
Oversampling Options Gambling behaviour (frequency, expenditure, or type) Demographics (male, young, lower socioeconomic) Other (family PG hx, smoker, impulsivity)
Extend the Study for as Long as Possible
While some people become PGs within a few years of gambling onset, others report that it took many years
A study of 5 years duration will only reliably identify the etiology for the subset of people who become PGs within a 4 year time period
Try to Conduct the Assessment at the Same Time of Year Each Time
Increases retention
Reduces seasonal artifact Gambling behaviour is somewhat seasonal, peaking in
the summer. Even though people are being asked about ‘past year’ behaviour, their current behaviour tends to distort their past year retrospective reports
Best to avoid summer and Xmas holidays
Have a Small Assessment Window
A long assessment period where people are assessed anytime within a several month period results in:
widely different inter-assessment intervals between people and within the same person (unless people do their assessment at the exact same time each period)
individuals being assessed at different times of year (seasonal artifact)
greater likelihood that an important change in gambling availability will be ‘spanned’ by the cohort
A Fixed Inter-Assessment Interval is Best(1 year being optimal)
Ensures people are assessed the same time each year
1 year interval capitalizes on the fact that a 12 month time frame is used in most gambling behaviour assessments Ensures ‘coverage’ of all behaviour during the course of the study Aids in memory recall. Even though you may be asking about the past 12
months, many people will use the last assessment (when they were last asked to evaluate this) as the demarcation point
Different inter-assessment intervals (e.g., 1 year -> 2 years) creates problems: Either have to change the questionnaire time frame each time (which is
problematic), or else not have ‘coverage’ of a certain period of time Difficult to attribute changes occurring during the longer interval to
changes in measured variables, as these changes could simply be due to a longer time for them to have naturally occurred.
Consider More Frequent Assessments for a Small Percentage of People
Yearly assessments of past year behaviour do not provide a fine grained chronology of gambling (in many ways a 5 yr longitudinal study is just a collection of 5 past-year retrospective reports)
Diaries (daily, weekly, monthly) or bi-annual assessments can help rectify this
Avoid Mixing Administration Formats
There are statistically significant differences in responses obtained via phone vs face-to-face vs mail vs online These differences most prominent for ‘sensitive’ questions Attributable to differences in anonymity, interviewer-interviewee
interaction effects, time to answer, ability to re-read, ability to go back, etc.
Self-administered computerized format is optimal Maximizes anonymity Respondent proceeds at their own pace Ensures consistency of administration and automated data
recording Less costly
Have a Comprehensive, Efficient, and Well-Tested Questionnaire
Needs to comprehensively survey all areas relevant to the etiology of gambling and PG
Avoid redundancies in the questions and/or scales
Make sure that the questionnaire is pre-tested for wording and robustness of the programming
Try to make the questionnaire ‘interesting’
Maximize Retention
Poor retention usually produces a nonrepresentative sample as drop-outs much more likely to be young, male, single, minority group, with significantly higher rates of pathology (substance use, mental health problems, etc)
This is a particularly serious problem if these individuals are your primary group of interest!
Post-hoc weighting does not necessarily correct this bias
Maximize Retention
Hire the Right People to do the Study
Probably the most important consideration
Participant retention most strongly related to personal engagement with people conducting the study
Preferable to have small number (e.g., 3 – 5) of full time staff rather than many part timers
Want staff who a) will continue throughout entire duration of project b) are good at working with people c) are highly organized d) get along with each other e) have different skill sets f) are from the region that is being assessed g) are neutral with respect to gambling
Maximize Retention
Offer Convenient Assessment Options
i.e., office-based, online, or mail-in
i.e., paper & pencil or computerized
Also helps to improve the response rate and thereby, the representativeness of your initial sample
Maximize Retention
Be attentive to the needs of each and every participant
Even though the project may be important and interesting to you, it is often emotionally draining, somewhat boring, and time consuming for participants!
This can be offset by having a positive interaction with staff who are professional, enthusiastic, supportive, and caring
The experience of each completed assessment shapes the person’s motivation for the next one
All participant complaints and suggestions have to be responded to in a timely way
Maximize Retention
Have an Office in Each Region
Increases public profile and legitimacy of project
Allows people to drop in and maintain contact & provide change of address
Maximize Retention
Incentives are Important for Both Participants and Staff
Participants: Minimum of $25/hr or $40/assessment Highest payment for initial participation (to improve response rate) and
for the 2nd assessment. Increasing amounts to take yearly inflation into account.
Staff: Significant monetary bonuses for goal attainment Setting goals to work toward is as important (if not more important) than
the financial reward for achieving these goals Praise and celebration of good work Allowing independence and flexibility in how they achieve the goals Utilizing a collaborative team approach
Maximize Retention
Use Staff Time Efficiently
2/3rds of people will respond with 1 invitation
1/3 require multiple reminders, and, the majority of these people tend to be the same people each time
Significant time and monetary savings are realized by analyzing the characteristics of your initial cohort (i.e., procrastinators; in-office people; people needing assistance) & concentrating your subsequent efforts on the people who need help/encouragement
Maximize Retention
Need a Comprehensive and Versatile ‘Contact Database’
Contains exhaustive list of ways of contacting the person Home phone, home address, cell phone, work phone, work address,
employer, home and work emails, vacation home address, social networking sites (e.g., Facebook), Chat systems (e.g., MSN), complete contact details for 2 people familiar with the person
Identifies person’s ‘category’ (special needs, procrastinator, etc.)
Records all contact attempts and responses
Contains an Appointment Scheduler
Records a myriad of other info useful to staff that can be analyzed to improve efficiency (much of which I’m not aware of myself!)
Maximize Retention
Use a Different Format and Message for each Reminder
Start with a personalized letter invitation (returned mail provides you with an early alert to participants who have moved)
Follow with an email invitation
Follow with a personalized postcard reminder
Follow with an email reminder
Follow with a phone call(s)
Maximize Retention
Mass Email Requires Special AttentionCheck with your ISP concerning
Spam filter triggers Capacity limitations (sending out; bounce backs)
Send Reminders at the Beginning of the WeekIn Canada, people are much less likely to complete
the assessment on weekends and holidays
Maximize Retention
Maintain Contact Between Assessments
Maintains ‘rapport’
Allows you to identify changed addresses before the assessment
Reminds people to contact you with their new address
‘Newsletter’ an efficient way of doing this
Maximize Retention
Have an Easily Remembered Project Logo
e.g. QERI, LLLP
On exterior office signage; website; all outbound communications (including cheques)
This ‘branding’ allows participants to easily identify project mailings (and distinguishes them from the large volume of junk mail/email they receive)
Maximize Retention
When these things are done right then you can expect most drop-off after the first assessment but decreasing drop-off in all subsequent assessments People have confidence it is a legitimate survey They are familiar with the survey They are confident they will get paid They have a history of positive interactions with staff
When these things are not done right then you may get increased rates of drop-off with each assessment Survey increasingly aversive History of negative interactions
Have a Website
http://wcmprod1.ucalgary.ca/lllp_data/homehttps://www.qeri.ca/
Provides means of communication for the cohort, funders, and the general public
Provides reassurance of legitimacy
Common storage site for data
Logistics
Success at conducting a large scale longitudinal study is not due to a few critical things, but a thousand little things and the ongoing ability to quickly identify and rectify the many issues that continually arise