a survey of reb systems in canada (careb agm montreal 2014) michael wilson (research ethics officer)...
TRANSCRIPT
![Page 1: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/1.jpg)
A Survey of REB Systems in Canada(CAREB AGM Montreal 2014)
Michael Wilson (Research Ethics Officer) &Dr. Brian Detlor (Chair)
McMaster Research Ethics Board (MREB)
![Page 2: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/2.jpg)
The Access Database
Year 2000 Year 2004
![Page 3: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/3.jpg)
MREB Active – Master Paper Record
![Page 4: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/4.jpg)
Born to Scan
Sharp 2700N copier / scanner
Close to my office
Archiving and emailing PDFs
![Page 5: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/5.jpg)
Program or Be Programmed!
![Page 6: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/6.jpg)
Coldfusion – Life is but a Dreamweaver
![Page 7: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/7.jpg)
On your mark…choose a vendor…
![Page 8: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/8.jpg)
UK – National Research Ethics Service
![Page 9: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/9.jpg)
So Many REB Systems?
One software system for UK research ethics review – developed by Infonetica
I developed, designed & programmed my own REB system
RFPs for larger systems i.e, > $100,000
IT projects have a high 40% - 70% failure rate
In-house system – or what happens if your developer leaves?
Our REB systems survey – what didn’t we know?
![Page 10: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/10.jpg)
REB Systems in Canada Survey
![Page 11: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/11.jpg)
Purpose of the Research Project
To obtain a descriptive snapshot of the human-participant REB systems landscape in Canada.
REB systems in Canada implemented? Key characteristics of successful REB systems? Satisfaction of stakeholders with REB systems? What works well & what could be improved? Context of systems & influences how fared? Recommendations on implementing REB systems
![Page 12: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/12.jpg)
Motivation
Our own “home-grown” REB system is at its maximum capacity We need a more robust system
Handle increases number of protocols Automate many tedious, monotonous tasks Better reporting capabilities Expedite the review of applications Eliminate the “master” paper record Better security / more reliable servers/ backups &
redundancy Better track revisions Provide researchers with a better experience
Creating protocols; obtaining status updates on the review process; submitting “change requests”
![Page 13: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/13.jpg)
Motivation
For a new and improved system we want to keep Design our forms and workflow in a way that works for our
Board Control changes system and make them quickly Low cost
DeLone & McLean Model of Information System Success
![Page 14: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/14.jpg)
Methods
Web survey sent CAREB members via CAREB listserv before & after Calgary AGM May 2013
Recruitment in person in Calgary
Survey instrument (Limesurvey on secure McMaster server) Quantitative and qualitative questions on REB background,
characteristics of REB system, user satisfaction, important REB system features Probing questions about the respondent’s REB system A very rich data set was collected – 64 complete
responses Many detailed qualitative comments
![Page 15: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/15.jpg)
Respondent Profile
Number of REBs per Institution
Type of Institution
One REB: 42%
> One REB: 58% 7%
67%
10%
2%
5%
9%Hospital basedUniversity basedUniversity and hospi-tal basedGovernmentPrivate or Inde-pendentIndustry basedCollege based
![Page 16: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/16.jpg)
Respondent Profile
Predominant Role of Respondent
14%
64%
9%
14%
REB ChairREB Administrator/Staff MemberREB MemberResearch Director or Administrator
![Page 17: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/17.jpg)
Type of REB System Currently Used
Current REB System
3%17%
28%
11%
45%
2%
34%
2% A stand alone ASP/Cloud sys-temA system integrated with my institution's Research Services systemsAn in-house developed systemAn outsourced system devel-oped by commercial (outside) vendorsA hybrid paper and electronic systemA regional REB system within a larger shared ASP/Cloud system A mostly paper-based system, with a desktop databaseA paper only system
![Page 18: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/18.jpg)
Current REB System
Amount of Time Using the Current REB System
1 - 3 years 3 - 5 years 5 - 8 years More than 8 years
Don't know
No answer 0%
5%
10%
15%
20%
25%
30%
35%
![Page 19: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/19.jpg)
What are the key characteristics of a successful REB system? Answers in rank order by frequency
1. Useful functionality (45 comments) E.g., generates reports; generates reminders; generates
statistics; creates meetings and agendas; facilitates robust and easy searching; tracks versions of documents; check status of a protocol
2. Efficient and meaningful processes and procedures (25 comments)
E.g., workflow meets the needs of all users; processes are streamlined; facilitates quick processing of protocols
3. Easy to use / User-friendly (24 comments) Intuitive forms; smart forms; minimal form navigation
![Page 20: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/20.jpg)
What are the key characteristics of a successful REB system? Answers in rank order by frequency (cont’d)
4. Sufficient support (18 comments) E.g., adequate number of admin staff; competent admin
staff; knowledgeable REB members; adequate IT support in terms of staff and infrastructure
5. Captures all information needed (9 comments)
6. Automates tasks wherever possible (9 comments)
7. Flexible (8 comments)
8. Integrates with other institutional systems (7 comments)
9. Provides online access and electronic submissions (4 comments)
10. Follows good project management practice (2 comments)
![Page 21: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/21.jpg)
How well do these characteristics compare with what’s in place? Not as well as one would like...
Functionality offered by the current REB system Permits researchers to submit an electronic application (protocol)
44% Handles annual renewals and amendments electronically 36% Handles serious adverse events reporting 25% Has an online reviewer system 30% Tracks applications through the review process 30% Serves as the master record 45% Permits data extraction for internal report purposes 41% Creates online statistics 20% Creates meetings, agendas, reports, reminders 28% Permits electronic signatures or variations of such 17%
![Page 22: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/22.jpg)
How satisfied are respondents with their current REB systems? Most are satisfied
Not Very Satisfied Satisfied Very Satisfied0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
How satisfied do you think REB staff are with your current REB system? (n = 54)How satisfied do you think re-searchers are with your current REB system? (n = 48)
![Page 23: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/23.jpg)
How satisfied are respondents with their current REB systems? Twice as many negative comments given than positive
ones
Negative comments (in rank order) concerned Usefulness (23 comments) Flexibility (10 comments) Support / Infrastructure (8 comments) Ease of Use (5 comments)
Positive comments (in rank order) concerned Usefulness (9 comments) Support / Infrastructure (9 comments) Flexibility (1 comment)
![Page 24: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/24.jpg)
Observations
“Satisfied” or “Very Satisfied” respondents utilized a wide variety of REB systems No single type of REB system determined
satisfaction Satisfied respondents utilized the full gambit of REB
systems expensive solutions & in-expensive solutions, vendor-based solutions & custom in-house solutions, paper-based solutions, desktop database solutions, &
full electronic solutions Why? What does this imply for practice?
![Page 25: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/25.jpg)
Observations
Why does no one type of REB system determine satisfaction? Satisfaction with an REB system seems to depend on the
context in which they are situated Critical factors include:
The number of protocols processed by the REB The greater the # of protocols, the more likely a full robust
REB system was needed The expectations of researchers and staff (e.g., lower
expectations lead to greater satisfaction) The amount of support available (both REB staff and IT support)
The more support available to handle administrative work and IT concerns, the greater the satisfaction
The willingness of an institution to supply this support is critical
![Page 26: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/26.jpg)
Observations
Certain factors can undermine satisfaction with implementing an expensive vendor-based REB solution Factors that inhibit satisfaction include:
high cost loss of control making timely changes to the
system loss of prior functionality, absence of needed
functionality, inclusion of unwanted functionality using processes and workflows that do not match
the way the institution works REBs should ensure that these factors are addressed prior
to implementation However, this is not always possible to do
![Page 27: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/27.jpg)
Other thoughts
Though electronic REB systems offer many benefits, they are, at the end of day, still just administrative systems They tend to offer more efficiency benefits (e.g.,
automated processes; online submissions; quicker tracking; easier reporting) than effectiveness benefits (e.g., better quality reviews)
“Researchers, reviewers and ethics staff are spared clerical, paper-heavy administriva in favour of more time and energy to focus on content. However, online systems will not compensate for poorly advised researchers, reviewer shortages or poorly informed ethics staff.”
“REBs and administrators focus a lot on improving efficiency, but I’d like to explore a system that improves effectiveness of REB processes.”
![Page 28: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/28.jpg)
Other thoughts
There is a desire for standardized software and harmonization of software
“There should be a coordinated effort to have all REBs using the same software.”
“Canada needs to develop one system only for multi-university research projects so that researchers are not being asked to fill in one for each university they have as participants.”
Change management is critical when implementing a new REB system
“There is an incredible amount of time involved from REB administrators to set up an efficient system (consulting, adapting system and policies and procedures, training etc.) that I think is often underestimated by senior administration. Managing change alone can be an enormous task and plans have to be in place to support implementation.”
![Page 29: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/29.jpg)
Recommendations?
Get your REB systems / IT project in writing – have a Project Charter signed by key stakeholders at the highest level
Look in-house first – your organization may have the talent and resources
Canadian REB system vendors and developers know the Canadian landscape
A web-based REB system solution reduces data entry, paper usage, improves efficiency & maximizes effectiveness
Integration with other internal systems is nice – but first have a system that first works for your REB, not some other office
![Page 30: A Survey of REB Systems in Canada (CAREB AGM Montreal 2014) Michael Wilson (Research Ethics Officer) & Dr. Brian Detlor (Chair) McMaster Research Ethics](https://reader035.vdocuments.net/reader035/viewer/2022062620/551a7fc7550346e0158b4934/html5/thumbnails/30.jpg)
Questions?
Detailed Survey results online and in Handout here
https://reo.mcmaster.ca/REBsurvey
Later: E-mail us
Journal publication forthcoming ...
Michael [email protected]
Brian [email protected]