neurosurgeons preferences for obtaining cme credits€¦ · neurosurgeons preferences for obtaining...
TRANSCRIPT
1
Neurosurgeons preferences for obtaining CME credits
Heather Hodge
EDRM 600 Introduction to Research Methods
Colorado State University
2
Introduction
As technology has evolved and provided new and innovative ways to deliver education,
continuing medical education (CME) providers have taken advantage of many of them. A
google search of the phrase “online continuing medical education” returned 66,100,000 results.
The Congress of Neurological Surgeons (CNS) provides close to 100 of those 66 million online
opportunities for continuing medical education to its members and to non-members alike.
There are a variety of reasons why physicians need CME including, but not limited to, for
hospital credentialing, state licensure, maintenance of certification and, in some instances, to
maintain membership in professional organizations. CME can be obtained in a variety of ways
including attending live, face-to-face courses (“traditional” CME) and participating in online
activities. CNS provides face-to-face opportunities in the form of an annual meeting and two to
three smaller regional courses annually. They also provide approximately 20 to 25 live online
courses per year on topics ranging from guideline updates to board review courses. Each of
these courses is archived and available for viewing for at least three years after the live iteration.
In addition, there is a variety of online education available in subspecialty topics such as Spine,
Cerebrovascular, Tumor, Trauma, Functional, Pediatrics, Peripheral Nerve and non-clinical core
competencies. Finally, CNS provides an online exam and self-assessment tool called SANS
which is available in a variety of subspecialties. Neurosurgeons are required to complete one of
three available larger SANS exams once every three years if they are participating in
maintenance of certification.
The purpose of this quantitative study is to determine whether U.S. neurosurgeons prefer
online educational activities or live educational activities. The research questions are: do the
networking opportunities and lack of distractions afforded at face-to-face meetings deter
3
neurosurgeons from participating in online education? Do neurosurgeons perceive online
education as being of lower quality than live, face-to-face education? Are they aware that there is
so much online education available?
Background and Rationale
Research regarding physician participation in online CME focuses on the perception,
expectations, social interaction and effectiveness in adopting evidence based medicine as
compared to traditional CME. However, it is uncommon to see studies that address this topic in
detail by either specialty or by type of practice. Participation in online education by physicians
may vary by one, or the other, or both.
Sargeant et al (2004) noted that “research demonstrates positive outcomes of on-line
programs, including an increase in knowledge…However, although learners are satisfied with
on-line learning, the effectiveness of and satisfaction with interpersonal interaction are lower” (p.
228). Eighty-two percent of the participants in this study were family practice doctors. The
remaining 18% were specialists although the study did not identify what kind of specialists.
Fordis et al (2005) completed a quantitative, randomized controlled trial that the authors
concluded, “provides the first evidence at the individual physician level that Internet-based CME
can produce effectively measures changes in behavior as well as gains in knowledge…” (p.
1049). While participants in this study were solely primary care physicians, the authors
concluded that these results might be able to be generalized to other physician populations.
However, this study did not address any of the issues of interaction except to say that it appeared
that neither the online group nor the live meeting group reached out the experts involved with the
activity.
4
While interaction with faculty or experts is important, the topic of interaction with peers
must also be taken into consideration. Sargeant et al (2006) followed up their 2004 study by
looking more closely at the “instructor roles in enhancing online learning through interpersonal
interaction…” (p. 128). Basically, they set out to show that facilitators can help create dialogue
with and among participants. They acknowledge that physicians often consult with colleagues to
enhance learning. In addition, it was noted that, “At formal CME events, physicians use
unstructured time for interaction to solve practice problems, integrate new material, and
generally aid the process of applying knowledge to practice” (p. 129). There were also quotes
from two study participants that indicated they really value the social interaction they get at face-
to-face meetings. It’s unknown if this can be accomplished online. Because online discussions
are asynchronous, it places additional challenges upon the instructor to facilitate more
interpersonal communication based on a variety of learning theories. Participants likely don’t
want to wait until later to get the feedback they can get instantly at a face-to-face meeting. As
noted in the conclusion to this study is the fact that another challenge with online CME is that
there is often no interaction with the facilitator. Rather, online CME only involves interaction
with the content.
The purpose of Guan et al’s 2008 study was to investigate the relationship between online
social interactions in a CME activity and participation. Again, study participants were general
practitioners and family doctors. This exploratory study showed that time and lack of social
interaction was frequently listed as barriers to participation. Because online learning is often
asynchronous, it takes more time to participate and develop relationships. The authors also refer
to previous studies indicating that failures often occur on the social level rather than a technical
5
one in online learning. This notion reinforces the social constructivist theory that interaction and
dialogue are very important to learning.
Harris et al (2010) look at the future of online CME in terms of growth and characteristic
by analyzing annual report data from the Accreditation Council of Continuing Medical
Education (ACCME) along with a survey of publicly available CME web sites. The study
rightfully acknowledges that there is a disconnect between the ACCME data and physician
survey findings. ACCME data shows much faster growth in online CME than results from
physician surveys that suggest physicians still prefer traditional CME delivered live and face-to-
face. Despite several limitations to this study, overall it does indicate that 6.9% to 8.8% of CME
utilized in 2008 was online, (p. 6) which demonstrates persistent and stable growth. However,
online activities are still not dominating the CME marketplace. As technology continues to
improve, and is embraced by CME providers, participation in online CME should continue to
increase. A white paper on the future of professional medical education, conducted by for-profit
company Pri-Med in 2011, stressed the importance of all stakeholders in professional medical
education embracing and discussing the impact of technological trends in order to shape the
future of medical education and providing physicians with the tools they need to provide the best
possible patient care.
According to a 2012 survey conducted by two for-profit companies, ON24, Inc. and
MedData Group, 84.1% of the 971 physicians surveyed would prefer to attend CME training
online. However, only 6.4% said they actually do participate in “virtual education.” Information
on the specialties of the study participants was not included. In addition, important CME
provider types were left out of the survey. This study appears to focus on education provided by
6
pharmaceutical and medical device companies which is significantly different than education
provided by medical specialty societies and other ACCME accredited providers.
While there is a plethora of literature on physician participation in online CME, most of
the research was conducted with general practitioners who one could argue have different
responsibilities and schedules than some specialists. In addition, some of the data on actual
participation is conflicting. A survey design will be utilized to determine how many
neurosurgeons are participating in online CME, why they are or why are they not participating,
and why they prefer either online education or live education.
Research Methods
The cross-sectional survey design study of how neurosurgeons prefer to obtain their CME
credits began with a decision on what participants to study. A list of all neurosurgeons who
participated in live, face-to-face CME activities and online CME activities provided by the
Congress of Neurological Surgeons (CNS) over the past two years was gathered. Excluded were
those who attended the CNS annual meeting, unless they also participated in an online activity or
in another live, face-to-face activity. The annual meeting was excluded because members are
required to attend once every three years and therefore it is not always by choice that they attend.
The final, de-duplicated, alphabetized list included 2,257 individuals. Using systematic
sampling, every fifth person was chosen from the list to receive the survey. Creswell (2012)
stated that while systematic sampling is not as precise as the random numbers table, it is more
convenient “because individuals do not have to be numbered and it does not require a random
numbers table” (p. 143). The survey was sent to 451 individuals.
7
A web-based questionnaire was developed to collect data from participants and
administered through SurveyMonkey (see Appenix A). When the survey was sent via email,
participants were informed that this was part of a research project for a Master’s level class and
that they were randomly selected to participate but that participation was completely voluntary.
A series of attitudinal questions were asked, along with some open-ended questions to gather
more in-depth information on why individuals prefer online or live educational activities. One
concern regarding the use of web-based surveys is that they may be “biased towards certain
demographic groups that tend to use computers” (Creswell, p. 384). However, because most
neurosurgeons are computer literate and use computers in their daily work, this is not a concern
in this study.
Descriptive analysis of the all the data will assist identifying general trends regarding
why neurosurgeons participate in online and/or live activities, whether or not they prefer one
over the other and why, and whether or not they are aware of the amount of online education
available. If needed, inferential analysis will help to determine if the results from the sample
group are predictive of most neurosurgeons.
Results
The response rate to this survey was very low at 8% as shown in Table 1.
Table 1: Response Rate
Number of
Surveys Sent
Number of Surveys
Returned Undeliverable
Total
Surveys Sent
Total Surveys
Completed
Response
Rate
451 16 435 34 8%
Because the response rate was so low, a wave analysis was done to check for response bias. The
survey was open for two weeks so the responses from the first week were compared to the
responses of the second week. Results were similar, so there was no response bias. Table 2
8
represents the only demographic information collected in the survey, which is the number of
years in practice.
Table 2: Number of Years in Practice
Number of Years in Practice Percentage of Responses Number of
Responses
Less than 5 years 32.4% 11
5 to 10 years 29.4% 10
11 to 20 years 8.8% 3
Over 20 years 29.4% 10
Using Minitab software, a t-test was completed on the following questions to determine if
the mean is significantly greater than the target of 4 (which correlates to “agree”). The t-tests
were run using an alpha level of .1.
Table 3: Mean, Standard Deviation and T-test for Descriptive Questions
Question Mean Standard
Deviation T-test
CNS offers a variety of online opportunities to obtain
CME credits. 4.264706 0.93124 p=.053
CNS offers a variety of live, face-to-face opportunities
to obtain CME credits. 4.117647 0.68599 p=.162
I enjoy participating in online educational activities. 4.529412 0.61473 p=0.001
I enjoy participating in live, face-to-face educational
activities. 4.088235 0.93315 p=.293
GraphPad Software was used to calculate the chi-squared value on the following filter
question (Table 4) and dichotomous questions (Table 5-7).
Table 4: How do you prefer to obtain your CME credit?
Category Observed Expected # Expected
By participating in online
educational activities 24 8.5 25.000%
By participating in live,
face-to-face educational
activities
10 25.5 75.000%
Value (Chi –square) df Two-tailed p value
37.686 1 0.0001
9
Table 5: Do you believe that there are good networking opportunities at live, face-to-face
meetings?
Category Observed Expected # Expected
Yes 24 28 82.353%
No 4 2 5.882%
Don't know 6 4 11.765%
Value (Chi –square) df Two-tailed p value
3.571 2 0.1677
Table 6: Do you believe that there are good networking opportunities when participating in
online activities?
Category Observed Expected # Expected
Yes 7 2 5.882%
No 19 28 82.353%
Don't know 8 4 11.765%
Value (Chi –square) df Two-tailed p value
19.393 2 .0001
Table 7: To what extent do you feel that the quality of online education is as good as live,
face-to-face education?
Category Observed Expected # Expected
A great extent 19 10 29.412%
Some extent 14 10 29.412%
Little extent 1 14 41.176%
Value (Chi –square) df Two-tailed p value
21.771 2 .0001
In response to the open-ended question, “Why do you prefer to obtain CME credits from
online educational activities?” two themes were clear: convenience (30) and cost savings (11). A
third theme, educational/focused topics (6), was not quite as popular.
10
In response to the open-ended question, “Why do you prefer to obtain CME credits from
live educational activities?” one theme was clear: interaction (15). Two other themes,
educational (5) and no distractions (3), were also apparent.
Conclusions
Based on the analysis of the data in Table 3, neurosurgeons agree that CNS offers a
variety of online opportunities to obtain CME credits. However, there is not enough evidence to
conclude that neurosurgeons agree that CNS offers a variety of live, face-to-face opportunities to
obtain CME credits. These results were congruent with what CNS does offer, which is more
online opportunities than live opportunities. However, results were also surprising because many
do not participate in the online education. Based on this data, the answer to the research
questions regarding whether or not neurosurgeons are aware there is so much online education
available is yes, they are aware. It can also be concluded in Table 3 that neurosurgeons enjoy
participating in online education while there is not enough evidence to conclude that they enjoy
participating in live education.
In Table 4, there is an extreme statistical difference between what was expected in terms
of preference between online education versus live education with respondents preferring online
education. Based on the answers to the open-ended questions, the main reason participants
preferred online education was clearly because of convenience, with one participant stating,
“More convenient. Don't have to travel. Have little enough time with my family as it is, I'm not
going to leave town if I don't have to.” For those who preferred live education, the main reason
was the interaction. Responses included, “I enjoy interacting with and talking to other
Neurosurgeons” and “More interactive, less rushed.”
11
In both Table 5 and Table 6, the results regarding whether or not live and online
education provide good networking opportunities were not statistically different, with most
agreeing as expected that live meetings do provide good networking opportunities and online
meeting not providing good networking opportunities.
While it is apparent in Tables 4, 5, and 6, that most agree that the networking
opportunities are better at live meetings, it does not seem that this is a deterrent to participating
in online education, as many still prefer to participate in online education (Table 4).
In Table 7, most participants agreed to a great extent that they felt the quality of online
education is as good as live education, which is statistically different than what was expected. In
terms of the literature, research almost unanimously showed that there was a knowledge gain for
physicians when participating in online education. However, the research did not address the
issue of whether or not the quality of online education was perceived to be as good as live
education. Based on this data, the question of whether or not neurosurgeons perceive online
education as being of lower quality than live, face-to-face education is no.
This study is limited for a couple of reasons. One, the response rate was so low and two,
many of the participants are likely already participating in online educational activities. This has
implications for future research in that it would be better to use a list of all neurosurgeons instead
of one that only includes those who participated in either live or online activities. In addition, it
would be best to include questions regarding use of other providers of education instead of
limiting it to just offerings provided by the CNS.
The overall implications of this study are that neurosurgeons enjoy participating in online
education mainly for the sake of convenience. They find the quality of online education to be as
good as live education. However, the networking opportunities available at live meetings are
12
greater than those available online. It would be beneficial for providers of online education to
incorporate more discussion and interaction which social constructivists would argue is
important in all learning environments. While most interaction that is offered online is
asynchronous, which leads to challenges for some in terms of misunderstandings and not having
the time to check back later, with advances in technology, this could be resolved in a variety of
ways including asking participants to join a chat room at a specific time, or conducting a live
webinar or a conference call. For education that endures over time, these live discussion groups
can occur on a regular basis so that individuals who can’t participate one time may be able to
join at another time. By providing worthwhile networking opportunities, participant satisfaction
should increase which presents another opportunity for future studies.
13
References
Creswell, J.W. (2012). Educational Research (4th Ed.). Upper Saddle River, NJ: Pearson
Education Inc.
Fordis, M., King, J.E., Ballantyne, C.M., Jones, P.H., Schneider, K.H., Spann, S.J., Greenberg,
S.B.,... (2005). Comparison of the institutional efficacy of internet-based CME with live
interactive CME workshops. The Journal of the American Medical Association, 294(9),
1043-1051. doi:10.1001/jama.294.9.1043.
Guan, J., Tregonning, S., & Keenan, L. (2008). Social interaction and participation: formative
evaluation of online CME modules. Journal of Continuing Education in the Health
Professions, 28(3), 172-179. doi: 10.1002/chp.174.
Harris, Jr., J.M., Sklar, B.M.., Amend, R.W., & Novalis-Marine, C. (2010). The growth,
characteristics, and future of online CME. Journal of Continuing Education in the Health
Professions, 30(1), 3-10. doi: 10.1002/chp.20005.
ON24, MedData Group study reveals physician digital behavior (2012, April 17). Retrieved from
http://www.on24.com/press_releases/on24-meddata-group-study-reveals-physician-
digital-behavior/.
Pri-Med (2011, October). How will the open education movement and current healthcare trends
transform the future of professional medical education?. Retrieved from
http://www.scribd.com/fullscreen/69458537?access_key=key-27qdh3gvuum5mh6q7v8a.
Sargeant, J., Curran, V., Jarvis-Selinger, S., Ferrier, S., Allen, M., Kirby, F., & Ho, K. (2004).
Interactive on-line continuing medical education: physicians’ perceptions and
14
experiences. Journal of Continuing Education in the Health Professions, 24(4), 227-236.
doi: 10.1002/chp.1340240406.
Sargeant, J., Curran, V., Allen, M., Jarvis-Selinger, S., & Ho, K. (2006). Facilitating
interpersonal interaction and learning online: linking theory and practice. Journal of
Continuing Education in the Health Professions, 26(2), 128-136. doi: 10.1002/chp.61.
15
Appendix A
*How many years have you been in practice? o Less than 5 years
o 5 to 10 years
o 11 to 20 years
o Over 20 years
*How much do you agree or disagree with the following statements? Please read each
one carefully and describe how best you feel about the statement.
*How do you prefer to obtain your CME credit? o By participating in online educational activities
o By participating in live, facetoface educational activities
*Why do you prefer to obtain CME credits from online educational activities?
*Why do you prefer to obtain CME credit from live, facetoface educational activities?
Do you believe that there are good networking opportunities at live, face toface meetings
(i.e. the CNS annual meeting)? o Yes
o No
o Don't know
Do you believe that there are good networking opportunities when participating in online
activities? o Yes
o No
o Don't know
Strongly
agree Agree
Neither agree or
disagree Disagree
Strongly
disagree
CNS offers a variety of
online opportunities to
obtain CME credits.
CNS offers a variety of
live, facetoface
opportunities to obtain
CME credits.
I enjoy participating in
online educational
activities.
I enjoy participating in
live, facetoface
educational activities.
16
To what extent do you feel that the quality of online education is as good as live, face to
face education? o A great extent
o Some extent
o Little extent
17
Appendix B
Codebook for Neurosurgeons Preference in Obtaining CME
Variable 1 ID - Each participant who completed the survey is numbered 1-34
Variable 2 Years in practice: 4=over 20 years; 3=11-20 years; 2=5-10 years; 1=less than 5 years
Variable 3
CNS offers a variety of online opportunities: 5=strongly agree; 4=agree; 3=neither agree or
disagree; 2=disagree; 1=strongly disagree
Variable 4
CNS offers a variety of live opportunities: 5=strongly agree; 4=agree; 3=neither agree or
disagree; 2=disagree; 1=strongly disagree
Variable 5
Enjoy participating in online education activities: 5=strongly agree; 4=agree; 3=neither agree
or disagree; 2=disagree; 1=strongly disagree
Variable 6
Enjoy participating in live education: 5=strongly agree; 4=agree; 3=neither agree or disagree;
2=disagree; 1=strongly disagree
Variable 7 Preference on obtaining CME: 2=Online Education; 1=Live Education
Variable 8 Good networking opportunities in live meetings: 3=Yes; 2=Don't know; 1=No
Variable 9 Good networking opportunities in online education: 3=Yes; 2=Don't know; 1=No
Variable 10
Quality of online education is as good as live education: 3=a great extent; 2=some extent;
1=little extent
ID Years in
practice
CNS offers a
variety of
online
opportunities
to obtain
CME credits.
CNS offers a
variety of live,
face-to-face
opportunities
to obtain
CME credits.
I enjoy
participating
in online
educational
activities.
I enjoy
participating
in live, face-
to-face
educational
activities.
Do you believe
that there are
good
networking
opportunities at
live, face-to-
face meetings?
Do you believe that
there are good
networking
opportunities when
participating in
online activities?
To what
extent do you
feel that the
quality of
online
education is
as good as
live, face-to-
face
education?
1 1 4 4 4 3 2 2 3
3 1 5 4 3 3 3 1 2
5 1 5 4 5 4 3 1 3
6 1 5 5 5 5 3 1 3
7 1 5 5 5 5 3 2 2
12 1 4 4 4 5 3 1 2
17 1 4 4 4 3 3 3 3
20 1 4 4 5 4 3 3 3
23 1 4 4 4 4 3 2 2
33 1 5 4 5 2 3 1 3
34 1 4 4 4 4 3 2 3
2 2 5 3 4 5 2 3 2
8 2 5 3 5 3 3 2 2
9 2 4 4 5 5 3 1 2
24 2 2 3 4 2 1 1 3
25 2 5 4 5 4 3 1 2
26 2 5 5 4 4 3 1 3
28 2 5 4 4 4 2 1 3
29 2 4 3 5 5 3 3 3
30 2 4 4 4 4 3 1 3
31 2 5 4 5 4 2 3 3
16 3 4 4 4 4 3 1 2
18
21 3 5 5 5 5 3 2 3
27 3 2 4 5 4 1 1 3
4 4 5 5 5 5 2 1 3
10 4 3 4 5 4 1 1 2
11 4 3 5 5 5 3 1 2
13 4 5 5 5 5 3 1 3
14 4 5 5 5 4 3 3 3
15 4 5 5 5 5 3 3 2
18 4 4 3 5 4 3 2 2
19 4 5 5 5 5 2 2 2
22 4 2 3 4 2 1 1 3
32 4 4 4 3 5 3 1 1