Kara L. Hall, PhD Director, Science of Team Science (SciTS) Team National Cancer Institute (NCI) In collaboration with : Brooke Stipelman, PhD (Contractor, NCI) Janet Okamoto, PhD (Postdoctoral Fellow, NCI) Amanda Vogel, PhD (SAIC-Frederick, NCI)
The Added Value of Transdisciplinary Team Science: Findings from Multi-Methods
NORDP May 22, 2012
Overview
• Define Transdisciplinarity
• Impetus for Transdisciplinary Team Science • Share findings from a variety of different ways to consider
impact or “value” of a Transdisciplinary (TD) initiative
A Continuum of Disciplinary Integration
Unidisciplinary
Multidisciplinary
Interdisciplinary
Transdisciplinary
Researchers from a single discipline work together to address a common problem
Researchers from different disciplines work sequentially, each from their own discipline-specific perspective, with a goal of eventually combining results to address a common problem
Researchers from different disciplines work jointly to address a common problem. Some integration of perspectives occurs, but contributions remain anchored in their own disciplines.
Researchers from different disciplines work jointly to develop and use a shared conceptual framework that synthesizes and extends discipline-specific theories, concepts, and methods, to create new approaches to address a common problem
Within
Across
Disciplines
Adapted from Rosenfield, 1992
Advancing the Tobacco Field through Integration
•Candidate genes•GWAS•Functional studies
Phase II-III Trials•Existing meds•Novel compounds
•fMRI•PET•Neuropsych assessment
•Quit success•Therapeutic response•Withdrawal signs
Merging Biology with Behavior (Tobacco field)
Lerman, 2012
Goal: Development of targeted therapies for nicotine addiction
DCCPS Funded TD Initiatives
Examples of Strategies / Approaches for Examining Processes and Outcomes of Team Science
• Study Designs
– Case study – Quasi-experimental designs
• Data types
– Publications – Administrative – Participant surveys – Interview transcripts
• Methods/Approaches – Peer Review / Expert Judgment – Financial Analysis – Bibliometric methods – Visualization techniques – Network analysis – Integration of qualitative and
quantitative approaches – Interviews
Qualitative Evaluation of Transdisciplinary Research Centers
• TREC: 5 year initiative (2005-2010), $54 million total finding, 4 research centers and 1 coordination center
• One-on-one qualitative semi-structured interviews with 33 TREC I grantees (summer, 2010)
• Documented lessons learned, challenges, facilitating factors, strategies for success, impacts
• Thematic analysis, NVIVO QDM software
Vogel, Stipelman, Hall et al, 2011
Impact of Participating in a TD Research Initiative
Adoption of TD Ethic,
Approaches
New Boundary-Crossing
Collaborations
Scientific Progress
Institutional Culture Change,
Resource Development
Career Development, Advancement
Vogel, Stipelman, Hall et al, 2011
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
• Developed a TD Ethic – Believe in the value of TD approach, team work – Greater appreciation for other disciplines – Willingness to continue learning in other areas of science – Willingness to use approaches from other disciplines, including concepts,
theories, and methods
• Reported using a TD approach in their current and planned research, introduced or reinforced by TD initiative – Reported applying the above principles – Said their current research is more sophisticated, complex, and “holistic”
than their pre-TD initiative research (more variables and measures, from a variety of disciplines)
– Many used the word “transformed” to describe the impact of TD initiative on their attitudes about TD research, and the ways they conduct their research
Vogel, Stipelman, Hall et al, 2011
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
0
1
2
3
4
5
Uni Inter/Trans
Time 1
Time 2
Leve
l of
Agr
eem
ent
Changes in Research Orientation Over Time in TD Initiative
Key finding: Decrease in unidiscplinary orientation from year 1 to year 3 Method: Research orientation factor scores from investigator survey in year 1 and 3
Hall, Stokols, et al., 2008
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
Change in Cross-Disciplinary Integration Over Time for Developmental Proposal Ratings
0
10
20
30
40
50
60
70
80
UD MD ID
Percent 2006
2007
Key finding: Increase in cross-disciplinary integration of proposals from year 2 to 3 Method: Content analysis of pilot grant proposals developed during initiative
Hall, Stipelman, et al., 2009
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
CPHHD Network Survey
Nodes sized on brokerage scores
Inter/Transdisciplinary Orientation
Okamoto, Vogel, Hall, et al., 2011
Key finding: Investigators with a primarily TD orientation (vs. MD or UD) were more likely to serve as brokers, both between centers and within their own centers Method: Web-based social network and collaboration readiness survey delivered to CPHHD investigators Dec 2010-Jan 2011
1 5
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
•Established new research collaborations
-Across disciplines, both within and across institutions -Across departments, schools, and institutions – breaking down barriers -With community organizations, for translational work
Vogel, Stipelman, Hall et al, 2011
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
TTURC R01 Comparison Group
Co-Authorship Networks of TD Center Grants and R01 Investigator-Initiated Grants
nodes sized on times cited
•The TTURC co-authorship network has 1027 unique authors (almost double the R01 comparison group). •The many red links show that there is a high degree of coauthorship across the TTURC centers while the many gray links indicate a great deal of collaboration within the TTURC centers.
Adoption of TD Ethic and Approaches
New Boundary Crossing
Collaborations
Scientific Progress
• TD approaches accelerated advancements in research design and methods – Development of new conceptual frameworks and research questions
and applications of research methods in new areas – Development of new measures, instruments, and software
• Important findings in previously unexplored areas of science, e.g., – Intersection of sleep, obesity, and cancer – Intersection of residential neighborhood GIS indicators, young adult
food purchasing, young adult eating patterns
• Cross-fertilization of research areas via publications in journals and presentations at conferences in other, fields, e.g. obesity researchers presenting at cancer conferences can advance the science in multiple fields
Vogel, Stipelman, Hall et al, 2011
Annual Publications
TD center publications have longer start up period compared to R01s but become more productive over time
0102030405060708090
1 2 3 4 5 6 7 8 9 10
Publ
icatio
ns -A
ll
Project Year
Hall, Stokols, Stipelman, et al., 2012
Centers initial lag in number of publications is eliminated around Project Year 4.
Productivity of Transdisciplinary (TD) Center Grants and R01 Investigator-Initiated Grants
Cumulative Publications
050
100150200250300350400450500
1 2 3 4 5 6 7 8 9 10
Cum
ulat
ive P
ublic
atio
ns -
All
Project Year
TTURCR01_stackedR01_long
Method: Quasi-experimental design comparing number of publications of TD initiative with matched R01 projects from the tobacco field over 10-year period
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
Aver
age
Num
ber o
f C
itatio
ns
Average Number of Adjusted Citations in TTURC and Comparison Group Publications Over Time
Key finding: During TTURC years – TTURC investigators produce more highly cited publication than CG (Journal impact factors were similar across the two groups – subsequent analysis showed both groups publishing in the similar journals) Method: Comparison of TTURC and R01 investigators’ citations rates from entire corpus of publications 1996-2010.
Long R01 Comparison Group: Map of Science - Publication Counts
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
TTURC Group: Map of Science - Publication Counts
Adoption of TD Ethic and Approaches
New Boundary Crossing Collaborations
Scientific Progress
In Sum
• This brief snap shot highlights examples of impacts and added value to engaging in TD science initiatives, such as: – Conduct of more sophisticated and complex research – Development of novel areas of science – Greater publication productivity and impact – Greater breadth and faster spread of research across domains of
science • Broadly we see advances in science, individual careers, and
institutional culture
Acknowledgements
Current SciTS Team Members Kara Hall, PhD Janet Okamoto, PhD MPH Brooke Stipelman, PhD Amanda Vogel, PhD MPH
Past SciTS Members Annie Feng, Ed.D Stephen Marcus, PhD Louise Mâsse, Ph.D Richard Moser, PhD Daniel Stokols, PhD Brandie Taylor, MA William Trochim, PhD Patrick Weld, MSW MPA
Key Contributors Katy Borner, PhD Barbara Gray, PhD Glen Morgan, PhD Linda Nebeling, PhD, MPH, RD, FADA Angela Zoss, MA Discovery Logic Inc./Thompson Reuters TTURC Evaluation Working Group TREC Evaluation Working Group CPHHD Evaluation Working Group