searching beyond the rct - looking for sibling studies on qualitative, economic and process research...
Post on 12-Jan-2016
220 Views
Preview:
TRANSCRIPT
Searching beyond the RCT - looking for sibling studies on qualitative, economic
and process research
Searching beyond the RCT - looking for sibling studies on qualitative, economic
and process research
Faten Hamad and Christine UrquhartFaten Hamad and Christine Urquhart
Introduction
Systematic reviews – producing the evidence
…We have the evidence on what works (or doesn’t) – what happens next?
Finding qualitative, economic and process research evidence – preliminary findings of research to identify sibling studies associated with particular randomised controlled trials.
Aims and objectives of presentation are to:
Discuss searching strategies to find evidence beyond the randomised controlled trial
Present preliminary findings
Assess your reactions to the findings!
Background
Sibling studies is the name which has been chosen to indicate the relationship that groups a set of related studies (randomised control trial, qualitative, process and economic evaluations).
An intervention may be viewed as:
“a complex system where intervention itself is a fragile creature that is delivered in a social system
of interacting elements, such as an individual’s capacity, interpersonal relationships, institutional
setting and infrastructure.”
Search strategies and search filters
Existing work by Cochrane Information Retrieval Methods Group and others (e.g. Hedges team)
Aim to maximise recall (sensitivity), keeping precision reasonable, and ensuring specificity
Search strategies vary with the database – depends on index terms, which terms have to be added as free text terms…
Relevance judgements
Seed studies selection
We chose a range of topic areas, and dates for the large RCTs and chose five seed studies with different
characteristics. We will discuss the result of the following two of seed studies:
-Telemedicine and diabetes (a known RCT, with many known direct siblings, that could be used to validate
and checking the search strategies for their sensitivity).
- Tamoxifen for breast cancer prevention (known qualitative sibling, two RCTs involved).
Search strategies and databases under investigation
Different search strategies and different databases were explored for the reason of exploring the differences in search performance in regard to different study areas.
- Author-Subject search (using a very simple subject term combination) with each of the author names in the seed article in turn, in MEDLINE on PubMed.
-Related article search in MEDLINE on PubMed (for the seed article).
- E-library search (with a combination of ISI (WoS), OCLC WorldCat, OCLC Articles First, EBSCO Business Complete, and EBSCO International Bibliography) - simple subject term combination only, and limited to the first 300 documents retrieved.
Search strategies and databases under investigation-Cont.
- SCOPUS search (author-subject search as in MEDLINE on PubMed).
- CINAHL (author-subject search as in MEDLINE on PubMed).
- Cited reference in ISI, Web of Science (with the seed article as the reference).
- Cited reference search in SCOPUS.
- Cited reference search in CINAHL.
Results Table 1 IDEATeL study: search strategy and retrieval performance
search strategyRelevant/R Total
retrievedUnique
Relevant/ROdds Ratio
Related Search(PubMed) 25 186 5 1.01219
Author +Subject(PubMed) 23 157 4 1.09189
Citation(Web of Science) 14 57 1 1.86855
Subject search(e-library) 39 296 33 1.19589
SCOPUS Author subject 39 52 5 23.64179
SCOPUS citation 32 64 12 7.13514
CINAHL Author subject 17 18 0 100.85393
CINAHL citation 4 7 0 6.90196
Total relevant retrieved without duplicates 106
Total retrieved without duplicate 634
Total non-relevant retrieved without duplicates 528
Table 2 IDEATeL study search strategies retrieval per study type and odds ratio calculations.
search strategy RCTs Qualitative Economical evaluation
Process evaluation
R1 R/N2 OR3 R1 R/N2 OR3 R1 R/N2 OR3 R1 R/N2 OR3
Related Search(PubMed) 8 39 0.1032 10 21 0.2396 3 8 0.1887 4 13 0.1548
Author-Subject(PubMed) 8 39 0.1271 9 22 0.2534 2 9 0.1377 4 13 0.1906
Citation(Web of Science) 4 43 0.1990 8 23 0.7442 2 9 0.4755 0 17 0
Subject search(e-library) 25 22 0.2034 20 11 0.3254 9 2 0.8055 6 11 0.0976
SCOPUS Author - subject 17 30 2.9205 12 19 3.2551 2 9 1.1453 8 9 4.5812
SCOPUS citation 19 28 1.5692 8 23 0.8044 2 9 0.5139 3 14 0.4955
CINAHL Author subject 10 37 24.054 4 27 13.185 1 10 8.9 2 15 11.867
CINAHL citation 1 46 0.7391 3 28 3.6429 0 11 0 0 17 0
1 : Relevant retrieved. 2: Relevant not retrieved. 3: Odds Ratios.
Table 3 Tamoxifen study: search strategy and retrieval performance
search strategyRelevant/R Total Unique
Relevant/ROdds Ratio
Related Search(PubMed) 17 200 13 0.58277
Author+Subject(PubMed) 17 451 7 0.24573
Citation(Web of Science) 5 53 2 0.60155
Subject search (e-library) 59 288 54 2.316121
SCOPUS Author subject 72 229 47 4.76069
SCOPUS citation 4 59 2 0.41722
CINAHL Author subject 19 41 7 5.49701
CINAHL citation 1 4 0 1.87527
Total relevant R without duplicates 156
Total retrieved without duplicate 1028
Total non-relevant R without duplicates 872
Table 4 Tamoxifen study search strategies retrieval per study type and odds ratio calculations.
search strategy RCTs Qualitative Economic Process evaluation
R1 R/N2
OR3 R1 R/N2 OR3 R1 R/N2
OR3 R1 R/N2 OR3
Related Search(PubMed) 12 83 0.1098 3 31 0.0735 1 0 DIV/0
1 25 0.0304
Author-Subject(PubMed) 12 83 0.0463 1 33 0.0097 0 1 0 4 22 0.0582
Citation(Web of Science) 2 93 0.0677 2 32 0.1966 0 1 0 1 25 0.1258
Subject search(e-library) 26 69 0.1629 18 16 0.4864 0 1 0 13 13 0.4323
SCOPUS Author-subject 52 43 0.6470 10 24 0.2229 0 1 0 10 16 0.3344
SCOPUS citation 0 95 0 4 30 0.3685 0 1 0 0 26 0
CINAHL Author subject 15 80 1.1676 1 33 0.1887 0 1 0 3 23 0.8123
CINAHL citation 1 94 0.0750 0 34 0 0 1 0 0 26 0
1: Relevant retrieved. 2: Relevant not retrieved. 3: Odds Ratios.
Discussion
CINAHL author-subject search was the most effective search that can precisely retrieve direct and indirect siblings of a certain seed study in general with higher score in the case of the IDEATeL study in specific.
- IDEATeL, the number of retrieved records was 41 (with 19 relevant);
- Tamoxifen, the number retrieved was 18 (with 17 relevant);
CINAHL author- subject odds ratios were quite different, but this does not affect the fact this search strategy was the best search strategy, in terms of the chances of finding relevant material from a search, with proportionally fewer irrelevant items retrieved.
The SCOPUS author–subject strategy was the second best search performance.
For both the seed studies, the e-library subject search retrieved most of the unique studies.
Simple subject terms and / or the combination of simple subject terms and author names for each seed study, appeared to be the most effective method of retrieving most of the siblings, outperforming citation searching (apart from SCOPUS citation with the IDEATeL seed study).
Conclusions and future works
The analysis indicates that there is neither a winner in the search strategies nor for the databases.
The CINAHL author-subject performed well (in terms of precision).
The SCOPUS author-subject search performed the next best, (higher recall for the relevant studies for both seed studies).
The e-library author-subject search produced a good number of relevant studies (and a high proportion of unique items as
well).
Conclusions and future works
Further research will examine how expanding the number of databases and changing the selection will affect the relative performance of the e-library Meta-lib search.
Further work is needed to identify how grey literature, conference proceedings and thesis and dissertation material can be obtained efficiently, as Web of Knowledge found some of the direct sibling publications for the IDEATel studies that could not be obtained on any of the other databases used.
Thank you for Listening
top related