Slide 1

SRDR Quarterly Training Brown Evidence-based Practice Center Brown University December 5 th, 2014 1:00pm-2:00pm Best Practices: Improving the efficiency and effectiveness, and consistency of data entry SRDR was initially developed by the Tufts EPC and currently maintained by the Brown EPC under contract with AHRQ (Contract No. HHSA 290-2007-10055-I & HHSA 290-2012-1200012-I ).

Slide 2

12/5/2014http://srdr.ahrq.gov2 Presentation outline Summarize common difficulties / challenges encountered by users entering data into SRDR Demonstrate best practices to mitigate these challenges

Slide 3

12/5/2014http://srdr.ahrq.gov3 Inefficient design of extraction forms Extraction forms with lots of empty space Unintuitive question titles on extraction forms Unrestrained extraction forms giving too much latitude to extractors Entering data prospectively into SRDR Pitfalls and limitations for copying and pasting data into extraction forms Losing track of data extractor progress, and questions they might have Entering data retrospectively into SRDR Many different file formats (MS Word, MS Excel, PDFs) Preparing results data for importing (coming feature) Challenges entering data into SRDR

Slide 4

12/5/2014http://srdr.ahrq.gov4 Exporting data from SRDR in a format usable to others (statisticians, analysts) or in a format ready to be used as evidence tables in appendix of an EPC report To be addressed in a future Webinar in January new features added to SRDR since the last training Webinar held in September Challenges exporting data from SRDR

Slide 5

12/5/2014http://srdr.ahrq.gov5 Compilations of strategies used by SRDR Users, EPCs, and the SRDR Team Recommendations that will enable you to overcome the challenges associated with entering data into SRDR Benefits of using best practices include: Reduce time and effort required to enter data into SRDR Improve accuracy and consistency of data collected Increase accessibility of data for other SRDR contributors and public users Take advantage of SRDR Tools and features Best Practices

Slide 6

12/5/2014http://srdr.ahrq.gov6 Best Practices for Designing Efficient Extraction Forms

Slide 7

12/5/2014http://srdr.ahrq.gov7 Background Key Questions (KQ) entered into an SRDR project should parallel the KQs addressed in your systematic review or EPC Report. Each KQ can only be associated with a single (primary) extraction form Several KQs may use the same extraction form Challenge Since a KQ can only be associated with one (primary) extraction form, collection of certain data across all eligible studies for the KQ (i.e. study quality information for multiple study designs) can result in extraction forms with long lists of mutually exclusive questions (i.e questions relevant to only one type of study design) that could be unwieldy to navigate Create accessory forms to complement primary extraction form

Slide 8

12/5/2014http://srdr.ahrq.gov8 Best Practice Create accessory extraction forms to complement the data collected on a primary extraction form (each accessory extraction form will require adding a new KQ to your project) Note: You will need to describe in the Project notes field of the Project Information page of the SRDR project, which KQs were added to the project to create accessory extraction forms and the KQs from the report they address. Benefit Creating accessory extraction forms will shorten the number of data fields that an extractor need enter data into for any one extraction form. Create accessory forms to complement primary extraction form (Cont.)

Slide 9

12/5/2014http://srdr.ahrq.gov9 Example: Denoting addition of accessory extraction forms in Project Notes field.

Slide 10

12/5/2014http://srdr.ahrq.gov10 Background A question title (data header) is required for each question added to the extraction form Displayed as column (spreadsheet) headers in simple and advanced export of data from SRDR Challenge It is difficult to understand arbitrarily named or unintuitive question titles (e.g., BMI_X instead of Average BMI). This difficulty adds time required by the data extractor to interpret the meaning of the question and presents confusion to other users. Use self-explanatory question titles on extraction forms

Slide 11

12/5/2014http://srdr.ahrq.gov11 Best Practice Question titles should be self-explanatory and include: units and/or specific data ranges / formats (i.e. only whole number values; or DD/MM/YYYY date format) Benefits Minimize data extraction errors Improve accessibility of data by users Use self-explanatory question titles (Cont.)

Slide 12

12/5/2014http://srdr.ahrq.gov12 Example: Intuitive and unintuitive question titles Self-explanatory: Not self-explanatory:

Slide 13

12/5/2014http://srdr.ahrq.gov13 Background Users can add a list of suggested arm and outcome titles to the Arm and Outcome tabs of SRDR extraction forms. During individual data abstraction, data extractors are given the option to use these pre-specified arm and outcome titles or create their own. Challenge When extractors have the latitude to name the arms and outcomes on their own, they may come up with different names (e.g., Cardiovascular disease could be named as: CVD, CV disease, etc.. ). Variations in the naming of arms and outcomes is unwanted and require time to standardize. It will also confuse users of the data unfamiliar with the project. Pre-specify arm and outcome titles for data extractors to use on the extraction form

Slide 14

12/5/2014http://srdr.ahrq.gov14 Best Practice Step 1: The project lead should pre-specify a list of the arm and outcome titles and then add the titles from this list for other extractors to use as suggested arms and outcomes on the Arms and Outcomes tabs of the SRDR extraction form. Step 2: Train data extractors in the team to use suggested arms and outcomes, unless it is absolutely necessary to create a new name (and then the new name should be added to the suggested list). Benefits Reduce data entry time Reduce spelling errors, case errors and unwanted redundancy Reduce data cleaning needs Pre-specify arm and outcome titles for data extractors to use on the extraction form

Slide 15

12/5/2014http://srdr.ahrq.gov15 Example of the unwanted variation that can result in Outcome titles

Slide 16

12/5/2014http://srdr.ahrq.gov16 Background Users may choose from the following selection of question types to add to an SRDR extraction form: Text Dropdown menu Multiple choice (select one) aka. Radio button question Multiple choice (select any) aka. Checkbox question Matrix of choices (only one answer allowed) aka. Matrix of Radio buttons Matrix of choices (select more than one answer) aka. Matrix of Checkboxes Matrix of dropdowns (specify option in each cell) Challenge SRDR extraction forms relying primarily on text, dropdown menu, and multiple choice questions could result in very long forms. Long forms are a challenge to use because of excessive vertical scrolling which increases the time needed by data extractors to enter data. Shortening extraction forms using dropdown and matrix-type questions

Slide 17

12/5/2014http://srdr.ahrq.gov17 Best Practice When possible, consolidate multiple questions into a single matrix question. (e.g. when the same set of descriptive statistics and other information is asked for multiple risk factors on the Baseline tab of SRDR extraction forms, the individual questions can be consolidated into separate cells within a single matrix question). Benefits Any number of columns or rows of cells may be added to matrix questions on SRDR extraction forms enabling users to minimize the length of the form. Values entered into each cell of a matrix question are exported top separate columns of simple export spreadsheets allowing for easy analysis/isolation of data. Using Dropdown and Matrix-type questions (Cont.)

Slide 18

12/5/2014http://srdr.ahrq.gov18 Example contrasting vertical space used by matrix and non-matrix questions Multiple Text Qs Single Matrix Q Multiple Matrix Qs

Slide 19

12/5/2014http://srdr.ahrq.gov19 Background Arm and outcome descriptions can be added to arms and outcomes entered into the Arm and Outcomes tabs of SRDR extraction forms. Challenge When exporting data from SRDR, Arm Title and Arm Description (same for Outcome) are concatenated into a single character string (e.g. the title, Vitamin D, with description, consumed, will appear on export as Arm: Vitamin D [consumed]) to distinguish arms that may have the same title but different descriptions. This concatenated string may require additional processing steps before analysis. Save time cleaning data by creating arm and outcome description questions on your forms

Slide 20

12/5/2014http://srdr.ahrq.gov20 Best Practice Instead of entering information into arm description and outcome description fields on Arm and Outcome tabs, create a question on Arm Details, Outcome Details tabs to capture this information. Note: To use this best practice, Arm Details and Outcome Details tabs on extraction forms should be set to display questions for each arm or outcome, respectively. Benefit Less time needed to clean arm and outcome titles exported from SRDR Avoid using arm, outcome description fields on Arm, Outcome tab (Cont.)

Slide 21

12/5/2014http://srdr.ahrq.gov21 Example: Adding arm description value to arm title on Arm tab Arms tab of extraction form Simple export of Arms tab SRDR Simple Export tool used to export data entered into extraction form

Slide 22

12/5/2014http://srdr.ahrq.gov22 Example: Adding arm description value to arm title on Arm tab (Cont.) Export of extraction form using Report Builder tool

Slide 23

12/5/2014http://srdr.ahrq.gov23 Example: Adding arm description value to question on Arm Details tab Arm Details tab of extraction form Simple export of Arm Details tab SRDR Simple Export tool used to export data entered into extraction form

Slide 24

12/5/2014http://srdr.ahrq.gov24 Example: Adding arm description value to question on Arm Details tab (Cont.) Export of extraction form using Report Builder tool

Slide 25

12/5/2014http://srdr.ahrq.gov25 Background Questions asked on an extraction form (excluding questions concerning study design or study quality), are typically asked for a specific arm or outcome defined in a study. This is typically accomplished by creating questions for specific arms or outcomes on the form, or instead creating blank data fields into which miscellaneous arms or outcomes may be entered (the same applies when creating extraction forms in SRDR). Challenge An extraction form (e.g., paper, software) seeking to predefine all possible arms/outcomes could be lengthy and unwieldy. To accommodate unexpected arms/outcomes during data extraction, the extraction form will need to be revised (this problem also applies to SRDR when features are not used). Use SRDRs feature to dynamically create arm/outcomes questions in tabs

Slide 26

12/5/2014http://srdr.ahrq.gov26 Best Practice Rather than enumerating each question explicitly on an SRDR extraction form, use the dynamic arm/outcome question feature to ask a question on extraction form tabs ( Arm Details, Baseline, Outcome Details, Results, Adverse Events) for just the arms and outcome that are defined in a particular study. Pro-tip: After constructing an extraction form, always check to see how data fields are presented in simple and advanced data export. Also, pilot the extraction form. Benefits Minimizes the number of times questions need to be asked on a tab, resulting in a smaller extraction form and quicker navigation Extraction form will readily accommodate unforeseen arm/outcomes keeping the number of revisions to the extraction form at a minimum Use SRDRs feature to dynamically create arm/outcomes questions in tabs (Cont.)

Slide 27

12/5/2014http://srdr.ahrq.gov27 Example: Extraction form that collects data for a predefined number of arms

Slide 28

12/5/2014http://srdr.ahrq.gov28 Example: Replicating the extraction form in SRDR and exporting data Simple export of Arm Details tab of SRDR extraction form

Slide 29

12/5/2014http://srdr.ahrq.gov29 Simple export of Arm Details tab of SRDR extraction form Example: Incorporating feature to dynamically create arm questions in form and exporting data 1.2. 3.

Slide 30

12/5/2014http://srdr.ahrq.gov30 Best Practices for Entering Data Prospectively into SRDR

Slide 31

12/5/2014http://srdr.ahrq.gov31 Definition of prospective data entry Copying data from the full text of studies included in a systematic review, directly into data fields on SRDR extraction forms. Method Manual entry: Requires typing or copying and pasting data values, individually, into data fields on the extraction form. Background

Slide 32

12/5/2014http://srdr.ahrq.gov32 Background Symbols, such as those listed below are commonly found in the full text of studies and can be typed or copied and pasted into extraction forms in SRDR Mathematical operations (, , , , >, ,