data management tools to measure the impact of core facilities

ARTICLE

Data Management Tools to Measure the Impact of Core Facilities

Diane B. Smith,1,2 Tracy L. Yarnell,1,2 Barbara J. Jibben,1,2 Linda E. Liou,3 Carolyn J. Hovde,3

and Julia Thom Oxford1,2,4,*1Biomolecular Research Center, 2Center of Biomedical Research Excellence in Matrix Biology, and 4Department of BiologicalSciences, Boise State University, Boise, Idaho 83725-1511; and 3Idaho INBRE Program, University of Idaho, Moscow,Idaho 83844-3025

The Biomolecular Research Center at Boise State University is a research core facility that supports the study ofbiomolecules with an emphasis on protein structure and function, molecular interactions, and imaging. Themission of the core is to facilitate access to instrumentation that might otherwise be unavailable because of thecost, training for new users, and scientific staff with specialized skills to support early-stage investigators, as well asmore established senior investigators. Data collection and management of users and their research output isessential to understand the impact of the center on the research environment and research productivity.However, challenges are often encountered when trying to fully quantify the impact of a core facility on theinstitution, as well as on the career success of individual investigators. This challenge can be exacerbated underthe conditions of unprecedented growth in biomedical research and shared core facility use that has beenexperienced at Boise State University, an institution of emerging research excellence. Responding to thesechallenges required new approaches to information management, reporting, assessment, and evaluation. Ourspecific data management, evaluation, and assessment challenges included 1) collection and management ofannual reporting information from investigators, staff, and students in a streamlined manner that did not lead toreporting fatigue; 2) application of software for analyzing synergy among programs’ management strategy andinvestigator success; and 3) consolidation of core facility management, billing, and reporting capabilities into 1cohesive system. The data management tools adopted had a beneficial effect by saving time, reducingadministrative burden, and streamlining reporting. Practices implemented for data management have facilitatedeffective evaluation and future program planning. The substantial burden of assessment requirementsnecessitates early consideration of a strategy for data management to allow assessment of impact.

KEY WORDS: assessment tools, biomedical research, evaluation, core facility

INTRODUCTION

The Biomolecular Research Center (BRC) at Boise StateUniversity is a research center with shared core facilitieslocated at an institution of emerging excellence in bio-medical research. The focus of the BRC is the study ofbiomolecules with an emphasis on proteins and theirmolecular interactions. The BRC is also the administrativecenter for multiple programmatic biomedical research grantawards that emphasize mentored career development andstudent training. To evaluate the performance of the centeras a whole, as well as its individual shared core facilities andspecific programs with respect to their contributions to both

the research and educational missions of Boise StateUniversity, we adopted specific evaluation and assessmentdata management tools and evaluation approaches. Thesemeasures were put in place so that the center could createrequired reports in a timely manner for the institution andfor individual funding agencies. These efforts supportprogram management and promote sustainability of thecenter, so the research infrastructure provided can continueto serve an integral role in the research and research trainingmission at Boise State. These practices may apply to otherprimarily undergraduate institutions with a rapidly growingand emerging biomedical research emphasis.

Within the BRC, small shared core facilities(,$200,000 of recharge income annually) support research.The National Institutes of Health (NIH) defines corefacilities as centralized shared resources that provide accessto instruments, technologies, and services, as well as expertconsultation for scientific and clinical investigators. 1 Corefacilities are discrete units within an institution and mayhave dedicated personnel, equipment, and space for

*ADDRESS CORRESPONDENCE TO: Julia Thom Oxford, BiomolecularResearch Center, Boise State University, 1910 University Drive, M.S.-1511, Boise, ID 83725, USA (Phone: 208-426-2395; E-mail: [email protected]).This article includes supplemental data. Please visit http://jbt.abrf.orgto obtain this information.https://doi.org/10.7171/jbt.20-3203-002

Journal of Biomolecular Techniques 32:63–73 © 2021 ABRF

mailto:[email protected]

mailto:[email protected]

http://jbt.abrf.org

https://doi.org/10.7171/jbt.20-3203-002

http://www.ABRF.org

operations. In general, core facilities recover the cost, or aportion of the cost, of providing services in the form of userfees that are charged to an investigator’s funds, often toNIHor other federal grants. 1 Instrumentation within the BRCcore laboratories supports recombinant protein expressionand purification, mass spectrometry, analytical instrumenta-tion for characterization of molecular interactions, histologyand microscopy, and cell/molecular biology investigations.Although the focus of the BRC is the study of biomoleculeswith an emphasis on proteins and their molecular interac-tions, the BRC has an additional level of emphasis onextracellular matrix molecules, cell-matrix interactions, andboth the biochemical and biomechanical properties ofextracellularmatrices and those used for biomimetic scaffoldsas part of one of the programs administered by the center, theNIHCenter of Biomedical Research Excellence (COBRE) inMatrix Biology Program.

In addition to the COBRE inMatrix Biology, the BRCis the administrative center for statewide and regionalprogrammatic biomedical research grant awards for whichBoise State University is a partner institution, including theNIH-sponsored Idaho Institutional Development Awards(IDeA) Network of Biomedical Research Excellence(INBRE) and the Mountain West Clinical TranslationalResearch Infrastructure Network programs. These pro-grams share the goals of workforce development, careermentoring, and expanding research capacity for biomedicaland behavioral research and are programs within the IDeAprograms at NIH.

The BRC also provides administrative support forequipment grants that provide instrumentation for theregional research community from the National ScienceFoundation (NSF) and the M. J. Murdock CharitableTrust. Additionally, the BRC administers a limited numberof individual researcher-initiated awards from agencies,including the American Heart Association, the NationalAeronautics & Space Administration, and NSF. Theseawards have made a significant impact on research andresearch infrastructure growth at Boise State.

The NIH INBRE program is designed to foster thedevelopment, coordination, and sharing of research re-sources and the expertise that will expand researchopportunities and increase the number of competitiveinvestigators in IDeA-eligible states. The program is alsodesigned to enhance the caliber of scientific faculty atresearch institutions and undergraduate schools to continueto attract talented students to augment the science andtechnology knowledge of the state’s workforce. Amultidisciplinary research program within a broad bio-medical thematic focus creates an environment thatstrengthens research expertise and infrastructure andfosters collaboration.

The NIH COBRE program strengthens biomedical orbehavioral research capacity in institutions from IDeAstates. COBREPhase I provides support to develop researchinfrastructure and to foster independence of juniorinvestigators. COBRE Phase II is designed to continue theprogress toward building an independent research centerthat is competitive for support fromNIH and other fundingagencies. The COBRE Phase III awards provide support formaintaining COBRE research cores developed duringphases I and II that are essential for the continuing conductof basic, clinical, translational, and community-basedresearch at the institution and to sustain a collaborative,multidisciplinary research environment for research pilotprojects and mentoring. Each phase takes place over a 5-yperiod. Boise State has recently completed phase I(2014–2019) and has transitioned to phase II (2019–2024).

The growth in biomedical research and shared corefacility use that we experienced required new approaches toinformation management for reporting, assessment, andevaluation that were not available at the universitypreviously. Here, we report on our approach to streamlineadministrative functions and increase the efficiency ofinformation management and the ability to assess andevaluate our center’s impact on investigators’ success, as wellas programmatic success. Specifically, the data managementchallenges included 1) collection and management ofannual reporting information from investigators, staff, andstudents; 2) implementation of software for analyzingsynergy between our program’s management strategy andinvestigator success; and 3) consolidation of core facility’smanagement, billing, and reporting capabilities into 1cohesive system.

The results indicate that the adoption of an effective datamanagement tools can lessen the administrative burden andallow a more effective day-to-day management of centeroperations while increasing the effectiveness of evaluation,assessment, and reporting. The substantial burden of datamanagement and assessment requirements for researchcenters and programs necessitates early consideration of athoughtful and meaningful strategy for data management.Flexibility must be maintained to allow continuous improve-ment to approaches used for assessment and evaluation inways that streamline reporting and dissemination ofoutcomes to all stakeholders. Here, we present lessonslearned for institutions of emerging research excellence.

MATERIALS AND METHODS

Acquisition of data documenting growth of researchexpenditures and awards at Boise State University

Information on research expenditures was acquired from theNSFNational Center for Science and Engineering Statisticswebsite (https://ncsesdata.nsf.gov/ids/).2

SMITH ET AL./DATA MANAGEMENT TOOLS

JOURNAL OF BIOMOLECULAR TECHNIQUES, VOLUME 32, ISSUE 2, JULY 202164

https://ncsesdata.nsf.gov/ids/

Acquisition of graduate program information

The 10th day of the fall semester enrollment numberswere reported by the Registrar’s Office at Boise StateUniversity.3

Collection and management of annual reportinginformation from investigators, staff, and students

The BRC assisted in the development of a customized datamanagement system to collect information from individ-uals to fulfill the reporting requirements of fundingagencies and the reporting requirements of Boise StateUniversity. Forms were created specifically for individualroles within the BRC and its associated programs.Students, research staff, faculty members, project investi-gators, and core managers completed unique forms withcustomized interfaces. This information was self-reportedannually, and collected information was automaticallydeposited into a database.

Implementation of software for analyzing synergy betweenprogrammatic components and investigator success

Publication data queries for grant P20GM109095 weremade through the MyNCBI software system, a tool thatretains user information and database preferences usingfeatures that save citations andmanage peer-reviewed articlecompliance with the NIH public access policy.4 Thisinformation was added to the bibliography section ofMyNCBI, and then bibliometric data were exported fromMyNCBI and imported into Gephi to derive the statisticalinformation on the coauthorship network.5 Visualization ofthe resulting network was rendered using the Forced Atlas 2algorithm.6

Consolidation of core facility’s management, billing, andreporting capabilities into 1 cohesive system

Agilent iLab Operations Software (iLab) was used for corefacility management and billing. The web-based system

FIGURE 1

Logicmodel. The logicmodel presents the long-term goals of the BRC and information about the resources available to thecenter (input), activities that are planned, types of output, the manner in which output is monitored, information that iscollected (output collection), and the anticipated effect of the planned activities for each of the aims of the center. ACS,American Chemical Society; AHA, American Heart Association; B2B, National Institutes of Health (NIH) Bridges to theBaccalaureate Research Training Program; CTR-IN, National Institutes of Health (NIH) Clinical Translational ResearchInfrastructure Network; EPSCoR, National Science Foundation (NSF) Established Program to Stimulate CompetitiveResearch; ITHS,National Institutes ofHealth (NIH) The Institute of Translational Health Sciences; LSAMP,National ScienceFoundation (NSF) Louis StokesAlliances forMinority Participation; NASA,National Aeronautics and SpaceAdministration;RAIN, Regional Allianceof InstitutionalDevelopmentAwards (IDeA)Networkof Biomedical Research Excellence (INBRE);SEED, American Chemical Society Project Summer Experiences for the Economically Disadvantaged Program; SRC, BoiseState University Summer Research Community; WWAMI, The University of Washington School of Medicine's WWAMI(Washington, Wyoming, Alaska, Montana and Idaho) Regional Medical Education.


65JOURNAL OF BIOMOLECULAR TECHNIQUES, VOLUME 32, ISSUE 2, JULY 2021

facilitates service requests, project management, billing, andinvoicing for shared research core facilities.

The combination of these methods streamlinedadministrative functions and increased efficiency of in-formation management and, in turn, our ability to assessand evaluate the center’s impact on investigators’ success,growth of research programs, and core facility sustainability.

RESULTS

Logic model

The design of evaluation strategies for programmaticassessment was outlined in a logic model. The logic modelshown in Fig. 1 represents the relationship betweenprogrammatic resources, planned activities, and the poten-tial outcomes of the program. The logic model includesinformation about the resources available to the center(input), activities that are planned, types of output, themanner in which output is monitored, information that iscollected (output collection), and the anticipated effect ofthe planned activities for each of the aims of the center. Thislogic model serves as a framework for monitoring andevaluation to assess the extent to which goals are achieved.

History

Boise State University has maintained a robust undergrad-uate teaching mission since its establishment and an ever-increasing focus on research growth since 2000. Growth inresearch programs from 1993 to 2019 exceeded a 10-fold

increase, as shown in Fig. 2. Fig. 2 illustrates the growth ofresearch activity over this time frame, with the annual levelof research expenditures at Boise State shown as a functionof year. Boise State received NIH IDeA funding starting in2001, and total NIH funding is shown in blue in Fig. 2. Fig.3 shows the levels of IDeA compared with total NIHfunding at Boise State. Representing a small percentage ofthe total funding, biomedical and behavioral research atBoise State also increased over this time. Future growth willbe due in part to the infrastructure investments made by theNIH IDeA Program to create shared core facilities.

Biomedical research growth fueled by NIH investments incore facility infrastructure

The NIH-sponsored Idaho INBRE program acted as thecatalyst for the creation of the BRC through theestablishment of infrastructure, funding, leadership, andmentorship starting in 2001. Like other institutions, BoiseState University established shared core facilities to supportthe growth of independent and collaborative research effortsby providing access to shared instrumentation. Accordingto Hockberger and colleagues,7 sharing services is drivenby the high cost of instrumentation and the technicalexpertise needed to operate this equipment. Conse-quently, researchers have become increasingly reliant oncore facilities to access state-of-the-art instrumentationand services in a safe, productive, and cost-effectivemanner.7 The increase in biomedical research at Boise

FIGURE 2

Research growth at Boise State. Annual totalresearch and development expenditures areplotted as a function of year (black open circleswith black line). Dashed lines represent the bestfit for the data points and projection for the nearfuture. Extramural grant awards are plotted foreach year (blue closed circles and blue line).Dotted lines represent the best fit for the datapoints and projection for near-future NIH fund-ing levels.


66 JOURNAL OF BIOMOLECULAR TECHNIQUES, VOLUME 32, ISSUE 2, JULY 2021

State is due in part to the availability of core researchfacilities for all researchers.

As an institution of emerging excellence, Boise StateUniversity is a developing doctoral research institution. Forthe academic year 2019–2020, 187 out of 382 studentswereenrolled in recently established Boise State Universitydoctoral degree programs that have the potential tocontribute to biomedical research. Several of the relativelynew programs have the potential to contribute to researchand research training with relevance to biomedical research,and these are shown in Table 1.

Collection and management of annual reportinginformation from investigators, research staff,

and students

Performance standards andmetrics for evaluation are criticalin the assessment of outcomes and evaluating the need andeffectiveness of core facilities.8 Evaluation of the growth inbiomedical research and BRC shared core facility userequired new approaches to information management.

A customized system was developed to collect in-formation from individuals. This information was essential

to fulfill the reporting requirements of funding agencies andBoise State University. Forms were created specifically forunique roles within the BRC and its associated programs.Students, research staff, faculty members, project investiga-tors, and core managers completed unique forms withcustomized interfaces. This self-reported information wascollected annually, checked for accuracy and completenessby the program directors and then by the INBRE andCOBRE evaluators, and deposited into a database. Thecollection of information and having the ability to store it ina searchable database were critical to allow management ofinformation as the center data volume grew over time.Additionally, the system enabled longitudinal reports andanalysis of trends over time (Table 2).

Collaboration between the Idaho INBRE program andthe COBRE program for information collection reducedthe reporting burden on participants who were required toreport much of the same information to several entities byproviding 1 custom system for self-reported information.The custom system allowed for program-specific informa-tion tailored to individuals based on assigned roles. Theexpectation was that reduction of reporting fatigue wouldresult in more reliable self-reported data.

FIGURE 3

NIH funding at Boise State University. NIHfunding from 2001 to 2019 is shown for eachyear. Blue indicates total NIH funding, and grayindicates the fraction of NIH funding from IDeAprograms for each year.

T A B L E 1

Boise State doctoral programs contributing to biomedical research

Name of programYear of

establishmentEnrollmentFall 2019

Electrical andComputer Engineering 2005 37Biomolecular Sciences 2012 33Materials Science and Engineering 2012 48Computing 2016 42Ecology, Evolution, and Behavior 2017 22Biomedical Engineering 2019 5

T A B L E 2

Information collected from individuals

Research milestones PresentationsCore facility usage PublicationsMentoring Training informationResearch support information ParticipantsPeer review panels Inventions, patent, licensesConference attendance Accomplishments



Collection and management of annual reportinginformation uses the following steps: 1) create a yearlyroster of participants and designate appropriate role(s), 2)assign appropriate modules/forms, and 3) request that allindividuals on the roster complete assigned modules/formsand update personal profile by a specific date. Self-reportedinformation is collected in a central database by a pre-establishedreporting year, and database information can be exported invarious ways depending upon need and specific question. Thegathered information is used for federal reporting requirementssuch as the Research Performance Progress Report (RPPR) andthe Scientific InformationReporting System (SIRS). Additionalreports to key stakeholders such as the External AdvisoryCommittee, Boise StateUniversity Institutional Center Report,annual activity reports for faculty and staff, renewal of externallyfunded programs, and other reports can also be generated asneeded. Fig. 4 illustrates the workflow for individual datacollection and the flow of feedback for improvements to theprocess for future years.

Application of software for analyzing synergy betweenprogrammatic components and investigator success

Social network analysis can be used to map collaborationand produce evidence for effective team science.9 Weapplied social network analysis methods to analyze theoutputs of research from the participant user base.Datawereretrieved for a 5-y period from MyNCBI, resulting in 96publications that cited one of our programmatic grants, theCOBRE in Matrix Biology (P20GM109095). Publicationdata were exported from MyNCBI and imported intoGephi to derive statistical information on the coauthorshipnetwork.5 Visualization of the resulting network shown inFig. 5 was generated using the Forced Atlas 2 algorithm.6

Nodes representing each of the 377 authors within thecoauthorship network were sized based on the number ofpublications for that specific author, ranging from 1 to 15for our network. Lines connecting nodes (edges) wereweighted to indicate the strength of connection; the thickerthe edge, the more connections between those 2 nodes.

FIGURE 4

Workflow for the collection of self-reportedinformation. Collection and management ofannual reporting information uses the followingsteps: create a yearly roster of participants,designate appropriate role(s), assign appropriatemodules/forms, request all individuals on rostercomplete assigned modules/forms and updatepersonal profile by a specific date, collect self-reported information in a central database by apre-established reporting year; and exporteddatabase information in various ways dependingonneed. Information is used for federal reportingrequirements such as the RPPR and the SIRS.Additional reports are created from this databaseto disseminate information to other key stake-holders such as the External Advisory Committeeand the Boise State University InstitutionalCommittee for Centers and Institutes. The in-formation is also useful at the time of renewal ofexternally funded programs.



Analysis of core facility staff within the network was includedin addition to individual researchers that represent users of theshared core facility to assess the role that shared core facilityscientists play in supporting researchproductivity representedby publication in peer-reviewed journals.

Core facility research staff members (orange nodes)support research productivity and output. The networkgrew during the 2014–2019 time frame, as shown bycomparing Fig. 5A, B. Output and productivity increasedover this time frame.

Consolidation of core facility’s management, billing, andreporting capabilities into 1 cohesive system

Core facilitymanagement and billing softwarewas adopted for theshared core facilities within the BRC. This approach streamlinedadministrative functions and increased efficiency of informationmanagement and the ability to assess and evaluate the center’simpact on investigators’ success and overall programmatic success.

With the adoption of core facility management andbilling software, the BRC more efficiently managed day-to-day operations as well as the processing of billing andinvoicing. The core facilities now provide a uniform andconsistent customer experience from initial contact to receiptof payment, as shown in Fig. 6. Details of projects aremanaged and maintained through this software. Communi-cations between the core and customers aremanaged throughthis system as well. Researchers who work independently onequipment have the flexibility and transparency of schedulingona shared calendar.There are variations of theworkflow thatinclude independent customer use of the facility as well astraining opportunities provided by the core.

Using our customized data collection and managementsystem, we could assess and evaluate the number ofresearchers that rely on the BRC to support their researchprograms and the productivity of investigators that utilizethe BRC.

FIGURE 5

Coauthorship Network 2014–2018. Social net-work analysis tools were used to generate thegraph shown.A) Network in 2014. B) Network in2019. Data were retrieved for a 5-y period fromMyNCBI, resulting in 96 publications that citedthe COBRE in Matrix Biology (P20GM109095)grant. Publication data were exported fromMyNCBI and imported into Gephi, and visuali-zation of the resulting network was generatedusing the Forced Atlas 2 algorithm. Nodesrepresent 377 authors, sized based on the nodedegree, or the number of publications for eachnode ranging from 1 to 15. Edges (lines connect-ing nodes) are weighted to indicate the strengthof connection; the thicker the edge, the moreconnections between those 2 nodes. Corefacility research staff scientists are representedby orange nodes, and young investigators whoreceive enhanced career mentoring are repre-sented by dark-blue nodes. All other authors areindicated by lighter shades of blue and includestudent authors and others (senior investigatorsand collaborators). The number of nodes in-creased from 2014 to 2019, as shown bycomparing the network structure in (A) (insetshows 2014 data) to (B) (2019 data).



BRC researchers and users of shared core facilitiesincreased from 2014 to 2018

Since 2014, we tracked the growth of users of shared corefacilities that support biomedical research. Figure 7 showsthe increase in the number of biomedical investigators aswell as the increase in investigators using the BRC each yearover the 5 years from 2014 to 2018.

Productivity of investigators that utilize the BRC

Researchers that utilize the BRC cite both the Idaho INBREand COBRE in Matrix Biology grant awards. Instilling theimportance of award citation and training in the correctmanner in which to cite the funding source allowed theprograms to meet rigorous compliance requirements.Additionally, we were able to conduct comprehensive

searches in NIH databases for journal articles, therebyenabling the collection of information in a systematicmanner that did not rely on individual self-reporting toobtain the majority of cited work. Figure 8 illustratesgrowth over time of cited peer-reviewed journals from 2014to 2018. The center’s administrative staff diligentlymonitors publications to ensure that NIH public accesspolicy is observed and that all investigators funded by theNIH submit publications to the National Library ofMedicine’s PubMed Central.10

Shared core facility management software

With the adoption of core facility management and billingsoftware, the BRCmore efficiently managed day-to-day facilityoperations as well as the processing of billing and invoicing.

FIGURE 6

BRC customer workflow. The shared researchcore facilities standardized the workflow re-quired for the customer interface. This workflowsupportedmore efficient management of day-to-day operations and billing/invoice processing.Investigators experience uniform and consistentsupport from initial contact to receipt ofpayment.



The core facility management software includes arobust reporting system that is available for the generation ofreports delineated by an individual investigator, the coremanagement team, or at an institutional level to address aspecific aspect of the core facility. Reports can span specificperiods of time, compare individual functions within a core,generate figures, and export detailed information in aComma Separated Values (CSV) file format. These data canbe filtered at a granular level by time frame, funding source,research lab, individual, institution, or equipment use. Thedata can also help to aid in the evaluation of the rate review

process, analyze equipment usage, increase the understand-ing of core staffing needs, and provide detailed informationabout individual users or lab activity. Reports, figures,spreadsheets, and data can be generated and used forprogram evaluation. The reports can also provide valuabledata to the institution, stakeholders, or potential grantees.Streamlining the process for core facility management andoffice administration allows for a more effective assessmentof the core facility. The office administrators benefit fromhaving a cohesive, transparent, and automated way ofinvoicing customers and processing payments.

FIGURE 7

Research activity from 2014 to 2018. Datamanagement tools were used to monitor thenumber of users of shared core facilities forbiomedical research. An increase in the numberof biomedical investigators (participants, shownin blue bars), as well as an increase in the numberof investigators using the BRC (red bars) and theBiomedical Research Vivarium (BRV) (greenbars) each year, was observed over the 6 y from2014 to 2018.

FIGURE 8

Publication information from 2014 to 2018. Datamanagement tools were used to monitor thenumber of peer-reviewed publications for eachyear. Increasing numbers of peer-reviewedjournal articles that cite the grant awardP20GM109095 from 2014 to 2018 are shownby the blue bars in the graph (left-hand y axis).Mean numbers of papers per author are shownby the gray bars for each year (left-hand y axis). Aline indicates an increasing trend in the numberof authors per year over the 2014–2018 timeframe (right-hand y axis).



Analysis of the data assists the core facility personnelas they make informed long- and short-term businessdecisions. Figure 9 demonstrates the departments sup-ported by the BRC core facilities (Fig. 9A) and the servicesthat were used (Fig. 9B). This information is vital to theplanning of sustainability of shared research core facilities.

DISCUSSION

Lessons learned

Over time, we used several custom data management toolsto collect individual annual reporting information forstakeholders. Regardless of which tool we adopted, therewas always an administrative burden associated with it,especially for faculty, who were required to report muchof the same information to many entities. We recognizedthat most of the BRC-affiliated faculty members wererequired to report similar information to both the BRC andIdaho INBRE programs annually. To reduce the adminis-trative burden that individual reporting requires, wecollaborated with the Idaho INBRE program to create aunique custom database system. This system allowed forindividuals to report all requested information from bothprogram rosters into 1 database. Therefore, individuals onlyreport the same information—such as funding, presenta-tions, and publications—once while having program-specific information tailored to individuals based onassigned roles. Some information from the previous yearcarries forward and autopopulates the current year’s report,further reducing the reporting burden experienced by theinvestigator. We expected that we would obtain morereliable individual reports while simultaneously reducingreporting fatigue. After the first round of reporting with thenew system, feedback indicated that this approach was wellreceived by faculty members, and we plan to continue

collecting individual reporting information in the shareddatabase. Collection and management of annual reportinginformation uses the following steps: create a yearly roster ofparticipants, designate appropriate role(s), assign relevantmodules and forms, and request that all individuals on theroster complete assigned modules and forms and updatetheir personal profile by a specific date. Self-reportedinformation is collected in a centralized database by a pre-established reporting year, and database information can beexported in variousways depending onneed. As noted earlier,the data are used for federal reporting requirements such asthe RPPR and the SIRS. Additional reports to keystakeholders such as the External Advisory Committee, BoiseState University Institutional Center Report, annual activityreports for faculty and staff, renewal of externally fundedprograms, and other reports can be generated as needed.

Future directions

The substantial burden of data management and assessmentrequirements for research centers and programs necessitatesearly consideration of a thoughtful and meaningful strategyfor data management. Adoption of these tools and bestpractices lessens the administrative burden and increaseseffectiveness in managing day-to-day center operationswhile also increasing transparency of core facility activityand our ability to report more comprehensively, makingassessment and evaluation more meaningful. We willcontinue to improve our application of software tools fordata collection, assessment, evaluation, and informationretention. Dissemination of outcomes will progress in anefficient, streamlined manner to all stakeholders as ourcenter continues to grow.

Like all share core facilities, the BRC must investigatechanging dynamics over time at multiple levels, including

FIGURE 9

Distribution of customers supported by sharedcore facilities. Data management tools wereemployed to monitor the customer base forshared core laboratories within the BRC and thetypes of services used by investigators. A)Disciplines supported by the shared core labo-ratories. B) Laboratory usewithin the shared corefacility. A) Distribution of users and their specificdiscipline or department is illustrated as a piechart, indicating that users are distributed among5 general areas, Chemistry and Biochemistry,Health Sciences, Engineering and Physics, Bi-ologic Sciences, and External Users, whichinclude investigators from nearby 2- and 4-ycolleges and universities, as well as researchinstitutes and industry. B) Services provided by laboratories within the shared core facility includeMass Spectrometry, Biostatistics, Histology, andMicroscopy, Bioinformatics, Cell Culture andRecombinant Protein Expression, andOther Analytic Services, which includes the characterization ofmolecular interactions and protein structure. The relative size of thewedge is indicative of the percentage of the total users for each type of service.



overall programs such as the COBRE in Matrix Biology,individual laboratories within the BRC, specific researchprojects, and personnel, including junior investigators, whoplay the role of mentored young scientists. Social networkanalysis tools provide a more in-depth understanding ofrelationships among faculty, staff, and student participantsas we include analysis of the overall size of the researchcommunity and the connectivity within the network. Weanticipate that network analysis will yield information aboutthe role of core scientists within research projects of juniorinvestigators and their career development, as well as theimpact that program grants like the COBRE in MatrixBiology and the Idaho INBREprogramhave on establishingan environment that promotes research growth andproductivity at an institution of emerging excellence.

CONCLUSIONS

The BRC at Boise State University is a research center thatincludes core facilities and administers multiple program-matic and individual researcher-initiated biomedical re-search grant awards. These awards have made a significantimpact on research and research infrastructure at BoiseState, an institution of emerging excellence. The IdahoINBRE program acted as the catalyst for the creation of theBRC through the establishment of infrastructure, funding,leadership, workforce development, and mentorship. As aresult, Boise State increased its NIH-funded programs andreceived its first COBRE grant in 2014. The growth inbiomedical research and shared core facility use requirednew approaches to information management for reporting,assessment, and evaluation.

Data management approaches to support our rapidgrowth had a beneficial effect on time management,reduction of administrative burden, and efficiency ofreporting. In addition, the approach supported ease of datacollection, evaluation, assessment, and project managementdecisions. Adoption of best practices has lessened theadministrative burden, more effectively managing day-to-day center operations while increasing transparency of corefacility activity.

The substantial burden of data management andassessment requirements for research centers and pro-grams necessitates early consideration of a thoughtful andmeaningful strategy for datamanagement. Flexibilitymust bemaintained to allow continuous improvement to approaches

used for assessment and evaluation inways that streamline thedissemination of outcomes to all stakeholders.

ACKNOWLEDGMENTSThe authors thank the National Institutes of Health USA IDeA INBREand COBRE programs (Grants P20GM103408, P20GM109095, andC06RR020533) (National Institute of General Medical Sciences).Additional funding was provided by the Biomolecular Research Center,at Boise State University RRID:SCR_019174, National ScienceFoundation USA (Grants 0619793 and 0923535); the M. J. MurdockCharitable Trust; USA the Idaho State Board of Education, and theDuane and Lori Stueckle Endowment USA. The authors declare noconflicts of interest.

REFERENCES1. NIH grants and funding, US National Institutes of Health

FAQs web site. Available at: https://grants.nih.gov/grants/policy/core_facilities_faqs.htm#3597. published by the USDepartment of Health and Human Services, National Institutesof Health, Bethesda, MD, USA. Accessed June 18, 2020.

2. NSF National Center for Science and Engineering Statistics(NCSES). Available at: https://ncsesdata.nsf.gov/ids/. publishedby the National Science Foundation, Alexandria, VA, USA.Accessed January 25, 2020.

3. Boise state university enrollment reports. Available at: https://www.boisestate.edu/ir/data-and-reporting/university-enrollment-reports/. Published by Boise State University, Boise, ID USA.Accessed June 10, 2020.

4. My NCBI Help. Available at: https://www.ncbi.nlm.nih.gov/books/NBK3842/#My. Published by the National Center forBiotechnology Information, US National Library of Medicine,Bethesda, MD, USA. Accessed June 18, 2020.

5. Bastian M, Heymann S, Jacomy M. Gephi: an open sourcesoftware for exploring and manipulating networks. Interna-tional AAAI Conference on Weblogs and Social Media. 2009.Available at: http://www.aaai.org/ocs/index.php/ICWSM/09/paper/view/154.

6. Jacomy M, Venturini T, Heymann S, Bastian M. ForceAtlas2,a continuous graph layout algorithm for handy networkvisualization designed for the Gephi software. PLoS One.2014;9:e98679.

7. Hockberger P, Meyn S, Nicklin C, Tabarini D, Turpen P,Auger J. Best practices for core facilities: handling externalcustomers. J Biomol Tech. 2013;24:87–97.

8. Turpen PB, Hockberger PE, Meyn SM, Nicklin C, Tabarini D,Auger JA. Metrics for success: strategies for enabling corefacility performance and assessing outcomes. J Biomol Tech.2016;27:25–39.

9. Marchand GC, Hilpert JC, Bragg KM, Cummings J. Network-based assessment of collaborative research in neuroscience.Alzheimers Dement (N Y). 2018;4:433–443..

10. NIH public access policy web site, omnibus appropriations Act,2009. Available at: https://publicaccess.nih.gov/policy.htm. Publishedby the US Department of Health and Human Services, NationalInstitutes of Health, Bethesda, MD, USA. Accessed July 25, 2019.



https://grants.nih.gov/grants/policy/core_facilities_faqs.htm#3597

https://grants.nih.gov/grants/policy/core_facilities_faqs.htm#3597

https://ncsesdata.nsf.gov/ids/

https://www.boisestate.edu/ir/data-and-reporting/university-enrollment-reports/



https://www.ncbi.nlm.nih.gov/books/NBK3842/#My

https://www.ncbi.nlm.nih.gov/books/NBK3842/#My

http://www.aaai.org/ocs/index.php/ICWSM/09/paper/view/154

http://www.aaai.org/ocs/index.php/ICWSM/09/paper/view/154

https://publicaccess.nih.gov/policy.htm

data management tools to measure the impact of core facilities

Documents