the challenge for all of us informaon checking updates3 asian and paciﬁc rim research integrity...

1

AsianandPacificRimResearchIntegrity(APRI)NetworkMee<ng|HongKong|20thFebruary2017

Upda%ngfooterToupdatethefooter:•  Clickintothetextboxontheslide

masterpageandupdatetheinforma<on

Checkingupdates•  Thetextupdatesmaynotflow

throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides

Thechallengeforallofus

PloSMedicine2005doi:10.1371/journal.pmed.0020124

DavidDobbs,TheAtlan<c,18thSEPT2015

“Fi>y-threepapersweredeemed'landmark'studies.…Nevertheless,scienKficfindingswereconfirmedinonly6(11%)cases.”Nature483,531–533(2012)doi:10.1038/483531a

2






ThechallengeforjournalsRetrac<onsareevermorefrequentandscalewithimpactfactor

Source: Ferric C. Fang, F. C. & Casadevall, A. Retracted Science and the Retraction Index.

Infect. Immun. 79, 3855-3859 (2011).

3






ThechallengeforjournalsCorrec<onsareontherise,too

0

20

40

60

80

100

120

140

Publishe

dpe

ryear

Correc%ons&retrac%onsinNaturejournals

Corrigendum

Retrac<on

4





throughtoalltheslidelayoutsassomehavebeenplacedmanually.ThereforeyoumayneedtocheckandmanuallyupdateotherslidesWearenottalkingaboutresultsthatwere:

•  Falsified—misconductisaproblembutit’ss<llintheminority•  Wrong—legi<mateobserva<onsbutsubsequentworkdisprovesthe

hypothesis—that’showscienceismeanttowork!Wearetalkingaboutresultsthatare•  Poorlydescribedpreven%ngverifica%on—independentexperimenter

cannotobservethesameresultsundersimilarcondi<ons.•  Overstated—failuretoconsideralterna<veexplana<ons•  Misrepresented—dataclaimedtobemorerobustthantheyactuallyare,

CHERRYPICKING!•  Sloppy—failuretoaccountfor(orevenconsider)sourcesoferror,pooruse

ofsta<s<cs,poorcontrols

Thebiggestproblemisn’tfraudAlthoughthelionshareofretrac<onsarisefrommisconduct,poorreproducibilitydoesnot

5

So,whatarethesolu%ons?

6






Partofthesolu%oniseduca%onWemustn’ttakeitforgrantedthateveryoneknowswhatgoodresearchis

Casadevall,A.andFang,F.C.RigorousScience:aHow-ToGuide.mBio7,e01902-16(2016);doi:10.1128/mBio.01902-16

APentateuchforScien%ficRigour

7






Upda%ngimage

Toupdatethebackgroundimage,clickonthepictureplaceholdericon

1.  WindowsExplorerorFinderwillopen

2.  Findyourrequiredimageandclick‘Insert’

3.  Whencoverimageinserted,right-clickinthegreyareadirectlybelowthistextboxandselect‘Resetslide’

NOTE:

Ifanimageisalreadyinplaceandyouwanttochangeit:

1.  Selecttheimage,andhityourdeletekey

2.  Followsteps1-4above

Encouragecommunitystandards

•  Rapidimprovementintheefficiencyofperovskitesolarcellsleadtoasurgeinclaimsofeverhigherperformance.

•  Extraordinaryclaimsbasedonpoorexperimentaldesignleadtodemandsfromrefereesthatallclaimsbesubmiiedtothirdpartyverifica<onbeforepeerreview.

•  Independentverifica<onbeforepeerreview,enforcedbyjournals,isnowstandardprac<ce.

Casestudy:perovskitesolarcells

NaturePhotonics8,665(2014);doi:10.1038/nphoton.2014.212

8






Don’tciteanythingyouhaven’tread!Greenberg,S.A.Howcita<ondistor<onscreateunfoundedauthority:analysisofacita<onnetwork.BMJ339,b2680(2009);doi:10.1136/bmj.b2680

“Fig.5—ConversionofhypothesistofactthroughcitaKonalone.CitaKonsonstatementthataccumulaKonofβamyloid“precedes”otherabnormaliKesininclusionbodymyosiKsmuscle.StatementasfactissupportedthroughcitaKontopapersthatonlystateitashypothesis(forexample,references5to80,91to80,134to74)orsomeKmessupportedbycitaKontopapersthatcontainnostatementsaddressingit(forexample,references91to72,251to75;deadendcitaKons).ThisphenomenonmightbecalledcitaKontransmutaKon.”

©2009 by British Medical Journal Publishing Group

9






BeTermethodssec%ons,longermethodssec%ons

Eliminatelengthlimitsformethodssec<onsextendedmethodsonlineEncouragedtosubmit

MethodstoProtocolExchange

overepor<ngstandardschecklist

Increaserigourofsta<s<csSta<s<caladvisorsta<s<calconsultantsEduca<on–NatureMethodscolumn‘PointsofSignificance’

Datasharingreaffirmdatasharingpolicy–condiKonofpublica<ontowardsdatacita<onsimplementedsourcedataScien<ficData–publica<onofdatadescriptorstofacilitatereuseandcredit

Bi-direc<onallinksbetweenProtocolsandAr<cles

Neteffect:•  Lengthincreasedbyaverage25%upto50%atsomejournals•  Methodsfullyintegratedwithar<cle

online–htmlandPDFNomore“supplementarymethods”

10






BeTermethodssec%ons,longermethodssec%ons

CellPresship://www.cell.com/star-methods

11





throughtoalltheslidelayoutsassomehavebeenplacedmanually.ThereforeyoumayneedtocheckandmanuallyupdateotherslidesIregularlytellyoungresearchersthatonewaytodecidewhether

somethingitethicalornotistoaskthemselves,

“IfthisbehaviourorpracKceweretobecomepublicknowledge,wouldIhaveanyreasontofeeluncomfortableorashamedbyit?”

Andiftheansweris‘yes’oreven‘maybe’,don’tdoit!

Themainpointhereisthatpoorprac<cecanonlythriveinthedark.Andgoodprac%cecomesbydoingeverythingintheopen.

Thean%sep%cpowerofsunlight

12






Moretransparentrepor%nghip://www.nature.com/authors/policies/checklist.pdf

13






Encouragemoresharingofdata

14






SharingplaWormsneedn’tjustbeaboutsharing(oratall)Ifyourdataisn’tsavedinasecure,persistentrepositoryitmayaswellnotexist

A cloud based research data management system where you can:

Manage your research outputs privately and securely

Make your research outputs citable & discoverable

15





throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtocheckandmanuallyupdateotherslides•  Atleast28researchfundersgloballyhave

policiesormandatesrequiringarchivingofdataasacondi<onofgrants,including:•  Na<onalScienceFounda<on(NSF)•  Na<onalIns<tutesofHealth(NIH)•  WellcomeTrust•  BillandMelindaGatesFounda<on

•  Someoftheserequiredatatobelinkedtopublica<onsincluding:•  ResearchCouncilsUK(aspartofopenaccesspolicy)•  EngineeringandPhysicalSciencesResearchCouncil(EPSRC)

Source:Hahnel,M:Globalfunderswhorequiredataarchivingasacondi<onofgrants.Figshare.hips://dx.doi.org/10.6084/m9.figshare.1281141.v1(2015)

Whysharedata?Becausefundersincreasinglyrequireit

16






16

Whysharedata?

“Publishingresearchwithoutdataisadver5sing,notscience!”

—CameronNeylon,GrahamSteel,andothers.

Becausethat’swhatscienceshouldbeabout

17






18






Upda%ngimage

Toupdatethebackgroundimage,clickonthepictureplaceholdericon

1.  WindowsExplorerorFinderwillopen

2.  Findyourrequiredimageandclick‘Insert’

3.  Whencoverimageinserted,right-clickinthegreyareadirectlybelowthistextboxandselect‘Resetslide’

NOTE:

Ifanimageisalreadyinplaceandyouwanttochangeit:

1.  Selecttheimage,andhityourdeletekey

2.  Followsteps1-4above

OpendatainChina

“TheChinesegovernmenthasinvestednearly200million

yuan(US$30million)since2006onaplacormforsharing

corrosiondatafrom30field-tesKngstaKonscoveringstandardmaterialsin

environments(air,soilandwater)typicalofdifferentparts

ofthecountry.”—Li,X.G.etal.Nature527,441–442(2015).

Casestudy:China’sopencorrosiondataplaqorm

19






19

Butistheworldreadytoopenuptheirdata?

HasopendataarrivedattheBri5shMedicalJournal(BMJ)?Anobserva%onalstudyObjec%veToquan<fydatasharingtrendsanddatasharingpolicycomplianceattheBriKshMedicalJournal(BMJ)byanalysingtherateofdatasharingprac<ces,andinves<gateartudesandexaminebarrierstowardsdatasharing.Results…only7/157researchar<clessharedtheirdatasets,4.5%…ConclusionsDespitetheBMJ'sstrongdatasharingpolicy,sharingratesarelow.Possibleexplana<onsforlowdatasharingratescouldbe:thewordingoftheBMJdatasharingpolicy…requests[ending]upinresearchersspamfolders;andthatresearchersarenotrewardedforsharingtheirdata.—AnisaRowhani-Farid&AdrianGBarneiBMJOpen6,e011784(2016).

Orevenshareitonrequest?

20





throughtoalltheslidelayoutsassomehavebeenplacedmanually.ThereforeyoumayneedtocheckandmanuallyupdateotherslidesInMarch2016weranapilotrequiringauthors

ofpapersatfiveNaturejournalstodeclarehowreaderscanobtainthe‘minimaldataset’neededtointerpret,replicateandbuildonthefindingstheyreport.Bytheendof2016werolledthisouttoalljournalsintheNaturefamily.Aswellasbeinganinherentgood,wehopeto:•  Learnmoreabouthowdifferentdisciplines

sharedatadifferently.•  Learnhowwecanhelpthemmeetthe

challengesthey.•  Raiseawarenessandencouragemoreresearchintotheopen.

EncouragemoresharingofdataNaturenowrequiresauthorstotellreadershowtheycangetholdoftheirdata

Nature536,138(20016).Tolearnmoregotohip://go.nature.com/2bf4vqn

21






Andnotjustdata—materialsarejustasimportant!

“AcondiKonofpublicaKoninaNaturejournalisthatauthorsarerequiredtomakeuniquematerialspromptlyavailabletootherswithoutunduequalificaKons.”hTp://www.nature.com/authors/policies/availability.html

“A>erpublicaKon,alldataandmaterialsnecessarytounderstand,assess,andextendtheconclusionsofthemanuscriptmustbeavailabletoanyreaderofScience.”hTp://www.sciencemag.org/authors/science-editorial-policies?_ga=1.200322094.1663650950.1484202254

“OneofthetermsandcondiKonsofpublishingwithCellPressisthatauthorsbewillingtodistributeanymaterialsandprotocolsusedinthepublishedexperimentstoqualifiedresearchersfortheirownuse.”hTp://www.cell.com/cell/authors

22






WhatisPreregistra5on?“PreregistraKonseparateshypothesistesKngfromhypothesisgeneraKngresearch.Bothareimportant.However,thesamedatacannotbeusedtogenerateandtestahypothesis,whicho>enhappensunintenKonally.WithpreregistraKon,confirmatoryanalysesareplannedinadvanceinordertoretainthevalidityoftheirstaKsKcalinferences,andexploratoryanalysesarereportedasposthocinvesKgaKonsthatmightinspireconfirmatorytestsinfuturestudies.”—fromhips://cos.io/prereg/

Andfinally… pre-registra%on

23






23 Upda%ngfooterToupdatethefooter:•  Clickintothetextboxontheslide


Checkingupdates•  Thetextupdateswillnotflow

throughtoalltheslidelayoutsassomehavebeenplacedmanually.Thereforeyoumayneedtomanuallyupdateothertextboxessuchasthedividerslidesetc

Thankyou!谢谢！

EdGerstner印格致 —[email protected]

Furtherreading•  ‘TurningPoint’whitepaperonChineseresearchlandscape(2015)—

hip://www.nature.com/press_releases/turning_point.pdf•  JaneQiu,SafeguardingresearchintegrityinChina.NatlSci.Rev.2,

122–125(2015);doi:10.1093/nsr/nwv002•  Munafò,M.R.etal.Amanifestoforreproduciblescience.Nature

HumanBehaviour1,0021(2017);doi:10.1038/s41562-016-002

“Good Research Practice”

February 20, 2017 Jun Fudano, Ph.D.

(Tokyo Institute of Technology)

Three senses of “Goodness”

2

Traditional Theories of Ethics

3

Agent → Conduct → Consequences

Virtue Ethics

Deontological Ethics

Utilitarian Ethics

Line Drawing/Demarcation Problem

4

“How good is good enough for what purpose, for whom?”

“How good is good enough?”

The same line for every discipline?

Well-being Model 1

The first and fundamental cannon of engineering ethics

“Hold paramount the safety, health,

and welfare of the public”

What is welfare/well-being/happiness?

Two aspects of Ethics

Aspire to do Ought not to do

Aspirational Ethics Preventive Ethics

Positive Negative

External orientation Internal orientation

Tend to be uplifting Tend to be intimidating

7

Contact

Jun Fudano, Ph.D. Professor Institute for Liberal Arts Tokyo Institute of Technology W-9 63 2-12-1 Ookayama Meguro-ku, Tokyo 152-8552, JAPAN Tel.：+81-3-5734-2370 Fax：+81-3-5734-2844 E-mail: [email protected]

mailto:[email protected]

APRI, Prof. Mai Har Sham, The University of Hong Kong

Quantity, Quality and Quality

• Publish or Perish - Quantity • Impact factors and citations - Quality • Research impact - Excellent Quality • Promotions and Ranking

Academic Excellence: Quality Research is Responsible Research

APRI, Prof. Mai Har Sham, The University of Hong Kong

Research Data Management

Researcher: • Data management plan • Reliable data collection, usage and

interpretation • Practice good data recording • Data sharing

Institution: • Research data management policy • Data storage and archive

the challenge for all of us informaon checking updates3 asian and paciﬁc rim research integrity...

Documents