Neuroimaging Data Neuroimaging Data Provenance Using the LONI Provenance Using the LONI Pipeline Workflow Pipeline Workflow EnvironmentEnvironmentAllan MacKenzie-Graham IPAW2008Allan MacKenzie-Graham IPAW2008
Arash PayanArash Payan
Ivo DinovIvo Dinov
John Van HornJohn Van Horn
Arthur W. TogaArthur W. Toga
22
Provenance in NeuroimagingProvenance in Neuroimaging
Tools used and data described Tools used and data described must be adequately described must be adequately described and documentedand documented•Determining data qualityDetermining data quality•InterpretationInterpretation•ReproducibilityReproducibility•ReusabilityReusability•InteroperabilityInteroperability
22
33
First Provenance ChallengeFirst Provenance Challenge
33(Moreau et. al, 2007)
44
Provenance SystemsProvenance Systems
44(Moreau et. al, 2007)
55
Goals of the LONI Goals of the LONI Provenance SystemProvenance System
DescriptionDescription•DataData•ProcessingProcessing
ReproducibilityReproducibility•Across platformsAcross platforms•Across compilationsAcross compilations•Across software versionsAcross software versions
Ease of useEase of use55
66
Neuroimaging Data Neuroimaging Data ProvenanceProvenance
Neuroimaging data provenanceNeuroimaging data provenance
ProjectProject
SubjectSubjectSpeciesSpecies
AgeAge
SexSex
AcquisitionAcquisitionScannerScanner
OrientationOrientation
WeightingWeighting
Field StrengthField Strength
TRTR
TETE
TITI
77
Provenance EditorProvenance Editor
77
88
LONI PipelineLONI Pipeline
88
99
LONI Pipeline ModuleLONI Pipeline Module
99
1010
Workflow ProvenanceWorkflow Provenance <connections><connections> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_0" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_0" /> <connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_0" /><connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_0" /> <connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_0" /><connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_0" /> <connection source="AlignLinear.LinearTransformation_0" sink="AffinetoRigid.Full-affineTransformation_0" /><connection source="AlignLinear.LinearTransformation_0" sink="AffinetoRigid.Full-affineTransformation_0" /> <connection source="AffinetoRigid.Rigid-bodyTransformation_0" sink="Reslice.Input.airLinearTransformation_0" <connection source="AffinetoRigid.Rigid-bodyTransformation_0" sink="Reslice.Input.airLinearTransformation_0" />/> <connection source="Reslice.OutputReslicedVolume_0" sink="AnalyzeImagetoMNC.InputAnalyzeImageFile_0" /><connection source="Reslice.OutputReslicedVolume_0" sink="AnalyzeImagetoMNC.InputAnalyzeImageFile_0" /> <connection source="AnalyzeImagetoMNC.OutputMinc_0" sink="MINCMath.InputMincs_0" /><connection source="AnalyzeImagetoMNC.OutputMinc_0" sink="MINCMath.InputMincs_0" /> <connection source="MINCMath.OutputMinc_0" sink="PrintAllLabels.InputMinc_0" /><connection source="MINCMath.OutputMinc_0" sink="PrintAllLabels.InputMinc_0" /> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_1" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_1" /> <connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_1" /><connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_1" /> <connection source="AlignLinear.LinearTransformation_1" sink="Reslice.Input.airLinearTransformation_1" /><connection source="AlignLinear.LinearTransformation_1" sink="Reslice.Input.airLinearTransformation_1" /> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_2" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_2" /> <connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_2" /><connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_2" /> <connection source="AlignLinear.LinearTransformation_2" sink="Reslice.Input.airLinearTransformation_2" /><connection source="AlignLinear.LinearTransformation_2" sink="Reslice.Input.airLinearTransformation_2" /> <connection source="TrimImage.Outputimage_0" sink="Binarize.Input_0" /><connection source="TrimImage.Outputimage_0" sink="Binarize.Input_0" /> <connection source="CompileFinalResults.PValuesImage_0" sink="BinaryMask.Input_0" /><connection source="CompileFinalResults.PValuesImage_0" sink="BinaryMask.Input_0" /> <connection source="Binarize.OutputBinaryVolume_0" sink="BinaryMask.BinaryMaskFile_0" /><connection source="Binarize.OutputBinaryVolume_0" sink="BinaryMask.BinaryMaskFile_0" /> <connection source="BinaryMask.Outputmaskedvolume_0" sink="MaskedP-Values.Input_0" /><connection source="BinaryMask.Outputmaskedvolume_0" sink="MaskedP-Values.Input_0" /> <connection source="KL_MI_register.displacementfieldimage_1" sink="InitializeStats.ControlsList_0" /><connection source="KL_MI_register.displacementfieldimage_1" sink="InitializeStats.ControlsList_0" /> <connection source="KL_MI_register.displacementfieldimage_0" sink="InitializeStats.SubjectsList_0" /><connection source="KL_MI_register.displacementfieldimage_0" sink="InitializeStats.SubjectsList_0" /> <connection source="ProgramsFiles.Output_0" sink="InitializeStats.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="InitializeStats.ExecutableDirectory_0" /> <connection source="Options.Output_0" sink="InitializeStats.Options_0" /><connection source="Options.Output_0" sink="InitializeStats.Options_0" /> <connection source="KL_MI_register.displacementfieldimage_0" sink="ComputeDeformationTensors.InputList_0" /><connection source="KL_MI_register.displacementfieldimage_0" sink="ComputeDeformationTensors.InputList_0" /> <connection source="KL_MI_register.displacementfieldimage_1" sink="ComputeDeformationTensors.InputList_0" /><connection source="KL_MI_register.displacementfieldimage_1" sink="ComputeDeformationTensors.InputList_0" /> <connection source="ProgramsFiles.Output_0" sink="ComputeDeformationTensors.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="ComputeDeformationTensors.ExecutableDirectory_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="ComputeDeformationTensors.OutputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="ComputeDeformationTensors.OutputDirectory_0" /> <connection source="ProgramsFiles.Output_0" sink="ComputeVoxelwiseStatistics.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="ComputeVoxelwiseStatistics.ExecutableDirectory_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="ComputeVoxelwiseStatistics.InputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="ComputeVoxelwiseStatistics.InputDirectory_0" /> <connection source="Options.Output_0" sink="ComputeVoxelwiseStatistics.Options_0" /><connection source="Options.Output_0" sink="ComputeVoxelwiseStatistics.Options_0" /> <connection source="Slices.Output_0" sink="ComputeVoxelwiseStatistics.iSlice_0" /><connection source="Slices.Output_0" sink="ComputeVoxelwiseStatistics.iSlice_0" /> <connection source="ComputeDeformationTensors.Output_0" <connection source="ComputeDeformationTensors.Output_0" sink="ComputeVoxelwiseStatistics.SynchParameter_0" />sink="ComputeVoxelwiseStatistics.SynchParameter_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="CompileFinalResults.OutputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="CompileFinalResults.OutputDirectory_0" /> <connection source="ProgramsFiles.Output_0" sink="CompileFinalResults.Executionpath_0" /><connection source="ProgramsFiles.Output_0" sink="CompileFinalResults.Executionpath_0" /> <connection source="ComputeVoxelwiseStatistics.Output_0" sink="CompileFinalResults.DummySynch_0" /><connection source="ComputeVoxelwiseStatistics.Output_0" sink="CompileFinalResults.DummySynch_0" /> <connection source="CompileFinalResults.PValuesImage_0" sink="Results.Input_0" /><connection source="CompileFinalResults.PValuesImage_0" sink="Results.Input_0" /> <connection source="CompileFinalResults.TValuesImage_0" sink="Results.Input_0" /><connection source="CompileFinalResults.TValuesImage_0" sink="Results.Input_0" /> <connection source="Reslice.OutputReslicedVolume_2" sink="KL_MI_register.Targetimage_0" /><connection source="Reslice.OutputReslicedVolume_2" sink="KL_MI_register.Targetimage_0" /> <connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_0" /><connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_0" /> <connection source="Reslice.OutputReslicedVolume_1" sink="KL_MI_register.Targetimage_1" /><connection source="Reslice.OutputReslicedVolume_1" sink="KL_MI_register.Targetimage_1" /> <connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_1" /><connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_1" /> <connection source="skullstrippedatlas.Output_0" sink="TrimImage.Inputimage_0" /><connection source="skullstrippedatlas.Output_0" sink="TrimImage.Inputimage_0" /> </connections></connections>
1010
1111
Executable ProvenanceExecutable Provenance
Executable provenanceExecutable provenance
EnvironmentEnvironment
OptionsOptions
Input filesInput files
Output filesOutput files
Binary provenanceBinary provenance
Binary configurationBinary configuration
Configuration optionsConfiguration options
System configurationSystem configuration
ArchitectureArchitecture
Operating systemOperating system
CompilerCompiler
LibrariesLibraries
Script provenanceScript provenance
ShellShell
ScriptScript
Binary provenanceBinary provenance
1212
Alignlinear ProvenanceAlignlinear Provenance<binary name="alignlinear" version="5.2.5" package="AIR" uri="<binary name="alignlinear" version="5.2.5" package="AIR" uri="http://www.loni.ucla.edu/Software/Software_Detail.jsp?software_id=8" " compilationTime="2006-04-21T11:58:00">compilationTime="2006-04-21T11:58:00">
<systemConfig><systemConfig>
<architecture name="x86" vendor="Intel" model="T2400" processorFlags="" /><architecture name="x86" vendor="Intel" model="T2400" processorFlags="" />
<OS name="MacOSX" version="10.5.3" distribution="" kernelName="Darwin" <OS name="MacOSX" version="10.5.3" distribution="" kernelName="Darwin" kernelVersion="9.3.0" />kernelVersion="9.3.0" />
<compiler name="GCC" version="4.0.1" flags="-o -Lsrc -lRPW -lm -g -02" updates="" /><compiler name="GCC" version="4.0.1" flags="-o -Lsrc -lRPW -lm -g -02" updates="" />
</systemConfig></systemConfig>
<configOptions><configOptions>
<config name="AIR_CONFIG_GROUPS" value="1" /><config name="AIR_CONFIG_GROUPS" value="1" />
<config name="HAVE_LIBM" value="1" /><config name="HAVE_LIBM" value="1" />
<config name="AIR_CONFIG_AUTO_BYTESWAP" value="1" /><config name="AIR_CONFIG_AUTO_BYTESWAP" value="1" />
<config name="AIR_CONFIG_REP16" value="1" /><config name="AIR_CONFIG_REP16" value="1" />
<config name="AIR_CONFIG_VERBOSITY" value="0" /><config name="AIR_CONFIG_VERBOSITY" value="0" />
<config name="AIR_CONFIG_DECOMPRESS_COMMAND" value="/usr/bin/gunzip -c" /><config name="AIR_CONFIG_DECOMPRESS_COMMAND" value="/usr/bin/gunzip -c" />
<config name="AIR_CONFIG_PIX_SIZE_ERR" value=".0001" /><config name="AIR_CONFIG_PIX_SIZE_ERR" value=".0001" />
<config name="AIR_CONFIG_THRESHOLD1" value="7000" /><config name="AIR_CONFIG_THRESHOLD1" value="7000" />
<config name="AIR_CONFIG_OUTBITS" value="16" /><config name="AIR_CONFIG_OUTBITS" value="16" />
<config name="AIR_CONFIG_REQ_PERMS" value="2" /><config name="AIR_CONFIG_REQ_PERMS" value="2" />
<config name="AIR_CONFIG_THRESHOLD2" value="7000" /><config name="AIR_CONFIG_THRESHOLD2" value="7000" />
<config name="AIR_CONFIG_PIPES" value="1" /><config name="AIR_CONFIG_PIPES" value="1" />
<config name="AIR_CONFIG_DECOMPRESS" value="1" /><config name="AIR_CONFIG_DECOMPRESS" value="1" />
<config name="AIR_CONFIG_PATHS" value="1" /><config name="AIR_CONFIG_PATHS" value="1" />
<config name="STDC_HEADERS" value="1" /><config name="STDC_HEADERS" value="1" />
</configOptions></configOptions>
</binary></binary> 1212
1313
ReproducibilityReproducibility
Across PlatformAcross Platform•ICA workflowICA workflow
Across compilationsAcross compilations•MDA workflowMDA workflow
1313
1414
Independent Components Independent Components AnalysisAnalysis
1515
Different Architectures Different Architectures Yield Different ResultsYield Different Results
1616
Minimum Deformation AtlasMinimum Deformation Atlas
1717
Different Compilation Different Compilation Options Yield Different Options Yield Different
ResultsResults
1818
Complex Neuroimaging Complex Neuroimaging WorkflowWorkflow
1818
1919
Future DirectionsFuture Directions
Community involvementCommunity involvement• provenance.loni.ucla.eduprovenance.loni.ucla.edu
Make LONI Pipeline aware of provenance Make LONI Pipeline aware of provenance filesfiles• Read in provenance fileRead in provenance file• Display executable provenanceDisplay executable provenance• Append provenance informationAppend provenance information• Write out provenance fileWrite out provenance file
Visualize provenance filesVisualize provenance files• Interface similar to LONI PipelineInterface similar to LONI Pipeline• Invoke LONI Pipeline to recreate file or Invoke LONI Pipeline to recreate file or modify processingmodify processing
Provenance DatabaseProvenance Database• Database of workflowsDatabase of workflows
2020
AcknowledgementsAcknowledgements
Arthur W. TogaArthur W. Toga
• Director, Laboratory of Neuro ImagingDirector, Laboratory of Neuro Imaging Arash PayanArash Payan
• Lead Developer, LONI PipelineLead Developer, LONI Pipeline Ivo D. DinovIvo D. Dinov
• Assistant Professor, Laboratory of Neuro Assistant Professor, Laboratory of Neuro ImagingImaging
John D. Van HornJohn D. Van Horn
• Assistant Professor, Laboratory of Neuro Assistant Professor, Laboratory of Neuro ImagingImaging