Neuroimaging Data Neuroimaging Data Provenance Using the LONI Provenance Using the LONI Pipeline Workflow Pipeline Workflow EnvironmentEnvironmentAllan MacKenzie-Graham IPAW2008Allan MacKenzie-Graham IPAW2008

Arash PayanArash Payan

Ivo DinovIvo Dinov

John Van HornJohn Van Horn

Arthur W. TogaArthur W. Toga

22

Provenance in NeuroimagingProvenance in Neuroimaging

Tools used and data described Tools used and data described must be adequately described must be adequately described and documentedand documented•Determining data qualityDetermining data quality•InterpretationInterpretation•ReproducibilityReproducibility•ReusabilityReusability•InteroperabilityInteroperability

22

33

First Provenance ChallengeFirst Provenance Challenge

33(Moreau et. al, 2007)

44

Provenance SystemsProvenance Systems

44(Moreau et. al, 2007)

55

Goals of the LONI Goals of the LONI Provenance SystemProvenance System

DescriptionDescription•DataData•ProcessingProcessing

ReproducibilityReproducibility•Across platformsAcross platforms•Across compilationsAcross compilations•Across software versionsAcross software versions

Ease of useEase of use55

66

Neuroimaging Data Neuroimaging Data ProvenanceProvenance

Neuroimaging data provenanceNeuroimaging data provenance

ProjectProject

SubjectSubjectSpeciesSpecies

AgeAge

SexSex

AcquisitionAcquisitionScannerScanner

OrientationOrientation

WeightingWeighting

Field StrengthField Strength

TRTR

TETE

TITI

77

Provenance EditorProvenance Editor

77

88

LONI PipelineLONI Pipeline

88

99

LONI Pipeline ModuleLONI Pipeline Module

99

1010

Workflow ProvenanceWorkflow Provenance <connections><connections> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_0" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_0" /> <connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_0" /><connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_0" /> <connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_0" /><connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_0" /> <connection source="AlignLinear.LinearTransformation_0" sink="AffinetoRigid.Full-affineTransformation_0" /><connection source="AlignLinear.LinearTransformation_0" sink="AffinetoRigid.Full-affineTransformation_0" /> <connection source="AffinetoRigid.Rigid-bodyTransformation_0" sink="Reslice.Input.airLinearTransformation_0" <connection source="AffinetoRigid.Rigid-bodyTransformation_0" sink="Reslice.Input.airLinearTransformation_0" />/> <connection source="Reslice.OutputReslicedVolume_0" sink="AnalyzeImagetoMNC.InputAnalyzeImageFile_0" /><connection source="Reslice.OutputReslicedVolume_0" sink="AnalyzeImagetoMNC.InputAnalyzeImageFile_0" /> <connection source="AnalyzeImagetoMNC.OutputMinc_0" sink="MINCMath.InputMincs_0" /><connection source="AnalyzeImagetoMNC.OutputMinc_0" sink="MINCMath.InputMincs_0" /> <connection source="MINCMath.OutputMinc_0" sink="PrintAllLabels.InputMinc_0" /><connection source="MINCMath.OutputMinc_0" sink="PrintAllLabels.InputMinc_0" /> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_1" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_1" /> <connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_1" /><connection source="SkullStrippedmousedataNORMAL.Output_0" sink="AlignLinear.ResliceVolume_1" /> <connection source="AlignLinear.LinearTransformation_1" sink="Reslice.Input.airLinearTransformation_1" /><connection source="AlignLinear.LinearTransformation_1" sink="Reslice.Input.airLinearTransformation_1" /> <connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_2" /><connection source="skullstrippedatlas.Output_0" sink="AlignLinear.StandardVolume_2" /> <connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_2" /><connection source="skullstrippedatlasINJECTED.Output_0" sink="AlignLinear.ResliceVolume_2" /> <connection source="AlignLinear.LinearTransformation_2" sink="Reslice.Input.airLinearTransformation_2" /><connection source="AlignLinear.LinearTransformation_2" sink="Reslice.Input.airLinearTransformation_2" /> <connection source="TrimImage.Outputimage_0" sink="Binarize.Input_0" /><connection source="TrimImage.Outputimage_0" sink="Binarize.Input_0" /> <connection source="CompileFinalResults.PValuesImage_0" sink="BinaryMask.Input_0" /><connection source="CompileFinalResults.PValuesImage_0" sink="BinaryMask.Input_0" /> <connection source="Binarize.OutputBinaryVolume_0" sink="BinaryMask.BinaryMaskFile_0" /><connection source="Binarize.OutputBinaryVolume_0" sink="BinaryMask.BinaryMaskFile_0" /> <connection source="BinaryMask.Outputmaskedvolume_0" sink="MaskedP-Values.Input_0" /><connection source="BinaryMask.Outputmaskedvolume_0" sink="MaskedP-Values.Input_0" /> <connection source="KL_MI_register.displacementfieldimage_1" sink="InitializeStats.ControlsList_0" /><connection source="KL_MI_register.displacementfieldimage_1" sink="InitializeStats.ControlsList_0" /> <connection source="KL_MI_register.displacementfieldimage_0" sink="InitializeStats.SubjectsList_0" /><connection source="KL_MI_register.displacementfieldimage_0" sink="InitializeStats.SubjectsList_0" /> <connection source="ProgramsFiles.Output_0" sink="InitializeStats.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="InitializeStats.ExecutableDirectory_0" /> <connection source="Options.Output_0" sink="InitializeStats.Options_0" /><connection source="Options.Output_0" sink="InitializeStats.Options_0" /> <connection source="KL_MI_register.displacementfieldimage_0" sink="ComputeDeformationTensors.InputList_0" /><connection source="KL_MI_register.displacementfieldimage_0" sink="ComputeDeformationTensors.InputList_0" /> <connection source="KL_MI_register.displacementfieldimage_1" sink="ComputeDeformationTensors.InputList_0" /><connection source="KL_MI_register.displacementfieldimage_1" sink="ComputeDeformationTensors.InputList_0" /> <connection source="ProgramsFiles.Output_0" sink="ComputeDeformationTensors.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="ComputeDeformationTensors.ExecutableDirectory_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="ComputeDeformationTensors.OutputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="ComputeDeformationTensors.OutputDirectory_0" /> <connection source="ProgramsFiles.Output_0" sink="ComputeVoxelwiseStatistics.ExecutableDirectory_0" /><connection source="ProgramsFiles.Output_0" sink="ComputeVoxelwiseStatistics.ExecutableDirectory_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="ComputeVoxelwiseStatistics.InputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="ComputeVoxelwiseStatistics.InputDirectory_0" /> <connection source="Options.Output_0" sink="ComputeVoxelwiseStatistics.Options_0" /><connection source="Options.Output_0" sink="ComputeVoxelwiseStatistics.Options_0" /> <connection source="Slices.Output_0" sink="ComputeVoxelwiseStatistics.iSlice_0" /><connection source="Slices.Output_0" sink="ComputeVoxelwiseStatistics.iSlice_0" /> <connection source="ComputeDeformationTensors.Output_0" <connection source="ComputeDeformationTensors.Output_0" sink="ComputeVoxelwiseStatistics.SynchParameter_0" />sink="ComputeVoxelwiseStatistics.SynchParameter_0" /> <connection source="InitializeStats.OutputDirectory_0" sink="CompileFinalResults.OutputDirectory_0" /><connection source="InitializeStats.OutputDirectory_0" sink="CompileFinalResults.OutputDirectory_0" /> <connection source="ProgramsFiles.Output_0" sink="CompileFinalResults.Executionpath_0" /><connection source="ProgramsFiles.Output_0" sink="CompileFinalResults.Executionpath_0" /> <connection source="ComputeVoxelwiseStatistics.Output_0" sink="CompileFinalResults.DummySynch_0" /><connection source="ComputeVoxelwiseStatistics.Output_0" sink="CompileFinalResults.DummySynch_0" /> <connection source="CompileFinalResults.PValuesImage_0" sink="Results.Input_0" /><connection source="CompileFinalResults.PValuesImage_0" sink="Results.Input_0" /> <connection source="CompileFinalResults.TValuesImage_0" sink="Results.Input_0" /><connection source="CompileFinalResults.TValuesImage_0" sink="Results.Input_0" /> <connection source="Reslice.OutputReslicedVolume_2" sink="KL_MI_register.Targetimage_0" /><connection source="Reslice.OutputReslicedVolume_2" sink="KL_MI_register.Targetimage_0" /> <connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_0" /><connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_0" /> <connection source="Reslice.OutputReslicedVolume_1" sink="KL_MI_register.Targetimage_1" /><connection source="Reslice.OutputReslicedVolume_1" sink="KL_MI_register.Targetimage_1" /> <connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_1" /><connection source="skullstrippedatlas.Output_0" sink="KL_MI_register.Templateimage_1" /> <connection source="skullstrippedatlas.Output_0" sink="TrimImage.Inputimage_0" /><connection source="skullstrippedatlas.Output_0" sink="TrimImage.Inputimage_0" /> </connections></connections>

1010

1111

Executable ProvenanceExecutable Provenance

Executable provenanceExecutable provenance

EnvironmentEnvironment

OptionsOptions

Input filesInput files

Output filesOutput files

Binary provenanceBinary provenance

Binary configurationBinary configuration

Configuration optionsConfiguration options

System configurationSystem configuration

ArchitectureArchitecture

Operating systemOperating system

CompilerCompiler

LibrariesLibraries

Script provenanceScript provenance

ShellShell

ScriptScript

Binary provenanceBinary provenance

1212

Alignlinear ProvenanceAlignlinear Provenance<binary name="alignlinear" version="5.2.5" package="AIR" uri="<binary name="alignlinear" version="5.2.5" package="AIR" uri="http://www.loni.ucla.edu/Software/Software_Detail.jsp?software_id=8" " compilationTime="2006-04-21T11:58:00">compilationTime="2006-04-21T11:58:00">

<systemConfig><systemConfig>

<architecture name="x86" vendor="Intel" model="T2400" processorFlags="" /><architecture name="x86" vendor="Intel" model="T2400" processorFlags="" />

<OS name="MacOSX" version="10.5.3" distribution="" kernelName="Darwin" <OS name="MacOSX" version="10.5.3" distribution="" kernelName="Darwin" kernelVersion="9.3.0" />kernelVersion="9.3.0" />

<compiler name="GCC" version="4.0.1" flags="-o -Lsrc -lRPW -lm -g -02" updates="" /><compiler name="GCC" version="4.0.1" flags="-o -Lsrc -lRPW -lm -g -02" updates="" />

</systemConfig></systemConfig>

<configOptions><configOptions>

<config name="AIR_CONFIG_GROUPS" value="1" /><config name="AIR_CONFIG_GROUPS" value="1" />

<config name="HAVE_LIBM" value="1" /><config name="HAVE_LIBM" value="1" />

<config name="AIR_CONFIG_AUTO_BYTESWAP" value="1" /><config name="AIR_CONFIG_AUTO_BYTESWAP" value="1" />

<config name="AIR_CONFIG_REP16" value="1" /><config name="AIR_CONFIG_REP16" value="1" />

<config name="AIR_CONFIG_VERBOSITY" value="0" /><config name="AIR_CONFIG_VERBOSITY" value="0" />

<config name="AIR_CONFIG_DECOMPRESS_COMMAND" value="/usr/bin/gunzip -c" /><config name="AIR_CONFIG_DECOMPRESS_COMMAND" value="/usr/bin/gunzip -c" />

<config name="AIR_CONFIG_PIX_SIZE_ERR" value=".0001" /><config name="AIR_CONFIG_PIX_SIZE_ERR" value=".0001" />

<config name="AIR_CONFIG_THRESHOLD1" value="7000" /><config name="AIR_CONFIG_THRESHOLD1" value="7000" />

<config name="AIR_CONFIG_OUTBITS" value="16" /><config name="AIR_CONFIG_OUTBITS" value="16" />

<config name="AIR_CONFIG_REQ_PERMS" value="2" /><config name="AIR_CONFIG_REQ_PERMS" value="2" />

<config name="AIR_CONFIG_THRESHOLD2" value="7000" /><config name="AIR_CONFIG_THRESHOLD2" value="7000" />

<config name="AIR_CONFIG_PIPES" value="1" /><config name="AIR_CONFIG_PIPES" value="1" />

<config name="AIR_CONFIG_DECOMPRESS" value="1" /><config name="AIR_CONFIG_DECOMPRESS" value="1" />

<config name="AIR_CONFIG_PATHS" value="1" /><config name="AIR_CONFIG_PATHS" value="1" />

<config name="STDC_HEADERS" value="1" /><config name="STDC_HEADERS" value="1" />

</configOptions></configOptions>

</binary></binary> 1212

1313

ReproducibilityReproducibility

Across PlatformAcross Platform•ICA workflowICA workflow

Across compilationsAcross compilations•MDA workflowMDA workflow

1313

1414

Independent Components Independent Components AnalysisAnalysis

1515

Different Architectures Different Architectures Yield Different ResultsYield Different Results

1616

Minimum Deformation AtlasMinimum Deformation Atlas

1717

Different Compilation Different Compilation Options Yield Different Options Yield Different

ResultsResults

1818

Complex Neuroimaging Complex Neuroimaging WorkflowWorkflow

1818

1919

Future DirectionsFuture Directions

Community involvementCommunity involvement• provenance.loni.ucla.eduprovenance.loni.ucla.edu

Make LONI Pipeline aware of provenance Make LONI Pipeline aware of provenance filesfiles• Read in provenance fileRead in provenance file• Display executable provenanceDisplay executable provenance• Append provenance informationAppend provenance information• Write out provenance fileWrite out provenance file

Visualize provenance filesVisualize provenance files• Interface similar to LONI PipelineInterface similar to LONI Pipeline• Invoke LONI Pipeline to recreate file or Invoke LONI Pipeline to recreate file or modify processingmodify processing

Provenance DatabaseProvenance Database• Database of workflowsDatabase of workflows

2020

AcknowledgementsAcknowledgements

Arthur W. TogaArthur W. Toga

• Director, Laboratory of Neuro ImagingDirector, Laboratory of Neuro Imaging Arash PayanArash Payan

• Lead Developer, LONI PipelineLead Developer, LONI Pipeline Ivo D. DinovIvo D. Dinov

• Assistant Professor, Laboratory of Neuro Assistant Professor, Laboratory of Neuro ImagingImaging

John D. Van HornJohn D. Van Horn

• Assistant Professor, Laboratory of Neuro Assistant Professor, Laboratory of Neuro ImagingImaging