1

Deep Project Management with XNATVanderbilt University

2013-08-20

Benjamin Yvernault,Bennett Landman, Brian Boyd,

2

RequirementsAt this point, you should know how to :

– Register and Log in to XNAT– See your projects and join projects– Create project / sharing data– Manage access permissions– Browse data and view image previews – Search for projects, subjects, and data– Download imaging data, processed, and stored meta-data– Delete scans / sessions / projects– Use the Pre-Archive– Upload scans through DicomBrowser– Quality Assurance

3

Set up your computerXnatsetup (basic) will on your computer : Install Python package to run the command tools :

Instructions on the wiki Install lxml and httplib2 using easy_install or pip Install Pyxnat from a git version :

git clone git://github.com/sburns/PyCap.git PyCapcd PyCappython setup.py install

Set up your .bashrc or .profile :Set up the XNAT variables :

export XNAT_USER = bob (your xnat username)export XNAT_PASS = bob (your xnat password)export XNAT_HOST = http://xnat.vanderbilt.edu/xnat

Add the tools to PATH :export PATH=/PathToMasimatlab/trunk/xnatspiders/Xnat_tools:$PATH

Remember!

4

After this short course, you will be able to:

• Get our code from Masimatlab• Set up your computer (python packages, …)• Download data from XNAT using command line tools• Upload data using XNAT command tools• Dcm4chee to send big DICOM database to XNAT• Use XNAT command line tools• Project mirroring XNAT->XNAT / full

5

Download from XNAT

You can download using what you learn from the previous course : Download file Manage file

You can use the Xnatdownload commandline tool developed by our lab that downloads resources from scans or/and assessors.

6

XnatdownloadPython commandline tool developed by our lab

Present in the masimatlab svn repository:trunk/xnatspiders/Xnat_tools/Xnatdownload

Different options to precise your download

Use: Xnatdownload -h to see the options

7

XNAT view

8

Upload Data to XNAT

DICOM can be uploaded using: DicomBrowser (previous course)dcm4che (next slides)

Xnatupload is a commandline tool developed by our lab to upload any resources to XNAT using three methods.

9

Dcm4che (1)Open Source clinical Image and object managementProgramed in JavaTutorial on how to upload : http://xnat.vanderbilt.edu in

Advanced resources.Join Masimatlab on NITRC to obtain the code for

Dicom4che Dcmsnd present in the

masimatlab/trunk/xnatspiders/dcm4che_tools/dcm4che-2.0.27/bin/ E.G: dcmsnd VandyXNAT@xnat.vanderbilt.edu:8104 /FullPathTemporatoryFolder/*

10

Remember!

XNAT reading DICOM

XNAT provides a DicomServer:Read information from the header Add the Dicom to the PreArchive with the right

project/subject/sessionXNAT uses tags from DICOM to identify where it

should be:Usually tag (0010,4000) Patient CommentsNeed to be:

“Project:ProjectIDXNAT;Subject:SubjectIDXNAT;Session:SessionIDXNAT”

E.G: Project:VUSTP;Subject:VUSTP4;Session:VUSTP4aCheck this link to know the tags : http://mindhive.mit.edu/node/1352

11

Dcm4che (2)

1. Use an IDE for java developers like Eclipse2. Create a new project and configure the java build path

to add all the externals JARs and Class folder from masimatlab

3. Change the SNARLdcmEdit.java file that will change in all of your DICOM the tag of identification for XNAT (0010,4000)

– Follow the steps in http://xnat.vanderbilt.edu “Sending DICOM to Vanderbilt XNAT” link

4. Send the DICOMs to XNAT using dcmsnd– dcmsnd VandyXNAT@xnat.vanderbilt.edy:8104 /path/*

12

Xnatupload

Python commandline tool developed by our labPresent in the masimatlab svn repository:

trunk/xnatspiders/Xnat_tools/XnatuploadIt can upload any resources (not only DICOM)Three methods in the option:

Files in the same folder, using specific namesFiles spread into a specific tree hierarchyUpload only on existing

subjects/sessions/scans/assessors

13

First Method of UploadThis method uploads only resources for scans.Options : --up1Conditions:

all the file are in the same folderName the files like :

Project-x-subject-x-experiment-x-scan-x-scantype-x-series_description-x-resource.extensionE.G: VUSTP-x-VUSTP5-x-VUSTP5a-x-301-x-T1-x-3W_T1-x-NIFTI.nii.gz

The project needs to exist first on XNATYou need to select a type of session (MR/CT/PET)

Call the script :Xnatupload -p VUSTP -d /tmp/vustp_data/ --up1 --sess MR

14

Second Method of UploadThis method uploads only resources for scans.Options : --up2Conditions:

One Folder with a specific structureThe structure needs to be :

Folder/subject/experiment/scan-x-scantype-x-series_description/resource/file.extensionE.G: MainFolder/VUSTP5/VUSTP5a/301-x-T1-x-3W_T1/NIFTI/file.nii.gz

The project needs to exist first on XNATYou need to select a type of session (MR/CT/PET)

Call the script :Xnatupload -p VUSTP -d /tmp/vustp_data/ --up2 --sess MR

15

Third Method of Upload This method uploads only (does not create any

subject/experiment/scan or assessor on XNAT) Options : -o/--onlyUpload Conditions:

One Folder with all the files in it (multi project)The files needs to have a name like :

Project-x-subject-x-experiment-x-scan-x-resource.extensionE.G: VUSTP-x-VUSTP5-x-VUSTP5a-x-301-x-NIFTI.nii.gz

Use the option -T to specify the kind of data (scans or assessors)The location needs to exist on XNAT

Call the script :Xnatupload -p VUSTP -d /tmp/vustp_data/ --only -T 1

16

XNAT view

17

XNAT command tools Xnatdownload and Xnatupload (already presented) Xnatreport

Creates report on project(s) from XNAT Xnatquery

Queries and displays XNAT at any level Xnatcheck

Checks if a resource exists on specific assessors or scansChecks the status of assessors

XnatSwitchProcessStatus Xnatsetup

18

XnatreportUse Xnatreport -h to see the options

E.G: Xnatreport –p VUSTP

19

Xnatquery

Help: Xnatquery –h

Xnatquery will display on the terminal all the objects at the level you asked.

Option --info for a report more focus on the processes. E.G: Xnatquery –p VUSTP --info

20

Xnatcheck Xnatcheck allows you to check the existence of a resource on a

project Help: Xnatcheck -h

21

XNAT view

22

XnatSwitchProcessStatus The status on an assessor gives information about the job

status of this process. Our system use this status to launch jobs.

XnatSwitchProcessStatus allows you to change the status of several assessors at a time.

Help: XnatSwitchProcessStatus -h

23

XNAT view

24

XnatsetupCommand tool to Setup your computer Different setups:

Basic :Install python packages: httplib2 / lxml / pyxnatSet up your .bashrc or .profile

Advance:Set up path for spiders

Redcap:Install python packages: PyCap and PandasSet up your .bashrc or .profile

Some specific spiders set up to check if you have everything to run them.

25

xnat_copy• Mirrors a Project from one XNAT instance to another• Copies files & attributes for all Subjects/Sessions/Scans

xnat_copy [options] project-cf recheck existing files-ca recheck existing attributes

26

Interactive Assignment• Create a NITRC account and join the Masimatlab team to get

the last version of masimatlab svn repository.• Download the code from masimatlab on NITRC• Set up your computer (manually or with Xnatsetup):• Download all the NIFTI from VUSTP using Xnatdownload• Upload the VUSTP data into a new project using Xnatupload

and one of the method• Check if a fMRIQA process in VUSTP doesn’t have a PDF• Xnatquery --info on VUSTP

27

Next time

• Create a project on REDCap and ask for API• What is a spider:

- source code - Which spiders already exist?- Run spiders from your computer

• ACCRE: run spiders (status / failed / ….)