submitting sequenced bacs robert buels sgn. purpose and scope provide well-organized archive of...

SubmittingSequenced BACs

Robert BuelsSGN

Purpose and Scope

Provide well-organized archive of sequencing data for the tomato genome sequencing project, including:

finished BAC sequences

chromatograms

assembly files

annotations

BAC Submission Guidelines

A comprehensive BAC submission format exists (see handout, pp. 4-5).

If we all follow this format, the data will be much easier to work with.

Problems

Not all submissions follow the submission guidelines.

Irregularities in naming, directory structure, submission file contents.

FTP site does not separate finished and unfinished sequences.

Result: data is harder to work with than it should be!

Problems

To address these problems:

We improved the directory structure of the FTP site.

Corrected existing submissions to follow submission guidelines.

We made a new submission system that checks submission guidelines.

New FTP Structure

tomato_genome/bacs/ chr01/ chr02/ ... chr12/ bacs.seq finished_bacs.seq README validate_submission.pl

New FTP Structure

tomato_genome/bacs/ chr01/ chr02/ ... chr12/ bacs.seq finished_bacs.seq README validate_submission.pl

all seqs

New FTP Structure

tomato_genome/bacs/ chr01/ chr02/ ... chr12/ bacs.seq finished_bacs.seq README validate_submission.pl

all seqs

finished seqs

New FTP Structure

tomato_genome/bacs/ chr01/ chr02/ ... chr12/ bacs.seq finished_bacs.seq README validate_submission.pl

all seqs

finished seqs

validation script

New FTP Structure

tomato_genome/bacs/ chr01/ unfinished/ finished/ C01HBa0088L02.all.xml C01HBa0088L02.seq C01HBa0088L02.tar.gz C01HBa0216G16.all.xml C01HBa0216G16.seq C01HBa0216G16.tar.gz ...

New FTP Structure

tomato_genome/bacs/ chr01/ unfinished/ finished/ C01HBa0088L02.all.xml C01HBa0088L02.seq C01HBa0088L02.tar.gz C01HBa0216G16.all.xml C01HBa0216G16.seq C01HBa0216G16.tar.gz ...

separate finished and unfinished

New FTP Structure

tomato_genome/bacs/ chr01/ unfinished/ finished/ C01HBa0088L02.all.xml C01HBa0088L02.seq C01HBa0088L02.tar.gz C01HBa0216G16.all.xml C01HBa0216G16.seq C01HBa0216G16.tar.gz ...

separate finished and unfinished

submission

New FTP Structure

tomato_genome/bacs/ chr01/ unfinished/ finished/ C01HBa0088L02.all.xml C01HBa0088L02.seq C01HBa0088L02.tar.gz C01HBa0216G16.all.xml C01HBa0216G16.seq C01HBa0216G16.tar.gz ...

separate finished and unfinished

submission

sequence

New FTP Structure

tomato_genome/bacs/ chr01/ unfinished/ finished/ C01HBa0088L02.all.xml C01HBa0088L02.seq C01HBa0088L02.tar.gz C01HBa0216G16.all.xml C01HBa0216G16.seq C01HBa0216G16.tar.gz ...

separate finished and unfinished

annotations

submission

sequence

New Submission System

Developed to automatically check submission formatting, analyze submissions, and publish them on the FTP site.

Also improves turn-around time since it is mostly automatic.

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz C01HBa0001A01/ chromat_dir/ edit_dir/ phd_dir/ seq_dir/ C01HBa0001A01.seq >C01HBa0001A01 ATGCCTACGAT...

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz C01HBa0001A01/ chromat_dir/ edit_dir/ phd_dir/ seq_dir/ C01HBa0001A01.seq >C01HBa0001A01 ATGCCTACGAT...

submission is tar.gz file, named for BAC

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz C01HBa0001A01/ chromat_dir/ edit_dir/ phd_dir/ seq_dir/ C01HBa0001A01.seq >C01HBa0001A01 ATGCCTACGAT...

submission is tar.gz file, named for BAC

self-named subdirectory

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz C01HBa0001A01/ chromat_dir/ edit_dir/ phd_dir/ seq_dir/ C01HBa0001A01.seq >C01HBa0001A01 ATGCCTACGAT...

submission is tar.gz file, named for BAC

self-named subdirectory

self-named sequence file

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz C01HBa0001A01/ chromat_dir/ edit_dir/ phd_dir/ seq_dir/ C01HBa0001A01.seq >C01HBa0001A01 ATGCCTACGAT...

submission is tar.gz file, named for BAC

self-named subdirectory

self-named sequence file

self-namedsequence

validate_submission.pl

Checks the formatting of your submission.

Avoids having to retry uploads to correct formatting.

~$ ./validate_submission.pl C01HBa0001A01.tar.gzC01HBa0088L02.tar.gz passed.

~$

On your machine...

Submission Process

1.Use validate_submission.pl to verify that your submission is properly formatted.

2.Upload with scp to upload.sgn.cornell.edu, using your login and password.

3.When finished, the submission cron job will notice it and process it.

Submission Process

4. If properly formatted, you will see it on the FTP site within 3 - 4 hours.

5. Otherwise, you will receive an email asking you to correct the format and retry the submission.

Questions?

Suggestions for improvement of FTP site?

Suggestions for submission process?