intermediate perl programming todd scheetz july 18, 2001
TRANSCRIPT
Intermediate Perl Programming
Todd Scheetz
July 18, 2001
Review of Perl Concepts
Data Typesscalararrayhash
Input/Outputopen(FILEHANDLE,”filename”);$line = <FILEHANDLE>;print “$line”;
Arithmetic Operations+, -, *, /, %&&, ||, !
Review of Perl Concepts
Control Structuresifif/elseif/elsif/else
foreach
for
while
Regular Expressions
General approach to the problem of pattern matching
RE’s are a compact method for representing a set of possible strings without explicitly specifying each alternative.
For this portion of the discussion, I will be using {} to represent the scope of a set.
{A}{A,AA}
{Ø} = empty set
Regular Expressions
In addition, the [] will be used to denote possible alternatives.
[AB] = {A,B}
With just these semantics available, we can begin building simple Regular Expressions.
[AB][AB] = {AA, AB, BA, BB}AA[AB]BB = {AAABB,AABBB}
Regular Expressions
Additional Regular Expression components* = 0 or more of the specified symbol+ = 1 or more of the specified symbol
A+ = {A, AA, AAA, … }A* = {Ø, A, AA, AAA, … }
AB* = {A, AB, ABB, ABBB, … }[AB]* = {Ø, A, B, AA, AB, BA, BB, AAA, … }
Regular Expressions
What if we want a specific number of iterations?
A{2,4} = {AA, AAA, AAAA}[AB]{1,2} = {A, B, AA, AB, BA, BB}
What if we want any character except one?[^A] = {B}
What if we want to allow any symbol?
. = {A, B}
.* = {Ø, A, B, AA, AB, BA, BB, … }
Regular Expressions
All of these operations are available in Perl
Several “shortcuts”
\d = {0, 2, 3, 4, 5, 6, 7, 8, 9}\w+\s\w+ = {…, Hello World, … }
Name Definition CodeWhitespace [space, tab,
new-line]\s
Wordcharacter
[a-zA-Z_0-9] \w
Digit [0-9] \d
Pattern Matching
Perl supports built-in operations for pattern matching, substitution, and character replacement
Pattern Matching
if($line =~ m/Rn.\d+/) {...
}
In Perl, RE’s can be a part of the string rather than the whole string.
^ - beginning of string$ - end of string
Pattern Matching
Back references…
if($line =~ m/(Rn.\d+)/) {$UniGene_label = $1;
}
Regular Expressions
$file = “my_fasta_file”;open(IN, $file);$line_count = 0;while($line = <IN>) {
if($line =~ m/^\>/) {$line_count++;
}}print “There are $line_count FASTA sequences in $file.\n”;
Pattern Matching
UniGene data file
ID Bt.1TITLE Cow casein kinase II alpha …EXPRESS ;placentaPROTSIM ORG=Caenorhabditis elegans; …PROTSIM ORG=Mus musculus; PROTGI=…SCOUNT 2SEQUENCE ACC=M93665; NID=g162776; …SEQUENCE ACC=BF043619; NID=…//ID Bt.2TITLE Bos taurus cyclin-dependent …...
Pattern Matching
Let’s write a small Perl program to determine how many clusters there are in the Bos taurus UniGene file.
Pattern Matching
Now we’ll build a Perl program that can write an HTML file containing some basic links based on the Bos taurus UniGene clustering.
Important:
http://www.ncbi.nlm.nih.gov:80/entrez/query.fcgi?cmd=Retrieve&db=Nucleotide&list_uids=GID_HERE&dopt=GenBank
Substitution
Pattern matching is useful for counting or indexing items, but to modify the data, substitution is required.
Substitution searches a string for a PATTERN and, if found, replaces it with REPLACEMENT.
$line =~ s/PATTERN/REPLACEMENT/;
Returns a value equal to the number of times the pattern was found and replaced.
$result = $line =~ s/PATTERN/REPLACEMENT/;
Substitution
Substitution can take several different options.specified after the final slash
The most useful areg - global (can substitute at more than one location)i - case insensitive matching
$string = “One fish, Two fish, Red fish, Blue fish.”;$string =~ s/fish/dog/g;print “$string\n”;
One dog, Two dog, Red dog, Blue dog.
Substitution
Example: Removing leading and trailing white-space
$line =~ s/^\s*(.*?)\s*$/$1/;
a *? performs a minimal match…it will stop at the first point that the remainder of the expression can be matched.
$line =~ s/^\s*(.*)\s*$/$1/;this statement will not remove trailing white-space, instead the white space is retained by the .*
Character Replacement
A similar operation to substitution is character replacement.
$line =~ tr/a-z/A-Z/;
$count_CG = $line =~ tr/CG/CG/;
$line =~ tr/ACGT/TGCA/;
$line =~ s/A/T/g;$line =~ s/C/G/g;$line =~ s/G/C/g;$line =~ s/T/A/g;
Character Replacement
while($line = <IN>) {$count_CG = $line =~ tr/CG/CG/;$count_AT = $line =~ tr/AT/AT/;
}$total = $count_CG + $count_AT;$percent_CG = 100 * ($count_CG/$total);
print “The sequence was $percent_CG CG-rich.\n”;
Subroutines
One of the most important aspects of programming is dealing with complexity. A program that is written in one large section is generally more difficult to debug. Thus a major strategy in program development is modularization.
Break the program up into smaller portions that can each be developed and tested independently.
Makes the program more readable, and easier to maintain and modify.
Subroutines
EXAMPLE:Reading in sequences from UniGene.all.seq file
Multiple FASTA sequences in a single file, each annotated with the UniGene cluster they belong to.
GOAL: Make an output file consisting only of the longest sequence from each cluster.
Subroutines
ISSUES:1. Want to design and implement a usable program2. Use subroutines where useful to reduce complexity.3. Minimize the memory requirements.
(human UniGene seqs > 2 GB)