www..uni-rostock.de ulf schmitz, statistical methods for aiding alignment1 bioinformatics...

Ulf Schmitz, Statistical methods for aiding alignment 1

www. .uni-rostock.de

BioinformaticsBioinformaticsStatistical methods for pattern searchingStatistical methods for pattern searching

Ulf [email protected]

Bioinformatics and Systems Biology Groupwww.sbi.informatik.uni-rostock.de



Outline

1. Expectation Maximization Algorithm

2. Markov Models

3. Hidden Markov models



Expectation Maximization Algorithm

• an algorithm for locating similar sequence patterns in a set of sequences

• suspected parts are then aligned• an expected scoring matrix representing the distribution of sequence

characters in each column of the alignment will be generated• the pattern is matched to each sequence and the scoring matrix values

are then updated to maximize the alignment of the matrix to the sequences

• this procedure is repeated until there is no further improvement




Seq1:Seq2:Seq3:Seq4:

.

.

.

Seq10:

100 nucleotides long

seq1: … … … … … … TCAGAATGCAGCATAG … … … … … … … … … … … …seq2: … … … … … … CGCATAGAGCATAGAC … … … … … … … … … … … … seq3: … … … … … … ACAGACAAAAAAATAC … … … … … … … … … … … …seq4: … … … … … … CATAGCAGATACAGCA … … … … … … … … … … … …

preliminary local alignment of the sequences

Columns not in motif provide background frequencies

ACATAGACAGTATAGAGAATCAGAATGCAGCATAGCAGCACATAGAGCAGCATAGTAGACCATAGACCGATACGCGCATAGAGCATAGACACGATAGCATAGCATAGCATTACAGATCAGCAAGAGCCGACAGACAAAAAAATACGAGCAAAACGAGCATTATCGTAGGGGACACAGATACAGACATAGCAGATACAGCATAGACATAGACAGATAGCAG

.

.

.

seq10:

provides initial estimates of frequencies of nucleotides in each motif column





Background Site column 1 Site column 2 …

G 0.27 0.4 0.1 …

C 0.25 0.4 0.1 …

A 0.25 0.2 0.1 …

T 0.23 0.2 0.7 …

1.00 1.0 1.0

The first column gives the background frequencies in the flanking sequence.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

.

.

.

seq10:

4x17 matrix




Each sequence is scanned for all possible locations for the site to find the most probable location of the site

•the EM algorithm consists of two steps which are repeated consecutively•step 1 - expectation step –

–column-by-column composition of the found site is used to estimate the probability of finding the site at any position in each of the sequences–these probabilities are used to provide expected base or amino acid distribution for each column of the site

Sequence 1 xxxxooooooooooooooooooxxxx|||| ||||||||||||||||||oxxxxooooooooooooooooo ||||| |||||||||||||||||ooxxxxoooooooooooooooo |||||| ||||||||||||||||

A

B

C

Use estimates of residue frequencies for each column in the motif to calculate probability of motif in this position and multiply by…

Backgroung frequencies in the remaining positions




seq1: ACATAGACAGTATAGAGAATCAGAATGCAGCATAGCAGCACATAGAGCAGCATAG16151413121110987654321

1.01.01.00

…0.70.20.23T

…0.10.20.25A

…0.10.40.25C

…0.10.40.27G

…Site column 2Site column 1Background

(for a in pos. 1) (for C in pos. 2)

for the next 14 positions in site

for G in flanking pos. 1 for A in flanking pos.2

for the next 82 flanking positions

Table of column frequencies of each base

100191631seq,1site PP25.027.0PP1.02.0P




seq1: ACATAGACAGTATAGAGAATCAGAATGCAGCATAGCAGCACATAGAGCAGCATAG

ACATAGACAGTATAGAGAATCAGAATGCAGCATAGCAGCACATAGAGCAGCATAG



16151413121110987654321

16151413121110987654321

16151413121110987654321

16151413121110987654321

.

.

.

seq10:

The probability of this best location in seq1, say at site k, is the ratio of the site probability at k divided by the sum of all other site probabilities.

P(site k in seq1) = Psite k, seq1 / Psite 1, seq1 + Psite 2, seq1 + … + Psite 85, seq1

The probability of the site location in each sequence is then calculated in this manner.




•step 2 – maximization step ––the new counts of bases or amino acids for each position in the site found in step 1 are substituted for the previous set

seq1: ACATAGACAGTATAGAGAATCAGAATGCAGCATAGCAGCACATAGAGCAGCATAG

16151413121110987654321

(e.g.) P(site 1 in seq1) = 0.01 and P(site 2 in seq1) = 0.02


.

.

.

seq10:

+0.01 A C A T A G A C A G T A T A G A

+0.02 C A T A G A C A G T A T AG A G




• This procedure is repeated for all other site locations and all other sequences.

• A new version of the table of residue frequencies can be build.

• The expectation and maximation steps are repeated until the estimates of the base frequencies do not change.

MEME (Multiple EM for Motif Elication) :• is a tool for performing msa‘s by the em method • see http://www.sdsc.edu/MEME/meme/website/meme.html




• the EM algorithm consists of two steps which are repeated consecutively

• step 1 - expectation step –– column-by-column composition of the found site is used to

estimate the probability of finding the site at any position in each of the sequences

– these probabilities are used to provide expected base or amino acid distribution for each column of the site

• step 2 – maximization step –– the new counts of bases or amino acids for each position in the

site found in step 1 are substituted for the previous set• step 1 is then repeated until the algorithm converges on

a solution



Markov chain models

– a Markov chain model is defined by:• a set of states• some states emit symbols• other states (e.g. the begin state) are silent• a set of transitions with associated probabilities• the transitions emanating from a given state define a

distribution over the possible next states



Markov chain models

• given some sequence x of length L, we can ask how probable the sequence is, based on our model

• for any probabilistic model of sequences, we can write this probability as:

• key property of a (1st order) Markov chain: the probability of each Xi

depends only on Xi-1

)XPr()X,,XXPr()X,,XXPr(

)X,,X,XPr()xPr(

112L1L11LL

11LL

L

2i1ii1

1122L1L1LL

)XXPr()XPr(

)XPr()XXPr()XXPr()XXPr()xPr(



Markov chain models1st order Markov chain



Markov chain models

Example Application

• CpG islands– CG-dinucleotides are rarer in eukaryotic genomes than expected

given the independent probabilities of C, G– but the regions upstream of genes are richer in CG dinucleotides

than elsewhere – CpG islands– useful evidence for finding genes

• Could predict CpG islands with Markov chains– one to represent CpG islands– one to represent the rest of the genome



Markov chain models



Markov chain models

Selecting the Order of a Markov Chain Model

• Higher order models remember more “history”• Additional history can have predictive value• Example:

– predict the next word in this sentence fragment “…finish __” (up, it, first, last, …?)

– now predict it given more history

• “Fast guys finish __”



Hidden Markov Models (HMM)

Hidden State

• We will distinguish between the observed parts of a problem and the hidden parts• In the Markov models we have considered previously, it is clear which state accounts for each part of the observed sequence • In another model, there are multiple states that could account for each part of the observed sequence

– this is the hidden part of the problem– states are decoupled from sequence symbols



Hidden Markov models

Markov model: Move from state to state according to probability distribution of each state and emit states visited:

Hidden Markov model: Move from stateto state in the same way, but emit a symbolaccording to probability distribution instead:



Hidden Markov Model

• Red square, match state • green diamond, insert state • blue circle, delete state

• Arrows indicate the probability of transition from one state to the next.



Hidden Markov Model

N * F L SN * F L SN K Y L TQ * W - T

A. Sequence alignment

B. Hidden Markov model for sequence alignment

BEG M1 M2 M3 M4 END

I0 I1 I2 I3 I4

D1 D2 D3 D4

Probability of sequence: N K Y L T

BEG -> M -> I1 -> M2 -> M3 -> M4 -> END0.33 * 0.05 * 0.33 * 0.05 * 0.33 * 0.05 * 0.33 * 0.05 * 0.33 * 0.05 * 0.5 = 6.1 * 10-10



Hidden Markov Models

Three Important Questions

• How likely is a given sequence?

• What is the most probable “path” for generating a given sequence?

• How can we learn the HMM parameters given a set of sequences?



Hidden Markov Models

HMM-based homology searching• formal probabilistic basis and consistent theory behind gap

and insertion scores • HMMs good for profile searches, bad for alignment (due to

parametrisation of the models)• HMMs are slow

HMMER - http://hmmer.wustl.edu/

Tools:

SAM - http://cse.ucsc.edu/research/comp/bio/sam.html



Outlook

• Machine learning• Clustering



Sequence Alignment

Thanx for your attention!!!

www..uni-rostock.de ulf schmitz, statistical methods for aiding alignment1 bioinformatics...

Documents

ulf schmitz

motif column slide

expectation step column

site sequence

column composition

em algorithm

aiding alignment2

4x17 matrix slide