Bioinformatics and Graphical Models:

Computation, approximation, and their value

MSR:Nebojsa Jojic, Vladimir

Jojic, Chris Meek, David Heckerman

UW:Jim Mullins, Mark

Jensen, Jerry Learn

Overview• Computational cost of usual algorithms

– State of the art– Phylogeny + alignment– Phylogeny + sequence modeling– Approximations and their pitfalls

• Recombination– Analogy to other ML domains– Graphical model– Experiments and computational cost

• Value of the computation– Potential applications– Drug discovery cycle– Value of time and clinical success– Market size and growth

• Discussion

Rational vaccine design(Jim Mullins et al)

• Rational design– Analysis of sequences to form a model of

virus evolution (phylogenies, etc.)– Develop vaccines that target as much

variability as possible

• Traditional design– Trial and error– Educated guesses

State of the art sequence analysis programs

• Example: – Rational AIDS vaccine design– Analysis of the envelope gene from a single patient in one visit– 200 sequences with 600 base pairs each– Overnight to align– 1-2 hours to 2-3 days to build a tree, depending on how much

search you are willing to do– This does not include modeling the inter-sequence

dependencies, coupling alignment and tree search, and it ignores recombination

• The total length of the HIV genome is 10000 and the number of samples is practically only limited by cost

Computational cost of a slightly more detailed analysis

• Metropolis search over all trees on 400 sequences of the full genome (10k) would last around 2 years on one machine

• Exact search intractable!

Approximation

• Free energy as a bound on negative log-likelihood

• Computation and approximation of the free energy:– Iterative conditional modes– Mean-field method– Structured variational techniques– (Loopy) belief propagation– Sampling techniques

• How tight is the bound?• What does the looseness translate to?

An example of the approximation issues

An example of the approximation issues

An example of the approximation issues:Tightness of the bounds

Variational technique Exact EM algorithm

Recombination

• In HIV, the rate of recombination has recently been estimated to be ¼ of the rate of mutation!

• Combinatorial explosion in inference

Similar situations in other domains where graphical models work well

• Occlusion in video

• Source interaction in audio

• Composition of images

“Occlusion” in audio

Speaker1 Speaker2

M 1-M* *

+

||

Retrieved Speaker1

Retrieved Speaker2

Epitome of an image

Input image

A set of image patches

Epitome

Layers from a single photograph

em

es

S1 s2 M

x

Modeling alignment and recombination by learning a library of gene patterns

sji-1 sj

i sji+1

xji-1 xj

i xji+1

r1={ACTGTCAGT}r2={ACGATC}

copy pattern 1, position 2 (letter C); insertion mutation

s1={(1,1), (1,2), (1,3), (1,3), (1,3), (1,3), (1,4),(1,5),(1,6),(2,1),(2,2),(2,3)}c1 ={ 1 1 1 0 0 0 1 1 1 1 1 1 }x1={ A C T C A T G T A A C G }

s2 ={(2,1), (2,2), (2,3), (2,4), (2,5), (2,6), (1,4), (1,5), (1,6) }c2 ={ 1 1 1 1 1 1 1 1 1 }x2 ={ A C G A T C G T C }

cji-1 cj

i cji+1

s - pattern positionc = 1 : copy letter

(with possiblemutation)

c = 0 : draw letterfrom a distributionunrelated to the

patterns

Conditionals:

p(xji|s

ji=(1,2),c=1)=f(xj

i,r1(2))=f(xji,C)

p(xji|s,c=0)=g(xj

i)

Example:

Patterns:

Observations and a hidden variable assignment:

Experimental results

Value of computation(from Tufts Center)

Growth

• Human viruses– West Nile– SARS– Hepatitis C– Polio– …

• Animal viruses– FIV – Pig, chicken and cow viruses

• Most bacterial diseases• Parasitic diseases• The first sign of success of rational design might trigger

great increase in the number of diseases tackled

How can MS/MSR be involved?

• MS: Architecture, platform, tools– Storage, transmission, computation– E.g., parallelizable computation on a single machine;

pear-to-pear networks for parallel computation on multiple machines

• MSR:– Helping to speed up the scientific progress leading to

the new opportunities for growth– Advising MS on the research direction in the

community and the future requirements for the platform