jeff shrager [email protected] - perception, parcellation, … · 2015-11-19 · thanks to...

1.  In which we discover that discovery requires a new kind of cognitive architecture.

2.  In which we discover that enabling scientists to use new kinds

of cognitive architectures requires a new kind of computational architecture.

3.  In which we discover that even simple cognition requires a new kind of cognitive architecture.

Perception, Parcellation, and Programming “Signposts to a Bridge Between Connectionist and Symbolic Systems”

We need to be able to program with Self-Organizing Probabilistic Partially-Overlapping Abstraction Hierarchies

Rationality (i.e., reason/symbolic computing) is the hallmark of intelligence. (The parable of the google car.)

One approach is to synchronize via “grounded” representations.

The challenge is how to compute flexibly with symbolic representations.

The problem with this is that abstractions can overlap on the ground.

Thanks to dozens of colleagues, esp. Jeff Elhai, Tim Finin, Arthur Grossman, Mark Johnson, David Klahr, Pat Langley, JP Massar, Al Newell, Andrew Pohorille, Bob Siegler, Herb Simon, Marty Tenenbaum, Mike Travers, and numerous students. Thanks too for support from CIWDPB, CMU, IBM, NASA, NIH, NSF, Stanford, and Xerox PARC.





Perception, Parcellation, and Programming “Signposts to a Bridge Between Connectionist and Symbolic Systems”

Partially-Overlapping Abstraction Hierarchies

P=.031

P=.031 P=.031 P=.031

P=.031

P=.031

P=.031

P=.031 P=.031

Probabilistic Partially-Overlapping Abstraction Hierarchies

The Goal:

Shrager, J. & Finin, T. (1982). An expert systems that volunteers advice. AAAI 82, Pittsburgh, PA. pp. 339-340.

Novices make errors, or know they need help, and are therefore helped by help systems or error reports. Experts don’t need help (or use help and references). Intermediate users don’t make errors, but can stagnate because they don’t know what they don’t know!

The First Computer Wizard

Modeling An Expert Consultant:

Traces of Users’ Action

Library of “Bad plans”

(Bad) Plan Recognizer

Advice Generator Advice

Expert Analysis


“KeyHole Goal Recognition”

The Problem:

People are really good at figuring out how fairly complex things work without either training or reading the manual.

How do people pull this off, and can we figure out how to design devices that are easy to learn instructionlessly?

Shrager, J. & Klahr, D. (1986). Instructionless learning about a complex device: The paradigm and observations. IJMMS, 25.

Instructionless Learning

The program: RIGHT 1, FORWARD 2

What the subject expected:

1

2


What the BigTrak did:

1

2

1: 1 min right turn

2


What the subject expected….


Her interpretation: “Oh, I see, it’s like doing the resultant or something….”




1

2

1: 1 min right turn

2


1. Observations are interpreted by current model. 2. If there are discrepancies, one new view is selected. 3. The model is updated by mixing in the view. 4. Coercion is carried out as needed in accord with new concepts introduced by the view. 5. The updated model may demand various actions and observations to be completed.

View Application: The Process:


A Model

Current “Mental Model”

Library of “Views”

Instructionless Experimenter

“SimTrak”

Experiment Planner

View Application

J Shrager (1987) Theory change via view application in instructionless learning. Machine Learning, 2: 247-276.


Conceptually coherent, possibly complex, units of partially abstract knowledge that can be incrementally “mixed into” an existing model (by “View Application”), updating the model in accord with the principles represented in the view.

Some Views in BigTrak Learning: Toy Deterministic Electrical device Non-deterministic Electronic … Vehicle Instruction following Clock face Vector addition Memory (remembering) and clearing the memory …

Update the theory in terms of Views. View Application


What the subject expected:

1

2

A Problem: View Application


1

2

1: 1 min right turn

2





The original SYMBOLIC model has NO RELEVANT CONTENT through which to recognize nor on to which to hang the new view!

1

2

1: 1 min right turn

2

View Application: The Problem:


Selfridge’s (1959) Pandemonium Paradigm

Finders Generators

Representation Specific Computation

Commonsense Perception

“Perception” is an active process that binds (synchronizes) cognition with the sensory-motor systems, and thus the real world.

This binding enables cognition the flexibility to discover and reason about novel features.


J Shrager (1990c) Commonsense perception and the psychology of theory formation. In Shrager & Langley (Eds.) Computational models of scientific discovery and theory formation. San Mateo, CA: Morgan Kaufmann.


Give biologists a program and they’ll make you program more and more.

The BioLingua Vision: Biologist as Programmer

But give them an integrated knowledge and programming environment, and teach them to use it, and you’ll change their lives!

(Not to mention saving yourself a lot of boring programming!)

BioBike

•  Integrate Genomic and Data Analysis Tools

•  Unify All Important Knowledge Bases

•  Integrate the Most Advanced Analytical Tools

•  Provide a Universal Programming Methodology

•  Provide Community Extensibility


BioBike

P=.031

P=.031 P=.031 P=.031

P=.031

P=.031

P=.031

P=.031 P=.031

BioBike

Cyanobacteria are 3.5 billion years old. They created the oxygen atmosphere. Algae and cyanobacteria create most of of the current oxygen atmosphere, and fix most of the greenhouse CO2. Algae form the base of the marine ecosystem.

Where they go; The planet follows!

Important Algae BioBike

•  Gene present in Prochlorococcus MED4 MED4 is naturally adapted to grow in high light.

•  Ortholog absent in Prochlorococcus MIT9313 MIT9313 is naturally adapted to grow in low light

•  Ortholog present in Synechocystis PCC 6803 In order to make contact with annotation and microarray data

•  Synechocystis PCC 6803 ortholog responds to high light Gene turns on by factor > 2 in response to high light

Look for:

How do cells control light response? I.e., What genes are related to the adaptation to high light?

BioBike

For each gene in ProMed4, Find all the gene’s functional orthologs, Find those from Syny6803, When there are not any Pro9313 genes in the orthologs, and there are any the 6803 orthologs and the expression ratio for the 6803 orthologs in the experimental data is >= 2, collect the 6803 orthologs in a list, called light-specific-genes.


BioBike

•  Integrate Genomic and Data Analysis Tools

•  Unify All Important Knowledge Bases

•  Integrate the Most Advanced Analytical Tools

•  Provide a Universal Programming Methodology

•  Provide Community Extensibility


BioBike

Integrated K/DB Layer

Unified Basic Concepts Layer

Computed Concepts Layer

BioLisp Scripting Layer

KEGG BioCyc

SMD Locally mirror important K/DBs

Remote Access Other K/DBs

Structures provided for important biological concepts: e.g., reactions, molecules, enzymes, experiments, expression-levels, etc.

An ever-expanding library of computations that produce complex, virtual, biological concepts, such as pathways, complexes, regulons, etc.

A simple programming language to be used by biologists to answer specific questions regarding the integration of their data with the concepts below.

Standard analytic tools, plus discovery tools that combine knowledge and data under user control.

GO

BioLingua Computational Biology Workbench BioBike

P=.031

P=.031 P=.031 P=.031

P=.031

P=.031

P=.031

P=.031 P=.031

BioBike

BioBike

COG

Integrated Knowledge Base Limited All Genes x All Genes x All Organisms Homology Table

Microarray DB

Organism Models

#$trichodesmium_erythraeum #$anabaena_variabilis_atcc29413 #$synechocystis_pcc6803 #$prochlorococcus_marinus_ccmp1375 #$anabaena_pcc7120 #$nostoc_punctiforme_atcc29133 o o o

BioBike

Inspectable Objects

Frame-Based Object Model

BioBike

BioBike

Count the genes of an organism. BioBike

Find the genes involved in glycolysis, and their reactions. BioBike

How many of those are transporters? BioBike

BioBike Biologist as Programmer The Biologists’ Reaction:

COG

Integrated Knowledge Base Limited All Genes x All Genes x All Organisms Homology Table

Microarray DB

Organism Models

#$trichodesmium_erythraeum #$anabaena_variabilis_atcc29413 #$synechocystis_pcc6803 #$prochlorococcus_marinus_ccmp1375 #$anabaena_pcc7120 #$nostoc_punctiforme_atcc29133 o o o

BioBike

An “Intelligent” Bioinformatic Reasoner BioDeducta

For each gene in ProMed4, Find all the gene’s functional orthologs, Find those from Syny6803, When there are not any Pro9313 genes in the orthologs, and there are any the 6803 orthologs and the expression ratio for the 6803 orthologs in the experimental data is >= 2, collect the 6803 orthologs in a list, called light-specific-genes.


BioDeducta

Language for Expressing Conjectures, and Platform for Analysis A. First Order Logic (FOL) representation B. Subject Domain Theory C. Biological Process (and entities) Ontology D. Visual query language. Goal Query

Subject Domain Theory

BioDeducta

Goal Query:

Subject Domain Theory:

New Terms

BioDeducta

Result: ?gene: #$PMED4.PMM0817 ?organism2: #$prochlorococcus_marinus_mit9313 ?experiment: HIHARA ?organism3: #$synechocystis_pcc6803 ?gene3: #$S6803.ssr2595 I.e., A low-light organism that has no ortholog to ?gene is prochlorococcus marinus pcc. 9313. Experiments were performed by Hihara on the organism synechocystis pcc 6803, and a high regulation ratio was discovered in those experiments on gene S6803.ssr2595, which is an ortholog of PMM0817. The annotation for PMM0817 reads: “possible high-light inducible protein”. (Matches the results from: Bhaya, Dufresne, Vaulot, and Grossman: Analysis of the hli gene family in marine and freshwater cyanobacteria. FEMS Letters, 2002, 205(2). PMM0817 is called hli17 in this paper.)

Goal Query: BioDeducta

Result: ?gene: #$PMED4.PMM0817 ?organism2: #$prochlorococcus_marinus_mit9313 ?experiment: HIHARA ?organism3: #$synechocystis_pcc6803 ?gene3: #$S6803.ssr2595

Goal Query:

+ “Explanation”

BioDeducta

Other things BioDeducta could figure out how to do:

Simulate natural or experimental “knockouts”.

Given inactivated reactions propose “bridging” reactions.

Construct pathway models (in likelihood order) that fit uArray data.

BioDeducta

BioDeducta Construct pathway models that fit uArray data.

Neighborhood search limits the search to subsystems thought to be relevant.

BioDeducta

P=.031

P=.031 P=.031 P=.031

P=.031

P=.031

P=.031

P=.031 P=.031

BioDeducta P=.031

HL -N -S -P -Ci

NblS

RR

Blue/ UV-A Photo- receptor

NblR

NblB NblA

Survival in High Light

Modification of PhotoSyn.

Represent an abstract theory.

cpcX hliA psbx ...

PsaX Degradation

Why do plants modify their photosynthetic apparatus in high light?

BioDeducta

Conceptually coherent, possibly complex, units of partially abstract knowledge that can be incrementally “mixed into” an existing model (by “View Application”), updating the model in accord with the principles represented in the view.

Some Views in Cell Biology: Transcriptional Regulation Operon Attentuation Chemical Cycle Transposon Insertion Feedback Regulation Allosteric Modulation Protein Assembly Signal Transduction

(aka. Schemas, Scripts)

Annotate the theory in terms of Views. BioDeducta

cpcX hliA psbx ...

HL -N -S -P -Ci

NblS

RR

Blue/ UV-A Photo- receptor

NblR

NblB NblA

Survival in High Light

Modification of PhotoSyn.

SIGNAL TRANS.

TRANSCRIPTION REGULATION

STRUCTURAL COMPENSATION

Annotate the theory in terms of Views.

PsaX Degradation

BioDeducta

(qxpr a increases high-light Nbls) (qxpr b increases -n Nbls) (view signal a) (view signal b) (qxpr e increases nbls nblr) (qxpr f increases nblr nbla) (view transcription-regulator e) (view transcription-regulator f) (qxpr h decreases (and nbla nblb) psaa) (view structural-modulation h) (qxpr i decreases (and high-light psaa) life) (qxpr j increases (and high-light (not psaa)) life) (view abstract-goal i) (view abstract-goal j) (qxpr k increases nbls rr) (qxpr l increases rr cpcb) (qxpr m increases rr hlia) (view transcription-regulator k) (view transcription-regulator l) (view transcription-regulator m) (qxpr n increases hlia modified-photosynthesis) ...

Step 1. Annotated theory in terms of Views. BioDeducta

Step 2. Find abstract pathways (by forward search). (33 solutions)

SIGNAL :: (INCREASES -P NBLS) TRANSCRIPTION-REGULATOR :: (INCREASES NBLS RR) TRANSCRIPTION-REGULATOR :: (INCREASES RR CPCB) TRANSCRIPTION-REGULATOR :: (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS) ABSTRACT-GOAL :: (INCREASES (AND HIGH-LIGHT MODIFIED-PHOTOSYNTHESIS) LIFE)

SIGNAL :: (INCREASES BLUE-UVA NBLS) TRANSCRIPTION-REGULATOR :: (INCREASES NBLS NBLR) TRANSCRIPTION-REGULATOR :: (INCREASES NBLR NBLB) STRUCTURAL-MODULATION :: (DECREASES (AND NBLA NBLB) PSAA) ABSTRACT-GOAL :: (INCREASES (AND HIGH-LIGHT (NOT PSAA)) LIFE)

BioDeducta

Step 3. Form qualitative predictions by simulation. ----------------- Pathway #1 ------------------ SIGNAL :: (INCREASES BLUE-UVA NBLS) TRANSCRIPTION-REGULATOR :: (INCREASES NBLS RR) TRANSCRIPTION-REGULATOR :: (INCREASES RR CPCB) TRANSCRIPTION-REGULATOR :: (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS) ABSTRACT-GOAL :: (INCREASES (AND HIGH-LIGHT MODIFIED-PHOTOSYNTHESIS) LIFE) Predictions: ((INCREASES NBLS RR) (INCREASES RR CPCB) (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS)) QSimulation: (INCREASES NBLS RR) (INCREASES RR CPCB) (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS) (INCREASES NBLS CPCB) (INCREASES RR RR) (INCREASES NBLS MODIFIED-PHOTOSYNTHESIS) (INCREASES RR CPCB) (INCREASES RR RR) (INCREASES CPCB NBLS) (INCREASES RR MODIFIED-PHOTOSYNTHESIS) (INCREASES CPCB CPCB) (INCREASES CPCB RR) (INCREASES MODIFIED-PHOTOSYNTHESIS NBLS) (INCREASES CPCB CPCB) (INCREASES MODIFIED-PHOTOSYNTHESIS RR)

BioDeducta

Step 5. Assign likelihoods to the pathways based upon the fit of qualitative predictions to regressions.

----------------- Pathway #1 ------------------ SIGNAL :: (INCREASES BLUE-UVA NBLS) TRANSCRIPTION-REGULATOR :: (INCREASES NBLS RR) TRANSCRIPTION-REGULATOR :: (INCREASES RR CPCB) TRANSCRIPTION-REGULATOR :: (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS) ABSTRACT-GOAL :: (INCREASES (AND HIGH-LIGHT MODIFIED-PHOTOSYNTHESIS) LIFE) Predictions: ((INCREASES NBLS RR) (INCREASES RR CPCB) (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS)) QSimulation: (INCREASES NBLS RR) = NIL (INCREASES RR CPCB) = NIL (INCREASES CPCB MODIFIED-PHOTOSYNTHESIS) = NIL (INCREASES NBLS CPCB) = - (INCREASES RR RR) = NIL (INCREASES NBLS MODIFIED-PHOTOSYNTHESIS) = NIL (INCREASES RR CPCB) = NIL (INCREASES RR RR) = NIL (INCREASES CPCB NBLS) = - (INCREASES RR MODIFIED-PHOTOSYNTHESIS) = NIL (INCREASES CPCB CPCB) = NIL (INCREASES CPCB RR) = NIL (INCREASES MODIFIED-PHOTOSYNTHESIS NBLS) = NIL (INCREASES CPCB CPCB) = NIL (INCREASES MODIFIED-PHOTOSYNTHESIS RR) = NIL Summary likelihood = -2

BioDeducta

----------------- Pathway #12 ------------------ SIGNAL :: (INCREASES BLUE-UVA NBLS) TRANSCRIPTION-REGULATOR :: (INCREASES NBLS NBLR) TRANSCRIPTION-REGULATOR :: (INCREASES NBLR NBLB) STRUCTURAL-MODULATION :: (DECREASES (AND NBLA NBLB) PSAA) ABSTRACT-GOAL :: (INCREASES (AND HIGH-LIGHT (NOT PSAA)) LIFE) Predictions: ((INCREASES NBLS NBLR) (INCREASES NBLR NBLB)) QSimulation: (INCREASES NBLS NBLR) = NIL (INCREASES NBLR NBLB) = NIL (INCREASES NBLS NBLB) = + (INCREASES NBLR NBLR) = NIL (INCREASES NBLR NBLR) = NIL (INCREASES NBLB NBLS) = + Summary likelihood = 2


BioDeducta

----------------- Pathway #17 ------------------ SIGNAL :: (INCREASES BLUE-UVA NBLS) TRANSCRIPTION-REGULATOR :: (INCREASES NBLS NBLR) TRANSCRIPTION-REGULATOR :: (INCREASES NBLR NBLA) STRUCTURAL-MODULATION :: (DECREASES (AND NBLA NBLB) PSAA) ABSTRACT-GOAL :: (INCREASES (AND HIGH-LIGHT (NOT PSAA)) LIFE) Predictions: ((INCREASES NBLS NBLR) (INCREASES NBLR NBLA)) QSimulation: (INCREASES NBLS NBLR) = NIL (INCREASES NBLR NBLA) = + (INCREASES NBLS NBLA) = NIL (INCREASES NBLR NBLR) = + (INCREASES NBLR NBLR) = + (INCREASES NBLA NBLS) = NIL Summary likelihood = 3


BioDeducta

Interactive Discovery: The Biologist’s Roles

•  Provide representations and biological concepts, possibly in abstract terms.

•  Focus search by providing initial models using the above representations and concepts.

• Guide search interactively: •  Focus attention on problematic aspects of the model.

•  Run discriminating experiments.

•  Make “hard” (subjective) choices.

BioDeducta

Interactive Discovery: The Computer’s Role

•  Deal with incomplete, ambiguous, overlapping, probabilistic, and abstract knowledge

•  Search the region near a given model using biologically-plausible operators and within given constraints

•  Produce explanations

•  Formulate discriminating experiments

The discovery system must be able to:

BioDeducta

BioBike reports and papers:

Elhai, J., Taton, A., Massar, J.P., Myers, J.K., Travers, M., Casey, J., Slupesky, M., Shrager, J. (2009) BioBIKE: A Web-based, programmable, integrated biological knowledge base. Nucleic Acids Research 2009; doi: 10.1093/nar/gkp354

Shrager J, Waldinger R, Stickel M, Massar J (2007) Deductive Biocomputing. PLoS ONE 2(4): e339. doi:10.1371/journal.pone.0000339

J Shrager (2007) The Evolution of BioBike: Community Adaptation of a Biocomputing Platform. Studies in History and Philosophy of Science, 38, 642-656.

JP Massar, M Travers, J Elhai, and J Shrager (2005) BioLingua: A programmable knowledge environment for biologists. Bioinformatics. 21(2), 199-207.

K Saito, D George, S Bay, J Shrager (2003). Inducing biological models from temporal gene expression data. Proceedings of the 6th International Conference on Discovery Systems. Sapporo, Japan.

L Chrisman, et al. (2003). Incorporating biological knowledge into evaluation of causal regulatory hypotheses. Proc. of the Pacific Symposium on Biocomputing (PSB2003). Hawaii.

J Shrager, P Langley, & A Pohorille (2002), Guiding revision of regulatory models with expression data. Proc. of the Pacific Symposium on BioComputing. World Scientific Press.

BioBike and BioDeducta

Biologist as Programmer Okay, so Where are They Now?

1.  The development of simple math as a clear setting. 2.  Simple addition ain’t so simple! 3.  Early Model: A competing “fast”/“slow” architecture. 4.  Late Early Model: Strategy choice 5.  Middle Model: Strategy Change 6.  Current Model: Based on “Systems Neuroscience”

1.  The problem: Strategy Change requires that need new concepts are constructed (as in Commonsense Perception).

2.  An approach: “Progressive Deeping” NN architecture that builds later-learned “high” level concepts from earlier-learned “low” level ones.

The research programme:

Strategy Choice

<A Little In-Sight>

Strategy Choice

Strategy Choice

Siegler, R. S. & Shrager, J. (1984). Strategy choices in addition and subtraction: How do children know what to do? In C. Sophian (Ed.), Origins of cognitive skills. Erlbaum.

Strategy Choice

Strategy Choice

Siegler, R. S. & Shrager, J. (1984). Strategy choices in addition and subtraction: How do children know what to do? In C. Sophian (Ed.), Origins of cognitive skills. Erlbaum.

Strategy Choice

Strategy Change

Strategy Change

Shrager, J. & Siegler, R. S. (1999). SCADS: A model of strategy choice and strategy discovery. Psychological Science.

Strategy Change

Arithmetic Concept Discovery

Strategy Change The Problem:


Cerebellum: Smooth (Cognitive skill) Sequencing

General Motor: Finger activity and cognitive skill sequencing

General Visual: Seeing fingers

IPS: Number concept; possibly the primary NN hidden layer

Hippocampus: Explicit number fact memory

General Auditory: Turning phonetics into internal representations, and possibly operating the echoic buffers

General Frontal: Activity initiation and in-process/attentional control

Internal Number

Echoic Memory

Phonetic Hearing Visual Fingers

Motor Fingers

QuasiMotor Attention

Phonetic Saying

How Complex Strategies Work in the Brain and in the World

Commonsense Perception in Strategy Change

New Strategies often Parse the Universe Differently!

Strategy Change




THE LEVELS NEED TO SELF ORGANIZE!

Selfridge’s (1959) Pandemonium Paradigm

These don’t self-organize in the right way!

Shrager, J. & Johnson, M. H. (1996). Factors influencing the emergence of function in a simple cortical network. Neural Networks, 9(6), 1119-1129. Elman, et al. (1999). Rethinking Innateness. MIT Press.

Cortical Parcellation

Shrager, J. & Johnson, M. H. (1996). Factors influencing the emergence of function in a simple cortical network. Neural Networks, 9(6), 1119-1129.

Cortical Parcellation

Being able to program with Self-Organizing Probabilistic Partially-Overlapping Abstraction Hierarchies

To achieve rationality, the hallmark of intelligence.

Which in turn allows you to synchronize abstractions via these grounded representations.

Which allows you to compute flexibly with symbolic representations.

Allows you to work with abstractions that overlap on the ground.

jeff shrager [email protected] - perception, parcellation, … · 2015-11-19 · thanks to...

Documents