2nd International Conference on Biomedical Ontology (ICBO’11)

Ontology-Based Analysis of Event-Related Potentials

Gwen Frishkoff12, Robert Frank2, Paea LePendu3,

& Snežana Nikolič1

1 Psychology & Neuroscience, Georgia State University2 NeuroInformatics Center (NCBO), University of Oregon

3 National Center for Biomedical Ontology (NCBO), Stanford University

http://nemo.nic.uoregon.edu

Why ontology-based analysis? Linking Data to Knowledge in Human Neuroscience

Ontology-based analysis of ERP data Data Information

• Pipeline for automated (and therefore objective)separation of ERP patterns and extraction of summary metrics for each pattern

Information Knowledge• Ontology to represent metrics in semantically structured

way so as to automatically classify & label ERP patterns within and across experiments

Overview

Why ontology-based analysis? Linking Data to Knowledge in Human Neuroscience

Ontology-based analysis of ERP data Data Information

• Pipeline for automated (and therefore objective)separation of brainwave (ERP) patterns and automated extraction of summary metrics, which are output to RDF

Information Knowledge• Ontology to represent data (in RDF) and automatically

(and therefore objectively) classify & label ERP patterns within and across experiments

Overview

“The plural of ‘anecdote’ is not ‘data’.”

— Roger Brinner (Economist)

Assertion #1: In a scientific domain, the priority should be to capture and track

assertions about data.

Corollary: To capture complex (and presently ill-defined) patterns in data, we need bottom-up (data-driven) analysis.

The plural of ‘data’ is not ‘knowledge’.

Assertion #2: To draw meaningful inferences from data, they must be linked to

a well-structured knowledge base

(ontology).

ONTOLOGY

DATA

INFORMATION

Data mining (i.e., analysis)

Knowledge engineering

Ontology mining?

The plural of ‘data’ is not ‘knowledge’.

Assertion #3: Ontology Semantic Structure. It cannot be automatically

extracted from data (or patterns in data). Cf. Searle’s Chinese Room argument…

Corollary: To build a valid ontology, we need top-down (knowledge-driven) methods

(ala BFO/OBO).

Introduction to ERP Domain (I):The Data = Measurements of Scalp EEG

EEGs (“brainwaves” or flunctuations in brain electrical potentials) are recorded by placing two or more electrodes on the scalp surface.

256-channel Geodesic Sensor Net ~5,000 ms

Introduction to ERP Domain (II):From EEG to Event-Related Potentials (ERP)

ERPs (event-related potentials) are the result of averaging across multiple segments of EEG, time-locking to an event of interest.

AVERAGE OVER (LOTS OF)

EEG SEGMENTS

EEG

ERP

Introduction to ERP Domain (III): Entities of Interest = ERP Patterns (in Data!)

ERP patterns characterized by three types of attributes:

(1) TIME latency of peak positive or peak negative potential (left) (2) SPACE scalp topography of this potential (right); and(3) FUNCTION experimental context in which these patterns are characteristically observed (e.g., presentation of visual stimulus)

120 ms

• Tried and true method for noninvasive brain functional mapping

• Direct measure neuronal activity• Whole-brain measurement (at scalp)• Millisecond temporal resolution• Portable and inexpensive• Important clinical applications (e.g., potential biomarkers for AD, presurgical planning)

• Recent innovations give new windows into rich, multi-dimensional patterns– Rich spatial info (high-density EEG)– Combined temporal & spectral info (JTF)– Multimodal (EEG/ fMRI/MEG) measures

1 sec

What’s great about ERPs …

If ERPs are so great….

Why are there so few meaningful applications in biomedicine?

And why so few (arguably no) cross-lab meta-analyses?

Problem #1: Patterns superposed in space & timeLATENT (INFERRED) PATTERNS

(THIS IS WHAT WE WANT TO TALK ABOUT)MEASURED DATA

(THIS IS WHAT WE ACTUALLY MEASURE/OBSERVE!)

Superposition

Everyone has one, and nobody likes to use anyone else’s.

Problem #2a: Patterns (actually, pattern labels) are like toothbrushes…

Prosody-specific negativityPhonological mapping negativity

Medial frontal

negativity

MEANINGFULNESS

RECOGNITION

POTENTIAL

fN400old-new

effect

N400 Effect

N300

410 ms

450 ms

330 ms

Consider a Hypothetical Database Query: Show me all the N400 patterns in the database.

Peak latency 410 ms

“CANONICAL N400”

Will the “real” N400 please step forward?

Problem #2b: Conversely, different scientists use the same label for incommensurable patterns.

Putative “N400”-labeled patterns

Parietal N400

≠≠

fN400

Parietal P600

Assertion #3: We cannot ground ERP meta-analysis in prior

literature (e.g., text mining). We need a reliable workflow for data

analysis & classification.

Summary: Motivation for NEMO• Lots of different — and equally valid! — methods for

pattern analysis• Inconsistent and subjective use of metrics and labels for

pattern summary and classification• No existing methods or tools to support ERP data

sharing and integration

Assertion #4: The best way to address these issues is to combine data-driven methods for pattern analysis with knowledge-driven methods for ontology development

and application (to interpret analysis results)

Neural ElectroMagnetic Ontologies

A set of formal (OWL) ontologies for representation of ERP domain concepts

A suite of tools for data-driven extraction and ontology-based annotation of ERP patterns

A database that includes publicly available, annotated data from our NEMO ERP consortium to demonstrate application of ontology for quantitative meta-analysis of results from studies of language and cognition

Why ontology-based analysis? Linking Data to Knowledge in Human Neuroscience

Ontology-based analysis of ERP data Data Information

• Pipeline for automated (and therefore objective)separation of ERP patterns and extraction of summary metrics for each pattern

Information Knowledge• Ontology to represent data (in RDF) and automatically

(and therefore objectively) classify & label ERP patterns within and across experiments

Overview

FROM DATA TO INFORMATION….

Extraction of meaningful

patterns (i.e., data analylsis)

ERP Pattern Analysis: Current Practice

N400 component

P3 component“Bumpology”

Bumpology^2?

NEMO Ontology-based Analysis: Overview

1. ERP Pattern Extraction

2. ERP Metric Extraction

3. RDF Generation (Data Annotation)

4. (Metadata Entry)

5. ERP Pattern Classification

1. NEMO Pattern Extraction

NEMO ERP Pattern Extraction Toolkithttp://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release

• NEMO_ERP_Pattern_Decomposition/• NEMO_ERP_Pattern_Segmentation/

Pattern Extraction I: DecompositionAdvantages:

• Data-driven• Automated/ Objective• Sensitive (able to separate

superposed patterns)

P100

N100

fP2

P1r/ N3

P1r/ MFN

100ms

170ms

200ms

280ms

400ms

Disdvantages:• Requires expertise (~vanilla

PCA)• Not used by majority of ERP

researchers

NEMO ERP Pattern Extraction Toolkithttp://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release

• NEMO_ERP_Pattern_Decomposition/

Pattern Extraction II: Segmentation

NEMO ERP Pattern Extraction Toolkithttp://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release

• NEMO_ERP_Pattern_Segmentation/

2. Metric Extraction

NEMO ERP Metric Extraction Toolkithttp://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release

• NEMO_ERP_Metric_Extraction/

Typical semi-structured representation of ERP data

Peak latency measurement (in ms)

ERP pattern (extracted from “raw” ERP data using PCA/ICA etc.)

Why ontology-based analysis? Linking Data to Knowledge in Human Neuroscience

Ontology-based analysis of ERP data Data Information

• Pipeline for automated (and therefore objective)separation of brainwave (ERP) patterns and automated extraction of summary metrics, which are output to RDF

Information Knowledge• Ontology to represent metrics in semantically structured

way so as to automatically classify & label ERP patterns within and across experiments

Overview

ONTOLOGY

FROM INFORMATION TO KNOWLEDGE….

NEMO Ontology-based Analysis: Overview

1. ERP Pattern Extraction

2. ERP Metric Extraction

3. RDF Generation (Data Annotation)

4. (Metadata Entry)

5. ERP Pattern Classification

Recall: Entities of interest (at Stage 1) = Patterns in Data

1 sec

TIME SPACE

FUNCTION Modulation of pattern features (time,

space, amplitude) in different experiment conditions

NEMO Ontology (in a nutshell)

L1: Brain

Physiological processes(BFO/OPB)

L3: Brain

Physiological data

(OBI/IAO)

3. RDF Generation

NEMO ERP Metric Extraction Toolkithttp://nemoontologies.svn.sourceforge.net/viewvc/nemoontologies/toolkit/release

• NEMO_ERP_Metric_Extraction/

# OWL Ontology Declaration / Import: GAF-LP1_NN_ERP_data<http://purl.bioontology.org/NEMO/data/GAF-LP1_NN_ERP_data> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/2002/07/owl#Ontology>.<http://purl.bioontology.org/NEMO/data/GAF-LP1_NN_ERP_data> <http://www.w3.org/2002/07/owl#imports> <http://purl.bioontology.org/NEMO/ontology/NEMO.owl>.

# Instance Declaration 000: GAF-LP1_NN_ERP_data<http://purl.bioontology.org/NEMO/data/GAF-LP1_NN_ERP_data> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.bioontology.org/NEMO/ontology/NEMO.owl#NEMO_0000495>.

Data annotation using RDF “Triples”

In natural language =

The data represented in cell Z (row A, column 1) is an instance of (“is a”) a peak latency temporal measurement (i.e., the time at which the pattern is of maximal amplitude)

Note that the predicate links an instance to a class within NEMO ontology.

In RDF form: <002> <type> <NEMO_0745000>Subject – Predicate –

Object

GOAL: Represent extracted information with rich, formal semantics that allow us to reason over data (both within and across datasets)

RDF Graph (“triples”)

ERP PATTERN CLASSIFICATION

5. Pattern Classification (I)(1) Temporal Criterion

(3) Functional Criterion

(2) Spatial Criterion

5. Pattern Classification (II)

RDF Data loads NEMO ontology

RDF Data is opened in Protégé ontology editing software

5. Pattern Classification (III)

HermiT Reasoner is used to generate inferences

5. Pattern Classification (IV)

Instance-level information (i.e., ERP pattern instances) are successfully classified!

Take-home messages1. For some biomedical applications it may important to capture L3

(DATA) as well as L1 (REALITY) explicitly, i.e., within the ontology

2. In linking the data to the ontology (e.g., for classification/labeling of patterns), it may be important consider data-driven methods for pattern analysis and metric extraction

3. An advantage of this approach is that we can generate relatively stable (non-controversial) representation of data (RDF artifacts), which we will archive and maintain — separate from, but linked to, the ontology — even as the ontology is uncertain & changing.

4. Further, robust representation of data across studies provides basis for valid quantitative meta-analysis, which provide high-quality evidence to inform pattern rules in the ontology

Ongoing Work & Open Issues• Evolving pattern rules to represent more

complex functional criteria (i.e., expt metadata)• Temporal reasoning (can we squeeze this into

DL/OWL?)• Representing uncertainty in pattern rules &

classification of pattern instances (beyond Evidence Codes?)

• Clinical applications: Pilot cross-lab work with aphasics (stroke & TBI patients with language disorders)

Funding from the National Institutes of Health (NIBIB), R01-MH084812 (Dou, Frishkoff, Malony)

NEMO Ontology Task ForceRobert M. Frank (NIC)Dejing Dou (CIS)Paea LePendu (CIS)Haishan Liu (CIS)Allen Malony (NIC, CIS)Jason Sydes (CIS)*Snezana Nikolic (PSY, GSU)

*emeritus

Acknowledgments

www.nemo.nic.uoregon.edu

NEMO EEG/MEG Data ConsortiumTim Curran (U. Colorado)Dennis Molfese (U. Louisville)John Connolly (McMaster U.)Kerry Kilborn (Glasgow U.)Charles Perfetti (U. Pittsburgh)

Special thanks to:Maryann Martone & associates (NIF)Jessica Turner (cogPO)Angela Laird (BrainMap)Sivaram Arabandi (OGMS)

YOU (BIO-ONTOLOGY

COMMUNITY)

Recent References• Frishkoff, G., Frank, R., LePendu, P., & Nikolic, S. (2011, in press). Ontology-

based Analysis of Event-Related Potentials. Proceedings of the International Conference on Biomedical Ontology (ICBO'11).

• Frishkoff, G., Frank, R., Sydes, J., Mueller, K., & Malony, A. (2011, subm). Minimal Information for Neural Electromagnetic Ontologies (MI-NEMO): A standards-compliant workflow for analysis and integration of human EEG. Standards in Genomic Sciences (SIGS).

• Liu, H., Frishkoff, G., Frank, R. M. F., & Dou, D. (2011, subm). Integration of Human Brain Data: Metric and Pattern Matching across Heterogeneous ERP Datasets. Journal of Neurocomputing.

• Frank, D. & Frishkoff, G. A. (2011, in prep.). The NEMO ERP Analysis Toolkit: Combining data-driven and knowledge-driven methods for ERP pattern analysis. Neuroinformatics.

• Frishkoff, G.A., Dou, D., Frank, R., LePendu, P., and Liu, H. (2009). Development of Neural Electromagnetic Ontologies (NEMO): Representation and integration of event-related brain potentials. Proceedings of the International Conference on Biomedical Ontologies (ICBO09). July 24-26, 2009. Buffalo, NY.