1

Doing Ontology Over Images

Barry Smith

What ontologies are for

3

what molecular function ?

what disease process ?

need for semantic annotation of data

4

need for semantic annotation of data

through labels (nouns, noun phrases)which are algorithmically processable

5

natural language labels

to make the data cognitively accessible to human beings

6

compare: legends for mapscompare: legends for maps

7

compare: legends for cartoons

9

ontologies are legends for data

10

ontologies are legends for images

11

what lesion ?

what brain function ?

12

xi = vector of measurements of gene i k = the state of the gene ( as “on” or “off”)θi = set of parameters of the Gaussian model......

ontologies are legends for mathematical equations

13

The OBO Foundry Idea

MouseEcotope GlyProt

DiabetInGene

GluChem

sphingolipid transporter

activity

14

annotation using common ontologies yields integration of databases

MouseEcotope GlyProt

DiabetInGene

GluChem

Holliday junction helicase complex

15

annotation using common ontologies can yield integration of image data

16

annotation using common ontologies can support comparison of image data

17

truth

18

simple representations can be true

19

there are true cartoons

20

a cartoon can be a veridical representation of reality

21

Cartographic Projection

22

maps may be correct by reflecting topology, rather than geometry

23

a fully labeled image can be aneven more veridical representation of reality

an image can be a veridical representation of reality

24

25

26

cartoons, like maps, always have a certain threshold of granularity

27

grain resolution

28

grain resolution serves cognitive accessibility

we transform true imagesinto true cartoons

29

there are also true cartoon sequences

30

31

Pathway diagrams are annotated dynamic cartoons

32

pathways can be represented at different levels of granularity

33

the jaw

34

35

Joint capsule

Netter

36

Mandible and condyle movement

37

Condyle position in fossa wrt location of disc

38

TMJ in jaw open and closed positions

39

Parts• 1 head of condyle F• 2 neck of condyle F• 3 disc B• 4 retrodiscal tissue B• 7 articular eminence F• 8 zygomatic arch F• 10 upper head of lateral pterygoid muscle F• 11 lower head of lateral pterygoid muscle F

Holes• 5 lower joint compartment B• 6 upper joint compartment B

Holes and Parts

40

ANTERIOR

Temporomandibular Joint (TMJ)

from Thomas Bittner and Louis Goldberg, KR-MED 2006

41

adjacency relations

No connectednessOnly (temporary) adjacency

Connectednessadjacencygraph

Adjacency relations

42

Frames of reference

Rigid = do notchange shape(bones)

A

BC

DE

F

The extension of the axis of the condyleintersects the fossa in region D

43

instances vs. types

44

two kinds of annotations

45

names of instances

46

names of types

47

pathway maps are representations of complexes of types

48

molecular images and radiographic images are

representations of instances

49

MIAKT system

50

51

52

53

54

Patient #47920

55

56

Mammography #31667

57

Mammography #31667

Medical-Image #44922

58

MRI-Exam #32388

Medical-Image #44922

Mammography #31667

Patient #47920

Breast #1388

Abnormality #86023