Is Abstraction a Key to Artificial

Intelligence?Lorenza Saitta

Università del Piemonte Orientalelorenza.saitta@uniupo.it

Premise

« Is abstraction a key to computing? »[Kramer, 2007]

The same question can be posed for Artificial Intelligence

« … [the process of] abstraction is the essence ofintelligence and the hard part of the problems beingsolved »

[Brooks, 1991]

Representation• Representation is critical in AI tasks• Search for the “best” representation

• One that facilitates performing a task/solving a problem• Often difficult to know a-priori which is the best

representation• Expert definition on the basis of domain knowledge, experience,

analogy, …• It is important to be able to easily change

representation when needed• Several types of representation changes have been

proposed in AI à Abstraction is one of them

Abstraction = Cognitive Organizational Principle

• Abstraction is a special kind of representation change, fundamental in human thinking

• Difficult to be precisely defined• Abstraction aims at reducing the complexity of the

perceived world

« … a ubiquitous function of the cerebralcortex, one in which many if not all of its areasare involved, is that of abstraction »

[Zeki, 2009]

“Were it not for the ability to construct useful abstraction, intelligent agents would be completely swamped by the real world”

[Russel & Norvig, 2010]

Abstraction’s Modus Operandi (1)

• Focalization on relevant information and removal of irrelevant details

• Aggregation/Grouping

4 1 Introduction

tion of abstract concepts has been investigated in Psychology and CognitiveSciences, notably by Barsalou and co-workers, who provide a theoretical andan experimental account of the issue [37]. An interesting connection can bedone with Computer Science, namely with the epistemological status of soft-ware and the basic skills needed for writing good programs6. As a matter offact, Kramer wonders whether “abstraction is the key to computing” [272],abstraction meaning here the capability of removing inessential details and toidentify a common “essence” inside variability.

This capability of going to the core of things is another fundamental aspectattributed to abstraction, namely the ability to focus on relevance. Objects,phenomena, events in the world are extremely rich in details and may be verycomplex. However, when solving a problem or executing a task, only someaspects of the reality are useful, and to take into consideration the whole wealthof details may be confusing. For instance, when planning an aerial trip, thephysical attributes of the aircraft, such as color or exact shape and sizes, areirrelevant and can be ignored. As another example, in Figure 1.3, a satelliteimage of downtown Torino is reported, where the buildings and monuments canbe seen. However, just to find one’s way around the city it is more convenientto reduce the information to the street network. By citing again Brooks [75],“... abstraction is the essence of intelligence and the hard part of the problembeing solved”.

Stampa Invia LinkIndicazioni stradali Le mie mappe Stampa Invia LinkIndicazioni stradali Le mie mappe

Fig. 1.3: Satellite image of the center of Torino (left): buildings and monuments are visible.The same area can be described by considering just the street network (right): this abstractmap is more convenient for moving around the city.

Actually, in trying to solve a complex problem it may sometime be a goodstrategy to proceed top-down, by starting with a coarse solution and then re-fining it. At each step of refinement new details are possibly taken into account,generating a sequence of solutions, each one more detailed than the previousone. In this case we may speak of a hierarchy of levels of abstraction, with the6 See Chapter 2.

Part–of(bicycle)

Member-of(forest) Functional relation

(computer, tennis set)

Computer

Keyboard

Mouse

Monitor

Body

Floppy

wheelpedal

saddle handlebar

wheel

Abstraction’s Modus Operandi (2)• Naming Equivalence classes of objects

• Discovery of new concepts(predicate invention)

Chair = Object with legs, a seat and a back

Hub

Community

Abstraction’s Modus Operandi (3)• Building Hierarchies

!

network models of aging [7-9]. One common approachto mining complex networks based on modularity isfirst to identify modules as knowledge building blocks,and then to use their organization to depict the knowl-edge contained in the networks. However, most moduleorganizations are limited to either a “vertical” or “hori-zontal” representation. A vertical relationship is repre-sented by a multilevel dendrogram that only describesthe inclusion/part-of relationships between modules atdifferent hierarchical levels [4,10,11], and the horizontalrelationship is a single-level graph that only shows howmodules are connected [7-9]. Neither of them providesan integrated view of the complex systems they repre-sent; consequently, it is difficult to further explore thesecomplex domains. In this work, we combine vertical andhorizontal relationships in order to organize the mod-ules into a multilevel pyramid, as illustrated in Figure 1.At each level, we describe the horizontal relationshipsby a network of modules that is by itself the abstractionof the network at a lower level [3]. In contrast, the verti-cal relationships, shown as links between layers, repre-sent the inclusion relationship between modules atdifferent levels. Using an abstraction pyramid, not onlycan domain experts gain a global multilevel view of acomplex system from two different perspectives (hori-zontal and vertical), but they can also investigate theinterconnection of the modules at a particular abstrac-tion level of interest in the hierarchy.

Our approach, named Pyramabs (Pyramid of abstrac-tions), identifies the modules and simultaneously con-structs the pyramid based on the network topology.Prior domain knowledge is not used. We tested Pyra-mabs on artificial random networks, a protein-proteininteraction network, and a metabolic network. We com-pared Pyramabs with other methods and verified ourresults based on those published in the literature andpublic databases.

ResultsThe two overarching goals of our work are to (1) pro-pose an alternative knowledge representation forimproved network interpretations, and (2) introduce anovel approach for extracting knowledge from networksand describing it using the new representation. Theabstraction pyramid discovered by Pyramabs does notreplace the known structure of ontology (e.g., the GeneOntology (GO)), but instead provides other informationthat may be missing. For example, an abstraction pyra-mid identified from a protein-protein interaction net-work could illuminate the protein interactions at variouslevels. Some vertical or horizontal relationships can pro-vide additional biological meaning that may not be char-acterized in the GO’s Directed Acyclic Graph (DAG)structure.We divide the analysis of complex networks into two

tasks: module discovery and module organization. Thenovelty of our two-way approach is derived from thesynergy of top-down and bottom-up clustering algo-rithms. This method identifies modules in a top-downfashion and constructs a hierarchy implied in a complexnetwork from the bottom up. In addition, it producesan abstraction of the network to different degrees at dif-ferent levels in the hierarchy. Our method can bedivided into three procedures: (1) computing the proxi-mity between nodes; (2) extracting the backbone fromthe network, represented by a spanning tree, and thenpartitioning the network based on that backbone; and(3) generating an abstract network. By iteratively apply-ing the same procedures to a newly generated abstractnetwork, we can disclose an abstraction hierarchyimplied in a complex network. The Pyramabs flowchartprovided in Figure 2 includes the following steps:

Step 1. Input a given network of nodes to Pyramabs.Step 2. Calculate the proximity between all pairs ofnodes and use as the link weights.Step 3. Normalize the proximity by computing thez-scores; then discard the links with a z-score belowa specified threshold to reduce the search space ofthe network.Step 4. Obtain the maximum-weight spanning treefrom the network and use as the backbone.

Figure 1 Illustration of vertical and horizontal relationships.Each circle represents a module. Vertical relationships and horizontalrelationships are denoted by dashed lines and solid lines,respectively. The thickness of a solid line increases with theimportance of the connection. The original network is at thebottom (Level 4). Higher-level networks are an abstraction, to acertain degree, of the next lowest network.

Cheng and Hu BMC Bioinformatics 2010, 11:411http://www.biomedcentral.com/1471-2105/11/411

Page 2 of 11

Analysis of Biological Networks[Cheng and Hu, 1997]

Level of Details (LOD) approach[Luebke et al, 2003]

Get this bookMorgan KaufmannAmazon.comBarnes&Noble.comBooks-A-MillionFind in a libraryAll sellers »

Related books

All related books »

Sponsored LinksBranding/Corporate FontsAll The Best Top Quality Fonts-Get 10% Off With Code TENPC10,Buy Now!www.type.co.uk

Pages displayed by permission of MorganKaufmann. Copyright.

Abstraction supports Robust Descriptions

• Reduction of computational complexity• Increasing in meaningfulness

Volume(x) ≤ a à Bike a < Volume(x) ≤ b à Car Volume(x) > b à Airplane

Has(two_wheels & open-body & handelbar& saddle) -> Bicycle

Has(four_wheels &body_with_windows) -> Car

Has(retractable_wheels&body&wings) -> Airplane

Task-dependent

Reusable

Basic Properties of Abstraction

Intensional Notion

122 5 Boundaries of Abstraction

Vehicle!

Land!Vehicle!

Sea!Vehicle!

Air!Vehicle!

Good transport!

People!transport!

Cart!Truck! Bus! Train! Car! Bicycle!

…… ……!

AB#918#RS#AH#708#SW#BN#387#LG#…………..#

is-a!is-a!

is-a!

Instance-of!

is-a!

Fig. 5.2: A possible hierarchical organization of the concept vehicle = “thing used for

transporting people or goods”. Transportation may occur on land, sea, or air. A vehiclecan be used to transport people or goods, and so on. The instances of car are specific cars,identified with their (Italian) plate.

nodes correspond to instances, i.e., particular objects that satisfy the descrip-tions. Each oval node adds some new property to the description of the fathernode, and is linked to this last by a “is-a” relation. Thus, nodes low in thehierarchy are more detailed then nodes up, and they provide more informa-tion about the objects that satisfy the properties. The lowest level contains theobjects themselves, which are, in fact, the most possible detailed descriptions.

The hierarchy can be read in two ways:

• Intensional view - Each node of the hierarchy is associated to a description.Climbing the tree, less and less detailed descriptions are found.

• Extensional view - Each node of the hierarchy is associated to a set of objects,exactly those objects that satisfy the description associated to the node inthe intensional view. Climbing the tree, larger and larger sets of objects arefound.

It is fundamental to understand that abstraction is only concerned with de-scriptions. Given a particular object, we have to distinguish what the object isfrom what we know (just want to keep) of the object. In fact, a given objectalways satisfies, for its very nature, the most detailed description possible. Ifwe do not need all the details of the complete description of the object, we mayresort to more abstract ones, moving up in the hierarchy, and finding less andless detailed descriptions.

Coverage

Abstraction

Abstraction ≠ Generalization

Less informative

Moreinformative

More general

Lessgeneral

Relative Notion

124 5 Boundaries of Abstraction

Without having the ambition to solve a longstanding problem (see Rosen’sview [417] in Section 2.1), we too came across the same problem, and wereforced to take a stance, even though only within our limited scope. After tryingunsuccessfully several definitions, we came up with the belief that finding ageneral rule to label something (an object, a concept, a word, ...) as “abstract”or “concrete” is without hope, and that we can only speak of abstraction as arelative notion, and not as an absolute one. In other words, all we can say isthat something is more abstract than something else. Then, abstraction has tobe considered as an equivalence relation that induces a partial order on entities.

In order to explain our choice, let us look at an example. In Fig. 5.4(a) apicture of a poppy field is reported. There are no clear and undisputable groundsfor labeling this picture as abstract or concrete. In fact, if we reason from thepoint of view of closeness with the reality, the picture is not the “true” poppyfield, and then it should be abstract (see also Fig. 2.2). On the other hand,if we judge from the point of view of abstract art, it has a close resemblancewith the original, and then it should be rather labeled as concrete. From thepoint of view of the ability to capture the essential aspects of the original,again we do not know what to say: maybe there are important details that thepicture did not capture (for instance, the pistils), or the image is even too muchdetailed (maybe, only the perception of a red field, as in impressionist art, wouldmatter). But, if we look at the picture in Fig. 5.4(b), and we compare picture

!"#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$!%#$

Fig. 5.4: (a) Picture of a poppy field. If we only have this picture, it is impossible to saywhether it is concrete or abstract. (b) The same picture in black and white. By comparison,this last is less informative than the colored one, because the information referring to thecolor has been removed; then picture (b) is more abstract than picture (a). [A color versionof this figure is reported in Fig. H.5 of Appendix H].

(a) with picture (b), we are immediately able to say that picture (b) is moreabstract. In fact, the information about the color has been removed, leavingthe rest unchanged. We want to stress that only the pictures are compared withrespect to the “more abstract then” relation, because the original poppy field,of course, did not change, as we have discussed in Section 5.1.2. We may noticethat picture (b) in Fig. 5.4 is more abstract than picture (a) even according to

Less abstractMore informative

(color information)

More abstractLess informative

(no color information)

Path in an Abstraction Space

“Ilvero modo etordine perdissegnar tutte leparti iemembra delcorpo humano”,

[Fialetti ,1608]

5.1 Characteristic Aspects of Abstraction 127

Fig. 5.6: From Fialetti’s “Il vero modo et ordine per dissegnar tutte le parti ie membra delcorpo humano”, 1608. One among a set of studies for drawing eyes. ( c• This artwork maybe protected by copyright. It is printed in this book in accordance with fair use principles.)

By taking the top-leftmost and bottom right-most drawings in Fig. 5.6, itis really hard, without looking at the intermediate steps, to relate them in anymeaningful way. However, the relation between the two clearly appears if weconsider tho whole process of stepwise transformations.

Abstraction has been considered a process also in Mathematics, where theconcept of number is reached, according to Husserl, through a counting processleaving aside all properties of a set of objects, except their numerosity. Lewis[299] defines explicitly abstraction as a process of removing details from theconcrete5. Finally, Staub and Stern’s approach to abstraction6 mixes the ideaof abstraction as a process and abstraction as a relative notion, as we do; infact, these authors claim that concepts are obtained by reasoning, startingfrom the concrete world. Along the reasoning chain abstraction increases, sothat the farther from the concrete world a concept is along the chain, the moreabstract. As an example, real numbers are more abstract than integers. Eventhough this approach shares with our view the ideas of process and relativityof abstraction, we do not arrive at the same conclusions as Staub and Stern,because they do not acknowledge the role of information reduction along theabstraction process.

Considering abstraction as a process raises two important issues. The first isto investigate whether the process has a preferential direction, and whether itis reversible. The second issue is the identification of the abstraction processes.Concerning the first issue, we must remember that we have defined abstractionas an information reduction mechanism, whatever this means. A part of theworld, namely a system S, contains, in nuce, all the features and details thatcan possibly be detected. It is then necessary to decide what features of thesystem are to be considered and measured, and which ones are not. The resultof this selection is the most detailed description dg of the system that we decideto keep, and also the most informative one. We call dg ground description. Ifwe remove from dg some features, we obtain a less informative, and hence more5 See Section 2.1.6 See Section 2.3.

Path in an Abstraction Space

“Ilvero modo etordine perdissegnar tutte leparti iemembra delcorpo humano”,

[Fialetti ,1608]

5.1 Characteristic Aspects of Abstraction 127

Fig. 5.6: From Fialetti’s “Il vero modo et ordine per dissegnar tutte le parti ie membra delcorpo humano”, 1608. One among a set of studies for drawing eyes. ( c• This artwork maybe protected by copyright. It is printed in this book in accordance with fair use principles.)

By taking the top-leftmost and bottom right-most drawings in Fig. 5.6, itis really hard, without looking at the intermediate steps, to relate them in anymeaningful way. However, the relation between the two clearly appears if weconsider tho whole process of stepwise transformations.

Abstraction has been considered a process also in Mathematics, where theconcept of number is reached, according to Husserl, through a counting processleaving aside all properties of a set of objects, except their numerosity. Lewis[299] defines explicitly abstraction as a process of removing details from theconcrete5. Finally, Staub and Stern’s approach to abstraction6 mixes the ideaof abstraction as a process and abstraction as a relative notion, as we do; infact, these authors claim that concepts are obtained by reasoning, startingfrom the concrete world. Along the reasoning chain abstraction increases, sothat the farther from the concrete world a concept is along the chain, the moreabstract. As an example, real numbers are more abstract than integers. Eventhough this approach shares with our view the ideas of process and relativityof abstraction, we do not arrive at the same conclusions as Staub and Stern,because they do not acknowledge the role of information reduction along theabstraction process.

Considering abstraction as a process raises two important issues. The first isto investigate whether the process has a preferential direction, and whether itis reversible. The second issue is the identification of the abstraction processes.Concerning the first issue, we must remember that we have defined abstractionas an information reduction mechanism, whatever this means. A part of theworld, namely a system S, contains, in nuce, all the features and details thatcan possibly be detected. It is then necessary to decide what features of thesystem are to be considered and measured, and which ones are not. The resultof this selection is the most detailed description dg of the system that we decideto keep, and also the most informative one. We call dg ground description. Ifwe remove from dg some features, we obtain a less informative, and hence more5 See Section 2.1.6 See Section 2.3.

5.1 Characteristic Aspects of Abstraction 127

Fig. 5.6: From Fialetti’s “Il vero modo et ordine per dissegnar tutte le parti ie membra delcorpo humano”, 1608. One among a set of studies for drawing eyes. ( c• This artwork maybe protected by copyright. It is printed in this book in accordance with fair use principles.)

By taking the top-leftmost and bottom right-most drawings in Fig. 5.6, itis really hard, without looking at the intermediate steps, to relate them in anymeaningful way. However, the relation between the two clearly appears if weconsider tho whole process of stepwise transformations.

Abstraction has been considered a process also in Mathematics, where theconcept of number is reached, according to Husserl, through a counting processleaving aside all properties of a set of objects, except their numerosity. Lewis[299] defines explicitly abstraction as a process of removing details from theconcrete5. Finally, Staub and Stern’s approach to abstraction6 mixes the ideaof abstraction as a process and abstraction as a relative notion, as we do; infact, these authors claim that concepts are obtained by reasoning, startingfrom the concrete world. Along the reasoning chain abstraction increases, sothat the farther from the concrete world a concept is along the chain, the moreabstract. As an example, real numbers are more abstract than integers. Eventhough this approach shares with our view the ideas of process and relativityof abstraction, we do not arrive at the same conclusions as Staub and Stern,because they do not acknowledge the role of information reduction along theabstraction process.

Considering abstraction as a process raises two important issues. The first isto investigate whether the process has a preferential direction, and whether itis reversible. The second issue is the identification of the abstraction processes.Concerning the first issue, we must remember that we have defined abstractionas an information reduction mechanism, whatever this means. A part of theworld, namely a system S, contains, in nuce, all the features and details thatcan possibly be detected. It is then necessary to decide what features of thesystem are to be considered and measured, and which ones are not. The resultof this selection is the most detailed description dg of the system that we decideto keep, and also the most informative one. We call dg ground description. Ifwe remove from dg some features, we obtain a less informative, and hence more5 See Section 2.1.6 See Section 2.3.

Reversible path Intermediate steps can be recovered

Encapsulation

Q1 = “How many cars are there in the line ?”

Details of the cars are hidden (they are irrelevant to the question)

Q2 = “How many red cars are there in the line ?”

Details of the cars must be recoverable

Information is not lost, in the abstraction process, but only hidden, shielded from the outside view

Abstraction OperatorsAll the operations described can be defined in terms of

Abstraction Operators

Input = I Output = OO = ω(I,θ)Goldstone and Barsalou [1998]

Giunchiglia and Walsh [1992]

[Korf, 80]

Goldstone and Barsalou [1998]

Giunchiglia and Walsh [1992]

[Korf, 80]

img2 = Thresholding(img1,τ)

Operators are implemented via some algorithms

Signals vs Symbols

• Abstraction process always moves from richer, “low-level” descriptions (concepts) toward “high-level” ones. During the process, only (hopefully) information irrelevant to the current task is removed from view.

• In particular, abstraction establishes a link between signals (at the lowest end of the spectrum) and symbols (at the highest end of the spectrum)

• Abstraction acts as a bridge between perceptual processing and symbolic thinking. It tames the complexity of the sensory input, keeps the important information, builds up intermediate concepts to reduce reasoning complexity, and provides us with a re-organized view of the input, ready to be interpreted in the light of our world model.

• We humans do not ascribe symbolic features to the sensory world, but we receive raw inputs (images, sounds, …). On the other hand, we do not interact with sets of weights, but with high level concepts and symbols

• Abstraction allows the early conflict in AI between numerical and symbolic approaches to be overcome. Both signals and symbols become necessary and cooperating aspects of both natural and artificial thinking.

Acquiring AbstractionsHow are abstract operators acquired?We do not know how we humans do it

Three levels of increasing difficulty

1. Given a set of predefined abstraction operators, we want to choose the best suited to a given situation• Usual approach in Machine Learning, Constraint Satisfaction problems,

Planning, Search, Problem Solving, ….

2. Learning an abstraction operator itself• Some approach in Model-based Diagnosis, Constraint Satisfaction, Planning,

Machine Learning, …• Deep learning

3. Meta-learning how an abstraction operator can be learned• This is for the future

Deep LearningDeep Learning

Data

acquisitionLearning

x f(X)

Feature

transfo

rm

atio

n

Feature

transfo

rm

atio

n

Feature

transfo

rm

atio

n

x z1

Classifierf(X)z

2

zh

…....

Intermediate representations

with increasing level of

abstraction and meaningfulness

(hopefully)

Learning Object Parts while Classifying

Groups of pixels (motifs) that occur frequently are memorized as features and reused in various parts of an imageExamples of learned object parts from object categories

Learning object partsFaces Cars Elephants Chairs

Classifier

Parts of objects

Segments

More complete parts of objects

Raw inputω1

ω3

ω2

Local Receptive Fields

• Each feature can connect only to a small region of the lower layer• (reduction in complexity) • Similar regions are merged (they share the weights)• The same features can be detected at different positions in the input

image • Reduction in the number of free parameters

How can intermediate features be created?

Pooling

• Goal: Robust to local distortion

• Approach: Group similar features together

to achieve invariance

Aggregation operator

Equivalence operator

An Architecture for Abstraction• Given a raw input (image, music,

written text, …) consisting of elementary signals (pixels, sonds, characters, ...), there are infinite ways of forming sequences of intermediate features. Deep learning uses the output of classification to select useful features (abstraction operators, in our language). The result is task-specific.

• We need a more general guiding principle:

The best abstractions are those tha are useful in the greater number of different tasks

Head

Torso

Arm

Human

Evolution• Multitasks Approach

• It is not possible to handle a large number of tasks at the same time

• A temporal dimension (evolution) has to be added• A storage to keep the history of learning is necessary

LTM = Repository of confirmed abstraction operators and new concepts

DLkI O

ωk1, …, ωkn

STM ω

ωi

I ODLh

ωh1, …, ωhm

LTM

ωj

ωk

Positive or negative reinforcement signal