Upload File

reasoning about space and robot perceptionpsantos/bremen2012.pdf · representation and reasoning:...

  • Home
  • Documents
  • Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular
110
Reasoning about Space and Robot Perception Paulo Santos FEI - São Paulo July 20, 2012 Paulo Santos ( FEI - São Paulo ) July 20, 2012 1 / 68

Upload: others

Post on 14-Jul-2020

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Reasoning about Space and Robot Perception

Paulo Santos

FEI - Satildeo Paulo

July 20 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 1 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 2 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 3 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 2: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 2 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 3 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 3: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 3 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 4: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 5: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 6: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 7: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Where is FEI

S Paulo SP FEI CampusFEI is the largest technological university in Brazil with over 8000 students

It has been a regional centre of technological development for theautomotive industry since the 50rsquos

It is now becoming a major centre for intelligent systems and robotics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 4 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 8: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 5 68

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 9: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

What I aim at

Cognitive Vision

Cognitive Robotics

Qualitative Spatial Reasoning

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 6 68

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 10: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

What is CogVisfrom ECvision network

A Cognitive Vision System can achieve the four levels of genericvisual functionality Detection Localisation RecognitionUnderstanding (role context purpose)and exhibits purposive goal-directed behaviour is adaptive tounforeseen changes and can anticipate the occurrence of objectsand events This is achieved through

I Learning semantic knowledge (form function and behaviours)I Retention of knowledge (about the cognitive system its environment

and the relationship with the environment)I Deliberation about objects and events including the cognitive system

itself

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 7 68

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 11: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

What is CogRob

Same thing as CogVis but with robots

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 8 68

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 12: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Qualitative Spatial Reasoning (QSR)

QSR = formalisation of spatial knowledge with primitive relationsamong spatial entities

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 9 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 13: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 10 68

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 14: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Protocol Learning Leeds UK

ldquo() learn descriptions of objects and events in an entirely autonomous way Our aim is zero human interference in the learningprocess and only to use non scene specific prior information The resulting models (object and protocol) are used to drive asynthetic agent that can interact in the real worldrdquo

References[1] P Santos D Magee A Cohn and D Hogg Combining multiple answers for learning mathematical structures from visual

observation In R Lopez de Mataras and L Saita editors Proc of ECAI pages 544ndash548 Valencia Spain 2004 IOS Press[2] C Needham P Santos D Magee V Devin D Hogg and AG Cohn Protocols from perceptual observations Artificial

Intelligence Journal 167103ndash136 2005[3] P Santos D Magee and C Needham Symbolically learned spatial attention Revista Controle e Automacao 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 11 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 15: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 12 68

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 16: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Assimilating knowledge from neuroimages inschizophrenia diagnostics

Neuroimage analysis

Statistical Pattern Recognition

A Multivariate Linear Framework Diagnostics

Spatial ontology

Combine it all

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 13 68

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 17: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Statistical Classifier

We use multivariate statistical classifiers to identify and analyse themost discriminating hyperplanes separating two image setsschizophrenia and control groups

Goal Analyse all features simultaneously (rather than by ROI)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 14 68

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 18: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Statistical Models

Enlargement of the ventricular systemAtrophy of the hippocampusCortical degeneration of the occipital parietal and frontal lobesEnlargement of the inter-hemispheric fissureAtrophy of the corpus callosum and hypothalamus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 15 68

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 19: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Statistical Models

(a) sagittal plane (b) coronal plane

Figure Effect size of the multivariate statistical differences comparing theintensity values described by the control and patient image models

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 16 68

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 20: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Evaluate the differences wrt t-tests (between models that are 3σ onboth sides) t = X1minusX2radic

σN1

+ σN2

Use snakes to single them out [4]

Map the findings on the Talairach Atlas

Build a space ontology for representing the neuroanatomicstructures[5]

Combine this ontology with meta-analises found in the literature(epistemological level)

Use Proteacutegeacute to navigate from the images to the ontology and backReferences

[4] PE Santos CE Thomaz D dos Santos R Freire JR Sato M Louzatilde P Sallet G Busatto and WF Gattaz Exploring theknowledge contained in neuroimages statistical discriminant analysis and automatic segmentation of the most significantchanges Artificial Intelligence in Medicine 49(2)105ndash15 2010

[5] P E Santos R Freire D N dos Santos C E Thomaz P C Sallet G F Busatto and A Cohn Qualitative Spatio-TemporalRepresentation and Reasoning Trends and Future Directions chapter A region-based ontology of the brain ventricular systemand its relation to schizophrenia IGI Publishing 2012

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 17 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 21: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 18 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 22: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 23: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 24: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 25: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Depth Profile Calculus

How much knowledge about a robotrsquos environment can be contructedfrom vision alone

How can we construct the knowledge about objects in the world fromsensor data of the robot

Construction of a qualitative spatial reasoning (QSR) system basedon sensor data

Use abduction for sensor data assimilation and deduction forpredictions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 19 68

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 26: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Simplified environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 20 68

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 27: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Spatial Reasoning about Robot Sensor Data

AttributesI Distance disparity sizeI Changes in the sensor data

RepresentationI Depth profiles and time pointsI Displacement between regionsI Mapping function between images and objects

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 21 68

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 28: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Depth Profiles

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 22 68

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 29: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Assimilating changes

Axioms of the system

lt Dynamic spatial rel gt larr lt desc sensor transition gt

lt Dynamic spatial rel gt larr lt obj minus obs relation gt

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 23 68

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 30: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Depth Profile Calculus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 24 68

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 31: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

DPC example

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 25 68

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 32: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

The model

Extract one horizontal depth profile of each scene from the visualdata

Objects in the scenes are represented as peaks

Axiomatise relations on the depth and size of these profiles as well asdisplacements [6 8]Embed the spatial relations as fluents in a Situation Calculus theory(in order to reason about perception and objectsrsquo motion)[7]

I with that we could prove sound and completeness of motion wrtperception

References[6] Paulo Santos Reasoning about depth and motion from an observerrsquos viewpoint Spatial Cognition and Computation

7(2)133ndash178 2007

[7] M Souchanski and P Santos Reasoning about dynamic depth profiles In Proc of the 18th European Conference on ArtificialIntelligence (ECAI) pages 30ndash34 Amsterdam The Netherlands The Netherlands 2008 IOS Press

[8] MV dos Santos R C de Brito H-H Park and P Santos Logic-based interpretation of geometrically observable changesoccurring in dynamic scenes Applied Intelligence 31(2)161ndash179 2009

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 26 68

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 33: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

in Bremen

Figure Rotunda scenario at Cartesium Building Un of BremenDevelop a 2-12D version of DPCReason about objectsactions in a seminar roomCollaboration with Mehul Bhatt Jakob Suchan and Arthita Ghosh

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 27 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 34: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 28 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 35: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 36: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Reasoning about Shadows in Robotics

Computer vision however has largely been filtering out cast shadowsas noise

Are they

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 29 68

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 37: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Illusory Motion from Shadows

(Loading)

Work on the perception of shadows (Kersten Mamassian Knill)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 30 68

ballBoxmov
Media File (videoquicktime)

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 38: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 39: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Reasoning about Shadows in Robotics

Making explicit the knowledge contained in cast shadows

Use it to reason about the robot environment

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 31 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 40: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 41: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Perception of shadows

ldquono luminous body ever sees the shadows that it generatesrdquo[da Vinci Notebooks of Leonardo Da Vinci Project Gutenberg (1888)]

From the light source viewpoint shadows are occluded by their casters

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 32 68

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 42: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Region Occlusion Calculus

MutuallyOccludesPO

NonOccludesEC

NonOccludesDC

TotallyOccludesTPPI

PartiallyOccludesPO1

PartiallyOccludesTPP

TotallyOccludesTPPI 1

TotallyOccludesEQ

PartiallyOccludesPO

TotallyOccludesNTPPI

TotallyOccludesNTPPI 1

PartiallyOccludesTPP1

MutuallyOccludesTPP 1

TotallyOccludesEQ

PartiallyOccludesNTPP

MutuallyOccludesEQ

MutuallyOccludesNTPP

MutuallyOccludesTPP

MutuallyOccludesNTPP1

1

1PartiallyOccludesNTPP

Figure ROC [9][9] D Randell M Witkowski and M Shanahan From images to bodies Modeling and exploiting spatial occlusion and motion

parallax In Proc of IJCAI pages 57ndash63 Seattle US 2001

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 33 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 43: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 44: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Perception of shadows

We model observer-caster-shadow within qualitative spatialreasoning ROC + an axiom about shadow

Shadow(s oScr L)harr PO(r(s) r(Scr))andTotallyOccludes(o s L)and

notexistoprimeTotallyOccludes(oprime o L)

ldquoa shadow is totally occluded by its caster from the lightsourceviewpointrdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 34 68

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 45: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Perception of shadows

With ROC + shadow axiom we proved a few theorems about1 ldquono shadow occludes its own casterrdquo2 ldquono shadow casts a shadow itselfrdquo3 ldquoif two shadows of distinct objects partially overlap then the objects will

be in a relation of occlusion wrt the light sourcerdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 35 68

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 46: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

and we can use it to do qualitative self-localisation

L

o2

3

4

5

4

3

2

11S

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 36 68

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 47: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

In practice

L

o2

3

4

5

4

3

2

11S

located(Region 1 ν o s)larr NonOccludesDC(o s v)

located(Region 2 ν o s)larr NonOccludesEC(o s v)

located(Region 3 ν o s)larr PartiallyOccludesPO(o s v)

located(Region 4 ν o s)larr TotallyOccludesTPPI(o s v)

located(Region 5 ν o s)larr TotallyOccludesNTPPI(o s v)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 37 68

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 48: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Qualitative regions for self-localisation

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 38 68

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 49: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

In practice

Qualitative robot self-localisation [10]

Relative depth from the observation of shadows

Threshold finding from qualitative regions [11]References

[10] Paulo Santos Hannah M Dee and Valquiria Fenelon Qualitative robot localisation using information from cast shadows InIEEE International Conference on Robotics and Automation pages 220ndash225 IEEE 2009

[11] V Fenelon P E Santos H M Dee and F Cozman Reasoning about shadows in a mobile robot environment undersubmission

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 39 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 50: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 40 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 51: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 52: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 53: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 54: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Puzzles and AI

Why puzzles could be interesting for AII Reasoning problems with small sizeI Synthetic domain we can focus on a studied feature forgetting

irrelevant factorsI Still they maintain enough complexity

Herbert Simonrsquos () analogy frequently used by John McCarthygames (puzzles) work as the drosophilas of AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 41 68

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 55: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Puzzles and AI

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 42 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 56: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 57: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 58: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Qualitative Spatial Reasoning (QSR)

Our final goal approximate Reasoning about Actions and Changeand AI planning to QSR

In this work we begin considering the formalisation an automatedsolution of a spatial puzzle involving strings and holes

Constraint we must find elaboration tolerant solutions

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 43 68

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 59: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Outline I

1 BackgroundWhere I am fromWhat I aim at

2 Robots Brain Shadows and StuffProtocol LearningAssimilating knowledge from neuroimagesReasoning about depthReasoning about Shadows in Robotics

3 Reasoning about Complex Spatial PuzzlesmotivationFormalising a puzzle

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 44 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 60: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 61: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

A spatial puzzle the Fishermanrsquos Folly

Initial state A (possible) goal stateit suffices with releasing the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 45 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 62: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 63: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 64: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 65: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Constraints

If you play with it yoursquoll soon find out that

Disks fixed to each string tip spheres can move along the string

The disks can pass through the post hole the spheres canrsquot

The disks cannot pass through the ring the spheres can

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 46 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 66: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 67: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Constraints

If you play with it yoursquoll soon find out that

The post base cannot pass through the ring

The sphere and the post cannot pass altogether through the ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 47 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 68: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 69: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 70: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 71: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 72: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

How does the solution look like

Representing the solution is crucial

The solution will be a plan ie a sequence of actions

What shall we consider as satisfactory solution

We are interested in a qualitative description of movementsIn the puzzlersquos brochure you can read things likeldquoFirst pass the disk through the post hole Then rdquo

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 48 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 73: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 74: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Holes

Holes are crucial There are 4 holes

Our first sort single-holed objects r s1 s2 ph

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 49 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 75: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 76: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Holes and their faces

Each hole h has two faces hminus h+

Given a face p minusp denotes the opposite faceminus(hminus) = h+ minus (h+) = hminus

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 50 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 77: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 78: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 79: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Long and regular objects

A second sort long objects = the string (str ) and the post (p)Knowing which holes they are crossing is relevant

Each long object l has two tips lminus l+

Third sort regular objects = disks d1 d2 and post base b

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 51 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 80: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 81: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Some (static) predicates

A tip can be linked to some objectlinked(strminus d1) linked(str+ d2) linked(p+ ph)

Some objects cannot_pass through some holes

cannot_pass(DS)cannot_pass(D r)cannot_pass(S ph)cannot_pass(SS)cannot_pass(pS)cannot_pass(phS)cannot_pass(rS)cannot_pass(b r)cannot_pass(b ph)cannot_pass(bS)cannot_pass(p ph)

for any disk D and sphere SPaulo Santos ( FEI - Satildeo Paulo ) July 20 2012 52 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 82: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 83: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(X ) = Y = list of crossings of long object X from Xminus to X+

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 84: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Schematic representation

The (w)hole picture could look like this

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

chain(str) = [s+1 ph+ s+2 ] chain(p) = [r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 53 68

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 85: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Passing a long object tip

Action pass_o(T P) = pass tip T to hole face P Examplepass_o(str+ phminus) should lead to chain(str) = [s+1 ph+ s+2 phminus]

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 54 68

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 86: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

The solution

chain(p) chain(str) next action(s) movesS0 [r+] [s+1 ph+ s+2 ] pass_o(str+ phminus)S1 [r+] [s+1 ph+ s+2 phminus] pass_o(p+ rminus) (1R)times 2

amp pass_h(ph rminus)S2 [ ] [s+1 r

minus ph+ r+s+2 r

minus phminus r+] pass_h(s2 rminus) (1L)S3 [ ] [s+1 r

minus ph+ s+2 phminus r+] pass_h(r ph+) (2R)+(2L)S4 [ ] [s+1 ph+ rminus s+2 r

+ phminus] pass_h(s2 r+) (1L)S5 [ ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 55 68

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 87: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

pass_o(str+ phminus)

Sphere1 Sphere2

PostH

Disk2Disk1

Base

Ring

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

[s+1 ph+ s+2 ] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 56 68

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 88: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

pass_o(p+ rminus)

Sphere1 Sphere2

PostH

Disk1

Base

Ring

Disk2

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ s+2 phminus] [s+1 rminus ph+ r+ s+2 r

minus phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 57 68

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 89: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

pass_h(s2 rminus)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ r+ s+2 r

minus phminus r+] [s+1 rminus ph+ s+2 phminus r+]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 58 68

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 90: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

pass_h(r ph+)

Sphere1

Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 rminus ph+ s+2 phminus r+] [s+1 ph+ rminus s+2 r

+ phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 59 68

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 91: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

pass_h(s2 r+)

Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base Sphere1 Sphere2

PostH

Disk1

Ring

Disk2

Base

[s+1 ph+ rminus s+2 r+ phminus] [s+1 ph+ s+2 phminus]

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 60 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 92: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 93: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 94: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 95: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Preliminary Prolog planner

Very primitive action theoryI Two actions pass_o and pass_hI Just one fluent chain(X ) (modified in all transitions)

Planner written in PrologI Simple handling of listsI Direct representation of predicates linked cannot_pass

What ifI A sphere could pass through the postI You could cut linksI Are there more solutions

Currently blind search heuristics can be interestingA commonsense heuristics try to avoid increasing complexity (morecrossings)

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 61 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 96: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 97: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 98: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 99: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 100: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 101: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

List-based solution

Not tolerant to elaborations (ie small elaborations in the domain donot necessarily imply small changes in the theory)

No possibility of proving correctness of the automated solutionsobtainedOverlooks formal aspects of the domain

I No proper ontology of the domainI Lacking the spatial properties of objectsI No rigorous definitions of actions and change (including indirect effects

of actions non-executable actions and the efficient representation ofactionsrsquo non-effects) - the frame problem

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 62 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 102: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 103: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 104: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 105: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Logic programming (with stable model semantics)formulation

Answer Set Programming (ASP) fashion

default negation provides a natural way for representing defaults (eglaw of inertia for solving the frame problem)

(logic) program rules provide a directional behaviour convenient forrepresenting causal effects (and for avoiding the ramification andqualification problems)

we use Situation Calculus as the underlying theory for actions andchange

Quantified Equilibrium Logic as underlying language that is a firstorder characterisation of stable model semantics

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 63 68

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 106: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Rope Ladder

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 64 68

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 107: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

On holes and strings

First attempt on holes+strings [12 13]

We consider planning with spatial knowledge and constraintsReferences

[12] P Cabalar and PE Santos Formalizing the fishermanrsquos folly puzzle Artificial Intelligence Journal 175(1)346ndash377 2011

[13] P Santos and P Cabalar The space within the Fishermanrsquos Folly playing with a puzzle in mereotopology Spatial Cognitionand Computation 8(1-2)47ndash62 2008

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 65 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 108: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 66 68

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 109: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Future (Dream) work

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 67 68

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle
Page 110: Reasoning about Space and Robot Perceptionpsantos/bremen2012.pdf · Representation and Reasoning: Trends and Future Directions, chapter A region-based ontology of the brain ventricular

Thanks

This author was partially supported by

FAPESP grant 201204089-3

CNPq Pq 2 grant 3033312011-9

Contact information

email psantosfeiedubr

webpage wwwfeiedubr~psantos

Paulo Santos ( FEI - Satildeo Paulo ) July 20 2012 68 68

  • Background
    • Where I am from
    • What I aim at
      • Robots Brain Shadows and Stuff
        • Protocol Learning
        • Assimilating knowledge from neuroimages
        • Reasoning about depth
        • Reasoning about Shadows in Robotics
          • Reasoning about Complex Spatial Puzzles
            • motivation
            • Formalising a puzzle

Ontology Training Examples. What does linked mean? Strategy serves retrieval, but not reasoning

RuleML 2015: Ontology Reasoning using Rules in an eHealth Context

Reasoning About Temporal Context Using Ontology and ...ebusiness.mit.edu/...ReasoningAboutTemporalContext.pdf · framework is suitable for reasoning on the Semantic Web. 1 Introduction

The Reasoning and Optimization Themeschmidt/COMP60332/2014-2015/reasoning_and...The Reasoning and Optimization Theme ... linear arithmetic ontology reasoning Applications: software/hardware

Ontology Reasoning for the Semantic Web and Its Application to Knowledge Graph

Ontology Engineering and Reasoning to Support Real World

Ontology Engineeringmkeet/OEbook/slides/L3DLOE19.pdfIntroduction Basic DL: ALC DL and FOL Reasoning services Summary Ontology Engineering Lecture 3: Description Logics Maria Keet email:

Using Ontology Reasoning to Classify Protein Phosphatases K.Wolstencroft, P.Lord, L.tabernero, A.brass, R.stevens University of Manchester

Ontology Reasoning with Deep Neural Networks - arxiv.org · us look into the problem of ontology reasoning with OWL RL. Ontology reasoning refers to a common scenario where the inference

Chapter4 Introducing Context and Reasoning in Visual Content … · Chapter4 Introducing Context and Reasoning in Visual Content Analysis: An Ontology-Based Framework Stamatia Dasiopoulou,

An ontology-based modelling and reasoning framework for

Large-scale Reasoning with a Complex Cultural Heritage Ontology (CIDOC CRM)

Commonsense Reasoning about Chemistry Experiments: Ontology and Representation

Semantic Web Part 4 The ontology layer 2: Reasoning in OWLrosati/semanticweb/slides5.pdf · Semantic Web Part 4 The ontology layer 2: Reasoning in OWL Riccardo Rosati Corso di laurea

The Reasoning and Optimization Themestaff.cs.manchester.ac.uk/~schmidt/COMP60332/2014-2015/reasoning...2 Why Optimization? 3 General practical remarks J ... linear arithmetic ontology

Ontology Summit 2017: AI, Learning, Reasoning, and Ontologies · 2/22/2017  · to further increase the effectiveness of ontology revision (Nikitina et al, 2012) Ontology Evolution:

A Novel Approach to Ontology ClassificationI · Ontology classification—the computation of the subsumption hierarchies for classes and properties—is a core reasoning service

Ontology-based Case-Based Reasoning (OntCBR) for Engineering … · Ontology-based Multi-layered Knowledge Framework [2006] Jae-Hyun Lee and Hyo-Won Suh, 2006, submitted "Ontology-based

Ontology-based reasoning techniques for multimedia ... · Ontology-based reasoning techniques for multimedia interpretation and retrieval Ralf Moller¨ 1 and Bernd Neumann2 1 Hamburg

The Reasoning and Optimization Theme - The …korovink/AR_O_intro_2014.pdfThe Reasoning and Optimization Theme ... linear arithmetic ontology reasoning Applications: software/hardware

A capability matching and ontology reasoning method for ...swe.whu.edu.cn/cnc_web/paper/9.pdfA capability matching and ontology reasoning method for high precision OGC web service

Integrating an Enterprise Architecture Ontology in a Case-based Reasoning Approach for Project Knowledge

Benchmarking Semantic Reasoning on Mobile …semantic-web-journal.net/system/files/swj1781.pdfspecific application scenarios, including reasoning task (e.g., ontology or rule-based

Ontology Based Context Modeling and Reasoning using OWL

Ontology Learning (Reasoning Web Summer School 2010)

Reasoning Performance Indicators for Ontology Design Patternsontolog.cim3.net/file/work/OntologySummit2014/2014... · Researching reuse in Ontology Engineering, speci cally ... measurement

Large-scale Reasoning with a Complex Cultural Heritage Ontology (CIDOC CRM) (paper)

A Novel Approach to Multimedia Ontology Engineering for Automated Reasoning over Audiovisual LOD Datasets

GoPubMed and beyond: Rules and reasoning for ontology-based literature search

Comparison of Ontology Reasoning Systems Using Custom Rules

spreading activation over ontology-based resources from personal context to web scale reasoning

Heterogeneous ontologies and hybrid reasoning for service ...Keywords: ontology, robotics, heterogeneity, hybrid reasoning 1 Introduction In this paper we address the challenge of

Ontology Reasoning for the Semantic Web and Its ...bj.bcebos.com/cips-upload/kg2/kg2_qgl.pdf · Ontology Reasoning for the Semantic Web and Its Application to Knowledge Graph

Ontology Reasoning V2008 - Department of Computer ... Reasoning...1 Ontology Reasoning Jon Atle Gulla Jon Atle Gulla Reasoning An ontology of this talk People Student Jie Bao present

Ontology Reasoning & Inconsistency · Ontologie-Management - 3.2.2009 - Ontology Reasoning & Inconsistency - Michael Prilop Zuerst: Beschränkung auf OWL-DL äquivalent zu SHOIN Beschreibungslogik