Zoe Demery
ᅠ
Image courtesy of James Russell
The Kakariki Modelexploration in parrots & children
What is exploration?
Burl, M. C., & Wetzler, P. G. (2011). Onboard object recognition for planetary exploration. Mach Learn, 84, 341–367. doi:10.1007/s10994-011-5239-6
What is exploration?
• Perceptual & motor interactions with objects
• To gather environmental information
Burl, M. C., & Wetzler, P. G. (2011). Onboard object recognition for planetary exploration. Mach Learn, 84, 341–367. doi:10.1007/s10994-011-5239-6
Why is exploration interesting?
Borst et al. (2009), Max-Planck Institute for Neurobiology: http://www.physorg.com/news168256071.html Perone, S., Madole, K. L., Ross-Sheehy, S., Carey, M., & Oakes, L. M. (2008). The relation between infants'™ activity with objects and attention to object appearance. Dev Psy, 44(5), 1242-8
Why is exploration interesting?
Borst et al. (2009), Max-Planck Institute for Neurobiology: http://www.physorg.com/news168256071.html Perone, S., Madole, K. L., Ross-Sheehy, S., Carey, M., & Oakes, L. M. (2008). The relation between infants'™ activity with objects and attention to object appearance. Dev Psy, 44(5), 1242-8
• Insight into underlying learning mechanisms
Why is exploration interesting?
• Little focus on how animals learn/process information
• Little systematic study or anthropocentric
• Just a random side effect?
Borst et al. (2009), Max-Planck Institute for Neurobiology: http://www.physorg.com/news168256071.html Perone, S., Madole, K. L., Ross-Sheehy, S., Carey, M., & Oakes, L. M. (2008). The relation between infants'™ activity with objects and attention to object appearance. Dev Psy, 44(5), 1242-8
• Insight into underlying learning mechanisms
Why is exploration interesting?
• Little focus on how animals learn/process information
• Little systematic study or anthropocentric
• Just a random side effect?
Borst et al. (2009), Max-Planck Institute for Neurobiology: http://www.physorg.com/news168256071.html Perone, S., Madole, K. L., Ross-Sheehy, S., Carey, M., & Oakes, L. M. (2008). The relation between infants'™ activity with objects and attention to object appearance. Dev Psy, 44(5), 1242-8
• Insight into underlying learning mechanisms
SELECTIVE, STRUCTURED & SENSITIVE
How do we study exploration?
Demery, Z. P., Chappell, J., & Martin, G. R. (2011). Vision, touch and object manipulation in Senegal parrots Poicephalus senegalus. Proc Roy Soc B, 278(1725), 3687–3693. doi:10.1098/rspb.2011.0374 Chappell, J., Demery, Z. P., Arriola-Rios, V., & Sloman, A. (2012). How to build an information gathering and processing system: Lessons from naturally and artificially intelligent systems. Behav Proc, 89(2), 179–186. doi:10.1016/j.beproc.2011.10.001
How do we study exploration?
Demery, Z. P., Chappell, J., & Martin, G. R. (2011). Vision, touch and object manipulation in Senegal parrots Poicephalus senegalus. Proc Roy Soc B, 278(1725), 3687–3693. doi:10.1098/rspb.2011.0374 Chappell, J., Demery, Z. P., Arriola-Rios, V., & Sloman, A. (2012). How to build an information gathering and processing system: Lessons from naturally and artificially intelligent systems. Behav Proc, 89(2), 179–186. doi:10.1016/j.beproc.2011.10.001
?
Why use kakariki as a model species?
ᅠ
What aspects of objects do kakariki
focus on during exploration?
all p < .01 (**)
Focus on corners
all p < .01 (**)
Focus on corners
Different surface transitions more likely to cue different object properties?
Focus on functional changes
Focus on functional changes
Basic knowledge about the properties of objects?
Sheet6
Page 1
1 2 3 4 50
5
10
15
20
25
30
35
40
0: rigid plastic cube 1: sand cube 2: hard sponge cube 3: soft sponge cube 4: sewn air cube
Trial number
Mea
n pr
opor
tion
of ti
me
spen
t exp
lori
ng e
ach
cube
in e
ach
tria
l
Focus changes with time
Sheet6
Page 1
1 2 3 4 50
5
10
15
20
25
30
35
40
0: rigid plastic cube 1: sand cube 2: hard sponge cube 3: soft sponge cube 4: sewn air cube
Trial number
Mea
n pr
opor
tion
of ti
me
spen
t exp
lori
ng e
ach
cube
in e
ach
tria
l
Focus changes with time
Start with the extremes, then gradually focus more on the intermediates?
Conclusions
Demery, Z., Rios, V. E. A., Sloman, A., Wyatt, J., & Chappell, J. (2010). Construct to understand: learning through Exploration. In Proceedings of the International Symposium on AI-Inspired Biology (pp. 59–61) Arriola-Rios, V., & Demery, Z. P. (submitted). Salient features and key frames: an interdisciplinary perspective on object representation. In Proceedings of AISB 2012.
• Exploratory behaviour can give us an insight how animals collect information
Conclusions
Demery, Z., Rios, V. E. A., Sloman, A., Wyatt, J., & Chappell, J. (2010). Construct to understand: learning through Exploration. In Proceedings of the International Symposium on AI-Inspired Biology (pp. 59–61) Arriola-Rios, V., & Demery, Z. P. (submitted). Salient features and key frames: an interdisciplinary perspective on object representation. In Proceedings of AISB 2012.
• Exploratory behaviour can give us an insight how animals collect information
• Multi-disciplinary approach needed to answer the question from different angles• We used kakariki as our model species, but also
children & robots• We looked at different levels of the question
(senses >> cognition >> behaviour)
Conclusions
Demery, Z., Rios, V. E. A., Sloman, A., Wyatt, J., & Chappell, J. (2010). Construct to understand: learning through Exploration. In Proceedings of the International Symposium on AI-Inspired Biology (pp. 59–61) Arriola-Rios, V., & Demery, Z. P. (submitted). Salient features and key frames: an interdisciplinary perspective on object representation. In Proceedings of AISB 2012.
• Exploratory behaviour can give us an insight how animals collect information
• Multi-disciplinary approach needed to answer the question from different angles• We used kakariki as our model species, but also
children & robots• We looked at different levels of the question
(senses >> cognition >> behaviour)• Exploration ‘strategies’ are structured,
selective & sensitive to particular environmental stimuli
Conclusions
Demery, Z., Rios, V. E. A., Sloman, A., Wyatt, J., & Chappell, J. (2010). Construct to understand: learning through Exploration. In Proceedings of the International Symposium on AI-Inspired Biology (pp. 59–61) Arriola-Rios, V., & Demery, Z. P. (submitted). Salient features and key frames: an interdisciplinary perspective on object representation. In Proceedings of AISB 2012.
Many thanks to...
- Jackie Chappell- Graham Martin- Jolyon Troscianko- Abi Phillips- Emma Tecwyn- Susannah Thorpe- Natural History Museum at Tring
- Aaron Sloman- Vero Rios- Nick Hawes- Jeremy Wyatt- Rustam Stolkin
- Sarah Beck- Ian Apperly- Alan Wing- Jack-in-the-Box Nursery
- Little Hippos Nursery- Nelson Junior & Infant School
www.vortices.com/zoe
www.vortices.com/zoe
ASAB Interdisciplinary Workshop 2012:
Physical Cognition & Problem-solving
June 27th - 28th 2012School of Biosciences
University of Birmingham
Invited speakers:
Dr. Joanna Bryson (Bath)
Prof. Ludwig Huber (Vienna) • Dr. Nicola McGuigan (Heriot-Watt)
Dr. Elva Robinson (York) • Prof. Murray Shanahan (Imperial)
Registration for this event is FREE
Abstract submission deadline: * May 11th 2012 *
Excellent networking opportunity for both post-graduate students and senior-level scientists working in the field of cognition
For further details please visit:http://jackiechappell.com/pcps-workshop -2012
Influence of the environment & the senses
Demery, Z. P., Chappell, J., & Martin, G. R. (2011). Vision, touch and object manipulation in Senegal parrots Poicephalus senegalus. Proc Roy Soc B, 278(1725), 3687–3693. doi:10.1098/rspb.2011.0374
Influence of the environment & the senses
Demery, Z. P., Chappell, J., & Martin, G. R. (2011). Vision, touch and object manipulation in Senegal parrots Poicephalus senegalus. Proc Roy Soc B, 278(1725), 3687–3693. doi:10.1098/rspb.2011.0374
Influence of the environment & the senses
Demery, Z. P., Chappell, J., & Martin, G. R. (2011). Vision, touch and object manipulation in Senegal parrots Poicephalus senegalus. Proc Roy Soc B, 278(1725), 3687–3693. doi:10.1098/rspb.2011.0374
A robot’s perspective on exploration
Colella, V., Klopfer, E., & Resnick, M. (2001). Adventures in Modeling: Exploring Complex, Dynamic Systems with StarLogo. Teachers College Press. Chappell, J., Demery, Z. P., Arriola-Rios, V., & Sloman, A. (2012). How to build an information gathering and processing system: Lessons from naturally and artificially intelligent systems. Behav Proc, 89(2), 179–186. doi:10.1016/j.beproc.2011.10.001 Demery, Z., Rios, V. E. A., Sloman, A., Wyatt, J., & Chappell, J. (2010). Construct to understand: learning through Exploration. In Proceedings of the International Symposium on AI-Inspired Biology (pp. 59–61) Arriola-Rios, V., & Demery, Z. P. (submitted). Salient features and key frames: an interdisciplinary perspective on object representation. In Proceedings of AISB 2012.
A robot’s perspective on exploration
Colella, V., Klopfer, E., & Resnick, M. (2001). Adventures in Modeling: Exploring Complex, Dynamic Systems with StarLogo. Teachers College Press. Chappell, J., Demery, Z. P., Arriola-Rios, V., & Sloman, A. (2012). How to build an information gathering and processing system: Lessons from naturally and artificially intelligent systems. Behav Proc, 89(2), 179–186. doi:10.1016/j.beproc.2011.10.001 Demery, Z., Rios, V. E. A., Sloman, A., Wyatt, J., & Chappell, J. (2010). Construct to understand: learning through Exploration. In Proceedings of the International Symposium on AI-Inspired Biology (pp. 59–61) Arriola-Rios, V., & Demery, Z. P. (submitted). Salient features and key frames: an interdisciplinary perspective on object representation. In Proceedings of AISB 2012.
Figure 1. Top view of the experiment where a robotic finger (sphere at thetop) pushes a dish-washing sponge, perpendicular to its widest axis, againsta pencil that serves as an obstacle (green cap). a) The contour of a deformeddish-washing sponge approximated by a series of splines, with the control
points placed by a human. b) The sponge instead represented by arectangular mesh. The original mesh was generated in the first frame when
the sponge was not deformed, whereas the mesh in this frame is theconfiguration predicted by the physics model. c) Hexagonal mesh, similar to
b).
do if an object becomes deformed to a shape unforeseen by the initialrepresentation?
3.2 REPRESENTING THE RELATEDENVIRONMENTAL PROCESSES
Once the agent has or can generate a representation for any shape ofthe object it may detect, the next step is to identify the key framesand unite them with appropriate functions.
We are not sure if our second behavioural experiment with thekakariki answers the question posed in the previous section, but itcertainly provides interesting issues to consider. We presented ourkakariki with five cubes of different deformabilities in a random or-der five times over different days. As we predicted in [7], they ini-tially explored the two extremes the most (i.e. the most rigid andthe most deformable cube), but their exploratory ’focus’ or ’strategy’changed. So in the second and third trial, the cube of the ’median’or intermediate deformability was explored significantly more thanall of the other cubes. Then in the final two trials, the cubes the nextinterval along (i.e. the second-most deformable cube and the second-most rigid cube) became more of a focus for the kakariki’ explo-ration. In conclusion, the exploratory strategy seems to change withtime, perhaps as more experience and gradually more specific knowl-edge is gained about the deformability of objects and different objectcategories.
Secondly, in the example of the dish-washing sponge, the follow-ing key frames can be identified:
1. The finger starts moving. At this point the force sensor does notdetect any relevant measure, but the command to move has beengiven and the vision (camera) begins to detect changes betweenframes, that is, that the position of the finger is changing. Thus,the first key frame would contain the finger, separated from thesponge, the sponge and the pencil.
2. The finger touches the sponge. At this point the force sensordetects an abrupt increase in one direction. On the side of vision,collision detection routines begin to detect a contact between thecircle that represents the finger, and one or two triangles in themesh that represents the sponge.
3. The finger stops moving. No more changes are detected.
Notice that these coarse key frames are the frames where thingschange in a very noticeable manner. It is possible to connect frames1 and 2 by using a function that describes the simple linear transla-tion of the circle (finger). Between frames 2 and 3, the same trans-lation function applies to the finger, but also the physics model getsactivated to deform the mesh as the finger pushes it; these two func-tions can predictively describe the observed movements. At frame3 no function or model is required anymore, because the executionof the command is over and there is no more movement. The scenehas ended. Seen in this way, the segmentation of the whole processinto smaller actions can be given by tracing back the activation anddeactivation of themechanisms required to generate the internal rep-resentation of each frame. Now each segment can be re-representedby a single symbol that makes reference to this. The whole sequencecan be described as something like:
1. Displace finger.2. Push sponge.3. Stop.
The agent can now choose between thinking of the command it ex-ecuted (e.g. translate), or the changes in the sponge (detected throughvision or touch), or combinations of both.
There are precedents to doing this type of segmentation, such as inthe work by [31], where the agent, Abigail, analyses a simple circle-and-sticks simulation of ping-pong games. Even for that highly sim-plified world, it was not trivial to unequivocally detect the points ofdiscontinuity that establish the beginning and end of an action. How-ever, Siskind was not using our concept of segmenting according themodel used for simulation. This concept is just an extension of theidea of a polynomial connecting two control points. Nonetheless,even though the use of splines to approximate curves is a widelyused technique, there is not a general technique that can automati-cally generate a spline from scratch to approximate any curve. Thestudy of the use of models to interpolate between frames, segmentand understand actions is a new field.
4 CONCLUSIONBy studying both artificial and natural agents, we can provide a fulleraccount of how an individual can most efficiently represent objectsand their affordances in their environment from the huge numberof sensory signals they receive. In this light, we must also considerwhat the requirements are posed by the external environment uponthe finite brain of the agent. Thus, we have briefly discussed twobehavioural experiments on parrot exploration of novel objects. Inconsidering natural behaviour and the possible underlying strategiesfor gathering information, we have described how a selection of keyelements from the environment can be used as a base for a representa-tion of objects. These key elements are connected through functions,which indicate how to obtain the value of other points. The samemechanism is used to represent actions, by identifying key frames,and finding the right physics model to interpolate between frames.By detecting the commands given, discontinuities in the behaviourof sensed signals, and the intervals of application of each model, it
A child’s perspective: the push-pull box
A child’s perspective: the push-pull box
• Are subtle changes in invisible functional cues (e.g. action, weight) for an object's properties attended to more than changes in non-functional visible cues (e.g. location, colour)?
• Is this attention even more pronounced for unexpected, within-type than expected between-type changes?
• Goal: get the ball out of the box
A child’s perspective: the push-pull box
• Are subtle changes in invisible functional cues (e.g. action, weight) for an object's properties attended to more than changes in non-functional visible cues (e.g. location, colour)?
• Is this attention even more pronounced for unexpected, within-type than expected between-type changes?
• Goal: get the ball out of the box
*
**
GLM: ** = p <.01 * = p < .05
Push-pull box: echoes kakariki data