A Hierarchy of Qualitative Representations for Space

Abstract

Research in Qualitative Reasoning builds and uses dis-crete symbolic models of the continuous world . In-ference methods such as qualitative simulation aregrounded in the theory of ordinary differential equa-tions . We argue here that cognitive mapping - build-ing and using symbolic models of the large-scale spa-tial environment - is a highly appropriate domain forqualitative reasoning research .We describe the Spatial Semantic Hierarchy (SSH), aset of distinct representations for space, each with itsown ontology, each with its own mathematical foun-dation, and each abstracted from the levels below it .At the control level, the robot and its environmentare modeled as a continuous dynamical system, whosestable equilibrium points are abstracted to a discreteset of "distinctive states ." Trajectories linking thesestates can be abstracted to actions, giving a discretecausal graph level of representation for the state space .Depending on the properties of the actions, the causalgraph can be deterministic or stochastic . The causalgraph of states and actions can in turn be abstractedto a topological network of places and paths . Localmetrical models, such as occupancy grids, of neigh-borhoods of places and paths can then be built on theframework of the topological network while avoidingtheir usual problems of global consistency.This paper gives an overview of the SSH, describesthe kinds of guarantees that the representation cansupport, and gives examples from two different robotimplementations . We conclude with a brief discussionof the relation between the concepts of "distinctivestate" and "landmark value ."

The Spatial Semantic HierarchyBuilding on recent progress in robot exploration andmap-building, we propose an ontological hierarchy ofrepresentations for knowledge of large-scale space .

'This work has taken place in the Qualitative ReasoningGroup at the Artificial Intelligence Laboratory, The Uni-versity of Texas at Austin . Research of the QualitativeReasoning Group is supported in part by NSF grants IRI-E~9216584and IRI-9504138, by NASA contract NCC 2-760,and by the Texas Advanced Research Program under grantno . 003658-242 .

Benjamin KuipersComputer Science DepartmentUniversity of Texas at AustinAustin, Texas 78712 USA

An ontological hierarchy shows how multiple repre-sentations for the same kind of knowledge can coexists .Each level of the hierarchy has its own ontology (theset of objects and relations it uses for describing theworld) and its own set of inference and problem-solvingmethods . The objects, relations, and assumptions re-quired by each level are provided by those below it .The dependencies among levels in the hierarchy help

clarify which combinations of representations are co-herent, and which states of incomplete knowledge aremeaningful .

In this paper, we formalize the computational modelof the cognitive map as developed by Kuipers andhis students (Kuipers 1978 ; Kuipers & Byun 1988 ;1991) . That theory was motivated by two insights fromobservations of human spatial reasoning skills and thecharacteristic stages of child development (Lynch 1960 ;Piaget & Inhelder 1967 ; Hart & Moore 1973) . First,a topological description of the environment is centralto the cognitive map, and is logically prior to the met-rical description . Second, the spatial representation isgrounded in the sensorimotor interaction between theagent and the environment .The Spatial Semantic Hierarchy (SSH) (Kuipers &

Levitt 1988) abstracts the structure of an agent's spa-tial knowledge in a way that is relatively indepen-dent of its sensorimotor apparatus and the environ-ment within which it moves. The following informallydescribes the knowledge at the different SSH levels,which will be described more formally below .

The sensorimotor system of the robot provides con-tinuous sensors and effectors, but no direct accessto the global structure of the environment, or therobot's position or orientation within it .

At the control level of the hierarchy, the ontology isan egocentric sensorimotor one, without knowledgeof fixed objects or places in an external environment .A distinctive state is defined as the local maximumfound by a hill-climbing control strategy, climbingthe gradient of a selected sensory feature, or dis-tinctiveness measure . Trajectory-following controllaws take the robot from one distinctive state to theneighborhood of the next, where hill-climbing can

Kuipers

find a local maximum, reducing position error andpreventing its accumulation .

" At the causal level of the hierarchy, the ontology con-sists of views, which describe the sensory images atdistinctive states, and actions, which represent tra-jectories of control laws by which the robot movesfrom one view to another . A causal graph of asso-ciations (V, A, V') among views, actions, and result-ing views represents both declarative and imperativeknowledge of routes or action procedures .

" At the topological level of the hierarchy, the ontologyconsists of places, paths, and regions, with connec-tivity and containment relations . Relations amongthe distinctive states and trajectories defined by thecontrol level, and among their summaries as viewsand actions at the causal level, are effectively de-scribed by the topological network . This networkcan be used to guide exploration of new environ-ments and to solve new route-finding problems . Us-ing the network representation, navigation amongdistinctive states is not dependent on the accuracy,or even the existence, of metrical knowledge of theenvironment .

" At the metrical level of the hierarchy, the ontologyfor places, paths, and sensory features is extended toinclude metrical properties such as distance, direc-tion, shape, etc . Geometrical features are extractedfrom sensory input, and represented as annotationson the places and paths of the topological network .Two fundamental ontological distinctions are em-

bedded in the SSH . First, the continuous world of thesensorimotor and control levels is abstracted to the dis-crete symbolic representation at the causal and topo-logical levels, to which the metrical level adds contin-uous properties . Second, the egocentric world of thesensorimotor, control, and causal levels is abstractedto the world-centered ontologies of the topological andmetrical levels .

Formalizing the levels of the hierarchy draws on dif-ferent bodies of relevant theory : the sensorimotor andcontrol levels on control theory and dynamcal systems ;the causal level on logic and stochastic transition mod-els ; the topological level on logic and simple topology ;the geometrical level on estimation theory and differ-ential geometry .The Spatial Semantic Hierarchy approach contrasts

with more traditional methods, which place geomet-rical sensor intepretation (the most expensive anderror-prone step) on the critical path prior to creationof the topological map (Chatila & Laumond 1985 ;Moravec & Elfes 1985) . The SSH is consistent with,but more specific than, Brooks' (1986) subsumptionarchitecture, particularly levels 2 and 3 .The SSH representational framework has been im-

plemented on several different simulated and physi-cal robots . Figure 1 (modified from (Kuipers & Byun1991)) shows how the control level definition of states

11 4 QR-96

and trajectories grounds the topological description ofplaces and paths, which in turn supports explorationand planning while more expensive sensor fusion meth-ods accumulate metrical information . When metricalinformation is available, it can be used to optimizetravel plans or to disambiguate apparently identicalplaces, but when it is absent navigation and explo-ration remain possible . Figure 2 demonstrates a frag-ment of behavior of an RWI B12 robot, using a ring of12 sonar sensors, as it follows control laws and identi-fies a distinctive place in the indoor office environment .

The Sensorimotor LevelThe robot has an objective location in the environ-ment, but it does not have direct access to a representa-tion of that location in an absolute frame of reference .Assume that the environment is two-dimensional, sothat the state of the robot has three dimensions : posi-tion (x, y) and orientation B . The vector of state vari-ables is x = [x, y, B]T . The robot also has a memory Mincluding symbolic descriptions of goals, beliefs, etc .,which can influence the choice of control law, hencebehavior .The robot has a vector of sensors providing input

s = [so . . . . s�_1]T and a vector of motor outputs u =1110 . . . . uk_1]T by which it can change its position inthe environment .The sensor values are a function of the robot's state,

ISO' . . ., s� _ 1 ]T = s = T (x) = T(x, y, 9) .

(1)All variables are piece-wise continuous functions oftime . This model treats the environment as static,with the only changes being to the robot's positionand orientation .The "physics of the environment" (or dynamics of

the robot),

[ .i, y, 9 ]T = x = 0(x, 11) = ~D(x, y, 9, uo, . . . uk_1)

(2)specifies how the state, and hence the sensory values,change with time as a function of the current state andthe motor outputs . The robot does not have directaccess to its state variables, but only to the sensoryinformation s(t) provided to it as it moves through theenvironment .

The Control LevelThe purpose of the SSH control level is to select andexecute control laws for travel through the environ-ment . During exploration, locally well-behaved fea-tures of the sensory input are identified and used toconstruct suitable control laws . During travel througha known environment, control laws are retrieved fromthe causal level of the cognitive map .During a particular segment i of reactive behavior,

the robot moves through the environment by settingits motor vector in response to its sensory inputs, ac-cording to a control law Xi .

[110, . . ., uk-1] = u = Xi(s) = Xi(so . . . . s,~_1)

;i

P5(Symm-

E4(Midlin

P4(Temp-Di

E3(Left-W.

P3(Symm-E

r'v;-Jn -..1

F h '.~ i

._ . .

. .

E23(Right-Wall)

E2(Left-Wall)El(Left-Wall) j

P2(Symm-Eq-2) "- P1(Temp-Disc)(b)

E5(Midline)

oh. - .

._ . . ~

!

P6(Symm-Eq-3)

E6(Midline)

P7(Temp-Disc)

E7(Left-Wall)

Figure 1 : Simulated NX robot applies SSH explorationand mapping strategy .

(a) The simulated NX robot uses range-sensors toexplore and map an environment (Kuipers & Byun1991} . The exploration and control strategies identifyrandom and systematic sensor errors, and thus providerobustness. (b) The topological map (fragment) iden-tifies places and paths with the distinctiveness mea-sures that define them (e .g ., equidistance from nearbyobstacles, discontinuous sensor changes), and repre-sents their connectivity relations . (c) The metricalmap consists of annotations on each place and path,and can be relaxed into a global 2D frame ofreference .

,vvv,r~wv11111~'lls~'~Ji

Figure 2 : Spot, a physical robot, applying SSH controlstrategies .

Spot moves along a right wall, identifies a distinctiveplace, turns left, and begins to follow the next wall .This figure plots the position of the robot and thesingle most relevant sonar reading, on a map of thecomdor it is exploring .

Although X ; is purely reactive (i .e ., determined bys), its selection depends both on the currently per-ceived environment, and on goals and other aspects ofthe robot's state not described at the control level . Forexample, sitting at an intersection, the robot's goalsdetermine whether to invoke a control law for a left orright turn, or to continue straight . For a given choiceof control law X;, equations (1), (2), and (3) define adynamical system that describes the behavior of therobot interacting with its environment .

Distinctiveness Measures

A critical step in our approach is the identification of adiscrete set of locally distinctive states within a contin-uous state-space . A locally distinctive state can be de-fined in terms of the behavior of a control law if we canidentify a continuous distinctiveness measure with anisolated locaa maximum in the current neighborhood .A distinctiveness measure (or "d-measure") d is

a continuous function d{s) -~ ~ . The set D ={d o , . . . d�,_ 1 } of distinctiveness measures depends onthe environment and sensorimotor system of the par-ticular robot . A d-measure can be used to define apoint-like distinctive state, such as the state equidis-tant from three obstacles and oriented midway betweentwo of them; or a path-like trajectory, such as the mid-line of a corridor . Pierce and Kuipers (1994) show howd-measures and control laws can be learned from un-guided experience .Each d-measure d has an appropriateness measure

ad(s) -~ [0,1] that specifies the degree to which d isuseful for control . a d need not be continuous, and itmay depend on goals or other aspects of the robot'sstate, as well as the sensory input stream s(t) to therobot . It is sometimes useful to think of a d-measured as having a prerequisite ~d(s) which must be truefor d to be defined . This can be subsumed by theappropriateness measure : ad(s) - ad(s) > e, for someuser-specified c >_ 0 .A neighborhood nbd(d) of the distinctiveness measure

d is a connected subset of the set of states where na{s)

Kuipers 115

is true . That is, a given distinctiveness measure mayhave several disconnected neighborhoods in differentparts of the environment .

Local Control LawsNavigation at the control level is an alternation be-tween two different types of control laws : hill-climbingcontrol laws to reach a nearby local maximum of a d-measure, and trajectory-following control laws to movefrom one part of the state space to another . One wayto express these is through a simple but general localcontrol law associated with a given d-measure d, spec-ifying a direction of change in the state space of therobot :

z = [x, y, B]T = k1 17d+ k2Nd

(4)where Vd is the gradient of d in the state space, andNd is a unit vector orthogonal to Od.

Since more than one d-measure d E D may be appro-priate during a given trajectory, we take the weightedaverage in the spirit of heterogeneous control (Kuipers& Astrom 1994) :

ic = EdED ad(t) [k 10d + k2 Nd]

(5)

EdeD ad(t)

where ad(t) is an appropriateness measure for d. Whend is not meaningful, ad(t) = 0 . Note that as the robotmoves, the effective number of participating local con-trol laws may change .

Other compositional approaches to control includepotential field methods (Arkin 1989 ; Slack 1993) andfuzzy control (Mamdani 1974 ; Kosko 1992) . Ap-propriateness measures and other parameters of thecontrol laws Xi may be acquired and optimized byfunction-learning methods including neural nets (e.g .(Pomerleau 1993)) and memory-based learning (Atke-son, Moore, & Schaal 1996 ; Moore, Atkeson, & Schaal1996) .Hill-Climbing . Starting in the state where a

trajectory-following control law terminates, identifythe applicable hill-climbing d-measure(s) . For a hill-climbing control law of the form (4), the k1 Vd termpoints the robot toward the local maximum, andk2 = 0. A distinctive state (x, y, 0) is the state ofthe robot when a hill-climbing control law termi-nates ; i .e ., when is = Vd = 0.

Trajectory-Following . Starting at a locally distinc-tive state, select and obey a trajectory-following con-trol law (or a sequence of local control laws) until itterminates . For a trajectory-following control lawof the form (4), the term k10d keeps the robot onthe desired trajectory, and k2Nd moves it along thetrajectory in one direction or the other dependingon the sign of k2 . A trajectory-following control lawterminates when is changes discontinuously (or veryquickly) . In equation (5) this would happen whensome ad(t) suddenly becomes zero while the corre-sponding control action [k, 17d + k2Nd] is non-zero .

116 QR-96

For example, a robot starts at one end of a corridor,facing "open space ." It takes a trajectory consisting ofopen-loop motion into the corridor it faces, then fol-lowing the midline to the end of the corridor . Uponreaching the end, the robot does hill-climbing to posi-tion itself equidistant from nearby obstacles .

Putting Control into Action

The local control law (5) provides a desired directionof motion z in state space, which must be translatedinto values for the robot's motor output variables u .

In simple cases, the dynamics of the robot (equation(2)) will have a pseudo-inverse (D- 1 so that, given xand a desired z, we can directly compute

u = (D-1 (x, ir) such that is = (D(x, u) .

(6)

In general (i .e ., for a robot with non-holonomic mo-tion constraints), there may be no way to achieve adesired z for a given state x (cf . (Latombe 1991)) . Insuch a case, we specify the control goal as a net changeOx to be obtained over some period of time . Then weassume the ability to plan a sequence of continuous ac-tions (e .g ., (Penberthy & Weld 1994)), or to retrieve apreviously developed control plan :

p = plan(x, Ox), such that u = p(x, t)

(7)

has the desired effect of reaching the state x + Ox.Note that, as with parallel parking, the intermediatestates of the plan p may be farther from the goal thanthe initial or final states . Further extensions will be re-quired to cope with pedestrians and other unexpectedobstacles .

Equation (5), along with either (6) or (7), providesan instance of the control law Xi required by equa-tion (3) . Thus, the robot's behavior during a singlehill-climbing or trajectory-following segment consists ofthe state-evolution of a particular dynamical system .Higher-level symbolic reasoning intervenes at the jointsbetween these segments to determine which dynamicalsystem controls the behavior .

The Causal LevelWhen a sequence of control laws - trajectory-following then hill-climbing - reliably takes the robotfrom one distinctive state to another, we abstract thesequence of control laws to an action A, and the twodistinctive states to the sensory images, or views, Vand V, obtained there . Their association is repre-sented by the schema (V, A, V') .When this abstraction can be applied across the en-

vironment, the continuous state space in which therobot is described as following the trajectories of a dy-namical system is abstracted to a discrete state spacein which the robot is described as performing a se-quence of discrete actions .

Views, Actions, and SchemasA view is a description of the sensory input vectors(t) = [si(t) . . . . s � (t)] obtained at a locally distinctivestate, (x, y, 0) . A view could be a complete snapshotof s(t), or it could be a partial description, consistentwith more than one value of s .An action denotes a sequence of one or more con-

trol laws which can be initiated at a locally distinctivestate, and terminates after a hill-climbing control lawwith the robot at. another distinctive state . A typ-ical action might consist of an open-loop trajectory-following control law to escape from the current neigh-borhood, then a closed-loop trajectory-following con-trol law to reach a new neighborhood, and finally a hill-climbing control law to reach a new distinctive state .A schema is a tuple (V A, V'), representing the

(temporally extended) event in which the robot takesa particular action A, starting with view V and termi-nating with view V' .

In the following, holds(V, so ) means that the view Vis observed in situation so ; do(A, so) means that actionA is initiated in situation so ; and result(A, s o ) denotesthe situation resulting after action A is initiated insituation so and terminates in a new distinctive state .The schema (V A, V') has two meanings :

declarative :

holds(V so) -> holds(V', result(A, so))imperative :

holds(V now) =:;, do(A, now) .

The declarative meaning is standard situation calculus(McCarthy & Hayes 1969) . The imperative meaning isintuitively clear, but not formalized .

Procedurally, in order for a complete schema(V, A, V') to be created from observations during be-havior, the partially filled schema (V A, nil) must bepreserved in working memory during the time requiredto carry out the action A to termination . In case ofinterruption, it may be that only the partial schemais stored in long-term memory . The partially filledschema (V, A, nil) lacks the declarative meaning of thecomplete schema, but has a restricted version of theimperative meaning :

Routines

imperative :

holds(V now) =~. do(A, now) .

A routine is a set of Schemas, indexed by initial view.It represents the sequence of actions and intermediateviews in a behavior that moves the robot from an initialto a final distinctive state . A routine can be used eitheras a description of the behavior, or as a procedure forreproducing it .

Consider the alternating sequence of views andactions Vo,Ao,V1,A,,Vz, . . .,Vn_1,An_i,Vn leadingfrom Vo to Vn ." A routine R is complete from view Vo to Vn if R

contains the schema (V;, Ai, Vi+1) for each i from 0ton-1 .

" A routine R is adequate from Vo to V if R containseither (l+, Ai, Vi+1) or (Vi , Ai, nil) for each i from 0ton-1 .An adequate routine supports "situated action" :

physical travel from state Vo to Vn within the envi-ronment (Agre & Chapman 1987) . It also general-izes naturally to causal graphs such as universal plans(Schoppers 1987), which are sets of rules specifying theactions to take at each state in a state-space to movetoward a given goal . In addition to situated action, acomplete routine supports cognitive operations such asmental review or verbal description of the route in theabsence of the environment .

The Topological LevelThe topological map describes the environment as acollection of places, paths, and regions, linked by topo-logical relations such as connectivity, order, contain-ment, boundary, and abstraction . Places, paths, andboundary regions are created from experience repre-sented as a sequence of views and actions . They arecreated by abduction, positing the minimal additionalset of places, paths, and regions required to explain thesequence of observed views and actions ." A place describes part of the robot's environment as

a zero-dimensional point . A place may lie on zeroor more paths . A place may also be defined as theabstraction of a region .

" A path describes part of the robot's environment,for example a street in a city, as a one-dimensionalsubspace . It may describe an order relation on theplaces it contains, and it may serve as a boundaryfor one or more regions . The two directions along apath are dir = -f-1 and dir = -1 .

" A region represents a two-dimensional subset of therobot's environment . The set of places in a regionshare a common property. A region may be definedby one or more boundaries, by a common frame ofreference, or by its use in an abstraction relation .

Co-occurrence Implies TopologicalConnectionsThe "current context" or "You-Are-Here pointer" de-scribes the current state of the explorer . The topo-logical level adds the current place, path, and 1-D di-rection to the current context . Simultaneous presenceof several descriptions in the current context implies atopological connection .

current-place (p) A current-view(v) -1 at(v, p)current-path(p) A current-direction(d) A

current-view(v) -+ along(v, p, d)current-place (p)

A

current-path(path)

-ion-path(p, path) .

The 1D topological order of places along the currentpath is inferred from a Travel action and the current

Kuipers

direction . We can also infer, or abduce, topologicalboundary and containment relations among regions,paths, and places .

Abduction to Places and Paths fromViews and ActionsThe definition of topological places is coupled with acategorization of actions into those that change thecurrent place, called Travel actions, and those thatleave the current place the same, called Turn actions .An action description includes a term representing theobserved magnitude of the corresponding control laws,from internal effort sensors such as odometry . Since anaction must begin and end at a locally distinctive state,not every magnitude of Turn or Travel is a meaningfulaction ." (V, (Turn a), V) means that place(V) = place(V)and a is monotonically related to the magnitude ofthe turn from B(V) to 8(V') . (place(V) denotes theplace where the robot observed V. Since places canhave identical views, it is not necessarily a functiononly of V .)

" (V, (Travel d), V)

means

thatplace(V) 0 place(W) if d :A 0, and d is monoton-ically related to the distance traveled from place(V)to place(W) .We use the following observations as the basis for

abduction of the connectivity properties of places andpaths, given sequences of views and actions . Sincedifferent places could provide the same sensory im-age, sophisticated inference and even physical travelmay occasionally be required to identify the currentplace from the current view (Kuipers & Byun 1988;Dudek et al. 1991) ." Every view is observed at a place .

bview 3place at(view, place)" A Turn leaves the traveller at the same place .

(V, (Turn a), V) -+ 3place [at (V, place) nat(V', place)](8)

" A Travel leaves the traveller on the same path, facingthe same direction . If the distance traveled is non-zero, the starting and ending places are different .(V, (Travel d), V') A d $ 0 -+3P1, P2 [PI t- P2 n at (V, pi) A at (W, P2)](V, (Travel d), V') --+3path, dir [along(V, path, dir) Aalong(V', path, dir)]The topological level supports an array of problem-

solving methods, augmenting graph search with heuris-tics based on the boundary and containment relations(not described here) that regions add to the topologicalmap .We have implemented a system that takes alternat-

ing sequences of views and actions from tours of a sim-ulated urban environment and builds causal, topologi-cal, and local 1-D metrical descriptions of the environ-ment .

118 QR-96

The Metrical LevelLocal 1-D GeometryObservations of the magnitudes of actions provideinformation about the local geometry of places andpaths . (V, (Travel d), V') provides evidence aboutthe distance between two places on the current path .(V, (Turn a), W) provides evidence about the anglebetween obstacles and/or paths at the current place .This information can be represented as 1-D (linear orcircular) metrical properties of the individual placesand paths in the topological map . These propertiesare accumulated incrementally by the same abductiveprocess that builds the topological map .

Local 2-D GeometryIf we take into account the fact that the topologicalmap is embedded in a 2-D space, we can incremen-tally accumulate local descriptions of place neighbor-hoods and path segments as 2-D manifolds . Occu-pancy grids (Moravec 1988 ; Konolige 1995), sonar tar-get maps (Leonard & Durrant-Whyte 1992 ; MacKenzie& Dudek 1994), and generalized cylinders (Nevatia &Binford 1977 ; Brooks 1981) are three representationsfor 2-D manifold descriptions of local place neighbor-hoods and path segments .

Global 2-D GeometryOnce the topological and local metrical descriptionsare sufficiently rich and reliable, these descriptions canbe relaxed into a global 2-D frame of reference (fig-ure 1(c)) . However, this representational transforma-tion is never on the critical path for exploration, map-learning, route-planning, or navigation .

GuaranteesA state s is localizable if, starting from s, there is areliable method for traveling to a distinctive state, andthus being localized within the topological map. Thelocalizable states are defined in terms of (a) the se-lection criteria for control laws, as embodied in theappropriateness measures ad(s), and (b) the basins ofattraction defined by those control laws, considered asdynamical systems . A state s is reachable if, startingat a localizable state, there is a reliable method forthe robot to travel to s . Using the framework definedabove, we can analyze which states in the physical en-vironment are localizable and/or reachable, giving var-ious levels of knowledge in the cognitive map .

DiscussionThe concept of "distinctive state" as used here for cog-nitive mapping appears to generalize certain aspects ofthe concept of "landmark value" as used for qualitativesimulation .Landmark values in QSIM corresponding to sign

changes, operating region transitions, or extremepoints in the behavior all represent individual real

numbers . Ordinal relations with these landmarks sup-port straight-forward qualitative hill-climbing, so theyare distinctive within their quantity spaces . The onlylandmark values with a different character are thosecorresponding to initial values, which represent uni-versally quantified variables ranging over a set definedby pre-existing landmark values .Both cognitive mapping and qualitative simulation

rely on abstracting a continuous underlying space to adiscrete set of objects with symbolic names and sym-bolic relationships . In spite of the differences betweenthe domains, I believe that the common structure willprove to be important.

ReferencesAgre, P. E ., and Chapman, D . 1987 . Pengi: An implemen-tation of a theory of activity. In Proc . 6th National Conf .on Artificial Intelligence (AAAI-87). Morgan Kaufmann .Arkin, R. C. 1989 . Motor schema-base mobile robotnavigation . International Journal of Robotics Research8(4) :92-112.Atkeson, C . G. ; Moore, A . W. ; and Schaal, S . 1996 . Lo-cally weighted learning . Artificial Intelligence Review . inpress.Basye, K . ; Dean, T. ; and Vitter, J . S. 1989 . Copingwith uncertainty in map learning . In Proc . 11th Int. JointConf. on Artificial Intelligence (IJCAI-89) .Brooks, R. A. 1981 . Symbolic reasoning among 3D modelsand 2D images . Artificial Intelligence 17:285-348 .Brooks, R. A. 1986 . A robust layered control system for amobile robot. IEEE Trans . on Robotics and AutomationRA-2(l) :14-23 .Brooks, R. A . 1990 . The behavior language ; user's guide.Technical Report Memo 1227, MIT Artificial IntelligenceLaboratory, Cambridge, MA.Cassandra, A. R. ; Kaelbling, L. P. ; and Littman, M. L.1994 . Acting optimally in partially observable stochasticdomains. In Proc . 12th National Conf. on Artificial In-telligence (AAAI-9l,), 1023-1028. Cambridge, MA: AAAIPress/The MIT Press .Chatila, R., and Laumond, J.-P. 1985 . Position referenc-ing and consistent world modeling for mobile robots . InIEEE International Conference on Robotics and Automa-tion, 138-170.Courtney, J . D., and Jain, A. K. 1994 . Mobile robot local-ization via classification of multisensor maps . In IEEE In-ternational Conference on Robotics and Automation, vol-ume 2, 1672-1678. Los Alamitos, CA : IEEE ComputerSociety Press.Dean, T. L., and Wellman, M. P. 1991 . Planning andControl. San Mateo, CA: Morgan Kaufmann .Dean, T. ; Basye, K . ; Chekaluk, R. ; Hyun, S . ; Lejter, M. ;and Randazza, M. 1990 . Coping with uncertainty in acontrol system for navigation and exploration . In Proc.8th National Conf. on Artificial Intelligence (AAAI-90).Cambridge, MA: AAAI Press.Dean, T. ; Angluin, D . ; Basye, K . ; Engelson, S . ; Kaelbling,L. ; Kokkevis, E. ; and Maron, O. 1992 . Inferring finite

automata with stochastic output functions and an appli-cation to map learning . In Proc . 10th National Conf. onArtificial Intelligence (AAAI-91). Cambridge, MA: AAAIPress.Dudek, G. ; Jenkin, M. ; Milios, E. ; and Wilkes, D. 1991 .Robotic exploration as graph construction . IEEE Trans.on Robotics and Automation 7(6) :859-865 .Engelson, S. P., and McDermott, D. V. 1992 . Maps con-sidered as adaptive planning resources . In AAAI Fall Sym-posium on Applications of AI to Real-World AutonomousMobile Robots, Working Notes.Froom, R. 1991 . Acquiring effective knowledge of envi-ronmental geometry for minimum-time control of a mo-bile robot. In Proceedings of the 1991 IEEE InternationalSymposium on Intelligent Control.Froom, R. 1992 . Approximate maps for high-speed controlof a mobile robot. In Mobile Robots VII: Proceedings ofSPIE - the International Society for Optical Engineeringv. 1831 .Gat, E . 1991 . Alfa : a language for programming reactiverobotic control systems. In IEEE International Confer-ence on Robotics and Automation .Gladwin, T. 1970 . East is a Big Bird : Navigation andLogic on Puluwat Atoll. Cambridge, MA: Harvard Uni-versity Press.Hart, R. A., and Moore, G . T. 1973 . The developmentof spatial cognition: A review . In Downs, R. M ., andStea, D., eds., Image and Environment. Chicago: AldinePublishing Company.Hutchins, E. L. 1995 . Cognition in the Wild. Cambridge,MA : MIT Press.Konolige, K . 1995 . A refined method for occupancy gridinterpretation . unpublished draft manuscript .Kortenkamp, D., and Weymouth, T. 1994 . Topologicalmapping for mobile robots using a combination of sonarand vision sensing. In Proc . 12th National Conf. on Arti-ficial Intelligence (AAAI-9!,), 979-984. AAAI Press/TheMIT Press.Kosko, B . 1992 . Neural Networks and Fuzzy Systems.Englewood Cliffs, NJ : Prentice-Hall .Kuipers, B . J., and Astr6m, K. 1994 . The compositionand validation of heterogeneous control laws . Automatica30(2):233-249 .Kuipers, B., and Byun, Y. T. 1988 . A robust qualitativemethod for spatial learning in unknown environments . InProc . 7th National Conf. on Artificial Intelligence (AAAI-88). Los Altos, CA: Morgan Kaufmann .Kuipers, B., and Byun, Y.-T. 1991 . A robot explo-ration and mapping strategy based on a semantic hier-archy of spatial representations . Journal of Robotics andAutonomous Systems 8:47-63 .Kuipers, B. J ., and Levitt, T. 1988 . Navigation andmapping in large scale space. AI Magazine 9(2):25-43 .Reprinted in Advances in Spatial Reasoning, Volume 2,Su-shing Chen (Ed.), Norwood NJ : Ablex Publishing,1990 .Kuipers, B. J . 1978 . Modeling spatial knowledge. Cogni-tive Science 2 :129-153 . Reprinted in Advances in SpatialReasoning, Volume 2, Su-shing Chen (Ed.), Norwood NJ :Ablex Publishing, 1990 .

Kuipers 119

Latombe, J .-C . 1991 . Robot Motion Planning . Boston :Kluwer Academic Publishers .Lazanas, A., and Latombe, J.-C . 1992 . Landmark-basedrobot navigation . In Proc . 10th National Conf. on Arti-ficial Intelligence (AAAI-92), 816-822 . Cambridge, MA:AAAI Press/The MIT Press .Leonard, J . J ., and Durrant-Whyte, H . F . 1992 . DirectedSonar Sensing for Mobile Robot Navigation . Boston :Kluwer Academic Publishers .Lynch, K. 1960 . The Image of the City . Cambridge, MA:MIT Press .MacKenzie, P ., and Dudek, G. 1994 . Precise positioningusing model-based maps . In IEEE International Confer-ence on Robotics and Automation, volume 2, 1615-1621 .Los Alamitos, CA: IEEE Computer Society Press .Mamdani, E . H . 1974 . Applications of fuzzy algorithmsfor control of a simple dynamic plant . Proc . IEE 121:1585-1588 .Mataric, M. J .

1992.

Integration of representationinto goal-driven behavior-based robots . IEEE Trans . onRobotics and Automation 8(3):304-312 .McCarthy, J ., and Hayes, P . J . 1969 . Some philosophicalproblems from the standpoint of artificial intelligence . InMeltzer, B., and Michie, D., eds ., Machine Intelligence 4 .Edinburgh : Edinburgh University Press . 463-502 .Moore, A . W., and Atkeson, C . G . 1993 . Memory-basedreinforcement learning : converging with less data and lessreal time . In Connell, J . H ., and Mahadevan, S ., eds .,Robot Learning. Kluwer Academic Publishers .Moore, A. W . ; Atkeson, C . G . ; and Schaal, S . 1996 .Memory-based learning for control . Artificial IntelligenceReview. to appear .Moravec, H., and Elfes, A. 1985 . High resolution mapsfrom wide angle sonar . In IEEE International Conferenceon Robotics and Automation, 116-121 .Moravec, H . P . 1988 . Sensor fusion in certainty grids formobile robots . AI Magazine .Nevatia, R., and Binford, T . O . 1977 . Descriptionand recognition of curved objects . Artificial Intelligence8(1):77-98 .Ohya, A. ; Nagashima, Y . ; and Yuta, S . 1994 . Exploringunknown environment and map construction using ultra-sonic sensing of normal direction of walls . In IEEE In-ternational Conference on Robotics and Automation, vol-ume 1, 485-492 . Los Alamitos, CA: IEEE Computer So-ciety Press .Penberthy, J . S ., and Weld, D . S . 1994 . Temporal planningwith continuous change . In Proc . 12th National Conf. onArtificial Intelligence (AAAI-94), 1010-1015 . Cambridge,MA : AAAI Press/The MIT Press .Piaget, J ., and Inhelder, B . 1967 . The Child's Conceptionof Space. New York : Norton . First published in French,1948 .Pierce, D., and Kuipers, B . 1994 . Learning to explore andbuild maps . In Proc . 12th National Conf. on ArtificialIntelligence (AAAI-94) . AAAI/MIT Press .Pomerleau, D . A . 1993 . Knowledge-based training of ar-tificial neural networks for autonomous robot driving . InConnell, J . H ., and Mahadevan, S., eds ., Robot Learning.Kluwer Academic Publishers .

120 QR-96

Schiele, B ., and Crowley, J . L . 1994 . A comparison ofposition estimation techniques using occupancy grids . InIEEE International Conference on Robotics and Automa-tion, volume 2,1628-1634 . Los Alamitos, CA: IEEE Com-puter Society Press .Schoppers, M. J . 1987 . Universal plans for reactive robotsin unpredictable environments . In Proc . 10th Int . JointConf . on Artificial Intelligence (IJCAI-87) . San Mateo,CA : Morgan Kaufmann .Simmons, R., and Koenig, S . 1995 . Probabilistic robotnavigation in partially observable environments . In Proc .14th Int . Joint Conf. on Artificial Intelligence (IJCAI-95), 1080-1087 . San Mateo, CA : Morgan Kaufmann .Slack, M. G . 1993 . Navigation templates : mediating qual-itative guidance and quantitative control in mobile robots .IEEE Trans . on Systems, Man and Cybernetics 23(2):452-466 .Watkins, C . J . C . H .

1989.

Learning from delayed re-wards . Ph.D . Dissertation, King's College, University ofCambridge .