Learning to Predict Trajectories of Cooperatively Navigating Agents

Henrik Kretzschmar Markus Kuderer Wolfram Burgard

Abstract— The problem of modeling the navigation behaviorof multiple interacting agents arises in different areas includingrobotics, computer graphics, and behavioral science. In thispaper, we present an approach to learn the composite nav-igation behavior of interacting agents from demonstrations.The decision process that ultimately leads to the observedcontinuous trajectories of the agents often also comprises discretedecisions, which partition the space of composite trajectoriesinto homotopy classes. Therefore, our method uses a mixtureprobability distribution that consists of a discrete distributionover the homotopy classes and continuous distributions over thecomposite trajectories for each homotopy class. Our approachlearns the model parameters of this distribution that match,in expectation, the observed behavior in terms of user-definedfeatures. To compute the feature expectations over the high-dimensional continuous distributions, we use HamiltonianMarkov chain Monte Carlo sampling. We exploit that thedistributions are highly structured due to physical constraintsand guide the sampling process to regions of high probability.We apply our approach to learning the behavior of pedestriansand demonstrate that it outperforms state-of-the-art methods.

I. INTRODUCTION

Accurate models of the navigation behavior of interactingagents are a prerequisite for a variety of applications in diversefields including robotics, computer graphics, and behavioralscience. For instance, mobile robots that have models ofpedestrian navigation behavior can predict the trajectories ofnearby pedestrians and thus can navigate among pedestriansin a safe, efficient, and socially compliant way [25, 21], or,alternatively, they can mimic the behavior of pedestrians [13].In computer graphics, Guy et al. [6] proposed such modelsfor generating realistic simulations of pedestrians and traffic.

We propose an approach to learn a probabilistic model ofthe navigation behavior of cooperatively navigating agentsfrom observations of their trajectories. The decision processthat ultimately leads to the observed trajectories in continuousstate spaces often comprises discrete decisions such aswhether to pass other agents on the left or on the rightside. Furthermore, the navigation behavior of agents suchas humans and animals is not a deterministic process. Theyrather exhibit stochastic properties, i. e., their trajectories varyfrom run to run when they repeatedly navigate in the samesituation. Our approach therefore learns a model that capturesboth the discrete decision process of the agents as well asthe distributions over their continuous trajectories to explainthe observed behavior. It reasons about composite trajectoriesof the agents, i.e., the joint behavior of all the agents, by

All authors are with the Department of Computer Science, University ofFreiburg, Germany. This work has partly been supported by the GermanResearch Foundation (DFG) under contract number SFB/TR-8, and by theEC under contract number FP7-610603.

demonstrations

fD =∑

x∈D f(x)

|D|

P (ψ)

pψ1(x)

. . .pψn

(x)

samples

Ep[f ] = fDψ1 ψn

Fig. 1. Our method learns a model of the behavior of cooperativelynavigating agents from demonstrations. We learn the model parameters of amixture distribution over composite trajectories to capture the discrete andcontinuous aspects of the agents’ behavior. The learned model generalizes tonew situations and allows us to draw samples that capture the stochasticityof natural navigation behavior.

means of a mixture distribution. As illustrated in Fig. 1, thisdistribution comprises a discrete distribution that captures thediscrete choices in the navigation process of the agents. Thedistribution furthermore consists of continuous distributionsover trajectories that capture the variance of the observedtrajectories. We propose a feature-based maximum entropylearning approach to infer the distribution that matches theobserved behavior of the agents in expectation.

A key challenge of such a learning approach is the so-called forward problem, i.e., computing for a given modelthe expected feature values with respect to the distributionover the high-dimensional space of continuous trajectories.We propose to use Markov chain Monte Carlo (MCMC)sampling and exploit that the distributions over observedtrajectories of interacting agents are highly structured. Theuse of a spline-based representation of the trajectories makes itpossible to efficiently compute the gradient of the probabilitydensity, which allows our method to guide the samplingprocess to regions of high probability using the Hybrid MonteCarlo (HMC) [5] algorithm. Therefore, our method is ableto capture the stochasticity of observed trajectories, whichis in contrast to existing approaches that learn deterministicmodels that do not replicate well the stochastic behavior ofnatural agents such as humans and animals.

We applied our method to learning a model of the navi-gation behavior of pedestrians. Our experimental evaluationsuggests that our approach outperforms three state-of-the-artmethods. Furthermore, a Turing test demonstrates that ourmethod generates trajectories that appear more human-likethan those created by previous methods.

II. RELATED WORK

Atkeson and Schaal [3] developed one of the pioneeringapproaches to infer a mapping from state features to actionsto directly mimic observed behavior. More recently, Ngand Russel applied inverse reinforcement learning (IRL) torecover a cost function that explains observed behavior [16].Later, Ratliff et al. [19], Ziebart et al. [25], Kitani et al.[12] used these techniques to address a variety of problemsincluding route planning for outdoor mobile robots andlearning pedestrian navigation behavior. In particular, Abbeeland Ng [1] suggest to match features that capture relevantaspects of the behavior that is to be imitated. However, featurematching does not lead to a unique cost function. To resolvethis ambiguity, Maximum Entropy IRL [24] relies on theprinciple of maximum entropy [10] and, hence, aims at findingthe policy with the highest entropy subject to feature matching.These learning methods work well in discrete state-actionspaces of low dimensionality. However, they do not scale wellto continuous trajectories, especially when taking into accounthigher-order dynamics such as velocities and accelerations.Inspired by Abbeel and Ng [1] and Ziebart et al. [24], ourapproach aims to find maximum entropy distributions thatmatch the observed feature values. However, in contrast tothe abovementioned techniques, our approach reasons aboutcontinuous trajectories including their higher-order dynamics.

To learn a model from observed trajectories, several authorsassume utility-optimizing agents that try to minimize aparametrized cost function that captures relevant proper-ties of their continuous trajectories. For instance, whereasHoogendoorn and Bovy [9] penalize accelerations, driftfrom planned trajectories, and closeness to other agents,Arechavaleta et al. [2] minimize the derivative of the curvatureto model pedestrian navigation behavior. In contrast to theseapproaches, we do not assume utility optimizing agents.Instead, we model the navigation behavior using a probabilitydistribution over the trajectories since the observed trajectoriesare rarely optimal but rather exhibit stochastic variations.

Our previous work [13, 14] models cooperative behaviorby jointly predicting the trajectories of all interacting agents.The approach to reciprocal collision avoidance presentedby van den Berg et al. [22] ensures that a set of agentsnavigates without collisions. Helbing and Molnar [8] learnthe parameters of a potential field to model interactionbehavior from observations. Trautman and Krause [21] modelthe coupled behavior of agents using interactive Gaussianprocesses. Lerner et al. [15] present a data-driven approachto modeling the behavior of crowds, where agents behaveaccording to examples that are stored in a database ofpreviously recorded crowd movements.

In general, estimating the feature expectations with respectto a high-dimensional probability distribution over contin-uous trajectories is a challenging problem. Our previousapproach [13] approximates the feature expectations usingDirac delta functions at the modes of the distribution. Thisapproximation, however, leads to suboptimal models whenlearning from imperfect human navigation behavior since

its stochasticity is not sufficiently captured. In practice, thismethod under-estimates the feature values and thus favorssamples from highly unlikely homotopy classes. Vernaza andBagnell [23] constrain the features to have a certain low-dimensional structure. Kalakrishnan et al. [11] assume thedemonstrations to be locally optimal and sample continuoustrajectories by adding Gaussian noise to the model parameters.In this paper, we estimate the feature expectations usingHybrid Monte Carlo sampling [5], which allows for arbitrarydifferentiable features of the continuous trajectories. Ratliffet al. [18] use Hybrid Monte Carlo ideas for globally robusttrajectory optimization. In summary, our method is capableof accurately modeling natural and potentially non-optimalreal-world behavior.

III. LEARNING TO PREDICT COMPOSITE TRAJECTORIESOF INTERACTING AGENTS FROM DEMONSTRATIONS

The objective of our approach is to learn a model of thenavigation behavior of interacting agents from samples oftheir trajectories. To this end, we consider the product spaceof the trajectories of all the agents ai ∈ A, i. e., the space ofcomposite trajectories x = (xa1(t), . . . , xan(t)), where xai(t)is the trajectory of agent ai that continuously maps time t tothe continuous configuration of agent ai ∈ A at time t. Weuse cubic splines in R2 to represent the x and y positions ofthe agents. We refer to the space of all composite trajectoriesas X . We assume that the behavior of the agents can bedescribed in terms of a joint probability distribution p(x)that depends on a feature vector f comprising features thatcapture relevant properties of the composite trajectories.

A. Maximum Entropy and Feature MatchingTo learn observed behavior, we aim to model the dis-

tribution that underlies the empirical sample trajectories.Following Abbeel and Ng [1], we aim to find a model thatinduces distributions that match, in expectation, the featurevalues fD of the empirical trajectories D, yielding

Ep(x)[f(x)] = fD =1

|D|∑xk∈D

f(xk). (1)

In general, however, there is not a unique distribution thatmatches the features. Ziebart et al. [24] resolve this ambiguityby applying the principle of maximum entropy [10], whichstates that the distribution with the highest entropy representsthe given information best since it does not favor anyparticular outcome besides the observed constraints. Theresulting distribution and the gradient of its parameters arewell known in information theory and have been used toinfer trajectories in discrete [25, 12] and continuous [13, 23]spaces. In short, our goal is to find the distribution p? thatmaximizes the differential entropy

H(p) =

∫x

−p(x) log p(x)dx (2)

subject to feature matching. Applying constrained optimiza-tion maximizes the Lagrangian

H(p)−∑i

θi(Ep[fi]− fi,D)− α(

∫x

p(x) dx− 1) (3)

Humans Our method Kuderer et al. RVO Social forces

Fig. 2. Trajectories of four pedestrians predicted by four different methods in the same situation. Humans: Observed trajectories recorded in the testenvironment shown in Fig. 3. Our method: Samples drawn from the policy learned by our method replicate the stochasticity of the observed trajectories.Kuderer et al. [13]: The Dirac approximation favors samples from highly unlikely homotopy classes. RVO and social forces: Deterministic predictions.

with respect to the distribution p and the Lagrangian multi-pliers [θ1, . . . , θn] = θ and α, where the last term ensuresthat the probability integrates to one. Taking derivatives withrespect to p reveals that the desired probability distributionbelongs to the exponential family pθ(x) ∝ e−θ

T f(x).One can interpret the term θT f(x) as a cost function

that comprises the feature weights θ, where trajectories withhigher cost are less likely. Computing the parameters θ?

that maximize the Lagrangian is analytically not feasible.However, we can use the gradient of the dual function withrespect to θ, which is given by fD − Ep[f(x)], to applygradient-based optimization. The resulting distribution p?

equals the exponential-family distribution that maximizes thelikelihood of the empirical data D. The derivation of thediscrete maximum entropy distribution follows the same idea,where integrals are substituted by sums.

B. Modeling the Navigation Behavior of Interacting AgentsUsing Mixture Distributions

The decision process of interacting agents that ultimatelyleads to their trajectories in continuous spaces often comprisescontinuous and discrete decisions. The continuous decisionsaffect physical properties of the composite trajectories suchas velocities, accelerations, and clearances when passingobstacles and other agents. The discrete decisions made bythe agents, such as choices to pass obstacles and other agentson the left or on the right side, determine the homotopy classof the resulting composite trajectories.

Our approach aims to learn a model of the behavior thatcaptures both the continuous and the discrete decisions of theagents. To this end, our method models the behavior usinga mixture distribution that comprises a discrete probabilitydistribution P (ψ) over the homotopy classes ψ ∈ Ψ of thecomposite trajectories, and continuous distributions pψ(x)over the composite trajectories x of the corresponding homo-topy classes ψ. Note that the number of homotopy classesincreases exponentially with the number of agents. However,techniques that only consider relevant homotopy classes [14]scale to many agents. We assume that the discrete mixtureproportions P (ψ) and the component densities pψ(x) dependon features f hom : Ψ 7→ R and f phys : X 7→ R, respectively,that capture relevant properties of the navigation behavior.According to the previous section, to learn the navigationbehavior, we aim to find a model that induces distributionsover the continuous composite trajectories as well as overhomotopy classes that match the feature expectations of the

observed composite trajectories D, which leads to

Epψ [f phys] = f physD =

∑x∈D

f phys(x)

|D|, (4)

EP [f hom] = f homD =

∑x∈D

f hom(ψx)

|D|, (5)

where ψx is the homotopy class of x. Applying the principleof maximum entropy subject to feature matching leads topψ(x) ∝ e−θ

physf phys(x) and P (ψ) ∝ e−θhomf hom(ψ).

To capture the homotopy class of a composite trajectoryx, we integrate the derivative of the angle αba(t) betweenthe vector xb(t)− xa(t) and the vector (1, 0)T over timefor all agents a and b, which leads to κba =

∫tαba(t) dt.

This function effectively accounts for the rotation of thevectors xb(t)− xa(t), which is an invariant of all thecomposite trajectories that belong to the same class.

The following features capture physical properties of thenavigation behavior of interacting agents in terms of an intentto reach the target positions energy efficiently, taking intoaccount velocities and clearances when avoiding obstaclesand other agents:

f phystime (x) =

∑a∈A

∫t

1 dt, (6)

f physacceleration(x) =

∑a∈A

∫t

‖xa(t)‖2 dt, (7)

f physvelocity(x) =

∑a∈A

∫t

‖xa(t)‖2 dt, (8)

f physobstacle(x) =

∑a∈A

∫t

1

‖xa(t)− oaclosest(t)‖2dt, (9)

f physdistance(x) =

∑a∈A

∑b 6=a

∫t

1

‖xa(t)− xb(t)‖2dt, (10)

where oaclosest is the position of the closest obstacle toagent a at time t. Our approach aims to find the continuousdistributions pθ(x) over the composite trajectories that, inexpectation, match these features. The feature

f homangle(ψ) =

∑a∈A

∑b6=a

κba. (11)

captures decisions to avoid other agents on the left or onthe right. Similarly, in case we are able to recognize groupmemberships of agents, the following feature indicates if anagent moves through such a group. An agent that passes two

Fig. 3. Trajectories observed during one hour of interactions of four personsin our test environment.

members of a group on different sides moves through thecorresponding group. Therefore, we have

f homgroup(ψ)=

∑a∈A|{G∈G | ∃b, c∈G :b, c 6=a ∧ κbaκca<0}|,

where G is the set of groups of agents, which allows ourapproach to learn to which extent the agents avoid to movethrough groups. Furthermore, we allow the features f hom todepend on the distribution over composite trajectories of thecorresponding homotopy class. For example, the feature

f homml cost(ψ) = min

x∈ψθT f(x) (12)

captures the cost of the most likely composite trajectory xof homotopy class ψ, which allows the model to reasonabout the homotopy class the agents choose in terms ofthe cost of the composite trajectory that is most likelyaccording to the learned distribution pψ(x). Based on theresults of our previous experiments [13], we assume thatwe can compute this most likely composite trajectory usinggradient based optimization techniques. We furthermore foundthe optimization algorithm RPROP [20] to perform best.After having learned the physical behavior in terms of thedistributions pψ(x), we can evaluate the features f hom tolearn the discrete aspects of the behavior.

C. Computing Feature Expectations with Respect to Distri-butions over Continuous Trajectories of High Dimensionality

In general, inference about distributions over continuoustrajectories requires a finite-dimensional representation of thetrajectories. However, even computing the feature expectationswith respect to a finite-dimensional representation is notanalytically tractable in general. Monte Carlo samplingmethods yet provide means to approximate the expectationsusing a set of sample trajectories drawn from the distribution.In particular, Markov chain Monte Carlo (MCMC) methodsallow to obtain samples from high-dimensional distributions.These methods aim to explore the state space by construct-ing a Markov chain whose equilibrium distribution is thetarget distribution. Most notably, the widely-used Metropolis-Hastings algorithm [7] generates a Markov chain in the statespace using a proposal distribution and a criterion to accept orreject the proposed steps. This proposal distribution and theresulting acceptance rate, however, have a dramatic effect onthe mixing time of the algorithm, e. g., the number of stepsafter which the distribution of the samples can be considered

to be close to the target distribution. In general, it is difficultto design a proposal distribution that leads to satisfactorymixing. As a result, efficient sampling from complex high-dimensional distributions is often not tractable in practice.

We exploit the structure of the distributions over compositetrajectories to enable efficient sampling. First, the navigationbehavior of physical agents shapes the trajectories accordingto certain properties such as smoothness and goal-directednavigation, which are captured by the features. As a result,the distributions over the composite trajectories of the samehomotopy class are highly peaked. Exploiting the gradientof the probability densities allows us to guide the samplingprocess towards these regions of high probability. To this end,assuming that the physical features are differentiable withrespect to the parameterization of the trajectories allows usto compute the gradient of the density pψ(x). In particular,we use the Hybrid Monte Carlo algorithm [5], which takesinto account the gradient of the density to sample from thedistributions pψ(x). The algorithm considers an extendedtarget density pψ(x,u) to simulate a fictitious physical system,where u ∈ Rn are auxiliary momentum variables. Themethod constructs a Markov chain by alternating Hamiltoniandynamical updates and updates of the auxiliary variables,utilizing the gradient of the density pψ(x) with respect to x.After performing a number of these “frog leaps”, HybridMonte Carlo relies on the Metropolis-Hastings algorithm toaccept the candidate samples x? and u? with probability

min(

1,pψ(x?)e−

12u

?Tu?

pψ(x(τ))e−12u

τTuτ

), (13)

where the normalizer Zp in the distribution pψ(x) =pψ(x)/Zp vanishes.

The resulting generative model allows sampling compositetrajectories from the mixture distribution by means ofancestral sampling [4], as illustrated in Fig. 1. In practice, weinitialize a Markov chain for each homotopy class at its mostlikely composite trajectory, which we can compute efficientlyusing gradient-based optimization. Being able to efficientlysample from the continuous distributions pψ(x) allows usto compute the feature expectations, which is necessary forlearning.

D. Inferring the Target Positions of the Agents

When using our model to predict the trajectories of theagents in new situations, the target positions of the agentsmight be unknown. Applying Bayes’ theorem allows ourmodel to reason about their target positions [24]. After havingobserved the agents traveling from the composite positions Ato B along the composite trajectory xA→B , the probabilitythat the agents proceed to composite target C is given by

Pθ(C | xA→B) ∝ pθ(xA→B | C)Pθ(C)

∝e−θ

T f(xA→B)∫x∈XB→C

e−θT f(x)dx∫

x∈XA→Ce−θ

T f(x)dxPθ(C), (14)

0 10 20 30 40 50 600

20

40

iteration

|fph

ys−

Ep[f

phys]|

0 10 20 30 40 50 600

5

10

15

iteration

|fho

m−

Ep[f

hom]|

Fig. 4. Evolution of the norm and variance over the five folds of thediscrepancy between the feature expectations of the model and the empiricalfeature values while learning pedestrian navigation behavior. Top: Learningthe physical properties of the trajectories. Bottom: Learning the discretedecisions that determine the homotopy classes of the composite trajectories.

where XA→C and XB→C refer to the set of all compositetrajectories that lead the agents from A to C, and from B to C,respectively.

IV. EXPERIMENTAL EVALUATION

We applied our approach to the problem of learning amodel of pedestrian behavior. We considered two datasetsof trajectories of interacting pedestrians. The first dataset,depicted in Fig. 3, comprises one hour of interactions offour persons that we recorded using a motion capture system,leading to 81 individual composite trajectories. To distractthe persons from the navigation task, we made them read andmemorize newspaper articles at different locations that wereconsecutively numbered. At a signal, they simultaneouslywalked to the subsequent positions, which repeatedly gave riseto situations where the participants had to evade each other.The second dataset [17] comprises 12 minutes of trajectoriesof pedestrians interacting in a hotel entrance, leading to 71composite trajectories with three to five pedestrians each.

A. Cross Validation

We conducted a five-fold cross validation on the afore-mentioned datasets to evaluate how well the models learnedby our approach generalize to new situations. We comparedour approach to the approach of Kuderer et al. [13], thesocial forces algorithm by Helbing and Molnar [8], and thereciprocal velocity obstacles (RVO) introduced by van denBerg et al. [22]. Fig. 2 shows example trajectories of thedifferent methods. We evaluated the evolution of the dis-crepancy between the feature expectations and the empiricalfeature values on the training sets while learning the modelparameters of our approach. Fig. 4 shows that our methodis able to replicate the observed behavior in terms of thefeatures. To allow for a fair comparison, we used the sameset of features for all the methods. To optimize the parametersof the social forces method and RVO, we minimized the normof the discrepancy between the feature values as induced bythe methods and the empirical feature values using stochastic

0

5

10

method

|EP

[f]−

f D|

0 1 2 3 40

0.5

1

1.5

∆t [s]

erro

r[m

]

Our approach Kuderer et al. Social Forces RVO

5

10

method

|EP

[f]−

f D|

0 1 2 3 40

0.5

1

1.5

∆t [s]

erro

r[m

]

(a) Dataset by Pellegrini et al. (b) Motion capture dataset

Fig. 5. Cross validation. The results suggest that our approach bettercaptures pedestrian navigation behavior in terms of the features (top) andthe prediction error in meters (bottom) compared to the approach of Kudereret al. [13], the social forces method [8], and reciprocal velocity obstacles [22].Left: Results on the dataset provided by Pellegrini et al. [17]. Right: Resultson the dataset recorded using our motion capture system.

gradient descend. We additionally evaluated the parametersprovided by Helbing and Molnar [8] and Guy et al. [6], whichturned out to not perform better than the learned parameters.For all methods, we assumed that the target positions of theagents were the positions last observed in the datasets. Theresults of the cross validation, depicted in Fig. 5, suggestthat our method is able to capture human behavior moreaccurately than the other methods in terms of features andin terms of the prediction error in Euclidean distances.

B. Turing Test

When using models of human navigation behavior tosimulate agents in the context of computer graphics, it isoften important that the generated trajectories appear human-like. We carried out a Turing test to evaluate how human-like the behavior generated by our approach compares toother methods. We asked ten human subjects to distinguishrecorded human behavior from behavior generated by oneof the algorithms. We evaluated how well the subjectsperformed on a set of runs that was randomly drawn fromrecorded human demonstrations. We showed them animationsof trajectories that were either recorded from the humandemonstrations or from the prediction of one of the algorithms.In particular, we presented 40 runs to each of the humansubjects, where the trajectories were equally drawn from thehuman demonstrations, from the predictions computed byour approach, by the approach of Kuderer et al. [13], andHelbing and Molnar [8]. Fig. 6 summarizes the results. Thehuman subjects correctly identified 79 % of all the humandemonstrations, but they mistook 68 % of the predictions ofour approach, 40 % of the predictions of the approach ofKuderer et al. [13], and 35 % of the predictions of the socialforces algorithm for human behavior. In summary, the resultsof this Turing test indicate that the behavior induced by ourapproach is significantly more human-like than the behaviorinduced by the other two methods according to a one-sidedpaired sample t-test.

Humans Ourapproach

Kudereret al.

SF0

20

40

60

80

100Pe

rcen

tage

perceived as human perceived as a machine

Fig. 6. Turing test to evaluate whether the behaviors induced by our newapproach, our previous work [13], and the social forces model by Helbingand Molnar [8] qualify as human. The results suggest that the behaviorinduced by our approach most resembles human behavior.

C. Robot Navigation

The learned models are applicable to mobile robot naviga-tion in populated environments. To interact with pedestrians,the robot constantly maintains a distribution over the com-posite trajectories of the pedestrians and itself in the currentsituation and acts according to the predicted cooperativebehavior. This allows the robot to mimic the behavior of thepedestrians and to behave according to their expectations. Ourcurrent implementation computes the most likely compositetrajectory comprising 3 agents in less than 100 milliseconds,which enables mobile robot navigation in real time.

V. CONCLUSION

We presented a novel approach to learning a model of thenavigation behavior of cooperatively navigating agents suchas pedestrians from demonstrations. Based on observationsof their continuous trajectories, our approach infers a modelof the underlying decision process. To cope with the discreteand continuous aspects of this process, our model uses ajoint mixture distribution that captures the discrete decisionsregarding the homotopy classes of the composite trajectoriesas well as continuous properties of the trajectories, such ashigher-order dynamics. Our method computes the featureexpectations with respect to the continuous high-dimensionalprobability distributions using Hamiltonian Markov chainMonte Carlo sampling. A Turing test suggests that thepedestrian trajectories induced by our approach are perceivedas highly human-like. Furthermore, a cross validation demon-strates that our method generalizes to new situations andoutperforms three state-of-the-art techniques.

REFERENCES

[1] P. Abbeel and A. Ng. Apprenticeship learning via inverse re-inforcement learning. In Proc. of the International Conferenceon Machine Learning (ICML), 2004.

[2] G. Arechavaleta, J.-P. Laumond, H. Hicheur, and A. Berthoz.An optimality principle governing human walking. IEEETransactions on Robotics (T-RO), 24(1):5–14, 2008.

[3] Ch. Atkeson and S. Schaal. Robot learning from demonstration.In Proc. of the Fourteenth International Conference on MachineLearning (ICML), 1997.

[4] C.M. Bishop. Pattern Recognition and Machine Learning(Information Science and Statistics). Springer-Verlag NewYork, Inc., 2006.

[5] S. Duane, A.D. Kennedy, B.J. Pendleton, and D. Roweth.Hybrid monte carlo. Physics Letters B, 195(2):216–222, 1987.

[6] S.J. Guy, J. van den Berg, W. Liu, R. Lau, M.C. Lin, andD. Manocha. A statistical similarity measure for aggregatecrowd dynamics. ACM Transactions on Graphics (TOG), 31(6):190, 2012.

[7] W.K. Hastings. Monte carlo sampling methods using markovchains and their applications. Biometrika, 57(1):97–109, 1970.

[8] D. Helbing and P. Molnar. Social force model for pedestriandynamics. Physical Review E (PRE), 51:4282–4286, 1995.

[9] S. Hoogendoorn and P.H.L. Bovy. Simulation of pedestrianflows by optimal control and differential games. OptimalControl Applications and Methods, 24(3):153–172, 2003.

[10] E.T. Jaynes. Where do we stand on maximum entropy.Maximum Entropy Formalism, pages 15–118, 1978.

[11] M. Kalakrishnan, P. Pastor, L. Righetti, and S. Schaal. Learningobjective functions for manipulation. In Proc. of the IEEEInternational Conference on Robotics and Automation (ICRA),2013.

[12] K.M. Kitani, B.D. Ziebart, D. Bagnell, and M. Hebert. Activityforecasting. In Proc. of the European Conference on ComputerVision (ECCV), 2012.

[13] M. Kuderer, H. Kretzschmar, C. Sprunk, and W. Burgard.Feature-based prediction of trajectories for socially compliantnavigation. In Proc. of Robotics: Science and Systems (RSS),2012.

[14] M. Kuderer, H. Kretzschmar, and W. Burgard. Teaching mobilerobots to cooperatively navigate in populated environments. InProc. of the IEEE/RSJ International Conference on IntelligentRobots and Systems (IROS), 2013.

[15] A. Lerner, Y. Chrysanthou, and D. Lischinski. Crowds byexample. Computer Graphics Forum, 26(3):655–664, 2007.

[16] A. Ng and S. Russell. Algorithms for inverse reinforcementlearning. In Proc. of the International Conference on MachineLearning (ICML), 2000.

[17] S. Pellegrini, A. Ess, K. Schindler, and L.J. Van Gool. You’llnever walk alone: Modeling social behavior for multi-targettracking. In Proc. of the IEEE International Conference onComputer Vision (ICCV), 2009.

[18] Nathan Ratliff, Matt Zucker, J. Andrew Bagnell, and SiddharthaSrinivasa. Chomp: Gradient optimization techniques forefficient motion planning. In Proc. of the IEEE InternationalConference on Robotics and Automation (ICRA), 2009.

[19] N.D. Ratliff, J.A. Bagnell, and M.A. Zinkevich. Maximummargin planning. In Proc. of the International Conference onMachine Learning (ICML), 2006.

[20] M. Riedmiller and H. Braun. A direct adaptive method forfaster backpropagation learning: The RPROP algorithm. InProc. of the IEEE International Conference on Neural Networks(ICNN), 1993.

[21] P. Trautman and A. Krause. Unfreezing the robot: Navigationin dense, interacting crowds. In Proc. of the IEEE/RSJInternational Conference on Intelligent Robots and Systems(IROS), 2010.

[22] J. van den Berg, S.J. Guy, M.C. Lin, and D. Manocha.Reciprocal n-body collision avoidance. In Proc. of theInternational Symp. of Robotics Research (ISRR), 2009.

[23] P. Vernaza and D. Bagnell. Efficient high dimensionalmaximum entropy modeling via symmetric partition functions.In Advances in Neural Information Processing Systems (NIPS),volume 25, pages 584–592. 2012.

[24] B.D. Ziebart, A. Maas, J.A. Bagnell, and A.K. Dey. Maximumentropy inverse reinforcement learning. In Proc. of the AAAIConference on Artificial Intelligence (AAAI), 2008.

[25] B.D. Ziebart, N. Ratliff, G. Gallagher, C Mertz, K. Peterson,J.A. Bagnell, M. Hebert, A.K. Dey, and S. Srinivasa. Planning-based prediction for pedestrians. In Proc. of the IEEE/RSJInternational Conference on Intelligent Robots and Systems(IROS), 2009.