Umea University

Master Thesis

Validation toolbox for a Physics Engine

Author:

Emma Sundling

emsu0033@student.umu.se

Advisers:

Kenneth Bodin

kenneth.bodin@algoryx.se

Claude Lacoursiere

claude.lacoursiere@algoryx.se

June 2016

Physics Department

Examiner

Eddie Wadbro

eddiew@cs.umu.se

“Aim for the moon. If you miss you may hit a star.”

W. Clement Stone

Abstract

Master thesis 30hp

Validation toolbox for a Physics Engine

Physics engines become more and more common due to the rapid development and

increasing demand of simulations. With this comes a need of testing the engine, a way

to measure its performance, not only its speed but also its accuracy and stability. The

purpose of this thesis has been to create a set of benchmark tests. They aim to check the

physical aspects, especially mechanics, of the engine. A strategy and export functions

for the test results in order to automate the testing have also been developed.

The resulting tests became a beam on piles which analyses constraint stability, an

overdetermined system consisting of a static door on multiple hinges, a falling object

investigating the accuracy of the integrator, a box on an inclined plane for testing the

friction model, a single pendulum as well as a multibody pendulum checking constraint

accuracy and energy conservation, the Earth orbiting around the Sun which tests the

stability of the integrator and finally a cantilever beam that is a static test of a real

scenario. After the tests are performed the results are presented on an HTML-page. A

prototype of a Web application is also established as well as a set of scalar tests that can

be performed continuously, in order to follow trends or compare the engine’s performance

from time to time.

This thesis was initialized by Algoryx Simulation AB which also provided the engine,

AgX Dynamics, with the numerical method called Spook. It mainly performed well on

all tests. In order to build a fully general toolbox more tests need to be added such as

material interactions, scalable test with thousands of bodies, torque tests as well as more

complex scenarios, for example a scissor lift and robots. The work can also be extended

with more developed export functions, both to the Web and to documents. Hopefully

this thesis can be seen as a complement to the earlier efforts done in creating a general

set of benchmarks and automation framework for continuous integration and testing.

Sammanfattning

Examensarbete 30hp

Valideringsverktyg for en fysikmotor

Fysikmotorer blir mer och mer vanliga pa grund av den snabba utvecklingen och

efterfragan pa simuleringar. I och med detta okar ocksa behovet av att testa motorerna

och ett satt att mata prestandan, inte bara snabbheten utan ocksa noggrannheten och sta-

biliteten. Syftet med detta examensarbete har varit att skapa ett set av prestandatester.

De syftar till att testa de fysikaliska aspekterna av fysikmotorn, sarskilt inom mekanik.

En strategi och exportfunktioner for testresultaten for att automatisera testningen har

ocksa utvecklats.

De resulterande testerna blev en balk pa palar som analyserar stabiliteten hos villkoren,

ett overbestamt system bestaende av en statisk dorr pa flera gangjarn, ett fallande

objekt som granskar precisionen hos integratorn, en lada pa ett lutande plan som

testar friktionsmodellen, en enkel pendel samt en flerkropppspendel som kontrollerar

villkorsprecisionen och energikonservering, jordens bana runt solen som testar integratorns

stabilitet och slutligen en utskjutande balk som ar ett statiskt test av ett verkligt fall.

Nar testerna ar genomforda presenteras resultaten pa en HTML-sida. En prototyp av en

webb-applikation har ocksa utvecklats samt ett set med skalara tester som kan utforas

kontinuerligt for att folja upp trender och jamfora motorns prestanda over tid.

Det har examensarbetet initierades av Algoryx Simulation AB som aven tillhandahallit

fysikmotorn, AgX Dynamics, med den numeriska metoden Spook. Motorn presterade

overlag bra pa testerna. For att bygga en allman verktygslada behovs fler tester sa som

interaktion mellan material, skalbara tester med tusentals kroppar samt mer komplexa

simuleringar, t.ex. en saxlyft och robotar. Arbetet kan ocksa utokas med mer utvecklade

exportfunktioner, bade mot webben och som dokument. Forhoppningsvis kan detta ses

som ett komplement till de tidigare anstragningar som gjorts for att skapa ett generellt

set av prestandatester och ett automatiskt ramverk for kontinuerlig testning.

Acknowledgements

Firstly I want to thank my advisers Kenneth Bodin and Claude Lacoursiere for all the

help and support throughout this work. The road has certainly not been straight and

I am very grateful for all your patience, knowledge and encouragement. Besides my

advisers, I would like to sincerely thank Eddie Wadbro who always been ready with

answers and good advices, no matter when or where. I also want to express my gratitude

for all the help and support provided by the people at Algoryx. They have not only

contributed with knowledge and guidance but likewise in practical ways. Finally I would

like to thank my family and partner for their support both during this work and during

life in general.

iv

Contents

Abstract ii

Sammanfattning iii

Acknowledgements iv

Contents v

List of Figures vii

Physical Constants and Notation ix

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.2 Offshore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.3 Web and Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 Purpose and Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Preliminaries 7

2.1 Physics engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.1 Collision detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.2 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.1.2.1 SPOOK - Pendulum . . . . . . . . . . . . . . . . . . . . 10

2.1.2.2 SPOOK - Harmonic Oscillator . . . . . . . . . . . . . . . 14

2.1.3 Key features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2 Stable simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3 Test cases 24

3.1 Work flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.1.1 Pre-study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.1.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2 Test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

v

Contents vi

3.2.1 Beam — Constraint stability . . . . . . . . . . . . . . . . . . . . . 25

3.2.1.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.2.2 Door — Overdetermined system . . . . . . . . . . . . . . . . . . . 29

3.2.2.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.2.3 Falling Object — Accuracy . . . . . . . . . . . . . . . . . . . . . . 33

3.2.3.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.2.3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.2.4 Friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2.4.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.2.4.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.2.5 Pendulum — Constraint accuracy . . . . . . . . . . . . . . . . . . 42

3.2.5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.2.5.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.2.6 N-pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.2.6.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.2.6.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.2.7 Planetarium — Integrator stability . . . . . . . . . . . . . . . . . . 51

3.2.7.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.2.7.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.2.8 Cantilever beam . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.2.8.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.2.8.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4 Additional work 58

4.1 Result presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.2 Continuous testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.3 Interactive Web based Simulation . . . . . . . . . . . . . . . . . . . . . . . 60

5 Discussion 62

5.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.2 Continuous testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.3 Interactive Web based simulation . . . . . . . . . . . . . . . . . . . . . . . 63

5.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

A Strategy and Work flow 65

Bibliography 66

List of Figures

2.1 Different bounding volumes . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 The theoretical set-up of the pendulum. . . . . . . . . . . . . . . . . . . . 11

2.3 The steps of shake . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.4 Spring - position vs time, both simulation and manual stepping . . . . . . 15

2.5 Spring - position vs time using a distance constraint . . . . . . . . . . . . 15

2.6 Spring - position vs time using distance constraint with high frequencyand no damping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.7 Spring - position vs time using distance constraint and damping . . . . . 17

2.8 Spring - amplitude versus number of time steps for different frequencies . 19

3.1 Beam set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.2 Beam — maximum deviation vs time step . . . . . . . . . . . . . . . . . . 27

3.3 Beam — max deviation vs time for time step of 9.8 · 10−5 s . . . . . . . . 27

3.4 Beam — deviation for each box . . . . . . . . . . . . . . . . . . . . . . . . 28

3.5 Beam — maximum deviation vs number of boxes . . . . . . . . . . . . . . 28

3.6 Door set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.7 Door — deviation in total Fz compared to weight vs number of hinges . . 31

3.8 Door — Fx deviation for two hinges vs mass . . . . . . . . . . . . . . . . . 32

3.9 Door — deviation in total Fz compared to weight vs mass . . . . . . . . . 32

3.10 Falling Object — position vs time . . . . . . . . . . . . . . . . . . . . . . 35

3.11 Falling Object — error vs time . . . . . . . . . . . . . . . . . . . . . . . . 35

3.12 Falling Object — position vs time with altered time step . . . . . . . . . . 36

3.13 Falling Object — error vs time with altered time step . . . . . . . . . . . 36

3.14 Friction set-up number 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.15 Friction set-up number 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.16 Friction — θ vs µs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.17 Friction — the force ratio vs tan(θ) . . . . . . . . . . . . . . . . . . . . . . 40

3.18 Friction — anisotropy plot . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.19 Pendulum set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.20 Pendulum — length deviation vs time step . . . . . . . . . . . . . . . . . 44

3.21 Pendulum — energy deviation vs time step . . . . . . . . . . . . . . . . . 44

3.22 Pendulum — energy vs time . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.23 Pendulum — energy deviation vs time step, stiff system . . . . . . . . . . 45

3.24 Pendulum — length deviation vs mass . . . . . . . . . . . . . . . . . . . . 46

3.25 N-pendulum set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.26 N-pendulum — max deviation for each constraint . . . . . . . . . . . . . . 48

3.27 N-pendulum — max deviation vs time step . . . . . . . . . . . . . . . . . 48

3.28 N-pendulum — energy deviation vs time step . . . . . . . . . . . . . . . . 49

vii

List of Figures viii

3.29 N-pendulum — max deviation vs number . . . . . . . . . . . . . . . . . . 49

3.30 N-pendulum — energy deviation vs number . . . . . . . . . . . . . . . . . 50

3.31 Planetarium set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.32 Planetarium — the orbit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.33 Planetarium — energy vs time, longer simulation . . . . . . . . . . . . . . 53

3.34 Planetarium — distance vs angle . . . . . . . . . . . . . . . . . . . . . . . 54

3.35 Planetarium — energy deviation versus time step . . . . . . . . . . . . . . 54

3.36 Cantilever beam set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.37 Cantilever beam — deflection vs position on beam . . . . . . . . . . . . . 57

4.1 Screen shot of the presentation of the results . . . . . . . . . . . . . . . . 59

4.2 The result table of the continuous test . . . . . . . . . . . . . . . . . . . . 60

4.3 Screen shot of the friction simulation on the web . . . . . . . . . . . . . . 61

Physical Constants and Notation

Acceleration a (ax, ay, az) m/s2

Angle θ rad or degrees, o

Compliance ε

Damping τ

Force F (Fx, Fy, Fz) N

Friction coefficient µ

Gaussian gravitational constant k = 2π/365

Gravitational acceleration g m/s2

Mass m kg

Momentum p N s

Number N

Pi π = 3.14159..

Position x (x, y, z) m

Spring constant κ N/m

Static Friction Coefficient µs

Time t s

Time step h s

Velocity v (vx, vy, vz) m/s

ix

Chapter 1

Introduction

Simulations are an important part of today’s society. Not only for performance optimiza-

tion, but also for training, education, research and games. It is a way to study systems

that for some reasons are hard to examine in real life, for example scenarios in outer

space or in construction purposes since it is often cheaper to simulate a system than

to build a proper model. Simulations are also a way to construct virtual environment

training systems used for both airline and military pilots, car drivers and do not forget

its important role in the entertainment industry, especially for computer games [24]. It

is also a way to gain access to deep information about systems, such as all positions

and velocities. Many different frameworks for simulations have been developed, both

commercial and open source. They are often referred to as physics engines.

A physics engine is a software program that keeps track of the laws from the physical

world as well as handles forces and interactions between objects [5]. The engines are

often divided into game physics engines and scientific physics engines where the first one

focuses mainly on real-time simulations and the experience. For the scientific case it is

more important that the calculations are precise for the accuracy of the simulations. This

is the balance all engines face, the compromise between speed, stability and accuracy.

So far there are no standard for measuring the accuracy of a physics engine. Some

efforts have been made to compare engines, often in order to choose for a particular

task. Additionally benchmark tests have been developed but at present these concentrate

mainly on game engines and examine speed and memory usage rather than accuracy and

stability.

In this thesis a validation toolbox for a physics engine is developed. It examines the

accuracy of the simulations through a collection of benchmark tests. These simulations

check the engine’s performance in different areas such as constraint accuracy and stability,

friction, stability of the integrator and overdetermined systems. Guidelines for choosing

1

Chapter 1. Introduction 2

benchmark tests are also produced. The following part of Chapter 1 describes the

background of the thesis and a study of related work as well as goals, limitations of the

work and methods. In Chapter 2 the theory of the engine is discussed and thereafter, in

Chapter 3, each test case is presented with both theory, results and a short discussion.

Later, in Chapter 4, the additional work is displayed followed by a discussion and

conclusions in Chapter 5.

1.1 Background

This thesis is initialized by Algoryx Simulation AB which develops software for physics

based simulations. It was founded in 2007 as a spin-out from Umea University and

they are the producers of the AgX Dynamics. AgX Dynamics is a physics engine for

simulating a wide range of scenarios such as wires and cables, rigid multibody system

dynamics, vehicles and granular material for example [1]. Algoryx wishes a toolbox for

validation of the accuracy of their product where focus is on non-smooth phenomena like

impacts, contacts and dry friction for example. The aim is not only to validate what

works but also to detect flaws in order to improve the product and additionally to be

able to follow the performance of the simulations over time.

1.2 Related Work

There have been several studies performed with the aim to compare physics engines, e.g.

for choosing a suitable engine for a certain case. Seugling and Rolin [38] published a

master thesis where they perform nine run time tests in order to select a suitable physics

engine to use in a physics module. They are testing friction, gyroscopic forces, energy

conservation, constraint stability, accuracy, constraint scalability, scalability of contacts,

piling and complex contacts. The benchmarks cover the most common scenarios in games

and simple interactive simulations and most of the results are compared with analytical

solutions. Still some of the more interesting cases are only judged using visual inspection

and they do not put the engines to the test with a large amount of objects. In many

complex simulation scenarios there are thousands of objects rather than 20. A typical

training scenario for offshore simulations could be 2 to 3 ships, each with cranes on them

with hundreds of bodies and cables that are kilometres long created by hundreds of

elements. Also, due to the nature of the thesis, a part of the evaluation considers other

aspects such as included features and cost rather than the accuracy of the engine.

An often referred work is the evaluation performed by Boeing and Braunl [6] that is

inspired from the study of Seugling and Rolin. It is a qualitative study of free physics

Chapter 1. Introduction 3

engines using the Physics Abstraction Layer (PAL). The tests cover a free falling sphere,

a bouncing sphere, friction with a box on an inclined plane, chain of spheres, a collision

system and a stacking test. All tests are performed and chosen carefully but could be

developed with larger mass ratios which can cause ill-conditioning, more objects and

more complex scenarios to make things harder. Another similar study is the master

thesis performed by Hansson [20]. It covers the basic benchmarks with a free falling

box, collisions, a friction test and pendulum in order to test the constraint stability. All

tests are varied with different friction coefficients, masses etc. Especially the mass is

substantially increased in some of the tests. Even though this study is not that wide

each test has a purpose and is thoroughly tested. Similar to the work of Boeing it would

be useful to develop the tests further, towards more complex and realistic scenarios.

Mirtich and Canny [32] have published a work on an impulsed-based method for simula-

tions. Despite that the article is not about evaluation of physics engines they perform

several interesting benchmarks in order to test their method, especially for testing colli-

sion detection. Some of the test results are compared to analytical solutions and some

are visually inspected. These benchmarks may not alone found a decent validation of a

physics engine but they provide a good starting point. This is the direction I followed.

1.2.1 Robotics

Robotics is another area where there have been different comparisons of physics engines.

As Roennau et al. [35] claim in their work does detailed simulations of robotic systems

become more and more important in the development of the robotics area and its

applications. They focus their tests on requirements for robotics which includes collision

detection, complex collision detection, different stacking cases, constraint stability and

two friction tests. In some tests the calculation time is investigated, in others the deviation

from the analytical solution. Additionally they examine cases where the engines either

fail or pass, for example if a boxes can be stacked into a pile without it collapsing. This

is an interesting approach but it may contain weaknesses. For example, if it is only

examined if the pile is collapsing or not the engine may pass the test even though the

boxes are starting to penetrate each other or perform other unrealistic behaviour. In

general the work is interesting and the cases are very well described which makes them

easy to replicate. A similar comparison of physics engines for robotics is made by Erez

et al. [10]. In this work an attempt is made to develop tests that are universal and

differ from the regular benchmarks with rigid bodies in order to be more suited for

evaluating engines for robotics applications. They test grasping of a capsule with a hand,

a falling humanoid, a planar chain and 27 capsules falling on the floor. Over all the tests

investigate which time steps the engines can use before the simulations fail and the CPU

Chapter 1. Introduction 4

time but also energy and momentum are examined when possible. The conclusion from

the work is that each engine usually perform well on systems which it was designed for.

Even though the benchmarks are well performed and complex the work did not test

common features such as friction and scalability. The test with the dropped capsules

uses up to 250 capsules as an attempt to scale up the system but the simulation is only

run for 10 seconds which may not be the case in real scenarios. Scalability is more deeply

examined in the work published by Hummel et al. [21] where collision tests with 1 to 2500

objects are performed. This study also focuses on robotics and the additional tests are a

bouncing tennis ball, a chain of spheres, interpenetration tests and a scenario where a

screw is screwed into a nut. Apart from scaling up the scenarios this work also tries large

masses. All tests are of importance when testing a physics engine but as they conclude

in their work the benchmarks can be extended in order to check more features such as

collisions with non convex geometries and soft body dynamics.

1.2.2 Offshore

The offshore exploitation industry is another field of study where simulations come in

handy due the complex nature of the systems because the important aspect is wire

and anchor handling, involving very long lines under very high tension in addition to

hydrodynamics. One study performed by Metrikin et al. [30] compares different physics

engines, mostly with respect to simulations of the broken-ice fields. The tests performed

in this study are a free falling sphere, an ice floe colliding with a floater and energy

dissipation when two ice floes are colliding. Additional tests such as friction tests and

collision tests were suggested but not performed. All executed tests compared the error

towards the time step to examine which physics engine was most suited for simulations

of a broken-ice field. The study uses rather simple benchmarks which could be further

developed for a more general evaluation, as suggested in the conclusion. Despite this the

study is interesting since it deals with a specific and important area.

1.2.3 Web and Games

Apart from articles there are also web pages that present a set of test cases in order to

evaluate physics engines. One of them is a product review by Lander and Hecker [27].

It lists 12 benchmarks which test collision detection, large size ratios, constraints, large

mass ratios etc. The range of the tests are broad and especially the collision detection is

well tested. Furthermore are each test focused on testing one thing which makes it easy to

grasp the results. Unfortunately the results are not compared to analytical solutions and

the grading of the engines is subjective. This could be a result of the work being mainly

Chapter 1. Introduction 5

focused on game engines. A similar set of tests is published at the web pages of MBS

Benchmark, MultiBody Systems Benchmark [33], which is built on the work of Gonzalez

et al. [14] [15]. It aims towards producing a standardized set of problems and procedures

to measure computational efficiency and provide a platform where information can be

collected, organized and shared. The main focus is multibody dynamics simulations and

perhaps not physics engine principally. The simple pendulum, the N-bar mechanism,

Andrew’s mechanism, Bricard’s mechanism and fly ball governor are the tests presented

on the web page and all results are compared to analytical solutions. These cases are

in the group Basic Problems but there is also a group called Industrial Applications.

Some of the tests listed above are very simple but there are also a stiff system and an

over-constrained system. As many other studies mentioned the set of scenarios is a good

starting point but should be developed further to really test the simulations.

One of the more extensive work to produce benchmark tests is called PEEL, Physics

Engine Evaluation Lab, constructed by Pierre Terdiman. It is a collection of more than

250 tests that aims to evaluate and compare engines with focus on speed and memory

usage. Some of the tests are described in his blog Coder Corner and they cover rigid

bodies, stacking of different objects, joints, Raycasts, sweep tests among others [41]. As

mentioned this collection of tests evaluates the speed and the usage of memory and it is

mainly aimed at games applications. It does not consider the validation of the physics in

the simulations.

1.2.4 Conclusion

To summarize, there are several studies performed where some kind of tests are executed.

The most common benchmarks are the simple pendulum, a free falling body, collision

of two bodies and a chain of sphere, probably since they are simple to perform and the

results can be compared to the analytical solution. These cases are of course important

to test but it is crucial to keep in mind that if a simple scenario is accurately simulated

it does not imply that also complex cases are correct. This is why the tests need to cover

a wide range of cases, from simple ones to more complex ones. It is of course impossible

to enclose everything but the studies discussed earlier may provide a starting point and

a direction towards which tests that should be included. In most of the studies the

aim is to choose a physics engine for a certain case or to determine which engine is the

best. The aim of this thesis is not a comparison between different engines but rather a

validation of the accuracy of the physics in the simulations today and a possibility to

follow the change in performance during time. An additional goal is to develop a way to

automate the tests and result production.

Chapter 1. Introduction 6

1.3 Purpose and Goals

The overall purpose of this thesis is to develop a product for validating the accuracy and

stability of simulations performed by a physics engine and to create a set of benchmarks.

Since there are no similar work performed with the aim to create a method for validation

that is universal the description of the assignment and the goals are flexible and may be

changed during the project. The goals of the thesis consists of four points

• Create a set of benchmarks (10–20) for testing the accuracy of the simulations

• Design a strategy for creating new test cases which can be used with continuous

integration and testing

• Develop a product that performs the validation of the physics engine

• Develop an export functionality for visualization of the results, either to the World

Wide Web or as document, in collaboration with the staff on Algoryx

1.4 Delimitations

Since the number of possible things to test, both physical laws and numerical problems,

are infinite the extent of the work needs to be limited in order to fit the scope of a master

thesis (30 hp). This thesis is therefore restricted to 10–20 benchmark tests depending on

the proportions of the tests. Additionally the export functionality and the possibility to

use the Web is not developed from scratch but the existing tools is used together with

newly developed tools produced by Algoryx.

Chapter 2

Preliminaries

In this chapter there is an introduction to what a physics engine is and how it works,

both regarding collision detection and the integrator. The numerical method Spook [24]

is briefly explained using the pendulum and the harmonic oscillator. In the beginning

there is a reminder of the basics of physics and the chapter ends with some common

features and problems when simulating.

Newton’s laws of motion

Newton’s three laws of motion are considered to be the fundamentals in classical mechanics.

The Newtonian laws describe how objects behave when exposed to forces, i.e. the response

to the forces. Almost all textbook in mechanics states these laws, see for example Physics

for Game Developers by Bourg [7, p.1]. Below are the laws stated in vectorial form.

Remember that the mass is assumed to be constant.

1. An object will continue to be at rest as long as the net forces acting on it is 0. If

the object is in motion its velocity will not be changed. Mathematically it can be

written as ∑F = 0⇔ dv

dt= 0. (2.1)

2. The change of momentum (p) is equal in both magnitude and direction as the

applied force if the mass of the body is constant, or

F =d(mv)

dt=dp

dt. (2.2)

3. For a force FA there is always a counterforce that is equal in magnitude but opposite

in direction, i.e.

FA = −FB. (2.3)

7

Chapter 2. Preliminaries 8

The second law can be rewritten using the definition of momentum, p = mv. Since it is

assumed that the mass is constant this gives

F = mdv

dt= ma, (2.4)

where F is the resultant force [7, p.15]. These equations are ODE’s of motion and need

numerical integration. From here on, the time derivative will be denoted with a dot.

That is, the velocity will be written as x and the acceleration as x.

2.1 Physics engine

In real life bodies behave according to the Newtonian laws. In a simulation it is the physics

engine’s responsibility to make sure that the laws are obeyed to make the simulation

realistic. It is the engine that simulates these laws in the computer, it works as the

heart of the simulation [9, p.295]. There are several ways to describe the structure of the

engine, see both [9], [11], [20] and [22]. According to Erleben [11] the engine is described

to consists of a collision detection part and a simulation part. Both sections will be

described further in this chapter but shortly one can say that the collision detection

handles the geometry of the state, i.e. analyses if there are any collisions or contacts

while the simulation takes care of the physics related part with numerical integration,

forces, constraints and so on.

2.1.1 Collision detection

Contacts and impacts are an important part of a realistic simulation. It is one of the ways

that the objects can interact with each other. In real life the phenomenon is common,

it happens every time you grasp something, stand or walk and it is natural that two

solid objects do not occupy the same point in space. This is not the case in dynamics

simulations.

First, before the program can handle a collision, the intersection between two objects

needs to be detected, which requires a collision detection system. It is a pure geometrical

problem of determining which objects are in contact with each other or will be in the

short future time. This is a computational hard task. If a simulation has n bodies the

computer needs to perform, in principle, n(n− 1)/2 checks in worst case. In order to

reduce the work and speed up the process several methods have been developed. Usually

the process starts with a broad phase that filters out objects into a list of potential

collisions. This list then goes into the narrow phase that determine if there is a collision

and also extracts additional information.

Chapter 2. Preliminaries 9

The Broad Phase

As mentioned above the broad phase aims to find objects that are close enough to each

other for further testing [11]. A basic method for this is to use bounding-volume tests.

The idea is to use a simple geometries to cover the more complex objects, called bounding

volumes [42, p.193], and the benefit is that it is easier to perform collision tests between

the simpler primitives. Three different bounding volumes are shown in Fig. 2.1 inspired

from Kimmerle [23]. It is the bounding sphere, the AABB (Axis Aligned Bounding Box)

and the OBB (Oriented Bounding Box). The figure illustrates that the OBB is a better

approximation than the sphere but in return it is computationally more costly. Examples

of methods that can be used to test if pairs intersect are Exhaustive search, Sweep n’

prune and (multilevel) grids [11] as well as Space Partitioning and Model Partitioning

which are described in for example Bergen [42].

Bounding sphere AABB OBB

Better approximations, higher build and update cost

Smaller computational costs for overlap test

Figure 2.1: Different bounding volumes, the bounding sphere, the AABB (Axis AlignedBounding Box) and the OBB (Oriented Bounding Box) and their different propertiesregarding approximations and computational cost. Figure inspired from Kimmerle [23].

There are different ways to perform intersection tests. Firstly, to check if two objects

intersect using only their instant position is called a static intersection test [17]. Secondly

there is a dynamic intersection test which uses information about the motion of the

object and constructs a sweeping volume instead of a bounding volume [11].

When the broad phase is performed the list of possible colliding pairs are reduced

compared to the start. This list is then passed on to the next step in the process, the

narrow phase.

The Narrow Phase

The goal with the narrow phase is to determine whether a collision is occurring or not.

Additionally other information such as contact points, penetrating features and proximity

information can be gathered [11]. When using discrete time penetrations are unavoidable.

Chapter 2. Preliminaries 10

This phase also approximates the depth of these penetrations. The algorithms used for

the collision tests in this phase are usually more refined but also slower compared to

the ones in the broad phase. One algorithm that can be used is the Lin-Canny closest

features algorithm which is described by Mirtich [31]. The advantage of the method is

that it is fast, but on the other hand it has a problem terminating when two objects are

penetrating and its convergence may be poor in degenerate cases. Another approach to

the problem is to treat the object as a convex hull of a point set rather than focusing on

polyhedral features. Gilbert, Johnson and Keerthi (GJK) have designed an algorithm

with that approach [31]. The GJK algorithm computes the distance between convex

objects iteratively. It is a fast and functional method that can be used on general convex

objects. The drawbacks are that it is hard to understand and since it is iterative it is

also sensitive to numerical errors [42, p.121]. There are many alternative methods such

as the method of rotating calipers [9, p.464], the I-Collide method and the V-Clip (which

are developed from both the Lin-Canny and the GJK methods) [9, p.464] [31].

2.1.2 Simulation

As mentioned earlier the simulation module is responsible for the physics in the simulations.

In AgX the numerical method Spook is used for solving the differential algebraic equations

regarding the motion of the objects. For a complete description of Spook, see the PhD

thesis by Lacourisere [24].

2.1.2.1 SPOOK - Pendulum

Spook is a discrete time variational method derived from the least action principle [24].

The action is defined as the time integral of the Lagrangian which is the difference between

the kinetic energy and the potential energy. Shortly the principle can be explained as

the fact that a body always want as small action as possible or the least action possible.

A thrown ball, for example, will always have a parabolic path in the air since it gives the

least action. If the path would be irregular or bumpy this principle would be violated [4].

To simplify the description of Spook, let’s consider the pendulum. It is a bob of mass m

moving at a distance l from the origin under constant gravity g (the bob is affected by the

gravity force Fg). Often the motion is considered in 2D, Fig 2.2 shows an illustration of

the scenario. The governing equations for the pendulum can be obtained from Newton’s

second law, Eq. (2.4),

mx = Fc + Fg (2.5)

Chapter 2. Preliminaries 11

where x is the position vector of the bob and Fc is the radial force keeping the bob on

the path. Due to the attachment of the motion of the bob is restricted, it can only move

on a trajectory a specific distance from the attachment point, l0. This constraint can be

mathematically expressed as

g(x) = ||x|| − l0 = 0. (2.6)

This is a holonomic constraint since it is a strict equality and only depending on the

coordinates and time [24, p.395]. It is the force Fc that constrains the motion to the

allowed trajectory and it therefore needs to be directed radially towards the attachment

point, i.e.

Fc = λx. (2.7)

In order to get something that is directed radially inwards the constraint is used. The

steepest direction from the allowed trajectory g(x) = 0 is the gradient of the constraint,

∇g, denoted G. It is always perpendicular to the trajectory by nature. Therefore

Fc = GTλ. From here on let Fg = Fe, for external forces. If all equations are put

together it yields

mx = Fc + Fg = GTλ+ Fe

g(x) = 0.(2.8)

In 2D this is a system of 3 unknowns and 3 equations, i.e. a closed system, also called a

differential algebraic equation of index 3.

g(x)

l

m

Fc

v

Fg

θ

Figure 2.2: The theoretical set-up of the pendulum.

The system can be solved by a computer using discretization, both in space and in time.

Let xk be the position at time k and use the time step h which gives

mvk+1 = mvk + hFk

xk+1 = xk + hvk+1.(2.9)

Chapter 2. Preliminaries 12

Let Fk = GTkλh + Fe,k. Fe,k is the external force at time k and note the new definition of

λ. Putting everything together yields

mvk+1 = mvk +GTk λ+ hFe,k

g(xk+1) = 0

xk+1 = xk + hvk+1.

(2.10)

This is called Shake which is a method that is almost energy conserving. It preserves

energy like quantity E = E0 +O(h) for exponentially long time [19]. In order to solve

g(xk+1) = 0 the Newton-Raphson method can be used. Then a non-symmetric matrix

appears which can make the problem very hard. Additionally it can fail to find a solution.

In Fig. 2.3 the different steps of Shake are illustrated, first mvk is added to xk. Then

hFg is added and finally GTk λ. These three additions determine where the position xk+1

is situated and whether it fulfils the constraint or not. In the figure it does not cross the

constraint circle which is a drawback of this method, that it sometimes may not find

a solution. It happens when velocities are too high compared to constraint curvature.

There are projection methods that project back on the constraint surface but these

requires some computational work and are still not guaranteed to find a solution [25].

g(x)

xk

mvk

hFg

GTk λ

Figure 2.3: The steps of shake.

Spook is a way to handle this since it rewrites the constraint to

1

4(gk+1 + 2gk + gk−1) + ελ− τ

hGk+1 (xk+1 − xk) = 0 (2.11)

where ε is the relaxation of the constraint and τ is the damping. That is, the constraint

is not solved exactly but a small deviation is allowed, if the constraint is degenerate. ε is

also called the compliance. Recall thatxk+1−xk

h = vk+1 and put all equations together

Chapter 2. Preliminaries 13

and it will give

mvk+1 = mvk +GTk λ+ hFe,k

1

4(gk+1 + 2gk + gk−1) + ελ+ τGk+1vk+1 = 0

xk+1 = xk + hvk+1.

(2.12)

Eq. (2.12) is Spook with holonomic constraints and damping in velocity format [24,

p.98]. Also Gk+1 is approximated with Gk and gk+1 + 2gk + gk−1 ≡ 4〈gk〉 is linearized

around k. By doing this linearization and stabilization it is made sure that the constraint

surface is always crossed, i.e. it is always possible to find a solution.

If the damping is 0, then λ can be calculated from Eq. (2.11) as

λ = − 1

4ε(gk+1 + 2gk + gk−1) ≡ −

1

ε〈gk〉. (2.13)

Substituting Eq. (2.13) for the constraint force gives

Fc = GTk λ = −1

εGTk 〈gk〉. (2.14)

For the pendulum GTk is linear, i.e.

GTk =1

||xk||xTk . (2.15)

Eq. (2.15) can be compared to the constraint force for a spring, Fspring = −κ∆x. It is

proportional to the spring constant κ, a positive real number characteristic for the spring,

times the deviation from the equilibrium length. Since this force is also linear and by

comparing the forces it can be seen that the compliance (the relaxation parameter) ε is

related to the spring constant as ε = 1κ

Fc = −1

εGTk 〈gk〉

Fspring = −κ∆x.

(2.16)

To prevent division the solver is designed in a way that the division with ε is avoid but it

illustrates the aspect of the theory. This relaxation corresponds to arbitrary stiff forces

and the discretization makes them almost energy preserving, if there is no damping. It

is a crucial part of the method since they prevent the simulation from exploding. This

oscillations or crossing of the surface is similar to an idea by Hairer [18]. A method like

this is often called a penalty method since it penalizes violations of the constraint, to

keep it short.

Chapter 2. Preliminaries 14

In summary, Spook is a method that relaxes the constraint in order to be able to find

solutions in all cases, including constraint degeneracy and ill-conditioning. This decreases

the accuracy but makes the method stable. The simulation will oscillate around the

”true” constraint but with limited oscillations. Next an harmonic oscillator is considered

in order to further investigate the damping parameter.

2.1.2.2 SPOOK - Harmonic Oscillator

Let’s consider a simple spring with an attached ball with the force Fspring = −κ∆x. The

analytical solution for the spring, or harmonic oscillator, is

xz(t) = Ae−ζω0t cos(√

1− ζ2ω0t) (2.17)

where xz is the position in z-direction, A is the amplitude, ζ = τκ/(2√mκ) is the

damping ratio and ω0 =√κ/m is the undamped angular frequency [12], [43].

In order to get smooth and nice oscillations the following needs to be satisfied

h

√κ

m< 2. (2.18)

The mass of the ball, m, is 1 kg. To satisfy the condition in Eq. (2.18) the spring

constant is chosen to

κ = 0.014

h2. (2.19)

Additionally to simplify the scenario the gravity is put to 0.

First look at the case when the spring is only simulated using a force in every time step.

In this case there are no constraints, i.e. neither elasticity nor damping is present. Due

to this Spook works just like the Verlet stepper

vk+1 = vk + Fh

xk+1 = xk + vk+1h.(2.20)

Fig. 2.4 shows the similarities between Spook and Verlet. It illustrates the position of

the ball versus time for the two methods and it can be seen that the graphs follow each

other completely. Actually the difference is always 0.

Chapter 2. Preliminaries 15

0 1 2 3 4 5

−2

0

2

Time [s]

Pos

itio

n,z

Position vs time, spring with only forces

Simulation

Verlet

Figure 2.4: The position of the ball versus time both for the simulation and the Verletstepper in the spring simulation.

Next the spring is simulated by a distance constraint with an compliance, ε = 1κ , also

called constraint relaxation. The result is shown in Fig. 2.5. There it can be seen that

the distance constraint still simulates the oscillating spring but with a phase shift. This

case is similar to the constraint explained for the pendulum since it has no damping.

0 1 2 3 4 5

−2

0

2

Time [s]

Posi

tion

,z

Position vs time, spring with distance constraint

Simulation Verlet

0 1 2 3 4 5−1

0

1

Time [s]

Sim

ula

tion

min

us

Ver

let

Difference between the spring with distance constraint and the Verlet stepper

Figure 2.5: The position of the ball versus time both for the Spook simulation usinga distance constraint and the Verlet stepper in the spring simulation. Also the difference

between the methods is plotted.

Chapter 2. Preliminaries 16

It can also be interesting to see what happens when the spring oscillates faster than

the condition stated in Eq. (2.18) using the distance constraint. Fig. 2.6 illustrates the

position of the ball versus time and there it can be seen that the oscillations are uneven

and not as smooth as the previous cases. This examples clarifies the importance of an

appropriate time step in comparison to the velocity scale of the scene. If the bodies move

too fast compared to the time step their true movement cannot be simulated. This is

called aliasing, low frequency sampling of high frequency dynamics which gives a result

that makes no sense. Also worth noticing is that even though the time step is too large

for the velocities in the simulation the stability is still maintained.

0 1 2 3 4 5

−10

0

10

20

Time [s]

Pos

itio

n,z

Position vs time, spring with distance constraint, high frequencies

Figure 2.6: The position of the ball versus time in the spring simulation simulatedwith a distance constraint for the case of high frequency without damping. Stability is

maintained.

Finally numerical damping is imposed to the constraint to see its impact. The damping

parameter, τ is chosen to τ = 2h which provides critical damping for undesired frequencies.

In Spook the system is linearized which can cause constraint violations. The damping

is imposed to avoid these violations with the cost of energy conservation though not

as much as for projection methods, for example [24]. A damped system, even if it is

conservative, will not have conserved energy but in return the simulation is stable. A

thing with Spook is that the decay of its damping can be predicted as√αk

when θ < 2

where

α = 1− 4θλS(1 + 4θ)(λS + ε)

. (2.21)

k is integer numbers from 0 and up, θ = τ/h and λS = 1 is an eigenvalue of a matrix

S considering the constraint matrices and λs is also called the natural frequency [24,

p.101]. Fig. 2.7 shows the result of the damped spring, both from the simulation and

the analytical expression presented in Eq. (2.17). In the figure it can be seen that the

Chapter 2. Preliminaries 17

simulation follows the analytical expression but more importantly that it also follows the

damping rate√αk.

0 0.5 1 1.5 2 2.5 3

−1

0

1

2

Time [s]

Posi

tion

,z

Pos vs time spring with distance constraint and damping

Simulation

AnalyticalDamping

Figure 2.7: The position of the ball simulated with a distance constraint for the casewith damping in the spring simulation.

It is possible to theoretically determine if a numerical method is stable by looking at the

eigenvalues of the matrices involved. For the harmonic oscillator, consider the rewriting

of Eq. (2.20) which become[I −hI0 I

][xk+1

vk+1

]=

[I 0

0 I

][xk

vk

]+

[0

hF

]. (2.22)

In this case the force expression is known which makes it possible to write the equations

on the form Azk+1 = Bzk where

zk =

[xk

vk

](2.23)

and A, B are matrices. Azk+1 = Bzk can be solved for zk+1 by multiplying with the

inverse of A, i.e

zk+1 = A−1Bzk ≡ Hzk. (2.24)

The stability of this system is totally determined by the eigenvalues of H. From now on

the eigenvalues will be denoted as λ, not to confuse with the factor introduced to the

radial force for the pendulum and in the constraint. In order for the system to be stable

the eigenvalues need to be less than unity, i.e. λ < 1. Under certain conditions, see the

thesis by Lacoursiere, [24, p.102], it can be seen that the equation for the eigenvalues

reduces to

λ2 − 2λβ + α = 0 (2.25)

Chapter 2. Preliminaries 18

where

β = 1− 2(θ + 1)λS(1 + 4θ)(λS + ε)

α = 1− 4θλS(1 + 4θ)(λS + ε)

.

(2.26)

For β and α the following holds:

0 < β < 1 when θ > 1/2, ε > 0

0 < β < 1/3 when θ > 2, ε > 0(2.27)

0 < α < 1 when θ > 0, ε > 0. (2.28)

Eq. (2.25) has the roots

λ± = β ±√β2 − α. (2.29)

The modulus of this is |λ±| = β±√β2 − α. This can be examined in two different cases,

β2 − α < 0 for an oscillatory system and the case when β2 − α > 0.

First, consider the case when β2 − α < 0 which, as mentioned above, corresponds to an

oscillatory system and occurs when θ < 2 and ε > 0. Then the modulus can be calculated

as

|λ±| =√β2 +

(√α− β2

)2=√α. (2.30)

As stated earlier 0 < α < 1 when θ, ε > 0 and since it is the case in this regime

|λ±| =√α < 1, i.e. the system is stable.

Next β2 − α > 0 for which |λ±| = |α|. By examining different cases it is revealed that

also this type of system will have a modulus less than unity and therefore is stable. For

the complete analysis see the thesis by Lacoursiere, [24, p.103].

Finally, to further show some properties of Spook, let’s study the cases when ε is large

and also when it is small or 0. The case with a large ε > λS corresponds to a low

frequency system with the spring constant κ = 1/ε ≤ 1. α can then be approximated as

α = 1− 4θλS(1 + 4θ)(λS + ε)

≈ 1− 4θλS

(1 + 4θ) 4εh2(1+4θ)

= 1− 4h2θλ

4ε=

=ε− hτε

= 1− hτ

ε= 1− hτK = 1− θ(hω)2

(2.31)

where it is used that the perturbation ε can be written as 4εh2(1+4θ)

, see [24, p.100]. In

this case, a low frequency system, the aim is λ close to 1 in order to conserve energy and

for stable simulations. As can be seen in Eq. (2.31) α is one minus something small if

Chapter 2. Preliminaries 19

the parameters are chosen correctly. That is, λ =√α will be close to 1 in most cases for

large ε and the simulation will conserve energy and be stable.

The opposite case, ε ≈ 0, is a high frequency system with h2ω2 � 1. If ε is put to 0, α

can be written as

α = 1− 4θλS(1 + 4θ)λS

λS(1 + 4θ)λS

=

=1

1 + 4θ=

1

1 + 4τ/h.

(2.32)

That is, for ε = 0 the damping will be eigenvalue independent but the system is still

stable. If a non-zero ε is introduced it will add a small filtering to the system. It is a low

pass filter which removes high frequencies and ε represents the cut-off frequency. Fig.

2.8 shows this filtering property of Spook. The graph illustrates the amplitude of the

oscillations of the spring for different frequencies versus number of time steps. It can

be seen that higher frequencies are filtered more effectively than lower frequencies and

the damping can even be calculated in number of time steps. For example, for τ = 2h,

oscillations with a frequency mode higher than 50 steps per period will be all gone within

10 time steps.

0 10 20 30 40 50−2

−1

0

1

2

Number of time steps

Am

pli

tud

e

Amplitude for different frequencies vs number of time steps

Period = 100.0 steps

Period = 50.0 steps

Period = 10.0 steps

Period = 5.0 steps

Period = 1.0 steps

Period = 0.1 steps

Figure 2.8: The amplitude of the oscillations of the spring for different frequenciesversus number of time steps.

This simple spring example does not only explain how the stepper and the damping

works but also shows the filtering property of Spook. Note that for low damping and

low frequency, there is no numerical dissipation, as shown in Fig. 2.7. But choosing

τ = 2h for constraint stabilization guarantees stability, without any tuning requirements

and it is independent of masses. This example additionally provides a start for a stability

Chapter 2. Preliminaries 20

analysis. The stability of a method is extremely important but it is also a rather massive

topic, huge enough to itself cover a thesis. It is therefore just briefly discussed in this

chapter, for a deeper analysis see the thesis and the manuscript draft by Lacoursiere and

Linde [24], [25].

In summary Spook is method that filters high frequencies. This high energy dissipation

is similar to the property of the Generalized-α methods. They are stable implicit methods

with second order accuracy but that requires solving nonlinear system of equations [37].

Also, as long as the ratio between the damping parameter and the time step is small

enough, τ/h < 2, the system is not overdamped and the constraint will be satisfied

eventually. These properties makes Spook a stable tool even for complex cases but with

a cost of precision and some energy loss.

2.1.3 Key features

The everyday problem for a physics engine is the compromise between the following

• speed

• stability

• ”faithfulness”

• accuracy

• efficiency (accuracy versus work)

Speed

In order to create a realistic simulation the frequency of the images needs to be such that

the human apprehend it as realistic. To achieve that the refresh rate needs to exceed

20 Hz. The standard for games is near 60 Hz which also is the frequency for visual

comfort. In AgX the default time step is 1/60 s. This frequency gives little time for the

computational work required of the engine which is why it needs to be fast.

Stability

An important part of simulations is the stability of the integrator. If the simulation is run

for a long time time it is not eligible if the scenario explodes due to a constant increase

in energy. If, of course, no energy is added on purpose. For a conservative system the

energy should be constant or it could be oscillatory but with global bounds.

”Faithfulness”

”Faithfulness” means, in this case, that fundamental aspects of the laws of physics are

Chapter 2. Preliminaries 21

maintained as well as the damping introduced corresponds to physical damping. Shortly

one can say that the simulation should aim to replicate the real world. The solutions

provided by the physics engine is in fact the exact solution but of a slightly different

physical problem of the same nature [19].

Accuracy

Especially for simulations used in investigation purposes are forced to be accurate in

order to give the correct information. But accuracy is also important for other cases, for

example in training simulations. The human mind have an incredible understanding of

what is natural and reasonable even without having to measure things.

Efficiency

In many cases there is often possible to choose a numerical method that is very accurate

but the cost is often the speed. Therefore it is wanted to have as high accuracy as

possible with as little work as possible.

As mentioned the challenge is to compromise between these aspects. For example does

a stable method lacks in speed in many cases. Is is therefore important to choose the

correct method for the purpose depending on what is most important. Additionally it

is of desirable to avoid tuning as much as possible, i.e. to be able to simulate a wide

variety of scenarios without having to search for the perfect parameters [24][p.6-9].

2.2 Stable simulations

There are a large variety of things that can cause problems when simulating. In many

cases the troubles can be problem specific and requires knowledge about the physics in

the particular situation to solve. Some problems, though, are more general than others

and in this section some of them are mentioned and what to do about it.

Choice of time step

There are many problems that can occur when the time step is not chosen properly.

Consider for example the spring mentioned above where the frequency is too high. This

corresponds to choosing a too large time step in comparison to the velocity scale of the

system. The engine then fails to correctly simulate the oscillations of the spring. Also,

if a too large time step is chosen, the engine may not detect a collision between two

objects which can result in a tunneling, i.e. two solid objects going through each other.

For example a ball falling towards a floor. If the time step is too large the ball may

travel through the floor without the engine noticing it and preventing it from happening.

Some engine may have special methods for preventing this kind of events but not all.

The ball can also end up in the middle of the floor in one time step and this creates

Chapter 2. Preliminaries 22

large stabilization forces. These stabilization forces try to counter effect the constraint

violation and since they are large the simulation can be unstable. Additionally the choice

of time step can impose errors in the simulation. So, if the time step is too large it may

cause

• Large constraint errors, i.e. inaccurate simulations

• Tunneling

• Large interpenetrations which causes large stabilization forces and therefore unstable

simulations

The advantage of a large time step is that the simulation is fast. If the time step is too

small the simulation instead becomes slow [39].

Initial state

A properly chosen initial state provides a good start for the simulation. If two solid

bodies are initialized penetrating each other it will result in large forces and therefore also

high velocities. Other invalid initial states are unrealistic forces on motors for example

or unrealistic velocities. Additionally is it not good to initialize a system with large

constraint violations. All of these scenarios can result in unstable simulations [29].

Mass ratio

Another important thing to consider in simulations is the mass ratio. Very large mass

ratios can result in ill-conditioned matrices for the method to solve which can lead to

undesirable simulations. An example of a bad mass ratio is to attach an object of 109 kg

to a thin wire which weigh around 10−5 kg. This gives a mass ratio of 10−14 which can

be compared to machine precision that is 10−16. That means only 2 digits of accuracy

which will give all wrong results. It is therefore impossible for the computer so solve a

problem of this kind. [34].

Solver

In previous sections the numerical method Spook and its key features have been described.

It ended up with a system of equations that needs to be solved which can be done in

different ways. First there is the direct solver which solves the system ”exact” except

for rounding errors. The Gaussian elimination, for example, can be used for solving

linear system of equations. In AgX the direct solver is called Saber [26]. Next is the

iterative solver which generates an approximation of the solution and then improves the

approximation until a termination criteria or the maximum number of iterations are

reached. An example of this method is the Gauss Seidel method. The iterative solver can

have problems with stiff systems and large mass ratios where the simulation may end up

Chapter 2. Preliminaries 23

with rubber-like behaviour. But for non-linear cases the direct solver may have a hard

time solving the problem and then it can be a good idea to use the iterative solver [13].

For AgX dynamics the non-linearity is not a problem, since everything is linearized. Its

direct solver can have a problem, though, when the simulation contains many bodies

that are tightly connected.

The default solver in AgX is a hybrid solver which solves all constraint with the direct

solver, except the frictional contacts. It uses a direct method for joints and normals and

is coupled with iterative methods for friction, see Lacoursiere and Linde for details [25].

Chapter 3

Test cases

This chapter first describes the work flow of the thesis. Then the different test cases are

presented with both model set-up, a short theory introduction and the specifics of the

simulation. It is followed by the results of the test and all cases are summed up with a

short discussion.

3.1 Work flow

3.1.1 Pre-study

In the beginning of the work a pre-study was performed with information gathering

from articles, master’s dissertations, theses and web pages. The goal was to understand

how a physics engine works in general and which scenarios that can cause problems.

Furthermore it was investigated what has been done earlier regarding evaluation of

physics engines and which benchmarks that usually are used. The aim of the thesis is to

cover as wide range of test cases as possible but also to sort out the most useful ones

for the company. To be able to prioritize the test cases a number of interviews with

stakeholders at Algoryx were performed. Then it was discussed which cases that are

suitable and interesting for the company.

To be able to implement the platform an introductory course in AgX Dynamics was taken

since it is the engine provided for the work. The version 2.14.0.0 of AgX Dynamics is used.

Additionally, for writing scripts, some effort was put into learning the scripting language

Lua [28] which can be used to create simple simulations with AgX. The pre-study ended

up as a project plan.

24

Chapter 3. Test cases 25

3.1.2 Implementation

When the pre-study was finished the implementation of the platform began. The

simulations are written as script in Lua and the analyses are performed with Octave

3.8.2 which also produces the plots for the HTML-pages. First a few easy benchmarks

were implemented in order to learn how the programs work. Those were then evolved by

adding more complex scenarios. Simultaneously an export function was developed. Due

to limited time this was restricted to the web as an HTML-page. In parallel with the

implementation, the strategy for choosing and creating benchmark tests was developed

as well as the export and visualization functionality. Additionally, after the first few test

cases were almost done, the work with the report started.

When a couple of tests were done it was investigated how continuous tests could be

performed. Scalar trials for some of the test cases were made and put together in a

continuous test with additional result presentation. Also a prototype of a Web application

was tested in cooperation with the developers on Algoryx.

3.2 Test cases

Below is a presentation of all tests performed with the set-up, the results and a discussion.

All simulations are run with the default solver, i.e. the hybrid solver.

3.2.1 Beam — Constraint stability

The stability of lock constraints is the topic of this test case. It is examined by simulating

a beam situated on two piles. The beam is constructed using a number of boxes, i.e. it

is discretized as a chain of rigid bodies constrained to each other with lock joints. Fig.

3.1 illustrates the set-up of the model.

x

z

Figure 3.1: The set-up of the beam created by boxes situated on piles.

As can be seen in the figure the beam is situated on two piles which exert a lifting force

on the beam to counteract the gravity pointing in the negative z-direction. The wish is

Chapter 3. Test cases 26

as small height deviations as possible. In order to have stable simulations the damping

needs to be non zero, otherwise the beam started to oscillate and finally explode. In

the simulation both the constraint and the contact between the piles and the beam is

controlled regarding compliance and damping.

Two tests are performed, one where the maximum deviation in height is measured as

a function of time step and one where the number of boxes in the beam is varied. All

simulations are run for 1000 time steps for a fair comparison between the tests and all

objects are initially at rest. Due to the oscillations that may occur in the beginning of

the simulations the maximum deviation is measured for the last 50 time steps.

The parameters for the first simulations are:

• Mass/box: 2.5 kg

• Size of box (x, y, z): (0.125, 0.1, 0.1) m

• Number of boxes: 80

• Gravity (g): 9.81 m/s2

• Compliance (ε): 0

• Damping (τ): 2h

In this simulation the beam reaches a total length of 10 m, which is common length at

the steel beam market [40].

The second simulation investigates the effect of the number of boxes, i.e. the length of

the beam. The number goes from 10 up to 200 where 200 represents a beam of 25 m

and the simulation is run for 20 seconds. All other parameters are the same as in the

previous simulation and the time step used is 1/60 s.

3.2.1.1 Results

Fig. 3.2 illustrates the maximum deviation on the beam compared to the total length

versus the time step. As can be seen in the graph the deviations are overall small, smaller

than 0.1 ppm, except for a time step of about 10−4 s. For that time step the deviation is

approximately −0.2 · 10−6%. In the next figure, Fig. 3.3, the maximum deviation for

the beam simulated with the time step of 9.8 · 10−5 s is plotted versus time. There it

can be seen that the beam never stops oscillating which is the reason for that increased

deviation. For the other beams the oscillations eventually stopped and the simulations

Chapter 3. Test cases 27

stabilized. In the lower plot in Fig. 3.2 the deviation for larger time step is visualized

showing that for time step larger than 0.01 s the deviations start to increase. Since the

deviation is negative it implies that the beam is too low compared to the ideal height,

i.e. it has dropped down.

10−4 10−3 10−2 10−1

−0.2

−0.15

−0.1

−5 · 10−2

0

Time step [s]

Max

hei

ght

dev

iati

on

[10−6%

]Maximum height deviation compared to total length vs time step

Figure 3.2: The maximum deviation in height of the beam versus the time step.

0 2 · 10−2 4 · 10−2 6 · 10−2 8 · 10−2 0.1−1

−0.8

−0.6

−0.4

−0.2

0

Time [s]

Max

hei

ght

dev

iati

on[1

0−6%

]

Max height deviation vs time, h = 0.000098 s

Figure 3.3: The maximum deviation in height of the beam versus the time for timestep of 9.8 · 10−5 s

In the next figure, Fig. 3.4, the default time step is used and the deviation for each beam

element is plotted. It shows that the beam bends like a half moon, as expected, with

the largest deviation in the middle. The largest deviation is about −5 · 10−9 m and the

bending seems symmetrical.

Chapter 3. Test cases 28

0 20 40 60 80

−5

−4

−3

−2

−1

0

·10−3

Position on beam

Hei

ght

dev

iati

on

[10−

6m

]

Height deviation vs position on beam

Figure 3.4: The deviation for each box in the beam, i.e. the position on the beam,with default time step, 1/60 s.

In Fig. 3.5, the last figure, the maximum deviation of the beam height compared to

its total length is plotted versus the number of elements in the beam. Since the length

of each beam element and their weight are kept constant both the total length and

the total mass of the beam are increased as well. In the figure it is shown that the

deviation decreases rapidly with increasing number of elements, especially for 100 and

more elements which is longer than approximately 12 m.

101 102

−8

−6

−4

−2

0

·10−2

Number of elements

Max

hei

ght

dev

iati

onco

mp

ared

toto

tal

len

gth

[10−

6%

]

Maximum height deviation vs number of elements

Figure 3.5: The maximum deviation in height compared to the total length versusthe number of boxes constructing the beam. All values are negative.

Chapter 3. Test cases 29

3.2.1.2 Discussion

The stability of constraints is an important part of simulations. If the constraints start to

vibrate it can induce energy which can violate the energy conservation or even cause the

scene to explode. From the beam test it can be seen that the constraints in AgX are quite

stable with small height deviations. The deviations also appear to behave natural with a

symmetrical bending. The larger deviation for the time step of 9.8 · 10−5 s occurring

due to the oscillations could be a resonance phenomenon. That for just this time step

the beam experiences resonance. It could also be connected to the damping and bad

tuning of the damping parameter, see the Theory in Chapter 2. Maybe the problem

could be that the damping is put to a constant when instead τ/h should be constant.

The damping should act as a low pass filter removing all high frequencies which would

stabilize that beam is this simulation. If the damping is not tuned correctly this high

frequencies stay causing the beam to oscillate. Another explanation is an unknown safety

switch in the API preventing the user from using exactly 0 for the compliance to avoid

problems with the matrices. When the beam gets longer the deviations rapidly increases

as expected. It is therefore of great importance to choose time step wisely and also to

consider the number and dimensions of the elements in the beam for larger objects.

One way to extend the test is to make the beam even longer to really push the simulation

to the limits. It could also be of interest to change the size of the elements to see if a

beam with higher or lower resolution behaves differently. Additionally this stability test

could be complemented with a circular chain of constraint in order to see how constraint

violations in different directions are handled.

3.2.2 Door — Overdetermined system

Overdetermined systems, also called over-constrained systems, could cause problems for

multibody system dynamics. It is the case when the number of equations for the system

exceed the number of unknowns. The matrix involved in the equation system that needs

to be solved for the motion of the bodies in the simulation can be written with matrix

blocks as

H =

[M GT

G 0

], (3.1)

G =

[G(1)

G(2)

](3.2)

where M , G and 0 also are matrices. In the case of an overdetermined system G(1) = G(2),

which makes G row deficient. In Spook this is handled by introducing the compliance

Chapter 3. Test cases 30

in H instead of in the 0 matrix. It improves the condition number for the matrix and

makes the problem easier to solve.

To examine how the engine handles overdetermined systems a door on multiple hinges is

simulated. In theory it is only necessary with one hinge to keep the door up straight

but in reality a door usually has two or three hinges. Principally it is examined if the

engine can simulate an overdetermined system at all. Next the forces on the door from

the hinges are measured in the static case. Fig. 3.6 shows the set-up of the door and the

forces acting on it. When examining the forces in the z-direction it gives

0 = mg −N∑i=1

Fi,z (3.3)

where m is the mass of the door, N is the total number of hinges and Fi,z is the force

in z-direction for hinge i. In the ideal case when the hinges are exactly equal in both

properties and installation the forces in z-direction should be equally distributed between

the hinges. The deviation in this direction is calculated as

∆Fz =

∑Ni=1 Fi,z −mg

mg. (3.4)

x

z

mg

F1z

F2z

F1x

F2x

Figure 3.6: The set-up of the door with the forces acting on it.

If there are only two hinges the forces in the x-direction can be easily investigated as

0 = F1x + F2x ⇒ F1x = −F2x. (3.5)

That is, the forces are equal in magnitude but with opposite directions.

Chapter 3. Test cases 31

Two simulations are performed and the forces are measured on the static door. The

values of the compliance and damping are tuned until the best results were obtained.

First the number of hinges is varied. The parameters used in this simulation are:

• Time step: 1/60 s

• Mass: 45 kg

• Dimension of door (x,y,z): (1, 0.06, 2) m

• Gravity (g): 10 m/s2

• Compliance (ε): 10−13

• Damping (τ): 10−13

In the other simulation the door has two hinges and the mass of the door is altered in

order to see if it has an impact. All other parameters are the same as in the previous

simulation.

3.2.2.1 Results

First the number of hinges on the door is varied and the result is shown in Fig. 3.7. As

can be seen in the plot the deviation rapidly decreases towards 0 for increasing number

of hinges. The deviation is constant negative, i.e. the total force is too small compared

to the weight.

0 10 20 30 40 50 60 70 80

10−5

10−4

Number of hinges

Dev

iati

onin

tota

lFz

com

par

ed

tow

eigh

t[%

]

Deviation in total Fz compared to weight vs number of hinges

Figure 3.7: The deviation in total Fz compared to the weight of the door versusnumber of hinges. All values are negative.

Chapter 3. Test cases 32

In the second simulation the door constantly has two hinges but its mass is varied. Fig.

3.8 illustrates the deviation in x-direction, i.e. the difference in magnitude between the

forces from the two hinges, versus the mass of the door. The deviation is measured in

micro Newton (10−6N) and should be 0. It can be seen that the difference increases with

increasing mass. The last figure, Fig. 3.9, presents the deviation in total Fz compared to

the weight versus the mass. All deviations are negative. Also this plot shows a constant

negative deviation as the corresponding plot above. In this case the deviation seems to,

over all, increase with increased mass, in opposite to the above result.

0 100 200 300 400 500 600

0

0.1

0.2

Mass [kg]

Dev

iati

oninFx

[10−

6N

]

Deviation in Fx vs mass of door

Figure 3.8: The deviation in force in x-direction, i.e. |F1,x| − |F2,x|, versus mass ofthe door.

0 100 200 300 400 500 600

10−5.6

10−5.4

10−5.2

10−5

10−4.8

Mass [kg]

Dev

iati

onin

tota

lFz

com

par

ed

toth

ew

eigh

t[%

]

Deviation in total Fz compared to weight of door vs mass of the door

Figure 3.9: The deviation in total Fz compared to the weight of the door versus mass.All values are negative.

Chapter 3. Test cases 33

3.2.2.2 Discussion

As can be seen in the results Spook handles overdetermined systems without problem

since all simulations run with rather small deviations, the maximum is of order 10−5 %.

In fact it even seems as the simulation improves with increasing number of hinges, at

least the deviation decreases, see Fig. 3.7. This could be due to (3.3) since it is chosen to

not divide by the number of hinges, N , the forces may add up to a larger error. Another

possible deviation to investigate is ∆Fz =(∑N

i=1 Fi,z −mg)/N . When using only two

hinges and comparing the force in x-direction it can be seen that the deviation increases

with heavier doors. It therefore seems as mass has an impact in this case as confirmed

by both Fig. 3.8 and 3.9. Still the simulation does not explode. A door of 600 kg may

not be especially realistic, as well as a door with more than 200 hinges, but in this work

the goal is to test the engine, not to make a realistic simulation. The next step for this

case could be a swinging door instead of a static and how it is handled as well as analyze

the case with a different expression for ∆Fz.

3.2.3 Falling Object — Accuracy

This test case aims to evaluate the accuracy of the integrator used in the simulations.

The integrator is responsible for the calculations of the positions of the objects given the

forces that act on the bodies, as mentioned earlier.

Analytically the equation for a falling body is the following

xz(t) =1

2gt2 (3.6)

where xz(t) is the position (in z-direction) of the object with respect to time t. It is

assumed that the object is initially at rest at the origin, i.e. xz(0) = 0 and vz(0) = 0. The

equation is obtained due to that the only force acting on the body is the gravitational

force, i.e. xz = vz = g. By integrating this twice Eq. (3.6) is received. In the engine

time and space is discretized giving

vz = g ⇒ vz,n+1 − vz,nh

= g

xz = vz ⇒xz,n+1 − xz,n

h= vz,n+1

(3.7)

and time is discretized as tn = nh. Manipulating the above equations gives vz,n+1 =

vz,n + hg and xz,n+1 = xz,n + hvz,n+1. Provided that both initial conditions are 0 this

Chapter 3. Test cases 34

results in

vz,n+1 = nhg = [tn = nh] = tng

xz,n+1 = xz,n + h2(n+ 1)g = h2gn(n+ 1)

2.

(3.8)

From Eq. (3.8) it can be seen that the velocity should be reproduced exactly as the

analytical solution. For the position the situation is different. It can be written as

xz,n+1 =1

2gt2n +

1

2gtnh, (3.9)

i.e. the discretization error is the last term which depends both on time and the time

step and it is second order.

The model consists simply of a sphere, initially at rest, that is allowed to fall free under

constant gravity. Air resistance is neglected in both the simulation and the analytical

model in order to simplify the test case.

Two simulations are performed for this test case. First the simulation is performed with

the default time step for AgX. All parameters are:

• Time step: 1/60 s

• Mass: 1.0 kg

• Radius: 1.0 m

• Gravity (g): 9.81 m/s2

In the second simulation all parameters are the same but the time step is altered in order

to examine its impact on the deviations. Both simulations are run for 20 seconds.

3.2.3.1 Results

The following four figures shows the results from the simulations with the falling ball.

In the first plot, Fig. 3.10, the position of the sphere is plotted against time for both

the simulation and the analytical solution. From the graph it can be seen that both

curves have similar shape but the simulation is shifted in time. Fig. 3.11 shows the error,

i.e. the difference between the position obtained with the simulation and the analytical

solution as well as the velocity difference, versus time. There it can be seen that the

error for the position increases linearly with time and at the end of the simulation, at 20

seconds, the error is approximately 5 m. This is according to the theory presented above.

Chapter 3. Test cases 35

Additionally the error is positive which implies that the simulation is earlier than the

analytical solution. For the velocity there is no deviation.

0 2 4 6 8 10 12 14 16 18 20

−2,000

−1,500

−1,000

−500

0

Time [s]

Posi

tion

(z-d

irec

tion

)[m

]

Position in z-direction vs time

Simulation

Analytical

Figure 3.10: The position of the free falling sphere versus time for both the simulation(with default time step) and the analytical solution.

0 2 4 6 8 10 12 14 16 18 20

0

2

4

Time [s]

Posi

tion

erro

r[m

]

Position error (in z-direction) vs time

0 2 4 6 8 10 12 14 16 18 20

−0.5

0

0.5

1

Time [s]

Vel

oci

tyer

ror

[m/s

]

Velocity error (in z-direction) vs time

Figure 3.11: The error in position and in velocity for the free falling sphere simulationcompared to the analytical solutions versus time.

Fig. 3.12 and 3.13 illustrate the same plots as the previous two figures but for the second

simulation when the time step is altered in order to examine its impact on the result.

The impact of the time step is most obvious in Fig. 3.13 where the error deviates with

about 30 m between the largest and smallest time step. As is explained in the theory

Chapter 3. Test cases 36

the error depends both on the time and the time step. For the velocity there are no

deviations no matter time step or time.

0 2 4 6 8 10 12 14 16 18 20

−2,000

−1,500

−1,000

−500

0

Time [s]

Posi

tion

(z-d

irec

tion

)[m

]

Position in z-direction vs time

h = 0.1

h = 0.05

h = 0.025

h = 0.0125

h = 0.00625

Analytical

Figure 3.12: The position of the sphere versus time for both the simulation and theanalytical solution with different time steps.

0 2 4 6 8 10 12 14 16 18 20

0

10

20

30

Time [s]

Posi

tion

erro

r[m

]

Position error (in z-direction) vs time

h = 0.1

h = 0.05

h = 0.025

h = 0.0125

h = 0.00625

0 2 4 6 8 10 12 14 16 18 20

−0.5

0

0.5

1

Time [s]

Vel

oci

tyer

ror

[m/s

]

Velocity error (in z-direction) vs time

h = 0.1

h = 0.05

h = 0.025

h = 0.0125

h = 0.00625

Figure 3.13: The error in position and velocity for the free falling sphere compared tothe analytical solutions versus time for different time steps.

3.2.3.2 Discussion

Falling object is the simplest test case in the set but still important since it is a good way

to understand discretization and to investigate how accurate the integrator is. As the

Chapter 3. Test cases 37

theory describes, Spook is inaccurate since the error in the position is both time and

time step dependent. This is also confirmed by the results. But the velocity is ”exact”.

It can be argued that this test is more of an education than a performance test since free

falling bodies may not be a common to simulate. On the other hand it is the only test

that focus on the precision of the integrator which is why I see it as good complement.

3.2.4 Friction

Friction is an important part of our everyday life and it is essential to test how well it

can be simulated. The complicated thing is how to test it and since the topic is rather

extensive it is challenging to restrict the tests in order to make them understandable. In

this work the friction coefficient and the isotropy of the friction are investigated as well

as the Coulomb friction model. A box on an inclined plane is used as a model for this

test.

The Coulomb friction model states that

|Ft| ≤ µs|Fn| (3.10)

where Ft is the tangential force, Fn is the normal force and µs is the static friction

coefficient. Fig. 3.14 shows the forces and how they acts on the box.

x

z

mg

Fn

Ft

θ

Figure 3.14: The set-up of the box on an inclined plane with the forces acting on it.

From the geometry in Fig. 3.14 it can be seen that the forces can be expressed as

Ft = mg cos(θ), (3.11)

Fn = mg sin(θ). (3.12)

Chapter 3. Test cases 38

The equality in Eq. (3.10) arises when the plane reaches the maximum angle, i.e. the

angle when the box starts to move, which gives

Ft = µsFn ⇒ mg cos(θ) = µsmg sin(θ)⇒ tan(θ) = µs. (3.13)

In order to test the Coulomb friction model the angle when the box starts moving

is recorded for each friction coefficient. The angle is successively increased during

runtime until the box starts to move. The box is considered in motion when its velocity

exceeds a certain limit, γ m/s. According to Eq. (3.13) the angle should behave like

θ = tan−1(µs). Additionally the ratio of the forces is plotted against the tangent of the

angle, |Ft|/|Fn| = tan(θ) for all angles in order to see that the box is ”released” from the

plane at the correct angle. To investigate if the mass affects the friction in the simulation

this test is performed with different masses. Since there is no mass dependence in the

Coulomb friction model the results should not differ.

Finally it is tested if the friction behaves the same way in all directions, that is, if the

friction model is isotropic. The x− y plane is then inclined in all directions, not only

around the y-axis as in the previous tests. When tan(θ) = µs the ratio of the forces,

|Ft|/|Fn|, is measured. If the model is isotropic the ratio should be equal no matter in

which way the plane is tilted.

To simplify the simulations an active gravity vector is used, see Fig. 3.15, as suggested

by Seugling and Rolin [38]. When the plane is inclined around the y-axis there is only

one angle that is varied which is called θ. The active gravity vector is then

G = −g

sin(θ)

0

cos(θ)

. (3.14)

In the second case when the plane is inclined in all directions the active gravity vector

becomes

G = −g

sin(θ) cos(α)

sin(θ) sin(α)

cos(θ)

(3.15)

where α is varied between 0 and 2π.

Chapter 3. Test cases 39

x

z

G

θ

Figure 3.15: An illustration of the active gravity vector, illustration inspired fromSeugling and Rolin [38].

The parameters used in the first simulations are:

• Time step: 1/60 s

• Mass: 1 kg

• Dimension of box (x,y,z): (1, 1, 1) m

• Gravity (g): 10 m/s2

• Velocity limit (γ): 10−3 m/s

• Initial angle: 0 rad

For the other simulations the parameters are the same except that the friction coefficient

is kept at 0.5.

3.2.4.1 Results

Figures 3.16, 3.17 and 3.18 present the results from the simulations with a box on an

inclined plane. As mentioned in earlier the goal is to investigate how the engine handles

friction and the foundation for these tests is the Coulomb friction model. Fig. 3.16

shows the angle when the box starts moving, i.e. when its velocity is greater than the

limit value, versus the fiction coefficient. In the same graph the analytical solution is

plotted, θ = tan−1(µs), and the results seem to match rather well. When the coefficient

approaches 1 there is a noticeable difference, otherwise the graphs follow each other.

Chapter 3. Test cases 40

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

Friction coefficient µ

θ[r

ad]

Angle when the box starts sliding, θ, vs the friction coefficient µ

Simulation

Analytical

Figure 3.16: The angle, θ, when the box starts moving on the inclined plane versusthe friction coefficient compared to the analytical solution, θ = tan−1(µs).

In the second simulation the ratio of the forces is measured for all angles, θ, and the

graph is shown in Fig. 3.17. As can be seen in the figure the ratio stops increasing when

it reaches 0.5, which is the same value as the friction coefficient. This is in agreement

with the Coulomb friction model. The graph also confirms that the friction model is

mass independent in the physics engine. There are some differences before tan(θ) equals

the critical value 0.5 but the box is ”released” at the same angle and the ratio behaves

similarly for larger angles.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0

0.2

0.4

tan(θ)

|Ft|/|F

n|

|Ft|/|Fn| vs tan(θ)

1 kg

10 kg

102 kg

103 kg

104 kg

Figure 3.17: The force ratio, |Ft|/|Fn|, for the box on the inclined plane versus tan(θ)for different masses on the box.

Chapter 3. Test cases 41

The last figure, Fig. 3.18, is a polar plot of the force ratio measured when tan(θ) = µs

for three different numerical friction models. The cone friction model present an isotropic

behaviour since its plot is circular. For the other two models, the box friction model

and the scaled box friction model, the plots are shaped as an ellipse or a wavy circle

respectively. These friction models are therefore anisotropic.

0π

16π

26π

36π

46π

56π

1π

76π

86π

96π

106 π

116 π

0 0.2 0.4 0.6 0.8

Polar plot of |Ft|/|Fn| for different α angles

ConeBox

Scaled box

Figure 3.18: A polar plot of the force ratio, |Ft|/|Fn|, for the box on an inclined planemeasured when the angle satisfies tan(θ) = µs for different numerical friction models.

3.2.4.2 Discussion

AgX performs mainly well on the friction test. It replicates the expression for the friction

coefficient (θ = tan−1(µs)) nicely in the middle register but deviates a little at the

boundaries, i.e. where µs is close to 0 and 1. Another good property is that the floor

“lets go” of the box at the correct angle which can be seen in Fig. 3.17. The beginning of

the plot looks a bit odd though, since it is not a straight line but has a decrease around

0.3. For larger masses the decrease seems to be less. It is unsure what this can depend

on. It could be a property of the friction model. Finally the anisotropy is analysed. If

the correct friction model is used, the cone model, AgX shows an isotropic behaviour.

In cases where the isotropy is not important it may be more efficient to use one of the

other models but the opportunity for isotropy exists and works well. As mentioned in

the beginning of this chapter it would be a good idea to split this test to make it more

well-arranged.

Chapter 3. Test cases 42

The result of this test is rather similar as the the one obtained in the work of Hansson [20]

see Fig. 3.16, but it is better than the result in Boeing [6]. In the other two test,

comparable to the ones performed by Seugling and Rolin [38], AgX’s result seems even

better since it produces an isotropy cone with the right friction model.

3.2.5 Pendulum — Constraint accuracy

The pendulum is a test case that, similar to the beam, tests constraints but more precise

constraint deviations. This simulation uses a distance constraint and there is a constraint

between the world attachment point and the sphere. For a completely stiff system

there should be no deviation in the original length l but then the simulation explodes.

Therefore non-zero compliance and damping is introduced. The theory of how Spook

handles the pendulum is discussed in the previous chapter, Chapter 2. Additionally the

energy conservation of the simulations is investigated.

Initially the sphere is situated horizontally (at an angle θ = 90o). The model is shown

in Fig. 3.19. In the beginning of the simulation all energy in the system is potential

energy since the sphere starts from rest. This potential energy is then transformed into

kinetic energy due to gravity until the sphere reaches the bottom position where it has

its highest velocity. Next the process is reversed, the kinetic energy is transformed into

potential energy until the sphere reaches its highest position on the other end of the loop.

x

z

l

m

θ

Figure 3.19: The set-up of the pendulum.

During the simulation the distance between the attachment point and the sphere is

measured and normalized for a unit-less comparison. It is done according to

∆l =lmeasured − l0

l0(3.16)

Chapter 3. Test cases 43

where lmeasured is the measured constraint length in the simulation and l0 is the initial

distance.

The first simulation investigates the impact of time step. The parameters used are:

• Mass: 1.0 kg

• Length: 1.0 m

• Radius of sphere: 0.25 m

• Initial angle: 90o

• Gravity (g): 9.81 m/s2

• Compliance (ε): 10−13

• Damping (τ): h

All simulations are run for 22 seconds, i.e. a little more than 10 periods.

Secondly the impact of mass is examined. The default time step is used, 1/60 seconds, and

the masses are varied. All other parameters are the same as in the previous simulation.

3.2.5.1 Results

The two following figures present the result from the simulations with varied time step.

Fig. 3.20 shows the normalized deviation from the original length versus the time step.

It increases with the time step up to almost 7 % for a time step of 0.07 s. Since it is a

log-log plot with a slope of 2 it can be seen that the deviations are of second order, i.e.

the error is proportional to square of the time step.

In the second figure the energy deviation is plotted against the time step, see Fig. 3.21.

Also in this graph the deviation increases with increasing time step up to 7 % for the

largest time step, 0.07 s. In opposite to the above graph of the constraint deviation this

plot shows a behaviour towards linear convergence.

Chapter 3. Test cases 44

10−2.2 10−2 10−1.8 10−1.6 10−1.4 10−1.2

10−1

100

101

Time step [s]

Max

dev

iati

on

from

orig

inal

len

gth

[%]

Maximum deviation from original length vs time step

Figure 3.20: The deviation from the original length versus the time step for a non-stiffsystem of a pendulum.

1 2 3 4 5 6 7

·10−2

2

4

6

Time step [s]

En

ergy

dev

iati

on[%

]

Energy deviation vs time step

Figure 3.21: The deviation of the energy versus the time step for a pendulum with anon-stiff system.

Since both compliance and damping is used for these simulations the energy will decay.

For the energy deviation measurement done in the previous graph it is the deviations of

the oscillations that are considered, not the decay of the energy. Fig. 3.22 shows the

energy versus time for the simulation with a time step of 0.07 s and it clearly shows both

the oscillations which are measured and the energy decay.

Chapter 3. Test cases 45

0 5 10 15 20

6

8

10

Time [s]

En

ergy

[J]

Energy vs time

Figure 3.22: The energy versus time for a non-stiff system of a pendulum.

The above results can be compared to the same case with a totally stiff system, i.e. with

zero compliance and zero damping. In Fig. 3.23 the energy deviation is plotted against

the time step, as in Fig. 3.21. The graph shows that the pendulum explodes for all time

steps.

1 2 3 4 5 6 7

·10−2

2

4

6

·104

Time step [s]

Max

imu

mdev

iati

onfr

omori

gin

al

len

gth

[%]

Maximum deviation from original length vs time step, stiff system

Figure 3.23: The deviation of the energy versus the time step, for a completely stiffpendulum.

Fig. 3.24 shows the deviation of the length versus the mass for a simulation with the

default time step 1/60 s as well as the energy deviation. As the graph shows the mass

has no impact on neither the constraint violations nor the energy deviation.

Chapter 3. Test cases 46

0 200 400 600 800 1,000 1,200

0.35

0.4

0.45

Mass [kg]

Max

dev

from

org.

len

gth

[%]

Maximum deviation from original length vs mass

0 200 400 600 800 1,000 1,200

3

3.5

Mass [kg]

En

ergy

dev

iati

on

[%] Deviation in energy vs mass

Figure 3.24: The deviation from the original length versus the mass of the sphere inthe pendulum.

3.2.5.2 Discussion

The pendulum is thoroughly investigated in this thesis due to its simplicity as well as its

theoretical value. It is a rather simple test to set up but reveals a lot of information about

both constraint violations and energy conservation, especially when complemented with

the N-pendulum. For the single pendulum neither the constraint violations nor the energy

deviations vary more than 7 %. Also the plots in Fig. 3.20 and 3.21 show quadratic

convergence for the constraint deviations and linear convergence for the energy deviation,

as discussed in the Theory in Chapter 2. As mentioned in the results the energy oscillates

simultaneously as it decays due to the damping but the decay is neglected in this analysis.

If the damping is put strictly to 0 the simulation explodes and there is nothing left to

measure, as seen in Fig. 3.23. So the advantage with using non-zero compliance and

damping is stable simulations but the drawbacks are imposed constraint violations and

energy decay. It needs some tuning to find proper values for the compliance and damping

in order to get as stiff system as possible without having it exploding.

3.2.6 N-pendulum

This test is an extension of the pendulum test, it is a multibody pendulum. The aim

is to investigate the accuracy of the distance constraint in a simulation including many

spheres as well as the stability.

Chapter 3. Test cases 47

Fig. 3.25 shows the set-up of the pendulum with N bodies.

x

z

θ

Figure 3.25: The set-up of the N-pendulum, a multibody pendulum.

For simplification all initial distances are equal as well as the radius and masses of each

sphere. The parameters used in the first simulation are:

• Time step: 1/60 s

• Mass: 1.0 kg

• Radius: 0.25 m

• Initial angle: 90o

• Gravity (g): 9.81 m/s2

• Number of spheres: 26

• Compliance (ε): 10−13

• Damping (τ): h

Then the effect of time step is investigated, still with 26 spheres. All other parameters

are also the same. Finally the number of bodies is varied with a time step of 1/60 s.

3.2.6.1 Results

As with the pendulum this simulation investigates constraint violations and energy

deviations. For a multibody pendulum an extra factor is where the violation is largest.

Chapter 3. Test cases 48

Fig. 3.26 shows a bar plot of the maximum constraint deviation for each constraint when

using 26 spheres and the default time step, 1/60 s. From the figure it can be seen that

the largest violations occur for the spheres at the bottom of the chain as expected from

finite compliance.

0 5 10 15 20 25

0

5

10

15

Constraint

Maxim

um

dev

iati

on[%

]

Maximum deviation for each constraint

Figure 3.26: The maximum deviation of for each constraint in the pendulum.

Next the time step is varied, the results are shown in Fig. 3.27 and 3.28. In the first

figure it can be seen that the deviations increase with increasing time step, up to 40 %

for a time step of 0.02 s. The second figure shows that the energy deviation increases

with increasing time step.

0 0.5 1 1.5 2

·10−2

0

10

20

30

40

Time step [s]

Max

imu

md

evia

tion

[%]

Maximum deviation vs time step

Figure 3.27: The maximum deviation of the whole N-pendulum versus the time step.

Chapter 3. Test cases 49

0 0.5 1 1.5 2

·10−2

5 · 10−2

0.1

0.15

0.2

0.25

Time step [s]

En

ergy

dev

iati

on

[%]

Energy deviation vs time step

Figure 3.28: The energy deviation versus the time step for the N-pendulum simulation.

Finally different number of spheres are used and the plots from the simulations are shown

below. Fig. 3.29 illustrates the total maximum deviation for the whole pendulum versus

the number of bodies and it increases linearly. The last figure, Fig. 3.30 shows the energy

deviation versus the number and then the case is the other way, the deviation decreases

with increasing number of bodies. When N was larger than 100 the simulation exploded,

probably due to the high velocities obtained by the bodies at the end of the chain.

0 20 40 60 80 100

0

5

10

15

Number of spheres

Max

imu

mdev

iati

on[%

]

Maximum deviation vs number of spheres

Figure 3.29: The maximum deviation of the whole N-pendulum versus the number ofspheres, N .

Chapter 3. Test cases 50

0 20 40 60 80 100

0.5

1

1.5

Number of spheres

En

ergy

dev

iati

on

[%]

Energy deviation vs number of spheres

Figure 3.30: The energy deviation versus the number of spheres, N , for the N-pendulum.

3.2.6.2 Discussion

For the multibody pendulum the results are rather interesting. In Fig. 3.26, for example,

the maximum constraint violation does not occur for the last constraint, as one could

believe, but at the next last. The total potential energy increases linearly with the

number of links which, by itself, can generate very high kinetic energy at the end of the

chain. But this could also be some kind of whiplash that affects the next last sphere

most. Since it is a chaotic system it is hard to analyse. When varying the time step it

can be seen that the maximum constraint deviation increases with increasing time step,

as expected, but the energy deviation oscillates almost chaotic for different h. When

observing the simulations they do not explode but there may be introduced vibrations

not visible for the human eye for both the spheres and the constraints which contribute

to the energy increase. This could be due to too heavy damping and too large compliance

parameter but it could also be a bug in the script which haven’t been found. The last

test which varies the number of spheres shows that the deviations increase linearly with

increasing number. In this case it is worth remembering that the last body will have a

higher velocity at the bottom of the path since the total length of the chain of constraint

will be longer for larger number. Since the time step is kept fixed this high velocity

causes problems when N is too large. Interesting though, is that the energy deviations

are very low and even decreasing with increasing number of spheres. Due to the fact

stated above and the increasing constraint deviations this seems unlikely. This could

also be due to measurement errors or a bug.

Chapter 3. Test cases 51

3.2.7 Planetarium — Integrator stability

The aim of this test is to investigate the long time stability of the integrator. It is done

by simulating the Earth’s orbit around the Sun, a classical example often used to show

the stability, or instability, of a numerical method. In a stable simulation Earth should

have a rather constant energy. It should not constantly lose or gain energy which would

result in a spiral either in towards the Sun or out in space.

In order to simulate this scenario with reasonable period times and distances the system

is normalized and simplified. Earth is considered a point mass that orbits around the Sun

due to its gravity but the impact on the Sun due to the Earth is neglected. Additionally

forces from other bodies in space are ignored. Fig. 3.31 shows a sketch of the model

where x is the position vector and r = ‖x‖ is the distance between the Sun and the

Earth. Another simplification to the model is that it is performed in 2D, that is, the

z-direction is ignored in the simulation.

x

r

θ

v

Figure 3.31: A sketch of the planetarium, not to scale.

Two extra parameters are introduced in the model, the eccentricity ξ and the Gaussian

gravitational constant, k = 2π/365. Initially the Earth is situated at x0 = (1, 0, 0) with a

velocity of v0 = (−ξk, k, 0). In every step the Sun will exert a force on Earth calculated

as

F =−k2xr3

. (3.17)

The energy is then determined via

E =1

2m‖v‖2 +

k2

r. (3.18)

Chapter 3. Test cases 52

In addition to the energy the distance is also recorded versus the angle which then is

compared to the analytical solution which is

r(θ) =1

1 + ξ sin(θ). (3.19)

Eq. (3.19) is a normalized expression, related to Kepler’s first law [8].

The parameters used in the simulation are:

• Time step: 1/60 s

• Mass: 1.0 kg

• Eccentricity (ξ): 0.017

With this parameters one period, one lap around the Sun, takes about 363 seconds. The

simulation is run for 1200 s.

3.2.7.1 Results

If the engine’s integrator is stable, the energy for the Earth when orbiting around the

Sun should be rather constant. This can also be visualized by plotting the orbit as seen

in Fig. 3.32. It is a plot of the position vector in the x− y plane. It shows that the orbit

is round and there is no signs that the Earth starts to deviate from the orbit, e.g. spiral

inwards or outwards, which would be the case for an unstable simulation.

−1 −0.5 0 0.5 1

−1

−0.5

0

0.5

1

x

y

The orbit of the Earth and the Sun

Figure 3.32: The simulated orbit of the Earth around the Sun in the x− y plane.

Chapter 3. Test cases 53

The next figure, Fig. 3.33, presents a classical plot of the energy deviation versus time,

or more precisely the deviation from the initial value of the energy. It shows that the

energy oscillates around the initial value but does not increase or decrease over all, it

seems to be a pure oscillation. An additional long time test was also run which only task

was to warn when or if the energy oscillated more than 5 %. The test was run for 6.5

hours simulated time, i.e. 14.04 · 105 steps with the default time step of 1/60 s, and the

energy never exceeded the limited values during the whole run.

In Fig. 3.34 the distance r is plotted against the angle, θ which also is an oscillating

function. Both the simulation result and the analytical solution, see Eq. (3.19), are

plotted. Next to the result from the simulation is the analytical expression for the orbit

and it can be seen that the graphs are rather similar. Some deviations may be notices at

the peaks.

0 200 400 600 800 1,000 1,200

−2

−1

0

1

2

·10−2

Time [s]

En

ergy

dev

iati

on[%

]

Energy deviation of the system vs time

Figure 3.33: The energy of the planetarium system versus time.

Chapter 3. Test cases 54

0 5 10 15 200.98

0.99

1

1.01

1.02

Distance r [AU]

θ[r

ad

]

Distance between bodies r vs angle θ

Analytical solution Simulation

Figure 3.34: The distance between the Earth and the sun versus the angle.

Finally it is examined how the time step affects the energy deviations, see Fig. 3.35. A

part from a jump in the beginning the deviation seems to increase linearly with the time

step.

0 1 2 3 4 5

0

2

4

6

Energy deviation [%]

Tim

est

ep[s

]

Energy deviation vs time step

Figure 3.35: The energy deviation versus time step for the planetarium simulation.

3.2.7.2 Discussion

One of the strengths with Spook is its stability. As can be seen in the planetarium

test the energy oscillates but the oscillations are always bounded, they do not overall

decrease or increase. Not even in the long time test of 6.5 hours did the energy oscillate

Chapter 3. Test cases 55

with more than 5 %. Some numerical methods may be more accurate but they tend

to collapse the path due to energy decay or explode after a couple periods. This test

shows that AgX may not be very accurate but it is stable. Fig. 3.35 contributes to the

conclusion since a time step as large as 5 s is used and the deviation still does not exceed

8 %. The figure also illustrates that the systems shows linear convergence of the energy.

3.2.8 Cantilever beam

In this test the goal is to investigate how well a real beam is simulated. It is performed

by using a cantilever beam attached to the wall with an applied load at the free end,

see Fig. 3.36 which illustrates the set-up of the model. The model is a HEA beam of

steel which is a European Wide Flange H Beam [16]. It has the shape of a lying H but

is simulated using boxes, to compensate this the compliance and damping is tuned to

mimic a real beam as good as possible. The test takes place in uniform gravity pointing

in the negative z-direction.

P

l

x

z

Figure 3.36: The set-up of the cantilever beam, i.e. a beam attached to the wall witha load on the free end.

Due to the applied load the beam will bend. To analyse how well the beam is simulated

its shape is compared to the analytical expression for the beam deflection

h(x) =Px2

6EI(3l − x) (3.20)

where h is the deflection in z-direction, P is the applied load, x is the position on the

beam (provided that x = 0 is situated at the well where the beam is attached), E is

the elastic modulus, I is the second moment of inertia and l is the total length of the

Chapter 3. Test cases 56

beam [36]. The elastic modulus for steel is approximately 210 GPa [2] and the moment

of inertia for a HEA beam can be calculated to about 2.305 µm [3].

The HEA beam is chosen since it is one of the most commonly used beams. There are

different dimensions (see Telge Stalcenter [40] for example) and the one simulated in this

test case is called HEA 120. Its dimensions are; 114 mm high, 120 mm wide, 12 m long

and with a weight of 19.9 kg/m. Since the beam will oscillate in the beginning of the

simulation the deflection is measured when the velocity of the load is less than a limit γ

in the z-direction.

The parameters used in the simulation are:

• Time step: 1/60 s

• Number of beam elements: 60

• Dimension of element (x,y,z): (0.125,0.120,0.114) m

• Mass/element: 2.985 kg

• Load (P ): 10 kg

• Gravity (g): 9.81 m/s2

• Velocity limit (γ): 10−4 m/s

• Compliance (ε): 10−8

• Damping (τ): 10−5

The values of the compliance and the damping were tuned manually until the best result

was obtained.

3.2.8.1 Results

The following figure, Fig. 3.37, shows the result of the cantilever beam test, both the

analytical solution h(x), the simulation and the difference between them. It can be seen

that the simulation bends down more than the analytical solution, especially at the free

end of the beam. The main challenge with this test was to find the proper values for the

compliance and damping. If the system was made stiffer the beam either got too stiff,

i.e. the result got worse, or the simulation exploded. The damping, for example, could

not take any smaller values than 10−5, then the beam started vibrating and eventually

exploded, as described in the Theory in Chapter 2.

Chapter 3. Test cases 57

0 2 4 6 8 10 12

−0.15

−0.1

−5 · 10−2

0

Position on beam [m]

Hei

ght

dev

iati

on

[m] Height deviation vs position on beam

Simulation

Analytical solution

0 2 4 6 8 10 12

0

5 · 10−2

0.1

Position on beam [m]

Diff

eren

ce[m

]

Difference in height between analytical and simulation vs position on beam

Figure 3.37: The deflection of the cantilever beam versus the position for both thesimulation and the analytical expression (h(x)). The lower plot shows the deviation

from the analytical solution versus position.

3.2.8.2 Discussion

The cantilever beam test differs a bit from the other test since it is a steady state test

and it is a real scenario which can occur in the everyday life. Most of the other tests are

manipulated to really test the limits of the engine or to investigate certain properties

of the method. As can be seen in Fig. 3.37 the beam has the correct shape but is a

little too flexible. When attempts were made to improve the results by making the beam

more stiff it either got too straight or exploded. It seems like the vibrations of the beam

cannot be damped but instead increases until the constraints cannot hold together any

more. It could be due to incorrect tuning of the damping parameter. This test could

be developed by analysing the energy of the beam or to record the forces to see what

happens.

Chapter 4

Additional work

In this chapter the additional work is presented, i.e. a strategy, the result presentation,

continuous tests and the interactive web based simulation.

A strategy for choosing test cases and performing the tests have been developed throughout

the work of this thesis in order to give some guidelines in the future of testing physics

engine. The guidelines can be seen in Appendix A.

4.1 Result presentation

Each test presented above is executed via a bat-script situated in a folder with all the

necessary files. All folders and scripts form the toolbox. When a test is performed the

results are presented on an HTML-page in a browser with text and plots that both

shortly explain the theory and comment on the result. Fig. 4.1 shows a screen shot of

the presentation page for the pendulum test.

58

Chapter 4. Results 59

Figure 4.1: A screen shot of the HTML-page that presents the results when a test isperformed.

4.2 Continuous testing

An additional vision with this thesis was to construct continuous tests that could be run

for example every night in order to check the status of the engine and also record trends

over time. Since the result of these tests needs to be easy to grasp quickly, scalar tests

were developed. A scalar test is a test that investigates the scalar value of a variable

and if it is within the allowed interval. These test returns either true or false depending

on if they are passed or not. It is a way to avoid the need to study graphs in order to

validate if the test was passed. The set points for the scalar tests are chosen from the

performance of the engine in January 2016.

After all tests are performed the results are presented in a list together with coloured

icons that are green for pass and red for fail. The results are also saved as a DAT file

with the day’s date in order to be able to compare the performance of the engine over

time. Fig. 4.2 shows a screen shot of the list with the results.

Chapter 4. Results 60

Attached to this report is a document with a more detailed description of the tests, see

Tests - A short text about suggestions of continuous tests. The following list summarises

the document:

• Beam, height deviation should be less than 4.9 · 10−9 %

• Door, deviation in vertical force should be less than 0.02 %

• Falling Object, position should not deviate from the analytical solution more than

4.904 m after 20 seconds

• Friction, the ratio of the forces should be |Ft|/|Fn| = 0.5± 10−4 when the plane is

inclined in all directions (test of anisotropy)

• Pendulum, constraint deviation should no exceed 0.004 % from original length

• N-pendulum, maximum deviation of all constraint should be less than 16 %

• Planetarium, the energy should not oscillate more than 2.37 %

Figure 4.2: The resulting table of the continuous test with the test name, the value ofthe test variable and pass or fail result.

4.3 Interactive Web based Simulation

Finally a prototype of a World Wide Web application is developed. By using the web tool

produced by Algoryx a simulation can be run on a web page and simultaneously interact

with the user. That is, the user can influence the simulation while it is running on the

Web. Additionally run time plots can be performed and documentation can be added.

Fig. 4.3 shows a screen shot of the web simulations. It illustrates how the application

Chapter 4. Results 61

is built and how the simulation and plot fit on the same page. Documentation can be

added as an extra slide in the simulation window.

Figure 4.3: A screen shot of a web simulation of the box on an inclined plane withrun time plot of the force ratio versus the angle.

Chapter 5

Discussion

This section discusses the general aspects of this thesis such as strategy, export function-

ality, continuous testing, Web application and the chapter ends with a conclusion listing

possible developments of the work and thoughts for the future.

5.1 General

Creating a toolbox is not just about putting tests together, performing them and analysing

the result. Since the vision is a toolbox that can be used in the future and not only for

this thesis some kind of standard and strategy need to be developed. In this report a try

to create a strategy for choosing and performing a test case have been made. The key

idea is to start with a question that the test should aim to answer. If a topic is picked

first the test can easily grow very fast and it can be hard to see its purpose in the end.

It is also of great essence that the question is not too general. One single test can for

example not find out how good an engine is on collision handling, then that test will be

very large and hard to grasp.

In my opinion it is better to have more tests that each answers one question. For example,

the friction test in this report may be split in two or even three different test cases. One

test could investigate the anisotropy of the friction model, another analyse the ratio of

forces versus the tilting angle and the last could check if the friction coefficient is handled

correctly. This is not done due to limited time. Additionally a toolbox needs a coherent

way of presenting the result, this needs to be established.

62

Chapter 5. Discussion 63

5.2 Continuous testing

The continuous tests in this report are rather engine specific since they aim to follow the

performance of AgX. There may be a way to create a set of tests that are more general

and can be used to compare engines but that was outside the scope. Additionally the

limits for the different parameters may need to be changed along with the progress of the

engine. I believe, though, that this is an important factor when testing the engine but

that it is sometimes forgotten. It seems to be more common to control the programming

related part such as memory leakage and crashes.

5.3 Interactive Web based simulation

As mentioned earlier the World Wide Web application is a prototype performed in order

to see what the outcome may be and if it could be a way of presenting the tests and

their results. Even though this part was outside the scope of the thesis the application

could be tested due to the great work of the developers at Algoryx. Even though the

interactivity of the simulations is a nice property it may not be desired for the testing

since then it is required that the cases are performed in the same way every time. If the

human factor is included the tests will not be replicable. It is something that should be

kept in mind if the web application is further developed in order to cooperate with the

test cases. Maybe it can be a complement to the tests when presenting the engine and

demonstrating its performance.

5.4 Conclusion

It is not an easy task to test a physics engine. The number of tests that can be done

is almost infinite. Although the focus in this thesis is physics, especially mechanics,

the subject is extremely wide. There exists no general set of tests that works for every

developer or customer or user. As discussed in Chapter 1 many tests, for example for

game engines, focus on speed and somewhat neglect the accuracy. Although it in many

cases is important to evaluate certain aspects of an engine, since not everything can be

tested, the engine always needs to compromise between speed, stability, “faithfulness”,

accuracy and efficiency, as explained in Chapter 2. An engine that is very fast may

therefore be rather inaccurate or unstable. If speed is the highest priority this may be

OK but it can be good to remember since exploding simulations tend to be fairly useful.

Chapter 5. Discussion 64

With more time this thesis can become a really large piece of work since there are many

ways to extend its scope. As mentioned above the number of test can be extremely large

as physics is a wide subject. A part from tests the export function also has development

potential. Two prototypes have been done, the HTML-page shown in Chapter 4 and

the web application presented in the same chapter. Both are just prototypes and can be

further evolved regarding design and maybe with different comments and text depending

on the results of the test. Another idea for the export functionality is to be able to

produce a PDF-document with the results after a performed test. Or maybe it could be

possible to get the results to the phone. The only limit is the imagination.

A topic that also is important but ended up outside the content of this work is version

handling. This could be especially crucial when using the same tests for a period in order

to see trends and follow the progress of an engine. It may then be of interest to keep

track of which version of the engine that was used, for comparison. Another aspect that

could be of significance is the possibility to compare engines. It was not the idea with

this thesis but it would be possible to make some changes to make it work.

As mentioned above and earlier there does not exist a general set of tests that analyses

everything for all physics engine. Some attempt have been made such as MBS Benchmark

and PEEL. This thesis is not general either, not even close. But it can be seen as an

attempt towards a universal set as well as an addition to the other works. Many more

tests need to be added in order to cover a wider part of the physics field. Examples

are material interaction such as a bouncing ball, constraint stability, conservation of

angular momentum such as a rotating box, impulses, buoyancy, torques, wires as well as

more complex cases such as vehicles, scalable constraints e.g. a scissor lift and robots.

Also larger tests are needed with thousands of bodies since this is the everyday life for

many engines. A part from more test the toolbox also needs to work for other engines

in order to be general. In summary, we are a long way from a general benchmark set

that investigates the physical aspect of an engine. But the ball is rolling and I believe

that there will be more effort put into this in the future, especially due to the rapid

development of physics engines and their increasing demand. Hopefully this report can

provide useful advices, ideas and work as a guideline in that future work.

Appendix A

Strategy and Work flow

Following are guidelines for a work flow when choosing and performing a test case

1. Determine the purpose of the test. A good idea is to first create a question and

then find a benchmark that answers the question.

2. Define what should be tested, i.e. which scenario that should be modelled (also

specify important parameters etc.) and what should be measured during the

simulation. Additionally reference data need to be found, e.g. an analytical

solution.

3. Model the test case and implement it. If needed, tune parameters to improve the

simulation.

4. Perform the simulation.

5. Report the result. Include 3D graphic, plots and text regarding the test case, the

model, the result and conclusions and possible explanations to the result.

65

Bibliography

[1] Algoryx Simulation AB. About algoryx. URL http://www.algoryx.se/company/

about-algoryx/. Accessed 2015-01-20.

[2] Broderna Edstrand AB. Roda katalogen - teknisk handbok, 1999. URL http://www.

begroup.com/upload/Sweden/Broschyrer/R%C3%B6da_Katalogen.pdf. Accessed

2015-06-04.

[3] R. Adams. Introduction to beam theory. URL http://paws.wcu.edu/radams/

intro_to_beam_theory.pdf. Course notes. Accessed 2015-06-04.

[4] D. D. Baumann. Principle of least action. URL http://www.damtp.cam.ac.uk/

user/db275/LeastAction.pdf. Cambridge University. Lecture notes. Accessed

2016-01-24.

[5] IBM Bluemix. Open source physics engines. URL https://www.ibm.com/

developerworks/library/os-physicsengines/. Accessed 2015-07-05.

[6] A. Boeing and T. Braunl. Evaluation of real-time physics simulation systems.

In Proceedings of the 5th International Conference on Computer Graphics and

Interactive Techniques in Australia and Southern Asia (GRAPHITE), pages 281–

288. Assoc Computing Machinery, Dec 2007. URL http://www0.cs.ucl.ac.uk/

teaching/VE/Papers/p281-boeing.pdf. Accessed 2015-06-09.

[7] D. M. Bourg. Physics for game developers. O’Reilly and Associates, 1005 Gravenstein

Highway North Sebastopol, CA, 1 edition, 1 2002. ISBN 0-596-00006-5.

[8] A. J. Brizard. Motion in a central-force field. URL http://www.dartmouth.edu/

~phys44/lectures/Chap_4.pdf. Saint Michael’s College. Lecture notes. Accessed

2015-12-20.

[9] D. Eberly. Game Physics. The Morgan Kaufmann series in interactive 3D technology.

CRC Press, 2 edition, 4 2010. ISBN 978-0-12-374903-1.

66

Bibliography 67

[10] T. Erez, Y. Tassa, and E. Todorov. Simulation tools for model-based robotics:

Comparison of Bullet, Havok, MuJoCo, ODE and PhysX. URL https://homes.cs.

washington.edu/~todorov/papers/ErezICRA15.pdf. Accessed 2015-01-21.

[11] K. Erleben. Module based design for rigid body simulators, 2002. URL http://www8.

cs.umu.se/kurser/TDBD24/VT05/articles/ErlebenModular2002.pdf. Accessed

2015-08-26.

[12] R. Fitzpatrick. Damped harmonic oscillator, 2013. URL http://farside.

ph.utexas.edu/teaching/315/Waves/node10.html. The University of Texas at

Austin. Accessed 2015-12-26.

[13] W. Frei. Solutions to linear systems of equations: Direct and

iterative solvers, 2013. URL https://www.comsol.com/blogs/

solutions-linear-systems-equations-direct-iterative-solvers/. COM-

SOL Blog. Accessed 2015-12-28.

[14] M. Gonzalez, F. Gonzalez, A. Luaces, and J. Cuadrado. A collaborative benchmark-

ing framework for multibody system dynamics. Engineering with Computers, 26:

1–9, Feb 2010. doi: 10.1007/s00366-009-0139-0.

[15] M. Gonzalez, D. Dopico, U. Lugris, and J. Cuadrado. A benchmarking system for

MBS simulation software: Problem standardization and performance measurement.

Multibody System Dynamics, 16:179–190, Sep 2006. doi: 10.1007/s11044-006-9020-8.

[16] BE Group. Information balk. URL http://www.begroup.com/

sv/BE-Group-sverige/Produkter/Stal_ror/Produktinformation/

Produktinformation-balk/. Accessed 2015-05-31.

[17] S. Hadap, D. Eberle, P. Volino, M. C. Lin, S. Redon, and C. Ericsson. Collision detec-

tion and proximity queries. URL http://www.cc.gatech.edu/~jarek/graphics/

papers/06CollisionCourseSIG04.pdf. Course notes. Accessed 2015-08-28.

[18] E. Hairer. Symmetric Projection Methods for Differential Equations on Manifolds,

volume 40. Springer Netherlands, 2000. doi: 10.1023/A:1022344502818.

[19] E. Hairer, C. Lubich, and G. Wanner. Geometric Numerical Integration, volume 31

of Springer Series in Computational Mathematics. Springer Berlin Heidelberg, 2001.

ISBN 978-3-540-30663-4.

[20] H. Hansson. Craft physics interface. Master’s thesis, Department of Electrical

Engineering, Linkoping Universty, 2007. URL http://liu.diva-portal.org/

smash/get/diva2:23270/FULLTEXT01.pdf. Accessed 2015-01-26.

Bibliography 68

[21] J. Hummel, R. Wolff, T. Stein, A. Gerndt, and T. Kuhlen. An evaluation of open

source physics engines for use in virtual reality assembly simulations. In Advances

in Visual Computing, Lecture Notes in Computer Science, pages 346–357. Springer

Berlin Heidelberg, 2012.

[22] U. Kargen. Integrering av 3d-fysikmotor i simuleringsramverk for telekrigdueller.

Master’s thesis, Department of Electrical Engineering, Linkoping University, 2009.

URL http://www.diva-portal.org/smash/get/diva2:210893/FULLTEXT01.pdf.

Accessed 2015-01-29.

[23] S. Kimmerle. Real-time collision detection for dynamic virtual environ-

ments - bounding volume hierarchies, 2005. URL http://www-ljk.imag.fr/

Publications/Basilic/com.lmc.publi.PUBLI_Inproceedings@117681e94b6_

1860ffd/bounding_volume_hierarchies.pdf. Course notes. Accessed 2015-09-18.

[24] C. Lacoursiere. Ghosts and machines: regularized variational methods for interactive

simulations of multibodies with dry frictional contacts. PhD thesis, Umea University,

2007. URL http://umu.diva-portal.org/smash/record.jsf;jsessionid=

9-EtOC74mNM8CBk_xvvfR0j95hrpUGVAr2NPs1SM.diva2-search2-vm?pid=diva2%

3A140361&dswid=-1888. Accessed 2015-09-03.

[25] C. Lacoursiere and M. Linde. SPOOK: a variational time-stepping scheme for rigid

multibody systems subject to dry frictional contacts. Manuscript draft UMINF 11.09,

Department of Computing Science, Umea University, SE 901 87 Umea, Sweden, Sep.

2011. URL http://www8.cs.umu.se/research/uminf/reports/2011/009/part1.

pdf. Accessed 2015-12-15.

[26] C. Lacoursiere, M. Linde, and O. Sabelstrom. Direct sparse factorization of blocked

saddle point matrices. In PARA 2010, Part II, volume 7134 of LNCS, pages 324–335.

Springer, 2012.

[27] J. Lander and C.Hecker. Product review of physics engines. URL

http://www.gamasutra.com/view/feature/196198/product_review_of_

physics_engines_.php?print=1. Accessed 2015-01-27.

[28] Lua. About, 2015. URL http://www.lua.org/about.html. Accessed 2016-01-23.

[29] Mathworks. Troubleshooting simulation errors. URL http://se.mathworks.com/

help/physmod/simscape/ug/troubleshooting-simulation-errors.html. Ac-

cessed 2015-12-28.

[30] I. Metrikin, A. Borzov, R. Lubbad, and S. Loset. Numerical simulation of a floater in

a broken-ice field – Part II: Comparative study of physics engines. In Proceedings of

Bibliography 69

the ASME 31st International Conference on Ocean, Offshore and Artic engineering,

pages 477–486. Amer Soc Mechanical Engineers, Jul 2012.

[31] B. Mirtich. Efficient algorithms for two-phase collision detection, 12 1997. URL

http://www.merl.com/publications/docs/TR97-23.pdf. Accessed 2015-09-11.

[32] B. Mirtich and J. Canny. Impulse-based simulation of rigid bodies. In Proceed-

ings of the 1995 symposium on Interactive 3D graphics, pages 181–ff, Apr 1995.

doi: 10.1145/199404.199436. URL https://graphics.stanford.edu/courses/

cs468-03-winter/Papers/ibsrb.pdf. Accessed 2015-01-27.

[33] MBS Benchmark: Multi-Body Systems Benchmark (University of La Coruna).

Benchmark Specifications. URL http://lim.ii.udc.es/mbsbenchmark/docs/

specifications.html. Accessed 2015-01-23.

[34] Bullet physics library. Physics tips, 2006. URL http://www.bulletphysics.org/

mediawiki-1.5.8/index.php?title=Physics_Tips. Accessed 2015-12-28.

[35] A. Roennau, F. Sutter, G. Heppner, J. Oberlaender, and R. Dillmann. Evaluation

of physics engines forr robotic simulations with a special focus on the dynamics of

walking robots. In 16th International Conference on Advanced Robotics (ICAR),

pages 1–7. IEEE, Nov 2013. doi: 10.1109/ICAR.2013.6766527.

[36] A. Ruina. Beam deflection formulae, 2013. URL http://ruina.mae.cornell.edu/

Courses/ME4735-2012/Rand4770Vibrations/BeamFormulas.pdf. Course formu-

lae. Accessed 2015-06-04.

[37] O. Bursi S. Erlicher, L. Bonaventura. The analysis of the Generalized−α methods

for non-linear dynamic problems. In Computational Mechanics, pages 83–104.

Springer Verlag, 2002. doi: 10.1007/s00466-001-0273-z. URL https://hal-enpc.

archives-ouvertes.fr/hal-00345290/document. Accessed 2016-05-11.

[38] A. Seugling and M. Rolin. Evaluation of physics engines and implementation of a

physics model in a 3-d authoring tool. Master’s thesis, Department of Computing

Science, Umea University, 2006. URL http://www.bulletphysics.com/ftp/pub/

test/physics/papers/SeuglingRolin.pdf. Accessed 2015-01-27.

[39] N. Souto. Video game physics tutorial - Part II: Collision de-

tection for solid objects. URL http://www.toptal.com/game/

video-game-physics-part-ii-collision-detection-for-solid-objects.

Accessed 2015-12-28.

[40] Telge Stalcenter. HEA-Balk, 2014. URL http://www.telgestalcenter.se/

hea-balk.html. Accessed 2015-06-04.

Bibliography 70

[41] P. Terdiman. Code corner - the evolution of PhysX. URL http://www.codercorner.

com/blog/?p=748. Accessed 2015-08-03.

[42] G. von Bergen. Collision detection in interactive 3D environments. The Morgan

Kaufmann series in interactive 3D technology. Morgan Kaufmann Publishers, 500

Sansome Street, Suite 400, San Francisco, 2004. ISBN 1-55860-801-X.

[43] Wikipedia. Harmonic oscillator. URL https://en.wikipedia.org/wiki/

Harmonic_oscillator. Accessed 2015-12-26.