programming robots
DESCRIPTION
Programming robots using visualisation, concurrency and coordination, using the Microsoft Robotics Developer Studio and the Visual Robotic Development Kit prototype held for the British Computer Society (BCS) in Cambridge, UK in 2006.TRANSCRIPT
ERO Technology Group
Programming RobotsProgramming with Visualisation, Concurrency and Coordination
Presented byAndreas [email protected] Date08. November 2006
Definition: Robot
A robot is an electro-mechanical device that can perform autonomous or preprogrammed tasks. A robot may act under the direct control of a human (eg. the Canadarm on the space shuttle) or autonomously under the control of a programmed computer.
Wikipedia1. Mechanical device programmed to perform tasks
any machine that can be programmed to carry out instructions and perform particular duties, especially one that can take over tasks normally done by people.
2. Imaginary machine like humana machine that resembles a human in appearance and can function like a human, especially in science fiction
3. Person like a machinesomebody who works or behaves mechanically, showing little or no emotion and often responding to orders without question
4. Traffic lightSouth Africa a set of automatic traffic light
Encarta Dictionary
Definition: Robot
A robot is a device, hard- or software with the capability of sensing and (re)acting.
Robotics Market PotentialService and consumer markets just emerging
• Remote assistance/presence• Assistive• Facilities maintenance• Security• Education• Entertainment
More applications are required to improve the market
• A lack of reusability keeps robot developers endlessly re-solving the same software problems
• Concentrating on the mechanics of robotics rather than the science of robotics
• Concurrency and complexity plague all software engineering and especially robotics
• Development requires too much effort and knowledge
0
10000000
20000000
30000000
40000000
50000000
60000000
70000000
Mar
ket
Siz
e $1
,000
s
1995 2000 2005 2010 2025
Year
Worldwide Robotics Market Growth
Home
Mediacal/Welfare
Public Sector
Bio-Industrial
Manufacturing
* Source Japan Robotics Association
Robotics Market Potential
0
10000000
20000000
30000000
40000000
50000000
60000000
70000000
Mar
ket
Siz
e $1
,000
s
1995 2000 2005 2010 2025
Year
Worldwide Robotics Market Growth
Home
Mediacal/Welfare
Public Sector
Bio-Industrial
Manufacturing
* Source Japan Robotics Association
Development Trend
2005
Rea
l vac
uu
m c
lean
er
2007 2010
2015 2020
Ro
bo
t w
ith
inte
gra
ted
Co
mm
un
icat
ion
Cap
abili
ties
1
Ro
bo
t lo
adin
g D
ish
was
her
2
To
urg
uid
e3
Ro
bo
t ad
apti
ng
to
ind
ivid
ual
Nee
ds
1 Filtering unnecessary Information2 Household Devices (Microwave, Fridge)3 Learning about new Environments by giving a Tour to it, after that it can re-give that tour.
Ro
bo
t h
elp
ing
han
dic
app
ed P
eop
le
Co
mp
anio
n f
or
eld
erly
Peo
ple
Gam
ing
Ro
bo
ts
Iro
nin
g R
ob
ot
Pro
acti
ve R
ob
ots
Ro
bo
ts u
nd
erst
and
ing
hu
man
Act
ivit
ies
24h
Ass
ista
nt
Increasing Software Challenges
* Source EURON Workshop 2005
Microsoft & Robotics
Systems
• PC Ecosystem (desktop, web, mobile, home)
• XP, XP Embedded, CE (real-time)
Development environment and tools
• Programming IDE• Debugging and Opmization
Existing applicable Technologies
• Speech SDK• Real-time Communications SDK• RFID SDK
Experience Groups
Microsoft Research
• Enabling Technologies• Human Robot Interaction• Personal Robotics• Educational Robotics• Programming Environments
Microsoft Robotics Groups
• Microsoft Robotics GroupRedmond
• Center for Innovative RoboticsCarnegie Mellon University
• External Research OfficeCambridge
New Requirements
• Ultra Heterogeneity• Distributed environments• Dynamic Configuration• Context-Awareness• Personalization• Extensibility• Reliability• Security• Privacy protection• Usability• Autonomy• …
Ubiquitous computing environments should be deployed incrementally
• Living space is not a demonstration room• We like to replace existing objects
How can software infrastructures help to decrease the complexities?
New Requirements
Challenges to Address
Future Applications• Support & Care, Education, Entertainment
Personal Robotics• Human-robotics Interaction• Mediator between digital/real world• Natural and affective interaction (speech,
gestures, emotions)
New programming paradigms• Body inspired software architecture (Pisa)• Meta-programming models (Berlin, Stuttgart)
From nature to software models
What is needed that robots could be successfully integrated in our everyday life? What are key technical issues?
A typical first Approach
• Build a (another) middleware
• Abstract from the underlying hardware (HAL)> i.e. classes for sensors, actuators, and communication
• Unique programming model > Can use every concept the
runtime provides
• Use the .NET CLR and APICLR
Robot API
Control Application
ControllerSensors and Actuators
This wont always work!
Things that work fine on a PC don’t scale down
• Thread switches on a smart-phone or PDA are costly
• Some hardware controller don’t support threads at all
• Hardware not capable enough for a middleware (CPU, memory, battery, …)
• Even though PDAs/Smartphones are fairly powerful, CF has some several restrictions (good for some reasons, but not for us)
Now think of even smaller devices
• Embedded controllers, wrist-watches, sensor networks, smart dust
VRDK – A Research Prototype
• Building a compelling & engaging programmable environment
• Innovative, very easy, flexible programming environment for Robot control applications
• Accessible to non technical audience (children, nurse, elderly, machine operator)
• Enables a ‘Path’ from very simple (beginner, child) to professional (using
Visual Studio)
Abstracting complexity on different levels
Source Code Visual
Model-driven Development Approach
Platform-independent Model of the Program(VRDK Language)
Same programming modelfor different devices
Transformation Transformation
GenerationGeneration
.NET .NET CF XYZPlatform-specific realization of the program
Executable Executable ExecutableExecutable or interpreteron target platform
Full Tool Chain
Deploy and Run
Deploy and Run
The gentle Shortcut
VRDK Editor
Code Generation
VS.NET
Target Devices
Demo
Domain Specific Language
Visual N#
• Domain-specific graphical language• Extension of VRDK• Easy to design
C# / VB.NET
• General purpose programming language• Don’t write code anymore
N#
• Domain-specific textual language• Easy to read and write
Users can switch betweenboth notations
Code Translator
N# - A Textual Notation for Visual N#
ambient MyAmbient @ Personwhere filter ($1.Company == „Microsoft Research“),
filter($1.Location == „Cambridge"), filter($1.SecurityLevel > 2) {
discover Lights @ Light where distance(a), filter($1.Color == „green");
process OnLampAdd @ l = Lights.Added {l.On();
}process OnLampRemove @ l = Lights.Removed {
l.Off();}
a = 100;}
Demo
The Microsoft Robotics Studio
A lightweight concurrency and services oriented runtime• Handling of sensory input and controlling actuators• Based on synchronous message passing• Decentralized System Services (DSS) facilitating tasks and
basic services such as debugging, logging, monitoring, security, discovery, and data persistence
Authoring/development tools• Visual programming editor• Simulation• Message debugging
Technology libraries and basic algorithms• Code samples and documentation
Open Platform
Technology librariesRobot models Device services(OEM, ISV, IRV)
Applications and tools
Runtime Environment
Robotic ApplicationRobotic Application
RuntimeRuntime
Authoring Tools
Visual Studio Visual Studio
Supported manufacturers
• Robotics Connection• Mobile Robots• Coroware• KUKA• LEGO• fischertechnik• Robosoft• WhiteBox Robotics• Lynxmotions• Phidgets• iRobot Roomba • …
• Community platforms
Development Model
Application model
• Build complex systems from smaller, simpler decentralized services
Distribution model
• Applications are a collection of distributed services
• Discovery of functionality via contract and categories
• Self-organizing • Interaction via Messaging
Programming model
• Coordination of messages • No manual creation of threads, locks,
semaphores
ProgrammingProgrammingModelModel
DistributionDistributionModelModel
Application ModelApplication Model
Robotics Studio Architecture
Orchestration ApplicationOrchestration Application
Decentralized System Services
Con
cu
rren
cy
Con
cu
rren
cy
an
d
an
d
Coord
inati
on
C
oord
inati
on
R
un
tim
eR
un
tim
e ActivationActivation DiscoveryDiscovery
DiagnosticDiagnosticss
TerminalTerminal
StorageStorage
UXUX
Robotic Services
Robot Robot Model Model
Vis/Sim Vis/Sim
Services
Device 1Device 1
Device 2Device 2
……
Device Services
VisionVision
SLAM SLAM
……
Library Services
Messaging TransportMessaging Transport
Signal Signal Processing Processing
Hardware Abstraction LayerHardware Abstraction Layer
HA
LS
PL
Serv
ice
sO
rch
estr
ati
on
Layered and Distributed Application
Behavior Control LayerBehavior Control Layer
Orchestration Layer Orchestration Layer
Services Layer Services Layer
Signal Processing Layer Signal Processing Layer
Hardware Abstraction Hardware Abstraction Layer Layer
“Learns” and begins to favors specific behavioral characteristics. Interacts with the orchestration layer
to achieve the favored patterns.
Defines and facilitates communication patterns that coordinate information processing amongst lower
level services.
Provides functionality abstraction, computational encapsulation, failure isolation, distributed and concurrent execution, via software services .
Executes algorithms that require near real-time computation and deterministic time control.
Directly interfaces with the physical robotics hardware - Sensors and Actuators.
Service Composition
• Services aren’t just for devices
• Services can be compose to form other services
• Services can represent passive or non-existent devices
> Fused sensory data can be re-exposed as an independent service
MotorMotor EncoderEncoder
DriveDrive
Steering Steering ServoServo
TrajectoryTrajectory
PosePose
WheelWheel Base GeometryBase Geometry
Rob
ot
Mod
el
Serv
ices
Robot Model ServiceRobot Model Service
Runtime Environment
An application is a composition of loosely-coupled services concurrently executing across scopes • Interactions described using service contacts• Components are networked• Provides isolation between components
> Reliability – ability to restart independently, replaceable/updatable> Parallelism
Application Boundary
Service
Main Port
Main Port
Service
State
Code
Private Port
State
Code
Service
Main Port
State
Code
Service
Main Port
Private Port
State
Code
H/WH/W
UI
Service
Main Port
Private Port
State
Code
* Example: Model car + sensors
Communication & Coordination
• Typed Ports
• Messages sent to ports
• Arbiter handle messages> Persistent> Batch> Choice> Join> …
Activate ( Arbiter.CreateReceiver(false, p, delegate(int i)
{ Console.WriteLine(i) ;
}) );
Port<int> p = new Port<int>();p.post(42) ;
Runtime Reliability
• Reliability> Reservations> Guarantueed CPU bandwith
• Number of fixed threads in dispatcher> Set during creation of dispatcher> Not using CLR threadpool> Less overhead
• No Priorities for threads> Hard to persuade developer not to use priorities> So, priorities introduced for dispatcher
• FIFO Ports> 90,000 SOAP messages/s within a node (=multiple services)> > 3,000 SOAP messages/s among nodes
Main Port
Service
State
Code
Private Port
Runtime Environment
Concurrency and Concurrency and Coordination Coordination
RuntimeRuntime
Decentralized Decentralized System ServicesSystem Services
Service Development vs. Application Development
• Done only once • Driver related• Done by manufacturer• Distribute & reuse
Authoring Environment
Visual StudioVisual Studio
Robot Package
Tool B
ox
Tool B
ox
Robot Model Designer Robot Model Designer Code GeneratorCode Generator
Application Designer Application Designer
Service Contract Service Contract DesignerDesigner
Debugging Debugging servicesservices
Packag
es
IDE
Deployment Designer Deployment Designer Code DownloaderCode Downloader
Visualization and Visualization and SimulationSimulation
Standalone & Visual Studio integrated
Scales with programmer skill
• Helps beginners with drag/drop programming
• Assists advanced developer with service orchestration
• Conceptually consistent with conventional programming languages
• Context based toolbox
Support Rapid Application Development via visual programming, simulation and visualization.
Supports programming of distributed scenarios Extensible to support other hardware
Visual Programming Language
Robots as first class objectsData flow and orchestration
• Easy for novices
• Rapid prototyping for experts
• Implicit parallelism
• Generates C# code
• Formal checking of protocol interactions
• Deadlock detection, state exploration
ActivityMessage link
Activity
Demo
Simulation Capabilities
Supported KUKA robot
Community: Robot Swarm Heading Target Area
Community: Robot Swarm Simulation
Community: Maze Simulator
Outlook
• VRDK> Research prototype – not developed anymore
• Microsoft Robotics Studio> Available as CTP (November CTP, today at 5.00 am)> Improved visualisation> Package based deployment> Wrap up for final version> CCR / DSS maybe available as separate DL> http://msdn.microsoft.com/robotics/
• N# > Ongoing research> Publications etc. > Visual Studio integration in progress
Thank you or your attention.