concurrent computational systems edward a. lee professor, uc berkeley ptolemy project chess: center...

Concurrent Computational Systems

Edward A. LeeProfessor, UC Berkeley

Ptolemy Project

CHESS: Center for Hybrid and Embedded Software Systems

University of Arizona

Distinguished Lecture SeriesJanuary 13, 2005

Tuscon, Arizona

Lee, Berkeley 2

Abstract

Prevailing software abstractions are deeply rooted in the sequential imperative model, which at its root, is poorly matched to concurrent systems. Threads are a widely used natural extension of these abstractions to concurrent problems. Semaphores and mutual exclusion locks provide synchronization primitives offering some control over the relative timing of actions in threads. In this talk, I will argue that nontrivial concurrent programs based on threads, semaphores, and mutual exclusion locks are incomprehensible. I will outline a family of alternative concurrent models of computation, such as discrete events, process networks, synchronous languages, and dataflow, that promise more comprehensible programs. Most of these have been around for some time, and some of them have achieved widespread usage, but generally not in mainstream software. Indeed, several of them offer promising views of "computation" in a broader sense than the Turing-Church sense. They promise a view of computation that embraces mixed hardware-software systems and embedded systems that engage the physical world.

Lee, Berkeley 3

Standard Software Abstraction(20-th Century Computation)

initial state

final state

sequence f : State State

• Time is irrelevant• All actions are ordered

Alan Turing

Lee, Berkeley 4

Standard Software Abstraction: Processes or Threads

suspend

Infinite sequences of state transformations are called “processes” or “threads”

The operating system (typically) provides:

• suspend/resume• mutual exclusion• semaphores

resume

Lee, Berkeley 5

stalled for rendezvous

stalled by precedence

race condition

Standard Software Abstraction:Concurrency via Interacting Threads

Potential for race conditions, deadlock, and livelock severely compromises software reliability.

These methods date back to the 1960’s (Dijkstra).

Lee, Berkeley 6

A Stake in the Ground

Nontrivial concurrent programs based on threads, semaphores, and mutexes are incomprehensible to humans.

No amount of process improvement is going to change this.

• the human brain doesn’t work this way.

Formal methods may help• scalability?• understandability?

Better concurrency abstractions will help more

Lee, Berkeley 7A Story: Ptolemy Project Code ReviewA Story: Ptolemy Project Code Review

Lee, Berkeley 8

Ptolemy Project Code ReviewA Typical Story

Code review discovers that a method needs to be synchronized to ensure that multiple threads do not reverse each other’s actions.

No problems had been detected in 4 years of using the code. Three days after making the change, users started reporting deadlocks

caused by the new mutex. Analysis and correction of the deadlock is hard.

But code review successfully identified the flaw.

Lee, Berkeley 9

Code Review Doesn’t Always WorkAnother Typical Story

/**CrossRefList is a list that maintains pointers to other CrossRefLists.…@author Geroncio Galicia, Contributor: Edward A. Lee@version $Id: CrossRefList.java,v 1.78 2004/04/29 14:50:00 eal Exp $@since Ptolemy II [email protected] Green (eal)@Pt.AcceptedRating Green (bart)*/public final class CrossRefList implements Serializable { … protected class CrossRef implements Serializable{ … // NOTE: It is essential that this method not be // synchronized, since it is called by _farContainer(), // which is. Having it synchronized can lead to // deadlock. Fortunately, it is an atomic action, // so it need not be synchronized. private Object _nearContainer() { return _container; }

private synchronized Object _farContainer() { if (_far != null) return _far._nearContainer(); else return null; } … }}

Code that had been in use for four years, central to Ptolemy II, with an extensive test suite, design reviewed to yellow, then code reviewed to green in 2000, causes a deadlock during a demo on April 26, 2004.

Lee, Berkeley 10

And Doubts Remain

/**CrossRefList is a list that maintains pointers to other CrossRefLists.…@author Geroncio Galicia, Contributor: Edward A. Lee@version $Id: CrossRefList.java,v 1.78 2004/04/29 14:50:00 eal Exp $@since Ptolemy II [email protected] Green (eal)@Pt.AcceptedRating Green (bart)*/public final class CrossRefList implements Serializable { … protected class CrossRef implements Serializable{ … private synchronized void _dissociate() { _unlink(); // Remove this. // NOTE: Deadlock risk here! If _far is waiting // on a lock to this CrossRef, then we will get // deadlock. However, this will only happen if // we have two threads simultaneously modifying a // model. At the moment (4/29/04), we have no // mechanism for doing that without first // acquiring write permission the workspace(). // Two threads cannot simultaneously hold that // write access. if (_far != null) _far._unlink(); // Remove far } }

Safety of this code depends on policies maintained by entirely unconnected classes. The language and synchronization mechanisms provide no way to talk about these systemwide properties.

Lee, Berkeley 11

What it Feels Like to Use the synchronized Keyword in Java

Imag

e “

borr

ow

ed

” fr

om

an

Iom

eg

a a

dvert

isem

en

t fo

r Y2

K

soft

ware

an

d d

isk

dri

ves,

Sci

en

tifi

c A

meri

can

, S

ep

tem

ber

19

99

.

Lee, Berkeley 12

Diagnosis: Interacting Processes are Not Compositional

An aggregation of processes is not a process (a total order of external interactions). What is it?

Many software failures are due to this ill-defined composition.

Lee, Berkeley 13

remote procedure call

Distributed Version of 20-th Century Computation

Force-fitting the sequential abstraction onto parallel hardware.

Lee, Berkeley 14

“asynchronous” procedure call

Combining Processes and RPC – Split-Phase Execution, Futures,Asynchronous Method Calls, Callbacks, …

These methods are at least as incomprehensible as concurrent threads or processes.

Lee, Berkeley 15

Better Concurrency Models

Better concurrency models exist.

We examine a family of such models that we call actor-oriented models.

Lee, Berkeley 16

What is an Actor-Oriented MoC?

Actor oriented:

actor name

data (state)

ports

Input data

parameters

Output data

What flows through an object is

streams of data

class name

data

methods

call return

What flows through an object is

sequential control

Traditional component interactions:

Lee, Berkeley 17

Ptolemy II: Framework for Experimenting with Actor-Oriented Models of Computation

Director from a library defines component interaction semantics

Large, domain-polymorphic component library.

Basic Ptolemy II infrastructure:

Visual editor supporting an abstract syntax

Type system for transported data

Lee, Berkeley 18

The Basic Abstract Syntax

P ortP ort

A ctor A ctor

L inkR elation

A ctor

P ort

connection

connection conn

ectio

n

L ink

Link

A ttribu tes A ttribu tes

A ttribu tes

• Actors• Attributes on actors (parameters)• Ports in actors• Links between ports• Width on links (channels)• Hierarchy

Concrete syntaxes:• XML• Visual pictures• Actor languages (Cal, StreamIT, …)

Lee, Berkeley 19

Hierarchy - Composite Components

toplevel CompositeActor

transparent or opaqueCompositeActor

Actor

Relationdangling

Port

Portopaque Port

Lee, Berkeley 20

Abstract Semanticsof Actor-Oriented Models of Computation

Actor-Oriented Models of Computation that we have implemented:

• dataflow (several variants)• process networks• distributed process networks• Click (push/pull)• continuous-time• CSP (rendezvous)• discrete events• distributed discrete events• synchronous/reactive• time-driven (several variants)• …

Actor

IOPort

IORelation

P2P1

E1

E2

send(0,t) receiver.put(t) get(0)

token tR 1

Basic Transport:

Receiver(inside port)

execution control data transport

init()fire()

Lee, Berkeley 21

Models of ComputationImplemented in Ptolemy II

CI – Push/pull component interaction Click – Push/pull with method invocation CSP – concurrent threads with rendezvous CT – continuous-time modeling DE – discrete-event systems DDE – distributed discrete events DDF – Dynamic dataflow DPN – distributed process networks DT – discrete time (cycle driven) FSM – finite state machines Giotto – synchronous periodic GR – 2-D and 3-D graphics PN – process networks SDF – synchronous dataflow SR – synchronous/reactive TM – timed multitasking

Most of these are actor oriented.

Lee, Berkeley 22



Lee, Berkeley 23

Example Model of Computation:Discrete Events (DE)

DE Director implements timed semantics using an event queue

Event source

Time line

Reactive actors

Signal

Lee, Berkeley 24

Semantics Clears Up Subtleties: E.g. Simultaneous Events

By default, an actor produces events with the same time as the input event. But in this example, we expect (and need) for the BooleanSwitch to “see” the output of the Bernoulli in the same “firing” where it sees the event from the PoissonClock. Events with identical time stamps are also ordered, and reactions to such events follow data precedence order.

Lee, Berkeley 25

Semantics Clears Up Subtleties: E.g. Feedback

Data precedence analysis has to take into account the non-strictness of this actor (that an output can be produced despite the lack of an input).

Lee, Berkeley 26

Semantics Clears Up Subtleties: E.g. Zeno Systems

DE systems may have an infinite number of events in a finite amount of time. Carefully constructed semantics gives these systems meaning.

Lee, Berkeley 27

A Rough Sense of Discrete-Event Semantics: Metric Space Formulation

2/1),( yxdCantor metric:

where is the earliest time where x and y differ.

x

y

The set of signals with this metric form a complete metric space.

Generalizations of this metric handle multiple events at the same time.

Lee, Berkeley 28

Causality in DE

x

x

y

y

),(),( xxdyyd Causal:

),(),( xxdyyd Strictly causal:

,1Delta causal:

),(),( xxdyyd

A delta-causal component is a “contraction map.”

Lee, Berkeley 29

Fixed Point Theorem(Banach Fixed Point Theorem)

Let (S n = [T V ]n, d ) be a complete metric space and f : S n

S n be a delta causal function. Then f has a unique fixed point, and for any point s S n

, the following sequence converges to that fixed point:

s1 = s, s2 = f (s1), s3 = f (s2), …

This means no Zeno!

Current work: Other formulations (using generalized ultrametric spaces, ordinals, and posets) give meaning to a broader class of systems.

Lee, Berkeley 30

Semantic Challenges

Create distributed discrete-event systems.

Three families of techniques are known:

Conservative techniques (Chandy & Misra) Optimistic techniques (Time warp, Jefferson) Distributed consensus (Croquet, Reed)

Lee, Berkeley 31

Application of DE ModelingWireless Sensor Nets in VisualSense

VisualSense extends the Ptolemy II discrete-event domain with communication between actors representing sensor nodes being mediated by a channel, which is another actor.

The example at the left shows a grid of nodes that relay messages from an initiator (center) via a channel that models a low (but non-zero) probability of long range links being viable.

Lee, Berkeley 32



Lee, Berkeley 33

Example Model of ComputationContinuous Time (CT)

A signal has a value at all real-valued times.

Integrator used to define systems governed by ordinary differential equations.

Director implements a “solver” that constructs an approximation to the continuous-time behavior.

Lee, Berkeley 34



Lee, Berkeley 35

Heterogeneous ModelsHybrid Systems: CT + FSM

The FSM director can be combined with other directors to create modal models.

Lee, Berkeley 36

Semantic Subtleties

In hybrid systems, signals may have more than one value at a given time.

Discontinuities Transient states Traditional mathematical formulation of a

signal:x: Reals Realsn

has to be modified:x: Reals Naturals Realsn

with ensuing consequences for the theory.

Lee, Berkeley 37



Lee, Berkeley 38

Heterogeneous ModelsMixed Signals: DE + CT

DE model of a digital controller

CT model of mechanical system

DE signal

CT signal

Lee, Berkeley 39



Lee, Berkeley 40

Example Untimed Model of Computation:Process Networks (PN)

actor == threadsignal == stream

reads blockwrites don’t

Kahn, MacQueen, 1977

Lee, Berkeley 41

PN Semantics

A signal is a sequence of values Define a prefix order:

x y

means that x is a prefix of y. Actors are monotonic functions:

x y F(x) F(y) Stronger condition: Actors are continuous

functions (intuitively: they don’t wait forever to produce outputs).

x

y

)(xF

)(yF

Lee, Berkeley 42

PN Semantics of Composition (Kahn, ’74)

x yFxxF

yx

)(

Knaster-Tarski fixed point theorem:• Continuous function has a unique least fixed point• Execution procedure for finding that fixed point• Successive approximations to the fixed point

If the components are deterministic, the composition is deterministic.

Lee, Berkeley 43

Semantic Subtleties

Giving semantics to finite sequences in combination with infinite sequences.

Unifying fixed-point semantics with metric-space semantics (in order to give meaning to heterogeneous models that combine timed with untimed systems).

Lee, Berkeley 44

Application of PN and Ptolemy IIKepler – Scientific Workflows

Led by San Diego Supercomputer Center (SDSC), with participation from the SEEK ITR, DOE SciDAC SDM, and others, researchers are building on Ptolemy II a distributed workflow system called Kepler to “provide access to any resource, any data, any service, anytime, anywhere” via a distributed computing platform (aka the “Grid”) supporting “high performance computing, and federated, integrated, mediated databases within a user-friendly workbench / problem-solving environment.”

This effort is compatible with the objectives of NSF Cyberinfrastructure, UK’s e-Science Programme, and DOE’s SciDAC (Scientific Discovery through Advanced Computing).

Lee, Berkeley 45

Kepler Example: Harvesting Web Services

Example showing a web service wrapper (Thanks to Bertram Ludaecher, San Diego Supercomputer Center)

Lee, Berkeley 46

Kepler Example: Grid Computations

Thanks toBertram LudäscherSDSC

Lee, Berkeley 47

ORB

This model integrates real-time data feeds from a ship (the Roger Revelle)

Thanks to Bertram Ludäscher, SDSC

Kepler Example:Real-Time Data Feeds

Lee, Berkeley 48

KEPLER/CSP: Contributors, Sponsors, Projects(or loosely coupled Communicating Sequential Persons ;-)

Ilkay Altintas SDM, ResurgenceKim Baldridge Resurgence, NMI Chad Berkley SEEK Shawn Bowers SEEKTerence Critchlow SDM Tobin Fricke ROADNetJeffrey Grethe BIRNChristopher H. Brooks Ptolemy II Zhengang Cheng SDM Dan Higgins SEEKEfrat Jaeger GEON Matt Jones SEEK Werner Krebs, EOLEdward A. Lee Ptolemy II Kai Lin GEONBertram Ludaescher SEEK, GEON, SDM, BIRN, ROADNetMark Miller EOLSteve Mock NMISteve Neuendorffer Ptolemy II Jing Tao SEEK Mladen Vouk SDM Xiaowen Xin SDM Yang Zhao Ptolemy IIBing Zhu SEEK •••

Ptolemy IIPtolemy II

Thanks to Bertram Ludäscher, SDSC

Lee, Berkeley 49



Lee, Berkeley 50

Synchronous Models of Computation

Director finds a value (or absent) at each “tick” of a global clock

Feedback is resolved by finding a fixed point.

Signal has a value or is absent at each tick of the “clock.”

Semantic foundation based on Kanster-Tarski fixed point theorem on Scott topologies.

Lee, Berkeley 51

Languages Based on theSynchronous Model of Computation

Lustre (and SCADE) Esterel Signal Statecharts (and UML state machines) Argos …

Lee, Berkeley 52



Lee, Berkeley 53

Dataflow Models of Computation

Computation graphs [Karp & Miller - 1966] Process networks [Kahn - 1974] Static dataflow [Dennis - 1974] Dynamic dataflow [Arvind, 1981] K-bounded loops [Culler, 1986] Synchronous dataflow [Lee & Messerschmitt, 1986] Structured dataflow [Kodosky, 1986] PGM: Processing Graph Method [Kaplan, 1987] Synchronous languages [Lustre, Signal, 1980’s] Well-behaved dataflow [Gao, 1992] Boolean dataflow [Buck and Lee, 1993] Multidimensional SDF [Lee, 1993] Cyclo-static dataflow [Lauwereins, 1994] Integer dataflow [Buck, 1994] Bounded dynamic dataflow [Lee and Parks, 1995] Heterochronous dataflow [Girault, Lee, & Lee, 1997] …

Many tools, software frameworks, and hardware architectures have been built to support one or more of these.

Lee, Berkeley 54

Ptolemy II Software ArchitectureBuilt for Extensibility

Ptolemy II packages have carefully constructed dependencies and interfaces

PN

CSP

CT

DE FSM

SD

F

Kernel

Data

Actor Math

Graph

Lee, Berkeley 55

Conclusion

Threads are a poor concurrent MoC There are many better concurrent MoCs The ones we know are the tip of the iceberg Ptolemy II is a lab for experimenting with them Specializing MoCs can be useful Mixing specialized MoCs can be useful. This is a rich research area.

concurrent computational systems edward a. lee professor, uc berkeley ptolemy project chess: center...

Documents

software reliability

mainstream software

concurrent systems

concurrent problems

code review doesnt

multiple threads

ptolemy project code

state state time