cougaar design case study: mandelbrot gui application todd wright feb 5 th, 2007

Cougaar Design Case Study:Cougaar Design Case Study:Mandelbrot GUI ApplicationMandelbrot GUI Application

Todd Wright

Feb 5th, 2007

2

OverviewOverview

• These slides present ten alternate designs for an example Cougaar application, a “Mandelbrot” fractal GUI

• Each design explores various tradeoffs:– Design complexity– Modularity and how the modules (plugins/agents)

interact with one another– Parallel / Distributed processing support

• As we go, we’ll summarize what we’ve learned and outline basic design patterns

3

• Basic idea:– The user submits an image calculation request

– Cougaar application code (plugins & agents) compute the image data

– The image is displayed to the user in a GUI

• The example image is a “Mandelbrot” fractal– Given an (x, y) range and image size, e.g.

• Range: (-1.5, -1.0) to (1.5, 1.0)

• Image: 1024 x 768

– Compute the image using the “Mandelbrot” algorithm• Simple math

• Entirely compute-bound (possibly network-bound if we make it distributed)

Mandelbrot ApplicationMandelbrot Application

AgentsAgents

Nodes

I/O

GUI

PluginsPlugins

4

Design Comparison MatrixDesign Comparison Matrix

• For all the following designs, we’ll rank each design based on the following scales of 1-to-10, with 10 being ideal:

1. Simplicity– How easy is the code to understand?

2. Modularity– Can we easily replace parts of our solution with alternative implementations?

3. Scalability– Can we distribute our solution across multiple hosts?

4. Inter-job Parallelism– Can separate jobs run in parallel?

5. Intra-job Parallelism– Can a single job be subdivided and run in parallel?

6. Adaptability– Can we customize the behavior, e.g. using policies or runtime metrics?

• This will allow us to better see the tradeoffs between the designs.

5

Design 1: Just a ServletDesign 1: Just a Servlet

• Design: Do everything in a self-contained Servlet:– Listens for browser HTTP requests

– Computes image data in the servlet “doGet” thread

– Writes the image result as a JPG

• Characteristics:– Easy to implementation and configuration

– Few Cougaar dependencies (no need for a blackboard or other plugins)

– No synchronization or threading issues (runs in the Servlet request thread)

Servlet

public void doGet(..) { read params compute image data write image as JPG}

Node

http

6

Analysis: Design 1Analysis: Design 1

Simplicity 10 Nearly all the code is focused on the Mandelbrot math and UI display code – very little Cougaar “plumbing”. This is ideal for “quick & dirty” applications.

Modularity 0 Unable to reuse code in a pluggable way.

Scalability 0 Single-host.

Inter-Job Parallelism 0 No prioritization. Multiple servlet calls can occur at the same time, but they’ll blindly compete for the same CPU.

Intra-Job Parallelism 0 Can’t decompose task, so we can’t take advantage of multiple CPUs or hosts.

Adaptability 0 Hard-coded behavior.

7

Design 2: Servlet UI + Calculator ServiceDesign 2: Servlet UI + Calculator Service

• Design: Move the “compute()” code out of the Servlet and into a separate Component– Primarily a refactor of the prior design

– Use a service to advertise the “compute()” method

– This is a typical solution for wrapping library code

• Characteristics:– Still fairly easy to implement and configure

– Improved modularity:• Can replace UI code while keeping calculator code (e.g. make popup Swing UI)

• Can replace calculator code while keeping UI code (e.g. compute different fractal design)

– No threading or synchronization issues (runs in the Servlet request thread)

Servlet

public void load() { calc = getService(Calc..);}public void doGet(..) { read params calc.compute(..); write image as JPG}

Node

http

Calculator

public void load() { advertise(Calc, this);}

public byte[] compute (..) { compute image data}

8


Simplicity 9 Still fairly easy to understand. The Cougaar “plumbing” is limited to narrowly-defined Service APIs.

Modularity 2 We’ve separated the display plugin from the calculator plugin, making each pluggable.

Scalability 0 Same as in design 1.

Inter-Job Parallelism 0 Same as in design 1.

Intra-Job Parallelism 0 Same as in design 1.

Adaptability 0 Same as in design 1.

9

Design point: Inlined code v.s. ServicesDesign point: Inlined code v.s. Services

• Key design points:– Design 1:

• Summary: The plugin directly calls the inlined / library code• Benefit: Easy to implement, self-contained• Downside: Difficult to switch between alternate library implementations,

awkward to share non-static library instances between plugins

– Design 2:• Summary: One plugin advertises a service, other plugin(s) obtain and use it• Benefit: Supports shared, pluggable services, cleans up the code• Downside: Must refactor / wrap library code into service API(s), plus add

new plugins to advertise these services

• Example of interest:– Plugin “A” advertises a “WindowManagerService” and pops up an empty

Swing Panel– Subsequent plugins obtain this service and add their Swing “JComponent”

panels to the service by calling an “add(..)” method (instead of popping up their own windows)

– The “window manager” plugin decides where to place the sub-frames

10

Design 3: Servlet UI & Blackboard-based Design 3: Servlet UI & Blackboard-based Calculator PluginCalculator Plugin

• Design: Instead of a service, publish the request on the blackboard– Use non-blocking blackboard operations (pub/sub) instead of a blocking method call

• Characteristics:– The “calculate()” method runs in a separate plugin thread

• We’re using the blackboard as both a communication and thread-switch layer

• We no longer have a simple, blocking “calculate()” service API

– We now have a blackboard representation of the Job• Defines our data-oriented “API” between our plugins

• Other plugins can observe this interaction (e.g. for debugging, management, etc)

Agent

Servlet

public void doGet(..) { Job job = new Job(params); publishAdd(job); job.waitForCompletion(); write image as JPG}

Node

http

Calculator

public void setupSubs() { subscribe to Jobs}public void execute() { for all added Jobs { compute job notify of completion }}

Blackboard

Job

11


Simplicity 8 Still fairly easy to understand. The Cougaar “plumbing” is limited to narrowly-defined Service APIs. The Servlet requires some awkward “wait/notify” code.

Modularity 3 Slightly improved because other plugins can observe the in-process Job status by watching the blackboard data.

Scalability 0 Same as in design 1 & 2.

Inter-Job Parallelism 2 Our “calculate()” now runs single-threaded in a plugin, instead of time-slicing across independent Servlet threads. This is better on single-CPU machines, but we still have no real control in multi-CPU environments.

Intra-Job Parallelism 0 Same as in design 1 & 2.

Adaptability 0 Same as in design 1 & 2.

12

Design Point: Services v.s. Blackboards Design Point: Services v.s. Blackboards


• Summary: Plugins interact through blocking service method calls

• Benefit: Easy blocking method APIs

• Downside: Method calls run in caller’s thread and are blocking. Use of callbacks to support non-blocking APIs requires awkward thread switching.

– Design 3:• Summary: Plugins interact through asynchronous blackboard pub/sub

operations

• Benefit: Non-blocking and parallelized, plugin “execute()” methods are single-threaded, “Job” state is visible on the blackboard

• Downside: Must reorganize code to fit the pub/sub “execute()” pattern. This can introduce “bookkeeping” state, where a service-based design would keep this state “for free” on the method-call stack.

• The prior example shows an awkward mixed design:– Servlet “doGet()” callbacks are blocking and must complete in that thread

– The blackboard is an asynchronous pub/sub interaction

– Hence the odd “job.waitForCompletion()” solution..

13

Design 4: Non-Blocking UIDesign 4: Non-Blocking UI

• Design: Replace Servlet UI with Plugin “Screensaver” UI– The servlet case is odd, in that the “doGet(..)” request method is a blocking, external Thread

call

– As a point of comparison, create a Plugin-based UI client that uses a non-blocking Cougaar thread and standard blackboard pub/sub operations

• Characteristics:– The UI plugin listens for a subscription change instead of a lock “notify()”

– This is a more standard Cougaar interaction pattern

– This approach isn’t applicable for our Servlet-based UI (but might fit a Swing UI)

AgentRequestor

public void setupSubs() { subscribe to Jobs publishAdd(new Job)}public void execute(..) { for all changed Jobs { write image as JPG }}

Node

Calculator

public void setupSubs() { subscribe to Jobs}public void execute() { for all added Jobs { compute image data publishChange(job) }}

Blackboard

Job

/tmp/out.jpg

write

14


Simplicity 9 Uses clear data-orientated interactions to invoke our plugins.

Modularity 3 Same as in design 3.

Scalability 0 Same as in designs 1- 3.

Inter-Job Parallelism 2 Same as in design 3. Slightly better because outstanding Requestor calls don’t block and tie up pooled threads.

Intra-Job Parallelism 0 Same as in designs 1 - 3.

Adaptability 0 Same as in designs 1 - 3.

15

Design Point: Mixed Services/BB v.s. all BBDesign Point: Mixed Services/BB v.s. all BB


• Summary: Servlet “doGet()” callback uses awkward lock wait/notify to detect blackboard work completion instead of an asynchronous subscription

• Benefit: Required due to limitations of blocking Servlet callback API• Downside: Awkward mixed-metaphor of wait/notify + subscription changes

– Design 4:• Summary: All interaction is through blackboard pub/sub options• Benefit: Easy integration via subscriptions, completely asynchronous• Downside: Not applicable in the Servlet case.

• Most applications fit entirely into the blackboard-friendly pub/sub pattern

• The design often gets awkward when plugins must interact with both blocking/callback services plus blackboard pub/sub operations– Typically results in awkward “todo” lists to switch threads– Ideally this can be avoided

16

Design 5: Separate Job/Result ObjectsDesign 5: Separate Job/Result Objects

• Design: Instead of changing the Job, publish a separate Result object– This makes it clear that the result is a separate data structure

– We’ll assume that we’re using the non-servlet “Requestor”, as in design 4

• Characteristics:– The subscriptions now look for different data structures (notice that arrows are “one-way”)

– The Result object should have a pointer to the Job, or have a shared “unique job identifier”

Agent

Requestor

public void setupSubs() { subscribe to Results publishAdd(new Job)}public void execute(..) { for all added Results { write image as JPG }}

Node

Calculator

public void setupSubs() { subscribe to Jobs}public void execute() { for all added Jobs { compute image data publishAdd(new Result) }}

Blackboard

Job

/tmp/out.jpg

writeResult

17


Simplicity 9 Arguably more clear due to separate data structure, or less clear due to added blackboard clutter.


Scalability 0 Same as in designs 1- 4.

Inter-Job Parallelism 2 Same as in design 4.

Intra-Job Parallelism 0 Same as in designs 1 - 4.

Adaptability 0 Same as in designs 1 - 4.

18

Design Point: Separate Results ObjectDesign Point: Separate Results Object


• Summary: Job has field for results data

• Benefit: Fewer blackboard objects

• Downside: Multiple writers to the same object, to fill in result slot

– Design 5:• Summary: Calculator publishes a separate Results object

• Benefit: Finer-grain subscriptions, “publishAdd” driven

• Downside: More blackboard objects

• This is more a matter of style

19

Design 6: Remote ProcessingDesign 6: Remote Processing

• Design: Transfer the job to a remote agent– Wrap the job in a relay

– We’ll assume that the “master” knows a-priori about the single “slave”

• Characteristics:– Can run the slave on a remote host (supports remote processing)

– Adds layer of Relay “wrapping” and processing code to do our data transfer

– Must transfer both the Job and its result-data (two-way comms instead of shared memory)

Master Agent

Servlet

Node 1

http Blackboard

Relay

Slave Agent

Calculator

Blackboard

Messages

JobRelay copy

Job

Node 2

20


Simplicity 7 Relay wrapping adds complexity. Can no longer see all activity on a single blackboard.


Scalability 3 Can offload jobs to remote worker.

Intra-Job Parallelism 2 Same as in design 5. We only have one worker, so the jobs run single-threaded on the workers CPU.

Inter-Job Parallelism 0 Same as in designs 1 – 5.

Adaptability 0 Same as in designs 1 – 5.

21

Design Point: Centralized v.s. DistributedDesign Point: Centralized v.s. Distributed


• Summary: Single agent with shared blackboard• Benefit: Plugins can assume that everything is on their local blackboard• Downside: Limited to single host

– Design 6:• Summary: Wrap job in Relay, transfer to remote agent for processing• Benefit: Distributed, partitions work and memory across hosts• Downside: Clutters plugin code with Relay “wrapping” and “addressing”. No

longer a shared memory, so Relays must transfer data back & forth.

• Relays (or similar mechanism) are used to transfer data between blackboards– Required because agents don’t support shared-memory blackboards– Anytime you make something distributed you run into well-known

distributed processing limitations (latency, robustness, etc)

• The next design separates the Relay wrapping/addressing from the non-transfer-related plugin work

22

Design 7: Remote Processing with DispatcherDesign 7: Remote Processing with Dispatcher

• Design: Introduce concept of “Dispatcher” Plugin– Separates Servlet/Calculator code from remote transfer code

– Still use relays to transfer jobs (an equivalent option is to use task/allocation)

• Characteristics:– Can implement different kinds of dispatch policies as pluggable “Dispatcher”s

– Adds job management control in the Dispatcher code

– One more layer of thread switching & indirection (but that’s often a good thing)

http

Master Agent

Servlet

Node 1

Blackboard

Relay

Slave Agent

Calculator

Blackboard

Messages

Job Relay copy

Job

Node 2

Dispatcher Receiver

23


Simplicity 7 Separate “Dispatcher” and “Receiver” plugins simplify our “Servlet” and “Calculator”. However, they’re yet more plugins to configure and understand.

Modularity 5 Can easily swap out dispatcher code to support more intelligent dispatching logic. Each plugin now has a more clearly-defined function.



Inter-Job Parallelism 0 Same as in designs 1 – 5.

Adaptability 0 Same as in designs 1 – 5.

24

Design Point: Use of “Dispatcher” PluginsDesign Point: Use of “Dispatcher” Plugins


• Summary: Domain plugins do Relay wrapping and addressing

• Benefit: Fewer plugins

• Downside: Clutters domain code, difficult to enhance

– Design 7:• Summary: Introduce “Dispatch / Receiver” plugins to handle Relay details

• Benefit: Cleans up design, supports pluggable dispatch options

• Downside: Adds more indirection, more objects on blackboard.

• The “Dispatcher” design is often a good idea, except in trivial cases where the added flexibility would be overkill.

25

Design 8: Load-balancingDesign 8: Load-balancing

• Design: Support multiple worker agents– Dispatcher can choose between slaves

• A job can be sent to any slave• This allows to balance work between our slaves

– Allow multiple, dynamic slaves• Slaves “register” with the master agent via a Relay• Slave “pulls” down job, replies with results, and pulls next job

– Add concept of separate relays for slave-to-master v.s. slave-to-master comms• Slave sends registration & results via its relay• Master sends new jobs via its relay• Creates more of a unified “comms channel”, for better error processing

• Characteristics:– Can balance jobs between slaves (if we have more jobs than slaves)

• Ideally one agent per CPU, distributed across hosts according to per-host CPU count• If we only have one job then this doesn’t help, since (in this design) we can’t reduce jobs

into smaller tasks

– Simple configuration via slave “register”, instead of hard-coding slave names in the master

• More adaptive – we can dynamically support added/removed slaves

Illustration on next slide..

26

Design 8: Load balancing (2)Design 8: Load balancing (2)

Master Agent

Servlet

Node 0

Dispatcher

Blackboard

Job AA

Node 1Slave1 Agent

ReceiverCalculator

Job AA

Blackboardfrom: Master

to: Master

to: Slave1

Node 2Slave2 Agent

ReceiverCalculator

Job BB


to: Master

to: Slave2from: Slave1 from: Slave2

http

Job BBhttp

27


Simplicity 4 The dispatcher logic is getting complicated. It must track dynamically added/removed slaves plus load-balance jobs between slaves.

Modularity 7 Just as pluggable as design 6, but now we benefit from factoring out our dispatcher code into its own plugin.

Scalability 7 Can run across an arbitrary number of hosts.

Intra-Job Parallelism 9 Can run one slave per CPU per host, which fully parallelizes jobs between CPUs.

Inter-Job Parallelism 0 Same as in designs 1 – 6. If we only have one job then it runs on a single CPU.

Adaptability 3 Can dynamically add/remove slaves, making this solution somewhat adaptive.

28

Design Point: Single v.s. Load-balancedDesign Point: Single v.s. Load-balanced


• Summary: Work is offloaded to a single remote worker• Benefit: Offloads work, relatively simple design• Downside: Only computes one job at a time, only supports a single

worker

– Design 8:• Summary: Work is dispatched to one of many workers• Benefit: Load-balances work, supports an arbitrary number of

workers• Downside: More complex design, must choose which slave to send

work to. Parallelism is limited to our job backlog.

• The load-balanced solution is a general-purpose, parallelized “grid” computer

• However, we’re still limited by the granularity of our Jobs.

29

Design 9: Fine-Grained Parallel processingDesign 9: Fine-Grained Parallel processing

• Design: Divide the job into subtasks, allocate tasks to remote agents– Add concept of Job-to-Task decomposition

• New Expander plugin decomposes Job into Tasks

• These Tasks are published on the blackboard

• Expanded detects when all tasks have been completed, aggregates the result, and completes the job

– Can divide our Job into an arbitrary number of Tasks, but ideally this is guided by the Dispatcher’s knowledge of how many slaves we have

• Characteristics:– Maximum parallelism.

• We can split a single Job across an arbitrary number of slaves

• We are no longer limited by our Job backlog

– Note that a complex Job representation is required to support Task decomposition & incremental result updates

Illustration on next slide..

30

Design 9: Fine-Grained Parallel processing(2)Design 9: Fine-Grained Parallel processing(2)

Master Agent

Servlet

Node 0

Dispatcher

Blackboard

Job

Node 1Slave1 Agent

ReceiverCalculator

Task


to: Master

to: Slave1

Node 2Slave2 Agent

ReceiverCalculator

Task


to: Master

to: Slave2from: Slave1 from: Slave2

http Expander

Task 0

Task 1

Task N

31


Simplicity 2 Expander decomposition is complex. Ideally the level of decomposition and number of available slaves is (somehow) coordinated.


Scalability 9 Can scale a single job across multiple hosts.

Intra-Job Parallelism 9 Same as in design 7. Also, can schedule high-priority jobs across multiple slaves and low-priority jobs onto a single slave.

Inter-Job Parallelism 9 Can run a single job in parallel.

Adaptability 3 Same as in design 7.

32

Design Point: Load-balanced v.s. ParallelDesign Point: Load-balanced v.s. Parallel


• Summary: Entire jobs are load-balanced between workers

• Benefit: Offloads work, relatively simple design

• Downside: No intra-job parallelism (but separate jobs may run in parallel)

– Design 9:• Summary: Uses task decomposition and balanced remote task allocation

• Benefit: Highly parallelized, can parallelize a single job across multiple workers

• Downside: More complex design, must track and re-assemble subtask results. Only works if the job can be decomposed into independent, parallelizable subtasks.

• The primary tradeoff in this case is design complexity.

• This also assumes that we can decompose our Jobs into arbitrarily small subtasks, which is not true for all applications.

33

Design 10: Support Dispatch PoliciesDesign 10: Support Dispatch Policies

• All the prior designs featured hard-coded behaviors:– Hard-coded or parameterized list “slave” agents

– Simple allocation rule: allocate to next available slave

• As an enhancement, we could modify our plugins to support more complex, policy-based behaviors.

• Example Policies:– Timeout calculations and re-allocate to alternate slave

– Send same task to multiple slaves, to reduce latency and add fail-over

– Send multiple outstanding subtasks per slave, to reduce network latency effects (i.e. keep working while the results are being sent on the wire)

– Allocate according to slave host metadata (e.g. CPU speed, network latency, scheduling relative to other work)

• All of the above examples illustrate QoS adaptation

34


Simplicity 1 In general, policy-based and adaptive behaviors are difficult to implement.

Modularity 8 Same as in design 8. May be improved if it supports parameter-based / pluggable policies.



Inter-Job Parallelism 9 Same as in design 8. Note that we could add adaptivity to our non-parallel designs – parallelism is not required.

Adaptability 6 Excellent, depending upon what is implemented.

35

Design Point: Hard-coded behavior v.s. Design Point: Hard-coded behavior v.s. PoliciesPolicies

• Key design points:– Design 98 (and prior):

• Summary: Behavior is hard-coded or only supports trivial parameterization.

• Benefit: Good enough for most applications.

• Downside: Inflexible behavior.

– Design 10:• Summary: Add plugin behavior options controlled through policies

• Benefit: Pluggable / adaptive behavior

• Downside: More complex to implement.

• The introduction of policies and behavior options allows for a “smarter” application.

ConclusionsConclusions

37

Design Analysis SummaryDesign Analysis Summary

Design 1 2 3 4 5 6 7 8 9 10Servlet Service Blackboard Non-

blocking UI

Response Object

Remote Remote Dispatch

Load-balanced

Fine-Grain Parallel

QoS

Simplicity 10 9 8 9 9 7 7 4 2 1Modularity 0 2 3 3 3 3 5 7 7 8

Scalability 0 0 0 0 0 3 3 7 9 9Intra-Job Parallelism

0 0 2 2 2 2 2 9 9 9

Inter-Job Parallelism

0 0 0 0 0 0 0 0 9 9

Adaptability 0 0 0 0 0 0 0 3 3 6

38

ConclusionsConclusions• The prior slides showed many different ways to build the

same application but with different system properties– The first couple designs are relatively simple– Subsequent slides supported parallelism but are more complex

– Each design is valid and ideal in certain environments

• Each “split” of code/data introduces design complexity:– Splitting a plugin into multiple plugins requires data coordination

between the plugins, requiring:• Coordination API (either a service or blackboard pub/sub)• Data structures (must be internally synchronized)

– Splitting data across agents requires data partitioning and transfer code

• Must decide which data resides on which agent(s)• Must transfer the data, typically via the blackboard (e.g. Relays)

39

Conclusions (2)Conclusions (2)• Service-based API are useful in limited cases

– Ideal for wrapping simple libraries (e.g. log4j)– Should be non-blocking and not require blackboard access

• Don’t block pooled threads• Requires a thread switch, otherwise you’ll get a blackboard “nested

transaction” problems• See the “todo” pattern and other (awkward) workarounds

• In contrast, blackboard interactions are non-blocking– This is good in that it switches threads and avoids blocking the plugin

when performing remote I/O, which increases parallelism– It’s bad in that the plugin code must support an asynchronous call and

subsequent “execute()” method resume when the result is published– The result is sometimes added “bookkeeping” state in the plugin, to

remember where prior async calls left off. This is effectively a “continuation”.

cougaar design case study: mandelbrot gui application todd wright feb 5 th, 2007

Documents

servlet design

compute code

ui code

calculator code

prior design

example image

image result

image size