A Distributed Component-based Software Architecture and Framework for

Laboratory Automation Systems

Venkataramanan Kuppuswamy

Meanwhile….

Guidelines for a good Lab Automation System s/w

• Extensible

• Not tied to a specific vendor or manufacturer

• Distributed across multiple computers or across different

process spaces, to enhance robustness & performance

• Responsive: Provides timely feedback to client & handles

multiple requests simultaneously

• Thread-Safe

Existing Software

• Commercial lab automation software

• Open source robotics frameworks. E.g. Orcos Robotics, Orca

Robotics & Robot Operating System (ROS).

• Other robotics frameworks such as Microsoft Robotics

Developer Studio (MSRDS)

• Open source microscopy software like μManager

• Device abstractions such as Interchangeable Virtual

Instruments

Drawbacks

• Expensive, closed source & not extensible

• Domain specific. E.g. μManager for Microscopy

• Do not provide a comprehensive design solution to develop

thread-safe & responsive software

Contributions of The Thesis

• Developed Component Oriented Distributed Architecture

(CODA)

• Developed CODA Framework

CODA

• Architecture to develop laboratory automation system software

• Based on autonomous components that could be distributed

• Provides design solution for developing thread-safe &

responsive components

• Provides a consistent way to create, use & destroy components

CODA Component

• Autonomous software module that performs an useful task, or

related set of tasks (control devices, imaging etc.)

• Can support multithreading

• Communicates by message passing. Every message has Body,

ACK Port & Operation Complete (OPC) Port

• Stateful

• Does not use shared memory

Component High Level Design

User LayerNotification Messages

Component API Messages

Notification Port Main Message Port

Exclusive Message Processing DispatcherNotification Mechanism

Component Infrastructure Layer

Component Implementation Layer Operation Requests

Notification Requests

State data

Message Validation

Operation Implementation

Component Specific Implementation

Operation Implementation

• Actions

• Quick, not prone to errors. Ex: updating state data of a device.

• Executed synchronously w.r.t dispatcher (hence thread safe)

• Tasks

• Lengthy, prone to error. Ex: device communication, reading from file,

database etc.

• Executed asynchronously, hence making component responsive

• Does not update state data directly, but utilizes message passing, hence

ensuring thread safety

Message Passing

User Software

Create Messages & Notif ication

Ports

User Component

Wait for ACK

OPC Notif ication

Action Message

Component

ACK Notif ication

Wait for OPC

Wait for ACK

OPC Notif ication

Task Message

ACK Notif ication

Interface Object

• Simplifies component use

• Used by clients to communicate with the component

• Provides an uniform way of accessing the component

irrespective of local or remote

• Local: Component in same process space as client

• Remote: Component in different process space (executable) on

same computer, or on a entirely different computer

Interface Object (cont.)

Interface Object

Create Message & Notif ication

Ports

Interface Object

Component

Component

ACK Notif ication

Wait for ACK

OPC Notif ication

API Message

Client Software

Client

API 'Wrapper Routine'

(Returns OPC Port)

Minimum Component Behavior

• Configure

• Setting values needed to successfully “Open” the component. Ex:

setting comm port for a serial device (e.g. stage controller)

• Open

• Setting up component for use. Ex: opening a serial port

• Close

• Cleans up component after use. Ex: closing a serial port.

• Component can be safely destroyed only after “Close”

Minimum Component Behavior (cont.)

Ready

Configured

Shutdown

OpenClose

Error Opening

Error Configuring

*Note: Configure refers to any of the family of Configure Messages

Configure

ConfigureError Configuring

CODA Summary

• An architecture to develop distributed component based

software that is responsive & thread-safe

• Based on stateful, asynchronous message passing components

• Components execute Actions & Tasks in response to messages

• Interface object simplifies component usage

• All components must implement a required minimum

behavior defined by CODA

CODA Framework

• Helps develop CODA compliant components

• Enables development of responsive, thread-safe & extensible

software.

• Develop code that conforms to Unified Modeling Language

(UML) state charts

• Very easy to make components remotely accessible

• Abstractions for devices such as stage controller & camera

CODA Framework (cont.)

• Concrete implementations for several common stage

controllers & IMAQdx compatible cameras

• Software developed using the framework is scalable with

number of cores (Microsoft Robotics Studio (MSRDS))

• Developed using MSRDS Concurrency Coordination Runtime

(CCR) & Decentralized Software Services (DSS)

• Compatible with LabVIEW

Framework Component High Level Design

Transition Execution

Transport

• Abstracts out details involved in passing message to

component.

• Deep clones every data that is passed to & from component –

ensures components are decoupled & completely autonomous

• Two implementations

• Local: Communicate with local components

• DSS: Communicate with remote components

Passing Message to a Local Component

Passing Message to a Remote Component

Camera Abstraction

Stage Controller Abstraction

Imaging Application - A Case Study

• Models a physical system consisting of a camera mounted on a

stage controller

• Focus Adjustment

• Auto Focus

• Focus Positioning

Without using the framework

• Camera Interfacing & state machine like control code

• Stage Controller interfacing & state machine like control code

• Imaging Application that talks to camera & stage controller

having state machine like control code

• Networking infrastructure to make components distributable

Using the Framework

• Stage Controller Implementation

• Camera Implementation

• Imaging Application talking to abstract stage controller &

camera

• Very little networking code

CODA Framework Summary

• Implements the concepts described in CODA

• Helps develop software that is distributed, responsive, thread-

safe & scalable

• Has device abstractions for camera & stage controller, and

concrete implementations for several manufacturers,

• Imaging Component

• Can be extended to support additional functionalities

Contributions of The Thesis

• Developed Component Oriented Distributed Architecture

(CODA)

• Developed CODA Framework

Future Work

• Improve the state machine design to include

hierarchical/composite states

• Device abstractions for laser, pumps etc.

• More subsystems such as multi-axis motion

• Strategy to prevent deadlocks

Acknowledgements

• Dr. Deirdre Meldrum, Dr. Roger Johnson, Mr. Dean Smith, Dr.

James Collofello & Dr. Hessam Sarjoughian

• All CBDA lab members & in particular Vivek, Inian & Sai

• Engineers at MSRDS forums

• My parents, sister & brother in law

• Friends, in particular Ganesh, Sriram, Parth, Venkatakrishnan,

Aravindan, Dilip, Raj, Vaibhav, Nikhil, Glenn, Sathish &

Tosha

Back Up Slides

Messages

• Has ACK port and Operation Complete (OPC) Ports

• Port: Receives message and transfers to another

module for processing

• ACK: Indicates a message has been successfully

received by component & sent for processing

• OPC: Indicates a processing of message has been

complete

Framework

• Talk a bit about UML state machines and robotics studio

• Advantages of state machines• Used robotics studio to get scalability.• State machines, deterministic• Already written lot of code that the developer

does not have to write• How actions and tasks are implemented• 2-3 minute video of the imaging software

Demo of a Imaging application

• TODO: Add video of the imaging application

About the demo Software

• Typical automation scenario

• Sophisticated

• Responsive

• Change deployment without code change

• Works with any camera & stage compatible with our

framework

Development & Test Strategy

• Developed the base abstract component

• Implemented a test component that covered all uses

cases for a simple use of a state machine

• Developed a test application and tested the test

component. Found issues & fixed it

• Developed a camera abstraction

• Developed an implementation for an Imaqdx compatible

camera

Test SuitesTest Suite Features Tested

TestComponent Tested the entire infrastructure. This includes testing the base component, base component state machine, state machine creation, state machine engine processing of tasks & actions, error handling, base component interface object, local & DSS transport.

StageControllerTest Functionalities of stage controller which includes open, Configure, Close, Velocity Move, Position Moves, Setting Properties such as Acceleration & Velocity, Stopping Motion & Homing

CameraTest Functionalities of camera which includes Open, Configure, Close, Setting Properties such as exposure time, Grabbing Images & Stop Grabbing Images

ImagingTest Functionalities of camera which includes Open, Configure, Close, Grabbing Images, Stop Grabbing Images, Focus Move, Stop Focus Motion & Focus Adjustment

Lab Automation System Software

• Controls & coordinates multiple different types of devices

• Ex: Motion control stages, cameras, lasers, pumps & valves

• Devices have own memory, processing unit, work independently &

have different manufacturers

• Certain devices & algorithms need specialized hardware and software

to run effectively

Imaging Component - A Case Study

Imaging Component Creation Steps

• Create all Main Messages & Internal Messages

• Create Main Message Port, Internal Message Port, State

Enumeration, Config Data & State Data classes

• Create new ImagingComponent deriving from base

component

• Compose with stage controller & camera interface objects

• Implement Open, Configure & Close methods

• Extend the base component state machine & add the imaging

component state machine

Lab Automation System

• Automation

• Increases productivity

• Lab automation Software interfaces with devices

• Devices have own memory, processing unit, work independently &

have different manufacturers

• Certain devices & algorithms need specialized hardware & software

Previous Work (cont.)

• μManager

• Open source, freely available software for automating microscope

operations

• Limited application domain

• Interchangeable Virtual Instruments

• Open instrument driver architecture

• Abstracts out common functionality available in instruments

• Used to automate test & measurement equipment

Action

User Layer

Create Message & Notif ication

Ports

User

Component Infrastructure

Layer

Component Implementation

Layer

Message Processing & Dispatcher

Notification Mechanism

Message Validation

Operation Implementation

Operation Request(Message)

ACK Notif ication

Wait for ACK

Operation(Message)

OPC Notif ication

State Change Notif ication()

[ Message Valid ? ]

Component Specific

Implementation

State Change Notif ication

Validate(Message)

API Message

Task

Component Infrastructure

Layer

Component Implementation

LayerNotification Mechanism

Message Validation

Operation Request( Message)

ACK Notif ication

StartTask( Message)

OPC Notif ication

[ Message Valid ? ]

Component Specific

Implementation

Message Processing & Dispatcher

Operation Implementation

Validate( Message)

API Message

Update State Data

UpdateStateData( Data)

State Change Notif ication State Change Notif ication()

State Change Notif ication()

State Change Notif ication

Task Finished

TaskFinished()

Message Processing

Transition Execution

Decentralized Software Services (DSS) Classes

• DSS Client: Communicates to a remote component by sending

messages to the DSS server which in turn sends it to the

component.

• DSS Server: Composes a remote CODA Component. Clients

of the component can access the it only through DSS server.

• DSS client & DSS server translate component request &

response messages to DSS requests & responses, & vice-versa

• Takes care of creating & safely disposing DSS resources

Abstract Base Component Class

• Abstracts out functionality needed by any component

• New components derive from the base CODA infrastructure,

and then add component specific details – this dramatically

decreases development time

• Derived components implement the required behavior & can

extend the base state machine and introduce new behavior

• Takes care of creating & safely disposing CCR resources

Abstract Base Component Class (cont.)

• Main Message Port

• Internal Message Port

• State Machine

• State Data

• State Enumeration

• Config Data

Abstract Base Component Interface Object

• Has API wrapper methods for Configure, Open & Close

• Has helper methods to talk to the transport

• Takes care of creating & safely disposing CCR resources

• Takes care of safely disposing the Transport

• Other interface objects derive from it & add component

specific API wrapper methods

Component Creation Factory

• Single point of component creation, irrespective of local or

remote

• Component Creation Creation Steps

• Create a new local/remote component with the given creation details

(Name, Type, IP Adress, Port etc.)

• Create a new Transport & bind it to the Component

• Create a new Interface Object & bind it to the Transport

• Returns the interface object to the client

Local Component Life Time Management

Remote Component Life Time Management

Imaging Component Creation Steps (cont.)

• Create new DSS client & DSS server classes & register the

API messages for remote transport

• Create a new interface object class that derives from the base

interface object class & add the new API message wrappers

• The Imaging Component can work with any camera & stage

controller compatible with the framework