ooi ci lca review august 2010 ocean observatories initiative instrument and platform agent...
TRANSCRIPT
OOI CI LCA REVIEW August 2010
Ocean Observatories Initiative
Instrument and Platform Agent Architecture (IPAA)
Steve Foley
Life Cycle Architecture ReviewLa Jolla, CA
OOI CI LCA REVIEW August 2010
Outline
• IPAA background
•Architectural Overview
•Release 1 Product Description Use Case Overview
•Technology Challenges and Achievements
•Progress in Elaboration Phase (from LCO to LCA)
•LCA Accomplished Use Cases
•Demonstrated Use Cases
OOI CI LCA REVIEW August 2010
IPAA Subsystem Overview
•Provides interface between instruments and the
rest of CI
•Configures and commands instruments
•Collects data from instruments and publishes
into CI
• Intended to be more production of drivers than
development of CI core concepts after initial data
plumbing is complete
OOI CI LCA REVIEW August 2010
Architectural Overview
•2 sides to the IPAA black box:
•AMQP protocol to talk to CI’s registries, services,
agents, UIs, etc.
•RS-232/Ethernet with instrument-specific text or
binary protocols to communicate with instrument.
•Between the 2 sides are the agent and driver that
provide access control, locking, translation, buffering
across slow links, instrument/driver specific logic
OOI CI LCA REVIEW August 2010
Instrument Agent Interface: CI Side
•Generic common interface for system-wide interaction
•Extensible at lower levels for instrument-specific work
•Based on getting and setting parameters and
executing commands
•Announces capabilities and uses registries for
discovery
•Potentially capable of transactions and sequences
OOI CI LCA REVIEW August 2010
Instrument Agent Interface
•Execute commands (CI and Instrument)
•Get/Set parameters (CI and Instrument)
•Get status (CI and Instrument)
•Get capabilities (CI and Instrument)
•Get translator function (CI)
•Get/Set CI lifecycle state (CI)
•Get registry entries (CI)
OOI CI LCA REVIEW August 2010
Test Instrument: SBE49•SeaBird Electronics SBE49 CTD has been our simple,
text-based test instrument for now, easy to mock up
and test. Some commands:•DS – Status & params•Polled sampling•PUMPON / PUMPOFF•TS – Take single sample•Autonomous sampling•START / STOP•NAVG=x – Samples to average•Settings•BAUD=x•OUTPUTFORMAT=x•…various coefficients to set
OOI CI LCA REVIEW August 2010
Instrument Driver Interface: Instrument Side
S>navg=16 # set a few parametersS>autorun=yS>pumpdelay=30S>ds # display some settingsSBE 49 FastCAT V 1.3a SERIAL NO. 0000number of scans to average = 16pressure sensor = strain gauge, range = 10153.0minimum cond freq = 3273, pump delay = 30 secstart sampling on power up = yesoutput format = raw HEXtemperature advance = 0.0625 secondscelltm alpha = 0.03celltm tau = 7.0real-time temperature and conductivity correction enabled, not applied to raw
dataS>outputformat=2 # Raw Tmp, Cond, Pres, SalS>ts # Test single sample290782, 2727.603, 524521, 1.6496S>start # Start streaming290782, 2727.603, 524521, 1.6496290785, 2727.598, 524521, 1.6496290781, 2727.610, 524521, 1.6496…
OOI CI LCA REVIEW August 2010
Instrument Driver Interface: Instrument Side
•Takes Get/Set/Execute based calls
•Translates to/from instrument protocol
•Maintains necessary protocol state/mode with
instrument
•Needs to know what commands are valid
•SBE49 has very little state to track, other
instruments will have more
OOI CI LCA REVIEW August 2010
Release 1 Product Description Use Case Overview
•Responsible for:
•Command An Instrument (#18)
•Supporting:
•Hello Instrument (#2)
•Direct Instrument Access (#19)
•Command a Resource (#20)
•Version a Resource (#24)
•Assess Access Policy (#31)
OOI CI LCA REVIEW August 2010
Command an Instrument Use Case• Initial use case is to command an instrument
•User determines commands to apply to an instrument
• Instrument Agent has already been created and
registered
•Commands are issued to Instrument Agent
•Commands have been redirected to Instrument Driver
specific to the instrument
•Commands are applied to the instrument by the driver
•Responses are passed back to the user
•Does not handle permissions or direct access
OOI CI LCA REVIEW August 2010
Status of Progress
•Elaboration period has been focused on
infrastructure for process management, data
flow, and proper interaction with registries
•End-to-end exchanges of messages demonstrate
complete data pathway and interface sanity
•“Command An Instrument” use case was
addressed and can be demonstrated.
OOI CI LCA REVIEW August 2010
Technology and Interface Choices
•AMQP between Agent and Driver
•May be changed during construction if
performance or situation requires it
•AMQP between Agent and CI registries
•Currently TCP with text protocol to serial device
(would be through a serial-to-Ethernet device)
OOI CI LCA REVIEW August 2010
Technology Challenges and Achievements
•Agent registration discoverable Instrument Agents
•Separation of agent and driver decoupled
processing
•Simulated device testing of data plumbing
•Protocol interaction with device science data flows
•Start data flowing from instrument, acquiring by
driver, and publishing to subscribers complete
data path coverage
OOI CI LCA REVIEW August 2010
Challenges – Registration
•Registries did not initially exist, but have been
developed with the Data Management team, and
their contents defined
• Instruments are the first working agents, so
plenty of work needed to be done to organize
registries
•Sequence is a bit involved, but easy to use at
higher levels
OOI CI LCA REVIEW August 2010
Challenges – Separation of Agent/Driver
•Having the driver and agent in the same process
was problematic for both CI interactions and
instrument interactions happening simultaneously
•Agent and driver communicate via AMQP messages
•Separate agent and driver allows for independent
development, testing, profiling, maintenance, and
improvement as needed
•Messaging layer is added complexity
OOI CI LCA REVIEW August 2010
Challenges – Simulation
•Did not initially have instruments
•Have multiple developers
•Need simple, repeatable, controllable, far end for
commands/queries
•We developed a simulator that handles very simple
requests from our code. Allows for a quick guess at
compatibility before a test instrument is ready.
•Definitely not a substitute for a real instrument
OOI CI LCA REVIEW August 2010
Challenges – Interacting with devices
•All instruments behave a little differently
• Initial simulator should fit protocol spec…but no
guarantees
•First cut of interaction with a sensor is to a mocked
up SeaBird Electronics SBE-49
•Allows for commanding an instrument and flowing
data
•Rest of instrument agent flows data through
completed data plumbing
OOI CI LCA REVIEW August 2010
Early Work Completed - RSN
While not on the Release 1 agenda, advance investigations were
made into RSN interfacing to keep pace with the rest of the project.
• Initial plans are for the interface to be Python (AMQP?) based,
SNMP may also be an option if needed
• Interfaces are being specified in detail
•Design seems to be inline with our future plans for platform
management
•RSN high-level integration design
•CG embedded device (Technologic Systems TS-7370) testing and
performance profiling
OOI CI LCA REVIEW August 2010
Early Work Completed - CGO
Work has been done to plan integration with the
CGO platforms.
•On-board controller will be Linux based embedded
ARM processor (Technologic Systems TS-7370)
•Python capable, early versions of code function
without modifications.
•AMQP library is slow, needs tuning
•More use case work needs to be done
OOI CI LCA REVIEW August 2010
One more Elaboration Iteration
• IPAA has one more iteration to go before IPAA
moves on to construction. This iteration will finish
up some plumbing:
•Support for direct access mode
•Streamline data flow
•Keep up with changing CI interfaces
OOI CI LCA REVIEW August 2010
Plan for Construction
• Integration
•More integration with RSN, possible prototyping
•Specify interfaces with CGO team, continue testing on
TS-7370
•Write preliminary acquisition drivers for CGO
platforms?
•Driver production
•Add drivers ADCP driver (binary format)
•Add Q330 data logger driver (via 3rd party Antelope sw)
OOI CI LCA REVIEW August 2010
Significant Risks•Performance on CGO embedded device
•Profiling indicates that OOI logic in python is fine for the
anticipated load, but AMQP library is not.
•Other libraries exist and may be worth testing.
• If Python is too slow, we may port of some code to C
•Common data format is still unspecified
•Simulators are only so good…will need real devices soon
•Some instruments are more difficult/involved than others
•Staffing – We need the people to write the drivers