RBNB DataTurbine (Ring Buffered Network Bus )

• Released under Apache 2.0 Open Source License• Solution for accessing both streaming and static data, from different vendor systems, via a common interface• Provides real high performance data streaming, 10Mb/sec, 1000 frames/sec on PCs• Supported by NASA SBIR, 15 years in development• NSF invested in supporting open-source development of the Data Turbine

– SDCI project, 2007-09,~ $850K

• It is one of just a handful (actually, 2) comprehensive solutions for managing streaming data

Motivating Projects• NEON – Ecology (you’ll see a working demo)

– http://neoninc.org • GLEON – Hydroecology (deployed, international)

– http://gleon.org/ • CREON – Coral reefs (development, deploy 2007)

– http://www.coralreefeon.org/ • MoveBank – Animal tracking (newly funded, NSF)

– http://www.princeton.edu/~wikelski/research/index.htm • Bridges and Civil Infrastructure – Engineering (testbed

status)– http://healthmonitoring.ucsd.edu/

• NEES – Earthquake Engineering (deployed)– http://it.nees.org/

• PRAGMA – Pacific Rim Applications and Grid Middleware Assembly (deployed)– http://pragma-grid.net

Integration of Heterogeneous Devices

Project NI cRIO

Campbell CR510

Apprise Templine

Davis weather station

Vaisala WXT510

Vaisala PTB210

Axis 241 (video)

Greenspan Dissolved Oxygen Sensor

GLEON X X X X X CREON X X X NEON X X X NEES X X PRAGMA X X X X X

More about the DataTurbine• Can be configured to feed data to several applications, including remote

servers• Supports multiple types of streams: real-time monitoring, video and

multimedia, telemetry, instant messages, etc. etc.• Can be accessed via URLs (e.g. can stream to browser); one can also write

to the server via browser• Can be mapped as a network drive (e.g. as a “Web Folder” opened in IE),

built-in support in Windows, Mac OS X, Linux, several other systems• Has a programmer API, and a developer community. .Net support available

(though Java is used more often)• Has direct connection with Matlab; M-files are provided with standard

distribution• Has several standard applications: rbnbAdmin, rbnbSource (signal

generator), rbnbPlot, rbnbChat• Scalable: DataTurbine servers can be interconnected to handle large

streams• Can manipulate the streams: fast forward or slow motion playback (TiVo-

like)• Secure access to DataTurbine Server, based on user credentials (under

development)

These are common streaming data infrastructure requirements

RBNB DataTurbine as infrastructure component

• Scalable, secure, programmable, versatile for different data types and vendor interfaces,developer community, with many applications written (e.g. data viewers and plug-ins)

• Typical scenarios:

DBMS

Monitoring and

management apps

CS loggers

NI loggers

Other proprietary or in-house

CS loggersLoggernet

DataTurbine in GLEON

Web sites and demos

• Main site: http://www.dataturbine.org• DASH demo: http://river.sdsc.edu/ucsddash• Geeting RBNB:

neesforge.nees.org/projects/rbnb/  • Real-time Data Viewer (RDV):

http://it.nees.org/software/rdv/index.php• JavaDoc:

C:\Program Files\RBNB\V3.1B1\Documentation\JavaDoc\index.html

RequirementsYou need:

– A computer running Linux, Windows, Unix, OSX or similar.– More importantly, it must have a working JVM, version 1.1 or

later. Different brands of JVMs should be fine (e.g. Sun, IBM, Jrocket, etc)

– Enough memory to hold the data you want– Enough disk to contain the archive you want– A network connection that's fast enough and reliable enough

for your needs.– We like Apple minis as minimal servers. With 2GB of memory,

they're fast and cheap. However, since all that matters is the JVM, you can use whatever you prefer.

– In general, more memory is good. A 32-bit JVM can use up to 3.5GB, and with a 64-bit JVM you can address as much as you can afford.

– If you have extreme needs, consider a 64-bit Sun box. We've had good results with their Niagara-architecture T2000.

Setting UpGet RBNB and install (double click Jar file)

RBNB-V3.1B1 has been successfully installed to:

C:\Program files\RBNB\V3.1B1

Usage-----

To use this product, start the Web server by either:

1) WINDOWS: using the shortcut provided on the desktop. 2) LINUX/UNIX/MACOSX: by executing the script:

C:\Program files\RBNB\V3.1B1/bin/Start_WebServer.sh

Once the Web server has been started, point your Web browser to:

http://localhost. . .

Inserting stream data in RBNB

• Configure the sensors. The data sources can produce either numeric or image data.

• According to RBNB Simple API (SAPI), create a program to insert stream data in RBNB server. Using simple API defined by RBNB, create a “source” program to insert the digital data values into RBNB. Documentations on SAPI and examples are included in the downloaded package (C:\Program Files\RBNB\V3.1B1\Documentation\JavaDoc\index.html)

Install RDV and verify data stream

• Real-time Data Viewer is the usual DataTurbine client

– Install it– Launch via JNLP, the Java Network Launch

Protocol. This requires that you write a small XML file defining the application, command line parameters and JVM options. An example can be found at it.nees.org/software/rdv/RDV.jnlp Note that deploying this requires a Java code signing certificate and the jarsigner program; you'll have to get your own or use our signed JAR files.

Now, the familiar ODM part…• Setup ODM database in HIS, and insert various

metadata information in it – at the moment, we automatically insert the sensor data only.

Site information, sensor type information, and other appropriate fields need to be populated.

• Configure the JAVA program ‘stream2db’ and run it.

– The program automatically inserts data values from the RBNB into ODM database when the new data arrives in RBNB server. It needs the name mapping between the sensor channels from the RBNB and column name and table names in ODM database.

• Monitor the database – Open the ODM in SQL Management Studio and edit tables.

Open DataValues table and see the values that came.• Configure web services over the ODM, and DASH

– We’ve done it before