neesgrid – a cyberinfrastructure for earthquake ... · stanford university (neesgrid...

Kincho H. LawStanford University

(NEESgrid Data/MetaData Efforts and Issues)

Kincho H. Law

Professor of Civil and Environmental Engineering

(Structural Engineering and Engineering Informatics)

Stanford University

February 28,2005

NSF EXCITED Workshop

(Exchanging CyberInfrastructure Themes in Engineering Design)

NEESgrid – A CyberInfrastructure for Earthquake Engineering Simulation


• NEESgrid System Integration Development

(NEESgrid - Bill Spencer, Jr. of UIUC – PI)

(Consortium effort – NCSA, ANL, USC’s ISI, U. Michigan, UC Berkeley, Stanford, and others…)

• NEESgrid’s Data and Metadata Efforts

(Data and MetaData Working Group)

• More on Data Issues

• Current Status

Outline of Presentation


George E. Brown Jr. Network for Earthquake Engineering Simulation (NEES)

Field Testing EquipmentUniversity of Texas, Austin

$2,937,036

Geotechnical CentrifugeUniversity of California, Davis

$4,614,294

Reconfigurable Reaction WallUniversity of California, Berkeley

$4,268,323

Fast Hybrid Testing LaboratoryUniversity of Colorado, Boulder

$1,983,553Multi-Axial Subassemblage Testing

SystemUniversity of Minnesota, Twin Cities

$6,472,049

Lifeline Testing FacilityCornell University

$2,072,716

Field Testing EquipmentUniversity of California, Los Angeles

$2,652,761

Three (two relocatable) Shake Tables

University of Nevada, Reno$4,398,450

System IntegrationUniversity of Illinois at

Urbana-Champaign$10,000,000Tsunami Wave Basin

Oregon State University$4,775,832

Dual (relocatable) Shake Tables and High Performance Actuators

State University of New York,University at Buffalo

$6,160,785$4,379,865

Consortium DevelopmentConsortium of Universities for

Research in Earthquake Engineering$1,999,907

Permanently Instrumented Field SitesBrigham Young University

$1,944,423

Modular Simulation LaboratoryUniversity of Illinois at

Urbana-Champaign$2,958,011

Multi-directional Testing FacilityLehigh University

$2,593,317

Large Uniaxial Shake TableUniversity of California, San Diego

$5,890,000

Geotechnical CentrifugeRensselaer Polytechnic Institute

$2,380,579

NEESit OperationsSDSC, University of

California, San Diego-$40,000,000


NEES Resources

Field Equipment

Laboratory Equipment

Remote Users

Remote Users: (K-12 Faculty and Students)

High-Performance Network(s)

Instrumented Structures and Sites

Leading Edge Computation

Curated Data Repository

Laboratory Equipment

Global Connections

(FY 2005 – FY 2014)

(Faculty, Students, Practitioners) Simulation

Tools Repository

• Enabling collaboration (laboratory experiments, field tests, simulations)

• Enabling data sharing, archiving and access by researchers (and public)


Vision of NEES (Vision of NEES (CollaboratoryCollaboratory))


NEESgrid!- CyberInfrastructure• To connect and facilitate experimentation/simulation (virtual

collaboratory) in earthquake engineering for US and the world

►Tele-Control Services and APIs

►Tele-Observation and Data Visualization

►E-Notebook

►Streaming data services

►DAQ and related services

►Data and Metadata Services

►Remote Collaboration and Visualization tools and services

►Core Grid Services, deployment efforts, packaging

►Simulation Component


Data/Metadata Working Group Member

Jean-Pierre Bardet University of Southern CaliforniaJennifer Swift University of Southern CaliforniaAndrei Reinhorn State University of New York, Buffalo(Data Sharing and Archiving Committee, DSAC)Ken Ferschweiler Northwest Alliance for Computational

Science and Engineering, Oregon State Univ.Lelli Van Den Einde University of California at San Diego Gokhan Pekcan University of Nevada, Reno(Coordinator of the Task Group)Hank Ratzesberger University of California, Santa Barbara Chuck Severance University of Michigan Bill Spencer University of Illinois, Urbana-Champaign(PI of NEES’s System Integration Project)Jim Eng University of MichiganJun Peng Stanford UniversityKincho H. Law Stanford University

Objective: To define a common approach and tools to enhance the sharing, access and utilization of the NEESgrid data repository


NEESdata

NEESpop

LocalRepositoryA

PI

CentralRepository

DataTeamlets

Data Acquisition

Workstation

AP

I

DataTeamlets

AP

I

Data/MDIngestTools

Data tools

Data viewers

Grid and Web Services

NEESgrid Data – Core Elements


Overall Data Modeling Efforts

NEES

Site A Site CSite B

Equipment People

Experiments Trials

Equipment People

Experiments Trials

Data Data Data

TsnumaiSpecimen

Shake TableSpecimen

GeotechSpecimen

CentrifugeSpecimen

Units Sensors Descriptions

SiteSpecificationsDatabase

ProjectDescription

Domain Specificmodels

Common Elements

Data / Observations


NEESgrid Experiment Data Flow

NEESGridData

Repository

ProjectBrowser

DataTurbine

DataIngestion

ExperimentControl

StreamingViewer

DAQC

D

SiteSpecific

ProjectRelated

ExperimentalSetup

ExperimentalElement

DataElement

Data Model

DAQDisk

StoredViewer


Evaluate existing data model representations and tools– Representation: E-R diagram, UML, NEESML, RDF, OWL– Tools: Protégé 2000, Rational Rose, Dezign

Evaluate existing data models for usability– Project Description: OrSt Model, Science Ontology– Sensors/Common Elements: Berkeley CUREE/Kajima, SensorML– Specimen/Structure Models: CIS/2, IFCs, SAC Data– Others: COSMOS

Visit, interview and discuss with experimental site personnel– Conference calls, UN Reno, UC Berkeley, UIUC, U Minnesota

Develop prototype models for feedback

Validate and test data models with saved experimental data

Data Model Development Effort by Stanford TeamData Model Development Effort by Stanford Team

A knowledge engineering approach:

(Shake Table/Large Scale Structural Tests – December 1, 2003)


Pre-ExperimentStage

ExperimentStage

Post-ExperimentStage

Simulations Simulations Simulations

Process: Activities

Data Model

InstrumentationSpecimen Units of Measure

Observations Pre-experiment and post-experiment data could be as valuable as the actual experiment itselfComputer simulations play a significant role towards the design of an experiment as well as for post-event investigations


Overview of Reference Data ModelOverview of Reference Data Model• Shake table and large scale structural tests• Simple and extensible, minimum burden to researchers • But sufficiently representative for earthquake

engineering experimentations

Data model representations:

Object Model– Has only 20+ main classes– Modeling tool: Protégé 2000– Backend storage: OWL (Web Ontology Language)

Relational Model– Has only 20+ tables– Modeling tool: Dezign– Backend storage: Relational database such as MySQL


Project

Task

EventGroup

Event

1 SensorSetup

DAQCableWaveFormSetup

OutputData

Sensor

Publication

Person

Organization

InfrastructureSetup

SepcimenSetup

PrimaryEquipment

Site

File

SoftwareSetup

DAQSystem

SetupFile

DAQSetup

Specimen

DAQChannel

RolePerson

WaveForm Software

Activity

ApparatusSetup Apparatus

DataElement

SiteInformation

Prototype Class RelationshipPrototype Class Relationship


Protégé Interface: Object ModelProtégé Interface: Object Model

Only 20+ main classes


OWL RepresentationOWL Representation


Relational Data RepresentationRelational Data Representation


Modeling of Specimen and SetupsModeling of Specimen and Setups

Universal modeling of specimen for all experiments is very difficult if not impossible

Goal is to provide ways to archive the data and information on the project and the experiment

Desirable formats and features: CAD drawings; sketched drawings and notes; photos; narrative description; electronic notebook; linkage of drawings, sensor locations to data, etc..


DAQChannel

DAQSystemSensor

DAQCable PC and/orExternal A/D

Sampling

ownermanufacturerserialNumberotherInfo

ownermanufacturerserialNumberotherInfochannelidfiltersamplingRategainoffsetexcitationVoltageunit

ownermanufacturerserialNumberotherInfocableidlengthconnectorType

ownermanufacturerserialNumberotherInfosensoridtypecalibrationInfolastCalibrationDateoutputQuantityminRangemaxRange

Modeling DAQ (data and setup)Modeling DAQ (data and setup)


Project Data Browsing


Data Services : Report Generator AssistanceData Services : Report Generator Assistance

RelationalDatabase:

MySQL

Jena JDBC

DBAccess.javaOWLAccess.java

DBDataRetrieval.javaOWLDataRetrieval.java

ReportGen.java

XML-Based DocumentsOWL (Web Ontology

Language)

Structured DataStroage

Based on Data Model

Standard Library forData Access

Present Common Interfacefor Data Access: e.g.

Access an Object by its ID

Retrieve Related Data:e.g. Data Related to

Instrumentation

Engineering ReportGenerator

DisplayNEES.java

OWLMapping.java DBMapping.java

Java Servlet Applications


Re: Prof. Andrei Reinhorn @ SUNY-Buffalo

Report Structure & Corresponding FunctionsReport Structure & Corresponding Functions

getSummary()getScope()getSetupOverview()getInfrastructure()getSpecimen()getLoading()getInstrumentation()getProcedure()getSchedule()getImplementation()getResults()getDataProcessing()getPredictions()getDiscussions()


Project

Task

EventGroup

Event

1 SensorSetup

DAQCableWaveFormSetup

OutputData

Sensor

Publication

Person

Organization

InfrastructureSetup

SepcimenSetup

PrimaryEquipment

Site

File

SoftwareSetup

DAQSystem

SetupFile

DAQSetup

Specimen

DAQChannel

RolePerson

WaveForm Software

Activity

ApparatusSetup Apparatus

DataElement

SiteInformation getSummary()

getSetupOverview()getInfrastructure()

getSpecimen()

getInstrumentation()

getLoading()

Data Model & Corresponding Functions


Front Page to Access Front Page to Access MiniMOSTMiniMOST ProjectProject


Retrieve SummaryRetrieve Summary


Scope and General PresentationScope and General Presentation


Infrastructure SetupInfrastructure Setup


InstrumentationInstrumentation


More on Data Issues…..Current Developments and Tools

• Facilitate experimenters and researchers generating and ingesting data into repository

• Allow browsing (and limited searching) and retrieving data/information about a project or experiment

• Support “No data generated by NEES would be lost” policy

But …..

“A data repository is only as good as what can be done with the data.”


Data Life Cycle

Data Production

Data Management

Data Consumption

• For how long?• What format? (standardization, usability)

• Heterogeneous data types (text, CAD, video, streams)• Raw, processed, derived, curated data• Non-proprietary data format and software (what to

archive?)• What to preserve? Everything? Selectively?

Need: data policies and short/long term plans for the cyberinfrastructure community

(not just the cyberinfrastructure but the community it serves!)


Consumers for the NEESgrid Data Repository:

• Experimenters – about generating reports, analysis of results

• Practitioners – about specific results for specific experiment on specific component (beam column joint)

• Researchers – about prior works for developing new ideas (new, novel active damping/sensing devices)

• Colleagues in other fields – data interoperability and sharing (IRIS’s seismological data sets)

• Faculty – developing teaching materials• Students – discovering new knowledge, mining archived

data • K-12 audience – about earthquakes and our civil

infrastructure systems


Preservation Planning

Administration

DataManagement

ArchivalStorage

AccessIngestProducer Consumer

Management

OAIS (Open Archive Information System) Functional Model

http://ssdoo.gsfc.nasa.gov/nost/isoas/

Current Focus and Deliverables

NEES Consortium

?

?


Capture Data/MetadataThroughout Data

Lifecycle

Data Models

Experiment Prep

Data Analysis

Data Publishing

Data Curation

Data Discovery and Reuse

Experiment Management

Data Monitoring


What have I learnt about NEESgrid (CI) data issues?

The amount of data (in a wide variety of formats) to be generated will be overwhelming – standardization is hard but some standardization is necessaryExpectations of what data models and repository can support are very high (and may not be easy to deal with in short term) Putting data in a computer (meaningfully) IS NOT an easy problem

A lot to be learnt among different disciplines – engineering, digital library, digital archive and preservation, library information science, information and communication technologies, ….Many related ongoing works: Skyserver, DSPACE, IRIS, GEON, BRIN, …..


Strong leadership and management team (not only technology developers) – coordinate activitiesTrue collaborative efforts – among developers, researchers, disciplines involvedDedication of community participants from the domain of concernPolicies and guidelines need to be well defined by the community, implemented and enforcedNeed to have “buy-in” by the researchers (or users of the CI)CI is not a pure “academic research” activity with much at stakes by the community at large

What have I learnt about NEESgrid (CI) issues?


Current Status

NEES: Officially opened Oct. 1, 2004 and Opening Ceremony at NSF Nov. 15, 2004NEES Organization: NEESr (Researchers), NEESInc(Management), NEESit (IT operations – SDSC)NEESgrid transition from NCSA to SDSC on Oct. 1, 2004NEESit to release interim version by end of March, 2005International collaborations/activities related to NEES:

KOCEF ($90 million effort in Korea) E-Defense (Japan)NCREE (Taiwan)European groups and others.


Rapidly Deployable CI and Data Problem

Source: A Proposal on “Innovative Use of IT in Post-Disaster Investigation,” EERI, Feb 2004


Acknowledgments

NEES’s Data and Metadata Task GroupNEES’s Data Sharing and Archiving CommitteeJoy Pauschke of NSF, Bill Spencer of UIUC, Chuck Severance of U. Michigan and Joe Futrelle of UIUC, Gokhan Pekcan of U. Nevada, Reno, Anke Kamrath, Lelli Van Den Einde of SDSC and many others …..Jun Peng, Stanford UniversityAny opinions expressed in this presentation are those of the presenter and do not necessarily reflect the views of the NEESgrid, NEESit, NEESInc, NSF and his collaborators.