Ocean Color Data Processing System:
Status
November
NASA s Goal
To make available the highest quality ocean color and sst data to the broadest user community in the most timely and efficient manner possible
•
• EExpertise:
Requirements for Success
• EExpertise: internal
highly integrated project structure with all
elements co located continuous communication
Requirements for Success
• EExpertise: internal and external
highly integrated project structure with all
elements co located continuous communication
Strong links with mission specific expertise MCST and
research community algorithms validation data new
products
Requirements for Success
• EExpertise
• IInfrastructureflexible data processing system that constantly upgrades
procedures technologies and equipment
Requirements for Success
• EExpertise
• IInfrastructureflexible data processing system that constantly upgrades
procedures technologies and equipment
SCIENCE drives the system rather than the
SYSTEM driving the science
Requirements for Success
Processing Cluster34 processing nodes 1.5 TB
9 storage nodes 17.1TB
Ingest Servers2 SeaSpace groundstations 78 GB9 storage nodes (1xx,2xx) 16.6 TB
Distribution Servers (ftp)6 storage nodes (2xx) 25.5 TB
1 server 250GB Distribution Servers (web)Production: large server 3 TB3 processing nodes (2xx, 3xx)
46 storage nodes (2xx, 25x) 303 TBDevelopment: 2 superstore nodes 10 TB
Testing Cluster8 test nodes (1xx) 480GB
7 processing nodes (2xx) 365GB2 storage nodes 2.4 TB
Network SupportSystems
Database ServerProduction: 1 large server 876 GB
Development: 1 superstore node 5.8TB
Backup Servers1 large server 876 GB+2 TB tape
13 storage nodes 15.6 TB
Extreme NetworksBlack Diamond 6816
Gigabit Ethernet switch280 ports
User Desktops
Cal/Val & QC Systems
Mission Operations Systems
Ocean Color Processing System
Overview
Development Servers1 processing node
2 storage nodes 2.4 TB
Ingest queue
table
L0
Ingest
process
L1A-L1B
(MOD_PR02)
L1B-L2
(MSl12)
MODISA L2
table
Archive - Distrib
Server
L3BIN
(l2bin)
MODISA L3-bin
table
Archive - Distrib
Server
MODISA L3-map
table
Archive - Distrib
Server
Operational MODIS-Aqua Data Flow (NRTPE)
Terra and Aqua
Software process
Hardware system
Database table
Ancillary input data
Sensor
CAL
Sensor
attribs
Atm
corr
Browser
CGI /
httpd
Ocean Color
Web Server
User
Community
MySQL
DB
Ozone
L3MAP
(smigen)
MET
MET,
Ozone, and
OISST data
are
dynamically
selected for
each L1A
granule
Product meta
data are
populated from
production DB
October 30, 2006
OISST
Ocean Color
Ingest
MODISA L0
table
Archive - Distrib
Server
ATT/EPH
Ingest
process MODISA atteph
table
Archive - Distrib
Server
L0-L1A
(MOD_PR01)
Geo-Location
(MOD_PR03)
MODISA L1
table
Archive - Distrib
Server
Full-resolution day-
and nighttime via
SEN (~60 GB per
day/mission)
NOAA Realtime
System
NASA EDOS
System
Missing
L0
process MODAPS L0
Archive
MODAPS L0
Archive
Granules not
received from
NOAA realtime
flow
• EExpertise
• IInfrastructure
• DDataMost efficient and ultimately most cost effective when
source data is available online for all needs including
processing reprocessing evaluation testing distribution
Requirements for Success
CZCS: Volume GB
Files
OCTS: Volume GB
Files
SeaWiFS GAC: Volume GB
present Files
SeaWiFS MLAC: Volume TB
Files
SeaWiFS LAC: Volume GB
Files
Current Source Data Holdings
MODIS AQUA Level A Subset now
Volume TB Files
MODIS AQUA Level approx TB year
Volume TB Files
MODIS TERRA Level A Subset now
Volume TB Files
MODIS TERRA Level since August
Volume TB Files
Current Source Data Holdings
• EExpertise
• IInfrastructure
• DData
• CCommunication
Requirements for Success
OceanColor
Web
oceancolor gsfc nasa gov
Consolidated
data access
information
services and
community
feedback
• EExpertise
• IInfrastructure
• DData
• CCommunication
• IIntuitive and efficient data distribution
Requirements for Success
• EExpertise: internal and external
• IInfrastructure
• DData
• CCommunication
• IIntuitive and Efficient data distribution
• CCommunity access to data
processing analysis tools
Requirements for Success
Extensive user support with
over sites active online
Forums posts
SeaDAS ported to the
Macintosh OS X including
new Intel architecture Also
runs on Linux Sun SGI
Redesigned GUI and
website are now much more
user friendly
New simple online installer
and four SeaDAS ftp mirrors
Australia Brazil Japan UK
SeaDAS lite option for
display analysis only
MB download
Modis Direct Broadcast
And High Resolution
processing module
User training workshops