Implementing WMS interface on Imagery Database (BDI)

Frédéric GUILLAUDDSI/DEV

EGOWS Meeting KNMI, June 2009

2

Overview

Context– Towards SOA for production and workstations (Marie-Françoise talk)– DWG Meteorology and WMS IE

Soprano imagery database (BDI) WMS implementation Issues Still to do Conclusion

3

Context

IT SOA pilot project Goals

– Improve our skill with SOA architecture and tools (ESB)– Improve our skill with OGC standards– Early identify issues– Make technical decisions

DWG meteorology IE on WMS– Calendars compatibles (Over 6 month by the end of 2009)– WMS BDI may be a service involved in IE experiments– To do this, WMS should be deployed in DMZ– This lead to security issues

4

Soprano Imagery database (BDI) : Before

RDBMS Oracle 3 days real-time data online (“blob” data type). Data is not archived. About 250 datasets Legacy catalogue content not always consistent … Native formats

– BUFR, TIFF-MeteoFrance (TIFF with - open - specific MF tags…) Native extent depends on dataset origin (MSG, METOP, other

geostationary platforms, radar data ..) Native CRS

– “Space View” (One-Point Perspective Projection ?) and “Polar Stereographic” Data access services

– Low level (application binary executable)– No API available– No remote access possible

5

Soprano Imagery database : product overview

Basic radar reflectivity measurements (for each radar) Mosaics

– Precipitation flux (through reflectivity conversion)– Precipitation accumulation (5’, 15’, …)– Forecasted intensities (2PIR algorithm)

6

Soprano Imagery database : product overview

Satellite data• From MSG, METOP, GOES platforms …• Composite CMS products, …

7

Soprano Imagery database : products overview

Composite products– Radar precipitation intensity calibrated with ground measurements

(ANTILOPE)– Satellite-Based fog diagnostics (CARIBOU)

8

WMS BDI features : Now

About 250 Layers GetMap returns only one LAYER at a time LAYER name is an internal identifier (tokens delimited by “_”, in

order to make easier mapping WMS – RDBMS data access Pre-defined STYLE : Each LAYER has a default colour map, which

is the official colour map on operational workstations at Meteo-France.

User defined styles are not yet supported (SLD=) If TIME not specified, GetMap returns the last image Outputs formats : image/gif; image/png Output CRS : only EPSG:4326 and Polar Stereographic at the

moment No support for metadata layers (e.g. pixel timestamp, pixel

quality, ...) but may be WCS scope ?

9

WMS Implementation : Reuse of legacy components

Data access component : “RDImage”– Application Binary Interface (executable)– Performs data extraction from BDI on criteria :

• Data type (radar, satellite …)• Production type (observation, forecast …)• Data origin (sensor platform, system, algorithm …)• Native domain covered by the dataset • Parameter (rain intensity, accumulation, radiance/wave length …)

Image processing component : “TransImage”– Application Binary Interface (executable)– Performs :

• Sampling a sub-domain• Format translation• Coordinate reference system operations (re-projection)

10

WMS Implementation

Meteo-France WMS framework– Implements WMS interface in PHP– Check on GetMap – GetCapabilities consistency– WMS 1.1.1 or 1.3.0

WMS BDI driver– Links GetMap parameters to RdImage & TransImage ones– Depending on GetMap parameters, invoke TransImage if necessary– Generate GetCapabilities response (XML)

11

WMS implementation overview

BDI (Oracle)

OCI

RDImage (.exe C/C++)

TransImage (.exe C/C++) Capabilities (C )

Meteo-France WMS Framework (PHP)

GetMap GetCapabilities

BDI Driver (PHP)

WMS

12

Issue : GetCapabilities layering

GetCapabilities response – Mainly product metadata– Describe product types (characteristics common to product instances)

GetCapabilities tree : Observations & Measurements approach– 3 Levels of layers :

• Process– Product type, Platform, Algorithm e.g Satellite MSG

• Feature Of Interest– Domain e.g Europe-Atlantic …

• Observed property– Parameter e.g IR 108

– Leaf layer is a named product type (queryable)

13

Issue : GetCapabilities layering

Layer granularity – Balance LAYER / Multi-dimensional LAYER (LAYER plus DIM)– E.g :

LAYER=SA__OBS__MSGC__V_EURATL__WV-62Or

LAYER= SA__OBS__MSGC__V_EURATL DIM_Wavelength=62 Or

LAYER=SA_OBS DIM_Platform=MSGC DIM_NativeDomain=V_EURATL DIM_Wavelength=62

Parameter names, units, … for DIMENSION usage– We need controlled vocabularies

E.g : DIM_Wavelength=62, 73, 108 , DIM_AccumulationTime=5, 15, 60 …DIM_BaseTime , DIM_Quality, DIM_PixelTimeStamp

The granularity of the layers will impact : – The efficiency of catalogue searching – The size of GetCapabilities response and performances

14

Issue : Keywords & Titles

Issue : Finding the layer name to perform GetMap request …– Layer names are quite complicated (internal identifiers), but unique :

• Ex : “RD_CPO_P__OBS__RD_ETRANGER__V_PIEMONT__REFLECTIVITE”– About 250 layers in GetCapabilities– At the moment “Title” == “Name” (that is bad !)

• difficult to be interpreted by a human… Title should be explicit – No abstracts, no keywords

To help catalogue searching, we are looking for keywords, “Controlled vocabularies”, “Ontology” …

– E.g “METEOSAT” “MSG”, “METEOSAT-8” “METEOSAT-9”, … Keywords should be hierarchic

– E.g “TEMPERATURE -- CLOUD -- TOP” Catalogue searching engine tested : GeoNetwork Possible keywords references :

– Global Change Master Directory (GCMD)– Common WMO code tables (Table C13, …)– AMS Glossary of Meteorology (GMET)– EUMETSAT , HMA (for satellite data) ?– Others ? To be defined ?

15

GeoNetwork evaluation as BDI catalogue

16

ISO 19128 – ISO 19115 relationship

ISO 19128 (WMS) Metadata elementproperty ISO 19115

Title CI_Citation.title

Abstract MD_DataIdentification.abstract

Keyword element in Keyword list MD_TopicCategoryCodeMD_Keywords

EX_GeographicBoundingBox EX_GeographicBoundingBox

BoundingBox EX_BoundingPolygon

DataURL MD_metadata.dataSetURI

Attribution CI_ResponsibleParty

Identifier and AuthorityURL CI_Citation.identifier

17

Issue : ISO 19128 – ISO 19115 relationship

How to keep consistency between WMS GetCapabilities (ISO 19128) and ISO19115 Metadata common fields ?

Where is the metadata reference or entry point ?– SOPRANO ISO 19115 metadata database

• To be created …• Partially harvested from VGISC metadata (probably not sufficient for BDI

because focussed on GTS products)– BDImage (or other databases) descriptive schema content

• Incomplete (keywords, abstracts are missing …)• Not always consistent (because of the database history…)• Difficult to change (because of backward compatibility)

18

Issue : Time

Time information– It would be useful to include product instance availability information

(ISO 19108) in GetCapabilities response :• Potential availability (time range plus step)• Real time availability (real time list : may be complex and CPU time –

consuming– There is a TIME parameter in GetMap, but no dedicated tags for TIME

information in GetCapabilities response WMS 1.1.1 and 1.3– GetMap : no support for multi-dimensional time (forecasted imagery)

e.g. : Forecasted precipitation intensity = f ( Base time, Validity Time )– TIME semantics may be ambiguous

e.g : Sampling Time, Validity Time, Result Time, Base Time … Possible approach :

– Using “Dimension” / “ExtendedCapabilities” ?

19

Issue : CRS identifiers

There are three methods to identify and describe CRS Namespaces & codes

– We need CRS identifiers not yet define in EPSG database and / or AUTO namespace, others …

• EPSG (not parametric) – Lot of codes to be defined ( all CRS in use at Meteo-France ? )

• AUTO2 (parametric) :– Universal Transverse Mercator AUTO2:42001– Transverse Mercator AUTO2:42002– Ortographic AUTO2:40003– Equirectangular AUTO2:40004– Mollweide AUTO2:40005

• Is it possible to define other codes ?– Ex Polar Stereographic for each prim meridian, “Space View”, .. ?

Reference (URL) to ISO 19111 documents Proj4 “input parameters – like” ?

20

Issue : Performances

It takes about 1 seconds to process full disk MSGC image “on the fly”– Reprojection appears to be very expensive– ”On the fly” processing probably not possible if many concurrent

requests. We probably need pre-defined CRS

– This would allow pre-calculation of the projection matrix for each CRS transformation (in TransImage)

– WMTS implementation ? But we’ll have to pre-compute tiles for each layer, CRS, Zoom Level, Format, Time …

– WMTS implementation is obviously a challenge for real-time Met data.

21

Left to do …

Lot of work on Metadata– Keywords, abstracts, ….– Link GetCapabilities response – ISO 19115 metadata

New output CRS New output formats (does GeoTiff make sense in a WMS ?) Support for multi dimensional times Support for metadata layers (pixel time stamp, pixel quality, … ?) Support for client style (SLD=) Improve performances

Limited set of CRS, projection matrix in TransImage component , WMTS Proj4 – GDAL migration of TransImage ( GEDAL is already in use on

SYNERGIE) TransImage as service Security topics to be investigated

22

Conclusion : BDI WMS works but …

To get really interoperable WMS, we need to agree on :

Metadata implementation rules (at least keywords) Colour Maps / Units of Measure

– E.g : Radar Data : Reflectivity (DBZ) – Intensity ( Kg m-2 / h) CRS policies (EPSG, AUTO2, …) How to manage multiple time dimensions Layer granularity for each product type ( balance LAYER – layer

DIMENSION ) Parameter names, units for DIMENSION usage

Thank you for your attentionquestions ?