Experiences in the Use of XML in Meteorology

Kirby James

Meteorological Office

http://www.bigfoot.com/~pkjames

Meteorology - A Global Science

WMO - World Meteorological Organization

• Agency of the United Nations

• Global Responsibilities (180+ countries)

• Co-ordination of Observations, Standards, Reference and Training

• Six Official Languages - English, French, Spanish, Russian, Arabic & Chinese

Reference Material Dictionaries & Glossaries

• ideal application for XML & style sheets

• multi-language support

• single source - multiple outputs in different formats – paper (RTF)– Web (HTML)– on-line database (XML)

Oxford English Dictionary

Oxford English Dictionaryhttp://algonquin.uwaterloo.ca/OED/

<E> ; ENTRY

<HG> ; HEADWORD GROUP

<HL>slate</HL> ; HEADWORD LEMMA

<MPR>sl<i>e&mac.</i><su></su>t</MPR> ; M PRONOUNCIATION

(<IPR> ; IPA PRONOUNCIATION

<IPH>sle&shti.t</IPH>)

</IPR>,

<PS>v.</PS> ; PART OF SPEECH

<HO>3</HO> ; HOMONYM NUMBER

</HG>

<LB>north.</LB> and<LB>Sc.</LB> ; USAGE GEOGRAPHIC

WMO - International Meteorological Vocabulary

• 800 pages• 3,000 entries (keywords)• four languages (English, French,

Spanish, Russian)• cost $100

WMO - International Meteorological Vocabulary

A3310

automatic picture transmission syn. APT

Direct broadcast of low resolution pictures taken from a satellite to ground stations equipped with appropriate receiving devices (B1200).

transmission automatique d'images syn. APT

Mode de diffusion directe d'images satellitaires à basse résolution aux stations terrestres dotées de l'équipement de réception nécessaire (B1200).

International Meteorological Vocabulary -

Conversion to XML

• original document was available in machineable form (.RTF)

• document had high level of structure

• conversion to XML was relatively straightforward (custom Perl scripts)

International Meteorological Vocabulary - Output

• primary requirement for Web output, in a number of different formats (e.g. single language, two language, etc.)

• requirement to minimise size of HTML files

• used James Clark’s Jade parser

• used DSSSL style sheets

International Meteorological Vocabulary

PARSER(Jade)

FAIL

DTD XML

OK

DSSSL

HTML RTF XML/SGML

UK Meteorological Glossary• already in machineable form (Quark Express)

• already highly structured

• single language (few foreign terms)

• requirement to output to Web format

• requirement for topic groupings

• requirement for mathematical formulae

Meteorological Glossary Hypertext in 1906

Classification - Topics

• Existing Classification of Glossaries

• Thermometer– BT = broader term (meteorological instrument)– NT = narrower term (Wet-bulb thermometer)– RT = related term (Thermograph)

• tree-like structure with cross-links

Glossary - Mathematics

• standards not widely accepted by users and vendors

• poor support of maths by browsers

• often in-line GIFs used

• used TeX embedded in <M> tags

• <M>$$ A = \pi r^2 $$</M>

Glossary - Output

Issues?

• American Meteorological Society Glossary

• 4 years work by involving 35 individuals

• 18 months over schedule

• cost in excess of $360,000

Meteorological Codes

• Code (noun)– a system of words, letters, figures or symbols,

used to represent others for secrecy or brevity,– a system of prearranged signals used in

transmitting messages,– a set of rules on any subject

Surface Observations

• SYNOP Code– 03772 11583 72314 10182 20119 40032 51031

69901 70282 868/1 81825 86645 90710 91125

– 03772 03 UK; 772 Heathrow– 72314 7 octas cloud 230° 14 knot wind– 10182 18.2°C air temperature– 20119 11.9°C dew point– 40032 1003.2 hPa QFF

XML (Synop) Code<!DOCTYPE OMF SYSTEM "http://zowie.metnet.navy.mil/~spawar/JMV-

TNG/XML/OMF.dtd">

<Reports TStamp="914456730">

<SYN Title='AAXX' TStamp='908539200' LatLon='37.483, 130.900’ BId='471150' SName='ULLUNGDO ISLAND’ Elev='223'>

<SYID WS='4'>16124 47115</SYID>

<SYG Ceiling='3000' Vis='5000' Wind='50, 8' T='17.6' TD='14.1' P='984.4' P0='1010.4' Pd='2 2.0'

Prec='9, 12' WX='1022' Tmm=', 18.9' Clouds='8552/'>11650 80516 10176 20141 39844 40104 52020 60092 71022 8552/ 333 10189

31017 55000 70126 92064</SYG>

</SYN>

</Reports>

Meteorological Code Tables<BUFR-TABLE ID="001003">

<TITLE>WMO Region or Geographical Area</TITLE>

<BODY>

<BN> 0 </BN> <BNAME> Antarctica </BNAME>

<BN> 1 </BN> <BNAME> Africa </BNAME>

<BN> 2 </BN> <BNAME> Asia </BNAME>

<BN> 3 </BN> <BNAME> America </BNAME>

<BN> 4 </BN> <BNAME> N America </BNAME>

<BN> 5 </BN> <BNAME> SW Pacific </BNAME>

<BN> 6 </BN> <BNAME> Europe </BNAME>

</BODY>

</BUFR-TABLE>

Logical Data Model for Meteorological Data

• map projection: – projection type,

– projection orientation,

– size and shape of the earth or other body.

• projection type: – latitude/longitude or

– instantaneous space view or

– polar orbiter or

– polar stereo-graphic or

– other.

Unexpanded SGML<!ENTITY MAP-PROJECTION '

<node><and> &PROJECTION-TYPE; </and> </node>

<node><and> &PROJECTION-ORIENTATION; </and> </node>

<node><and>size and shape of the earth </and></node> ' >

<!ENTITY PROJECTION-TYPE '

<node><or>lat-long</or></node>

<node><or>instantaneous space view</or></node>

<node><or>polar orbiter</or></node>

<node><or>polar stereo-graphic</or></node>

<node><or>other</or></node> ' >

Expanded SGML <N><A>Co-ordinate system</A>

<N><A>map projection</A>

<N><A>projection type</A>

<N><O>lat-long</O></N>

<N><O>instantaneous space view</O></N>

<N><O>polar orbiter</O></N>

<N><O>polar stereo-graphic</O></N>

<N><O>other</O></N>

</N>

<N><A>projection orientation</A>

<N><O>normal</O></N>

<N><O>transverse</O></N> . . . . . etc.

Documentation & Interface to Legacy Database

• Custom databases used to give acceptable performance for access to multi-dimensional data-sets

• XML used to document database calls (presented in HTML)

• same XML source used to drive Web-based ad-hoc query interface

Summary• benefits greatest when ‘data’ well structured

• cost of organising poorly structured data can be exceptionally high

• until recently lack of effective tools for preparation editing of XML source

• diminishing requirement for paper output reduces benefits of multiple output formats

http://www.bigfoot.com/~pkjames