external projects related to wg 6

Deutscher Wetterdienst

External Projects related to WG 6

COSMO Working Group 6: Reference Version and Implementation

Working Group Coordinator: Ulrich Schättler

06.09.2010 COSMO General Meeting 2010 2

Contents

Interoperability

Priority Project POMPA: Performance On Massively Parallel Architectures


Status of the Interoperability Programme

Ulrich Schättler DWD

Philipp Glatt MeteoSwiss


Idea of Interoperability

To support future collaborative effort in European Limited Area Modelling by:

Defining a standard output format

Providing software tools (adaptors) to convert LAM output to the standard format

Enabling European LAMs to run from appropriate alternative model fields

To display or use in NMHS post-processing system

To provide operational backup


GME IFSUM ARPEGE

COSMO UM_local ALADIN HIRLAM

Fieldextra

Read (Write) Grib 2and transform to COSMO-Model grid:only 2D fields

For Parameter List 1:ps, psml, t2m, td2m, …

Plans for the Adaptor


GME ARPEGE

COSMO ALADIN HIRLAM

INT2LM

For Parameter List 2:u, v, w, t, p, qx, …

Read (Write) Grib 2and transform to COSMO-Model grid:2D fields, full atmosphere,soil parameters

to be donestatus unclear

GFS?GSM?

Plans for the Adaptor


lum2lm (Grib1): Temperature (lowest COSMO Level) from

GME IFS

UMAemet

UMDWD


lum2lm (Grib1): PP from reference pressure (lowest level) from

GME IFS

UMAemet

UMDWD


lum2lm (Grib1): Wind speed (lowest level) from

GME IFS

UMAemet

UMDWD


In December 2009, the “Interoperability Team” met in Reading for discussing last years work and do some planing for 2010.

The first 2 deliverables were finally agreed: The Standard Format shall be the native grid and native parameters Documentation supplied with the standard format should be the

minimum required Some Grib 2 Issues were discussed: There is a problem with coding the

vertical coordinate / vertical grid of COSMO in Grib 2 (see tomorrows session on Postprocessing)

Workshop in Reading


Work in 2010 Vertical coordinate / vertical grid of the COSMO-Model

A COSMO-internal discussion about coding the vertical grid in Grib2 has been initiated early 2010

For the Gal-Chen height based hybrid coordinate it is possible to code the grid with typeOfLevel=118, the corresponding orography (as a 2D field) and vertical coordinate parameters

This is NOT possible for e.g. the SLEVE coordinate

The COSMO discussion led to a proposal to the WMO Commission for Basic Systems (IPET-DRC: Inter-Programme Expert Team on Data Representation and Codes). A new vertical coordinate type is proposed, where no vertical coordinate parameters together with a 2D field are given, but a full 3D field with the heights of every grid point is specified.

The proposal has been discussed also with Enrico Fucile (ECMWF)


Work in 2010 grib_api

grib_api has been established at some COSMO centres to be the official Grib(2) library (DWD, MeteoSwiss, ARPA-SIM Bologna)

Work is in progress to improve performance on NEC machines (together with NEC, MeteoFrance and ECMWF)

It has been implemented in FieldExtra for reading and writing. FieldExtra therefore can now produce the parameters of the standard output format list directly

Work is in progress, to implement it into INT2LM


Work in 2010 New Grib 2 test data set and documentation

A new Grib2 test data set, which has been produced by FieldExtra, has been provided. This still has the old (Grib1-style) coding of the vertical coordinate

No Grib2 test data has been provided with typeOfLevel=118, because this might not be the default case any more (if proposal is accepted)

No surface test data (according to the guidelines from the ET on Surface Aspects) has been provided up to now

The minimum required level documentation has not been updated according to the plans from last workshop

describe native grids

describe fields in the standard output list: what field is it, what does it represent in the generating model


Priority Project POMPA

Performance on MassivelyParallel Architectures

(To be confirmed by STC)

Project Leader Oliver Fuhrer MeteoSwiss


Computer architectures are about to change.

Multi-core, many-core systems are on their way.

Many people talk about General Purpose GPUs.

But first: Identify the problems of the COSMO-Model on existing machines!

Background: Future Computer Architectures

NEC SX-9

8 Procs

IBM pwr6

256 procs

Computations Dynamics 729.59 570.44

Computations Physics 506.18 220.45

Communications 115.61 207.69

I/O 124.43 108.40

% of I/O and Comm. 15 25


Fraunhofer / SCAI investigated LM_RAPS_4.0 on IBM BlueGene

The RAPS code is not as optimized on NEC SX-9 as the current operational model is

Timings for COSMO-DE 421 461 50 grid points, dt=25 s

Modern architectures surely are more similar to BlueGene than they are to SX-9

There are problems in the I/O and the overall scalability!

Investigations from Fraunhofer / SCAI on BlueGenewith RAPS 4.1


Scalability of COSMO-EU: 665 657 40, 24h


Scalability of COSMO-DE: 421 461 50, 21h


Scalability of COSMO-Europe: 1500 1500 50, 2.8 km, 3 h, no output


I/O: Accessing the disks and the global communication involved disturbes scalability heavily.

Although the communications besides I/O are almost all local, the speedup degrades when using many processors.

What cannot be seen on the pictures before: Although the speedup of the computations is not bad, the efficiency of the code is not satisfying:

NEC SX-9: 13 % of peak

IBM pwr6: about 5-6 % of peak

Cray XT4: about 3-4 % of peak (?)

The Problems of the COSMO-Code …


More Background

Swiss national initiative HP2C

High Performance High Productivity Computing

Plan: Enable software to run efficiently on new computer architectures (multi-core, many-core, GP-GPUs, etc.)

ETH Zürich is taking part with the COSMO-CLM: HP2C-CCLM

This project has 3 subtasks

Cloud-resolving climate-change scenarios

Preparing the CCLM and existing model extensions for future computers

Refactoring CCLM components for emerging architectures

HP2C-CCLM runs from Spring 2010 to December 2012


Finally: PP POMPA

POMPA is the „missing link“ between the Swiss national initiative and our consortium

POMPA gives additional resources in the optimization of the COSMO-Model for future computer architectures

POMPA can ensure that the operational COSMO-Model really benefits from the outcomes of HP2C-CCLM


Thank you

very much

for your

attention

external projects related to wg 6

Documents