using and programming echam6| a rst introduction · currently, the newest released version is...

Using and programming ECHAM6— a first introduction

S. Rast

[email protected]

September 26, 2014

mailto:[email protected]

Contents

1 Getting started with ECHAM6 5

1.1 Source code of ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2 Basic Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Input data for ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.1 Model resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.2 Files containing the data needed to run ECHAM6 . . . . . . . . . . . . . 10

1.3.3 Input namelists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Looking at your results 19

2.1 Basic visualization with ncview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2 Standard plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3 Basics about the netcdf data format . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3.1 cdl format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.2 ncdump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.3.3 ncgen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4 Climate data operators (cdo’s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.4.1 File formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.4.2 General structure of cdo commands . . . . . . . . . . . . . . . . . . . . . 26

2.4.3 Use of several operators in one command . . . . . . . . . . . . . . . . . . 26

2.4.4 Operators for getting information about files . . . . . . . . . . . . . . . . 27

2.4.5 Selection and renaming of variables . . . . . . . . . . . . . . . . . . . . . . 27

2.4.6 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.4.7 Conversion from spectral space to grid point space . . . . . . . . . . . . . 28

3

4 CONTENTS

3 Programming in ECHAM6 31

3.1 Simplified flow chart of ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2 Submodel interface in ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.3 Introduction of a submodel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4 Introduction of your own namelist . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.5 Read in fields from netcdf files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.6 Derived data types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.7 Tracers in ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.7.1 Data types associated with tracers in ECHAM6 . . . . . . . . . . . . . . . 39

3.8 Representation of 3–d variables in ECHAM6 . . . . . . . . . . . . . . . . . . . . 42

3.9 Working with 2 and 3–dimensional variables in ECHAM6 . . . . . . . . . . . . . 45

3.9.1 Global diagnostic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.9.2 Vertical coordinates: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.10 Recursive derived data types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.11 Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.12 Time control in ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.13 Radiation in ECHAM6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

List of listings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Chapter 1

Getting started with ECHAM6

ECHAM6 is an atmospheric general circulation model that was developed at the Max PlanckInstitute (MPI) in Hamburg over the last years and it is the youngest member of the “ECHAM”family. Predecessors are ECHAM3, ECHAM4 (that was never widely used), and ECHAM5. Itis a general circulation model describing the dynamics of the atmosphere. It is based on themodel of the European Centre of Medium–Range Weather Forecast (ECMWF). ECHAM6 iswritten in Fortran90 and it is a highly parallelized and vectorized code. Its structure is modularand allows for relatively easy coupling of new model components to the existing parts.

The goal of this short introductory course is to get you started with the technical aspects ofECHAM6. In particular, you will find a short description of how you can compile and run theECHAM6 code. We give a short overview of the input data and input variables that are necessaryto run ECHAM6. Some words about the data format of the output and its visualization finishthe first part. In the second part of the course, we give some more insight into the code ofECHAM6 and how you can change it for your purposes. In this part, we restrict ourselves tothe introduction of simple submodels which use tracers in the atmosphere. You will not findany discussion of the physics modelled by ECHAM6 (see [3]) nor the code structure of theland–surface submodel JSBACH in this introduction. The code of the land–surface submodelJSBACH is well separated from the ECHAM6 code and is therefore not subject of this shortintroductory course. A detailed discussion of the results and physics of ECHAM6 in the revision6.1 should start with the articles [6, 4, 2].

The ECHAM6 model is distributed in the framework of the MPI–M Earth System Model. Moreinformation about the model distribution can be found onhttp://www.mpimet.mpg.de/en/science/models/model-distribution/available-models.html

Currently, the newest released version is echam–6.2.00. This released version has a rather highclimate sensitivity and was never released as a component of the earth–system model mpiesm.The new version echam–6.3.00 will be released soon and will again have a climate sensitivitythat is comparable to the one of the echam–6.1.00 version. In this course, we use echam–6.2.01that is provided as a tar–file

blizzard.dkrz.de:~m218036/echam course 2014/echam-6.2.01.tar

You can access the supercomputer “blizzard” by

ssh -X -l <course\_account > blizzard.dkrz.de ←

5

http://www.mpimet.mpg.de/en/science/models/model-distribution/available-models.html

6 CHAPTER 1. GETTING STARTED WITH ECHAM6

Copy this version to your working directory blizzard.dkrz.de:/work/mh0010/<course account>

and unpack this version there with: tar xvf echam-6.1.01.tar

A directory echam-6.1.01 will be created. We will discuss the contents of its subdirectories inthe next section.

1.1 Source code of ECHAM6

The directory echam-6.1.06p3 contains the following files and sub–directories:

• aclocal.m4: machine specific compiler options

• configure: executable to create makefile

• configure.ac: file used to create configure script

• COPYING: license agreement

• INTSTALL: short description of compiling

• Makefile.in: template for makefile

• README: license agreement, information about changes with respect to predecessor version.

• target.∗: scripts and data basis for compilation with various compilers and machinearchitectures

• doc:

This directory contains available documentation about ECHAM6.

– codes.6: list of code numbers of output variables

– echam/echam6 scidoc.pdf: scientific documentation

– echam/echam6 userguide.pdf: comprehensive user guide including a description ofall input data, namelist variables, most important output files, and code structure.

– mpi-m sla 201202.pdf: Comprehensive license agreement

• source files and libraries of the ECHAM6 model:

– blas: BLAS (basic linear algebra subprograms). If no locally installed version isavailable on your computer, this version can be used.

– lapack: LAPACK (linear algebra package). If no locally installed version is availableon your computer, this version can be used.

– cdi: CDI library for reading/writing GRIB and netcdf format files.

– support: Support routines which are coded in the programming language c.

– unitrans: empty

– src: ECHAM6 source code files coded in FORTRAN90 for atmospheric part

– jsbach: ECHAM6 source code files coded in FORTRAN90 for land model JSBACH.

– include: The input for the ECHAM6 program is organized in namelists. Here youcan find all namelist definitions of ECHAM6.

1.2. BASIC COMPILATION 7

– lib: binary libraries after compilation.

• Directories containing scripts for running, compiling, and testing ECHAM6:

– config: Files related to the generation of a machine dependent makefile.

– m4: compiler related files

– util : containing special files for automatic creation of Makefiles

– contrib: Find a run script generator for CMIP5 type simulations in the subdirectorygenerate-scripts.

– run: Here you can find some example run scripts.

– test: collection of test scripts performing standard tests on ECHAM6. These testswork on a computer without queueing system only.

• bin : You will find the executable “echam6” in this directory after compilation.

1.2 Basic Compilation

Before you can run the ECHAM6–model you have to produce an executable program, this meansthat you have to translate the ECHAM6–code from the human–readable program source codeinto a machine–readable binary code. This binary code will depend on the architecture of themachine which you are using (parallel, vector or scalar architecture or a mixture of these three).The translation into the machine code is done by a compiler. According to the machine on whichyou are working, various directives or so–called options have to be passed to the compiler. Wedescribe the compilation of ECHAM6 on the supercomputer blizzard at MPI Hamburg. First,you have to load the right compilers:

Listing 1.1: Load compilers

module load IBM/xlf13 .1.0.8 IBM/xlC10 .1.0.7 ←

In a second step, go into the directory echam-6.2.01 and type:

Listing 1.2: basic compilation

./ configure ←

make -j 2 ←

The -j 2 option of the make command tells the compiler to use two processors for compilationin parallel. This speeds up the compilation procedure that takes several minutes. After thesesteps, a binary executable echam6 is generated in the directory echam-6.2.01/bin.

The source code consists of many files organized in modules and subroutines that depend oneach other. The compiler has to take these dependencies into account and start the com-pilation with the subprograms without any dependencies. The dependencies are listed inecham-6.2.01/src/Makefile whereas the echam-6.2.01/Makefile contains all compiler op-tions.

If you change the dependencies of modules and source code files or if you add a new sourcecode file, then you have to create a new Makefile in the src directory by typing the followingcommand being in the folder echam-6.2.01 before you execute the make command:

Listing 1.3: Dependencies: create new Makefile

util/createMakefiles.pl

It is not possible to execute this command in the util directory!


1.3 Input data for ECHAM6

The ECHAM6 program is a general circulation program of the atmosphere and thus needs datawhich act as initial and boundary conditions. The initial data contain information about thestate of the atmosphere at the beginning of a simulation whereas the boundary condition datacontain information about the properties of the Earth’s surface, the oceans, and solar irradiation.In addition, data describing properties of the atmosphere have to be provided. They are notboundary conditions but are described here as well for convenience. The ECHAM6 program willsimulate the state of the atmosphere on a certain grid, i.e. on certain locations that are spatiallydistributed in the atmosphere (spatial discretization). The initial state of the atmosphere is givenby the initial conditions. The simulation starts from this initial state and the simulation resultswill be such that they are consistent with the boundary condition data. The input data arestored in a central directory:

[blizzard.dkrz.de:]/pool/data/ECHAM6/input/r0002

This directory is directly accessible from thunder and contains resolution independent data,i.e. data that do not depend on the number of grid points in the atmosphere, the subdirectoryT63 contains data that depend on the model resolution. Before we can discuss the input files wewill explain the coordinate system of the model grid in greater detail and discuss the file formatin which the input files are given.

1.3.1 Model resolution

Horizontal resolution

The ECHAM6 program can handle the following horizontal resolutions although the input datafor some of these resolutions is not provided to date. The names of the resolutions are explainedlater in the text.

Horizontal resolution lon × lat nlon × nlat

T31 3.8× 3.8 96 × 48T63 1.9× 1.9 192 × 96T127 0.94× 0.94 384 × 192T255 0.5× 0.5 768 × 384

Some basic information about the correspondence of the number of grid points and the numberbehind the “T” is given in section 3.8.

Vertical resolution

The model uses σ–hybrid layers (σ–hybrid coordinates) in order to describe the vertical co-ordinate. The σ–hybrid coordinates are defined such that the model levels follow the terrain(orography) in the lower troposphere but become almost pure pressure coordinates in the highertroposphere. To this end, the levels depend on the surface pressure at each location on theglobe.

1.3. INPUT DATA FOR ECHAM6 9

= −90°

= 90°

0

Let λ describe the longitudes λ ∈ Λ := [0, 360[.

Let ϕ describe the latitudes ϕ ∈ Φ := [−90, 90].

Each point on the earth is then described by (λ, ϕ) ∈ Λ× Φ.

Let ps be the surface pressure: ps :

Λ× Φ× R+ → R+

(λ, ϕ, t) 7→ ps(λ, ϕ, t)at time t.

Then, we define L + 1 time and location dependent pressure levels pi, i = 1, L + 1 by a set ofL+ 1 coefficients (Ai, Bi), i = 1, L+ 1:

pi(λ, ϕ, t) = Ai + Bips(λ, ϕ, t) (1.1)

The coefficients (Ai, Bi)i=1,L+1 themselves do not depend on location or time. These pressurelevels define L layers ]pi, pi+1], i = 1, L in the atmosphere. The midpoints of these layers arepressure levels defined as:

pi := (pi + pi+1)/2 (1.2)

We can define new coefficients (Ai, Bi)i=1,L by

(Ai, Bi) := ((Ai + Ai+1)/2, (Bi + Bi+1)/2), i = 1, L (1.3)

and we get according to equations (1.1, 1.2, 1.3):

pi = (Ai + Bips + Ai+1 + Bi+1ps)/2 = (Ai + Ai+1)/2 + (Bi + Bi+1)ps/2 = Ai +Bips


Thus, (Ai, Bi)i=1,...,L describe the pressure levels at the midpoint of the layers as defined inequation (1.2). The most common numbers of levels for simulations with ECHAM6 includingtroposphere and stratosphere are L = 47 or 95 (middle atmosphere version).

The level 1 with p1 represents the uppermost level with the lowest pressure that is 0.01 hPa forthe “middle atmosphere version” comprising troposphere and stratosphere. The levels 47 or 95with p47 or p95 represent the lowest level with the highest pressure. The pressure of the lowestlevel is slightly lower than the surface pressure.

Near the upper limit of the model the levels are pure pressure levels (Bi = 0). In the lowerpart of the atmosphere the levels follow the terrain and are proportional to the surface pressure(Ai = 0).

Generally, p1 = 0 and pL+1 = ps.

Combinations of horizontal and vertical resolutions

The following combinations of horizontal and vertical resolutions are used:

Name Explanation Horizontal/vertical resolution

CR resolutions of tests or paleo–simulations T31L47 or T31L31LR low resolution T63L47MR medium (or mixed) resolution T63L95HR high resolution T127L95XR extra–high resolution T255L95

1.3.2 Files containing the data needed to run ECHAM6

All input files are in netcdf format. The netcdf format is a binary machine independent format.We will see later how we can display the variables of such a file graphically or how to generatean ASCII output of such files (see section 2.3).

In the following examples, we choose the resolution T63L47 but this may be replaced by anyresolution TnLl.

Initial conditions

The ECHAM6 model calculates “the dynamics of the atmosphere”. We can respresent thedynamical and physical state of the atmosphere as a point in a multidimensional space S madeup by all possible values of the variables describing this state. A trajectory ϑ in state space S isa mapping of a time interval [t1, t2] into this state space S. The ECHAM6 model approximatelycalculates ϑ(t) for each t ∈ [t1, t2] knowing the state ϑ(t1) ∈ S at the beginning of the timeinterval. We can assume that the trajectory is determined by a system of first order differentialequations without loss of generality since any higher order system can be transformed to a firstorder system formally (we introduce first derivates as additional unknowns e.g.). The mappingϑ : [t1, t2] → S includes a certain choice of initial conditions and shall be consistent with thegiven boundary conditions. This means that we have to provide a state ϑ(t1) as initial conditionto the model. In a deterministic world, only one single trajectory passes through each point


S ∈ S of the state space1. This condition of determinism means that the trajectory is completelydetermined by an initial condition ϑ(t1) and the choice of certain boundary conditions. On theother hand, in most cases of applications, we are not interested in a particular trajectory butmore in “typical” trajectories which should reflect the climate in a certain time interval [t1, t2]in some sense: If the time average of the state space variables is equal to the average of thesevariables at a certain time over an ensemble of trajectories starting in a ball around ϑ(t1) wewill say that the trajectory is representative for the climate state (hypothesis of ergodicity).For various applications there are balls around distinct initial conditions ϑ(t1) which give theappropriate mean values of the state variables for a certain climate. In general, it is not possibleto construct many files containing the right initial condition ϑ(t1) for every thinkable application.The idea is that the trajectories come themselves into a region of the state space S that containsthe most probable trajectories for a certain climate state if we impose the appropriate boundaryconditions. This behaviour is expected to be independent of the initial condition ϑ(t1) if we waitlong enough. Consequently, there is only one file containing initial conditions for each resolution,but it is required that the user performs a calculation of the trajectory for a sufficiently longtime until the trajectory comes into the region of the state space that represents best thetypical trajectories corresponding to the special boundary conditions. This may take years ofprecalculation of the trajectory before your actual simulation can start. The precalculationphase is also called the spin up phase. You can speed up the spin up phase by the use of arestart file that represents a state of the atmosphere that is as close as possible to the region ofthe state space S that you like to sample. We say that the model is in a steady state if the statevariables “fluctuate around” the mean state representing the climate. However, ECHAM6 canexhibit “multiple states” under certain circumstances. In that case, the average state dependson the initial conditions.

The following initial files provide a possible but not very probable state of the atmosphere forthe present day climate:

• atmospheric variables (T63/T63L47 jan spec.nc): approximate values for vorticity, diver-gence, temperature, and specific humidity for January.

• land surface variables(T63GR15 jan surf.nc): sea mask, glacier mask, lake mask, orogra-phy, orographic slope, snow depth, . . .

• hydrological discharge model (hdstart.nc): This is a resolution independent file. Never-theless, it is questionable whether the data are good at very high resolutions (T255).

The file name of the file containing the surface variables contains the acronym GR15. Thiscorresponds to a certain resolution of the ocean. The ocean resolution plays a certain role forthe land sea mask and has therefore to be known by the surface model.

Boundary condition files

There are various kinds of boundary condition files for ECHAM6. They may or may not de-pend on spatial resolution. An example of boundary conditions that do not depend on spatialresolution is the solar irradiance, on the other hand gridded ozone concentration files depend on

1Nevertheless, we know that the climate system shows chaotic behaviour. This means that the trajectoriespassing through neighbouring points S, S′ (“separated only by an ε > 0”) will be separated by any distance D > 0if we wait long enough.


the spatial resolution. Furthermore, the boundary conditions may be constant in time or reflectsome variability in time. E.g.: The solar irradiance could just be set to a time independentconstant or vary with the 11–year solar cycle. When boundary conditions have a seasonal cyclebut do not change with the year, they are called “climatological”. If the boundary conditionschange from year to year, they are called “transient”. Remark on aerosol optical proper-ties: There is a data set for the troposphere and in addition a second one for the stratosphere.The stratospheric aerosols are mainly of volcanic origin. Data describing the properties of theatmosphere are not boundary conditions but are described here for convenience.

Boundary conditions (climatological files). We give a list of the most important boundarycondition files in the T63–resolution here, a complete list can be found in the “user guide” [5].

T63 ozone CMIP5 1850-1860.nc mean seasonal cycle over the years 1850 to1860 of the ozone concentration for radia-tion calculation

T63TP04 piControl-MR sst 1850-2349.nc climatological sea surface temperature cal-culated from a coupled ocean–atmosphererun

T63TP04 piControl-MR sst 1850-2349.nc climatological sea ice.T63 aeropt kinne lw b16 coa.nc Aerosol optical properties of coarse mode

for thermal wavelengths.T63 aeropt kinne sw b14 coa.nc Aerosol optical properties of coarse mode

for solar wavelengths.T63 aeropt kinne sw b14 fin 1865.nc Aerosol optical properties of fine mode for

solar wavelenghts.

Remarks:

• Surface data like leaf aerea index, soil model data, vegetation ratio have to be provided toECHAM6, but they are discarded and interactively calculated by JSBACH.

• The concentrations of the other greenhouse gases like CH4, N2O, CFC’s, and also CO2 areassumed to be independent of the geographical location but may vary with altitude. Formethane and carbon dioxide, an interactive calculation using parameterized equations isalso possible.

• There is no “climatological” aerosol file for the aerosol optical properties of the fine modein the solar wavelength range although the pre–industrial aerosol concentration of 1865can be considered to be a kind of a climatology.

Boundary conditions (transient files).

T63/amip2/T63 amip2sst yyyy.nc Sea surface temperature of the year yyyybetween 1870 and 2008

T63/amip2/T63 amip2sic yyyy.nc Sea ice data.T63/ozone2/T63 ozone CMIP5 yyyy.nc Ozone concentration for the year yyyy be-

tween 1850 and 2008.T63/aero2/T63 aeropt kinne sw b14 fin yyyy.ncAerosol optical properties of the fine mode

for solar wavelengths. This data set un-derestimates the anthropogenic contribu-tion and should not be used in the future.


Parameter sets

In addition to the boundary condition files, some parameter sets are read by ECHAM6 fromfiles since they are too large to be included into the code directly. These parameter sets are notboundary conditions as they do not define the values of some variables calculated by ECHAM6on the boundary of the atmosphere. They rather determine the exact form of the equations thatare used by ECHAM6. These parameter files comprise absorption coefficients for the radiationcalculation, optical properties of clouds or properties of the land surface model.

It has to be underlined here that the optical properties of clouds are just basic optical propertiesof the droplets but since the clouds are interactively calculated and are therefore variable inspace and time, the optical properties of the clouds are not fixed boundary conditions. Incontrast to that, the aerosol optical properties are fixed from the outside. Nevertheless, from amathematical point of view, they do not represent boundary conditions like the solar irradiancethat is just the solar flux at the upper boundary of the atmosphere.

The run script echam6 T63L47.sh prepared for you already contains the links to all necessaryboundary conditions for the T63L47 resolution (see exercises).

1.3.3 Input namelists

In Fortran, you can provide the values of input variables in namelist groups (namelists), spec-ifying name and value of each variable. Several namelists are used to specify the ECHAM6input. There are two different files each containing several namelists: namelist.echam andnamelist.jsbach. The file namelist.echam contains all namelists that are relevant for the at-mosphere, whereas the file namelist.jsbach contains all namelists relevant for the land surfacemodel.

namelist.echam contains:

parctl: contains variables controlling the parallelizationrunctl: variables concerning the start, and the end of the program. This is the most importantnamelist for you.dynctl: variables related to the parameterization of the dynamics (diffusion e.g.)physctl: variables for switching on/off radiation, diffusion, convection, gravity wave drag, surfaceexchange, . . .radctl: variables controlling radiationsubmodelctl: variables controlling the call of submodels

There are further namelists for diagnostics and debugging.

namelist.jsbach contains:

jsbach ctl: general JSBACH switchesalbedo ctl: variables controlling the interactive surface albedo calculationcbalance ctl: variables related to the carbon cycledynveg ctl: variables concerning dynamic vegetation (change of vegetation types with time)

Syntax:


Listing 1.4: Namelist syntax

&<namelist name >

<varname >=<value >

/

We give an explanation of the most important namelist variables in the following.

PARCTL namelist

nproca=< n >: number of pieces into which the globe is split along latitudes.

nprocb=< n >: number of times the globe is split along longitudes. The resulting pieces fromthe cutting along longitudes and latitudes are then distributed upon different processors.

nprocio=< n >: number of i/o processors. nprocio= 1 is the default.

RUNCTL namelist

Variables related to the output files

out datapath=< dirname >: defines the directory to which the output files will be written; itis essential to put a slash at the end of the directory name.

example:

Listing 1.5: out datapath

out_datapath = ’/scratch/mpi/mh0010/m218036/

echam_course_2013/data/exp1/’

out expname=<′ name′ > : experiment name

out filetype=1[2] : determines the data format of meteorological output data (GRIB[netcdf]format, respectively); note that netcdf files are roughly twice as large as GRIB files

trac filetype=1[2] : GRIB[netcdf] format for tracer output files

ldebugio=T[F] : switches on[off] the debugging of input and output.

ldebugev=T[F] : switches on[off] the debugging of events. An event is an action to be done byECHAM6.

Variables related to the start mode of the model

lresume=F[T]

F initial run, starting from initial filesT Start a rerun, this means: continue some experiment using restart files

so as if you did the experiment without interruption (see later)


Variables related to time. For computer models, time is not a continous time axis but adiscretized sequence of times. At each of these times a certain action may be performed by themodel. We call an event any action that has to be performed by ECHAM6 at certain times.

The dates on the time axis are separated by delta time seconds. In the following table, themaximum possible time step for selected resolutions are listed. For longer time steps, themodel becomes instable in the respective resolution. These default values are set in the modulemo time control.f90.

resolution T63L47 T127L95

default delta time/s 600 240

default delta time/min 10 4

The time step can be set in the RUNCTL namelist by delta time=< dt > where < dt > is aninteger number giving the time step in seconds. Nevertheless, the standard time step should bechanged in special cases only.

Before we can discuss time variables further, we have to know that ECHAM6 distiguishes be-tween a “computer experiment” or “simulation” and a “run”.

Experiment (simulation): Time integration of atmosphere dynamics with ECHAM6 the du-ration of which is determined by scientific reasoning. Such an experiment may require theintegration over a long time period so that this integration has to be split up into severalchunks in order to fit into the queue of your computer.

Run: Any (sub)period of a whole experiment that is short enough to be integrated by one callto ECHAM6 in one job on your computer.

dt start = dt start (six comma separated integer numbers)

defines the start date of an experiment (not of the actual piece of an experimentthat you run in one job). The format is year, month, day, hour, minute,

second

This is the start date that characterizes your experiment and must not bechanged under any circumstances when you perform a rerun starting at a laterrestart date.

dt stop = dt stop (six comma separated integer numbers)

defines the stop date of an experiment; should be the final date of your exper-iment. There are possibilities to stop your actual run before dt stop is reachedin order to have shorter jobs (runs) on a computer. It has the same format asdt start. It can be changed safely when you are doing a rerun.

You can stop your program before dt stop is reached by setting one of the following vari-ables:

no days = no days (integer number)

number of days of an experiment counted from dt start on (and not from aneventual restart date on)


no steps = no steps (integer number)

as no days, but counting integration time steps

no cycles = no cycles (integer number)

stop after the no cycles’th times writing the restart files. There is a possibilityto force the model to go through the complete initialization of the model againby setting lforcererun=.true.. This is the default. If you do not like togo through the initialization again (what would be the standard), you have toactively set lforcererun=.false.. Works together with put rerun. The defaultvalue is no cycles = 1, meaning that the program stops after the first time itwrote restart files.

Variables related to repeated events

There are events like writing the output, writing restart files or calculating the radiation whichhave to be done in regular time intervals. The description of the times at which such repeatedevents have to be performed is not trivial and often only testing different settings leads to yourdesired result.

The following format is used for variables describing regular events (the types of the variablesare given in parenthesis):

interval(integer), unit(string), adjustment(string), offset(integer)

• interval:

integer number describing the time interval between two events in the unit ofunit.

• unit:

one of ’steps’, ’seconds’, ’minutes’, ’hours’, ’days’, ’months’, ’years’.Note that you have to write 1,’months’, ... !

• adjustment:

one of ’first’, ’last’ or ’exact’ (do not use exact!) The adjustment decideswhether the event is triggered at the beginning or end of the interval spaced byinterval unit:

-

*

time0h

day1 day2

1, ’days’,’last’1, ’days’, ’first’

last time step of day

0h

model time steps

• offset:


offset shifts by offset seconds the dates of the events relative to the start datebefore any adjustment by the adjustment variable is done. Difficult to use, tryto do your experiments with

Listing 1.6: offset

offset = 0

Special timing of events

ECHAM6 knows so-called “present-date events” and “next date events”. A present date eventwill be executed as explained so far. Next date events will be executed at the time step after acorresponding present date event.

Example: Writing restart files is a “present date” event. Consequently:

Listing 1.7: putrerun

putrerun = 1, ’days’, last , 0

means that the restart file is written at the last date of a day.

Unfortunately, there is no list of “present or next date events”.

Important event variables

• putrerun:

Write restart files. If lforcererun=.true., the model goes through the wholeinitialization phase again if it continues the run.

• putcheckpoint:

Write restart files at this time interval. This variable does not stop the modeltogether with no cycles.

• putdata:

Writing frequency of data to the output files.

Starting a rerun. The restart files are written to files with namesrestart out expname < yyyymmdd >< hhmmss > < fname >.nc. Here, < yyyymmdd >is the year, month, and day for which the restart file is written. Similarly, < hhmmss > isthe hour, minute, and second of the restart date. Actually, if y is the year, m the monthand d the day in the month, < yyyymmdd >:= y × 10000 + m × 100 + d. In a similar way,< hhmmss > is defined. That means, that the 1st of January 200, 00:00:00h a.m. wouldbe 20001010. ECHAM6 expects restart files to be read under the following standard name:restart < out expname > < fname >.nc.

When you like to continue an experiment at a certain date for which rerun data are available,you can do it in the following way:


• Copy the restart files into your run directory or establish a corresponding link. The namingconvention of restart files is the following: restart $out expname< fname >.nc. Here,< fname > is usually echam, jsbach, surf, veg, co2.

• Set lresume = .true. in the RUNCTL namelist.

• Make sure that dt start is either not present in the RUNCTL namelist or has still the (old)start date of the beginning of the experiment.

• Run the ECHAM6 model.

Chapter 2

Looking at your results

Visualization of your results is important when the simulations are done. There are varioustools available to display viewgraphs of your data, depending on the output format which canbe either GRIB (gridded binary) or netcdf format. In general, you would like to have a verysimple and fast method to see whether your run produces reasonable results. Later, you will dosome more sophisticated analysis of the data and you probably will use different methods of theones presented here.

2.1 Basic visualization with ncview

ncview is a tool with graphical user interface for having a first look at your data. Call it byncview and the file name of the netcdf–file you like to visulize.

2.2 Standard plots

Standard postprocessing provides a first analysis of your runs. These scripts do not work withthe original output but with some standard “postprocessed” files ATM,BOT out expname <yyyy >. Most of the example run scripts include the automatic generation of these files, butyou can generate them manually with a script that is called after.sh and will be providedwith the exercises. The viewgraphs are generated by the script POSTJOB (also provided with theexercises). In this script, you have to modify the experiment name, the first and last year ofyour experiment and the path to the data. In that case, standard plots are created that includea comparison with era40 analysis data. It is also possible to compare two different experiments.

2.3 Basics about the netcdf data format

The netcdf data format is a machine independent binary format. No compression of data isperformed. It is designed for efficient data handling (fast retrieval of small amounts of datafrom large files). The data can be stored in a “self explanatory” manner, meaning that you canexplain your data in the netcdf file itself (units, definition of the physical quantity etc.).

19

20 CHAPTER 2. LOOKING AT YOUR RESULTS

The programs ncdump and ncgen are available to translate data from binary format to humanreadable cdl format or even to fortran or C code which will create the respective netcdf files.

ZZZ

ZZZ

ZZZ

ZZZ

ZZZ

ZZ

-

=

human readable

format (cdl)binary format

netcdfncdump

ncgen

fortran/C

interfacencgen

fortran 90/77

or C

programm

We will discuss the cdl–format and the ncdump and ncgen programs only.

2.3.1 cdl format

The cdl format is the basis for the creation of a netcdf file. It is also the only format in which youcan read the content of a file in a text editor. In particular, if you like to see the explanation ofa certain variable in the netcdf file, you will read it generally in cdl format. We therefore discussthis format in more detail here. The purpose of netcdf–files (and cdl–files) is the representationof tabulated values of functions on a multidimensional space, i.e.

f :

Ω ⊂ Rd → Rx 7→ f(x)

, d = 0, 1, 2, 3, 4, . . .

In general, we wish to represent functions with a maximum d = 4, the components of x ∈ Rd

representing longitudes, latitudes, levels, and time. The components of the d-dimensional vectorx are called dimensions in the cdl format. The function f is called a variable.

As an example we consider the surface pressure ps:

ps :

Λ× Φ× R+ → R+

(λ, ϕ, t) 7→ ps(λ, ϕ, t)

In this case, the longitudinal and latitudinal angles λ, ϕ and time t are called “dimensions” ofthe variable ps.

In the declaration part at the beginning of a cdl file, the dimensions and variables are definedand in a second part, the values of all dimensions and variables are listed. More precisely, acdl file has the following general structure (You may fill in tabs, blanks, and newlines for betterreadability.):

Listing 2.1: netcdf syntax

netcdf <name >

[dimensions:

<dims >]

[variables:

2.3. BASICS ABOUT THE NETCDF DATA FORMAT 21

<vars >]

[data:

<dat >]

• < name >:

The name of your data set. This name with the additional extension .nc willbecome the name of the binary netcdf file when you convert it from cdl formatto netcdf format. The name can consist of any ASCII characters, but no leading’’ is allowed.

• < dims >:

This is a comprehensive list of the dimensions of all the variables in the data setand the number of values they will take.

Listing 2.2: dimensions

<dim 1> = <n1 >, <dim 1>= <n2 >, ..., <dim m> = <nm >;

at most one of < n1 >, ..., < nm > can have the value ’unlimited’, meaningthat the number of values it is allowed to take is arbitrary. Later on, you mayconcatenate files only with respect to this so–called record dimension with specialprograms.

e.g.:

Listing 2.3: example for dimension listing

dimensions:

lat= 10, lon= 10, time=unlimited;

• < vars >:

The definition of variables is done by their data type, their name and a dimensionspecification:

Listing 2.4: variable definition

<type > <varname1 >[(<dim1 >[,<dim2 >, ...])],

[<type ><varname2 >(...) ];

The most important types are:

– < type >:

char characterint 32-bit signed integerdouble IEEE double precision floating point (64bit)real IEEE single precision floating point (32bit)

– < varnamei >:

The name of a variable is composed of any ASCII characters, butmust begin with a letter and must not contain ’;’, ’:’, ’’ or otherelements of the cdl language; there seems to be no restriction con-cerning the length of the variable names.


– < dimi >:

The dimensions of the variable. These are one of the dimensionsdefined before.

Listing 2.5: Example: definition of a variable

double t(time , lon , lat)

There is the possibility to add explanations or definitions of the variables or the dataset inso–called attributes. These attributes can be used for making the netcdf file self–explanatory.You should extensively use these attributes. There is even a standard (the COARDS standard)that determines a good practice of which attributes should be present in every netcdf file. Theapplication of this standard makes it easier for someone else to read and understand your data.

The syntax for the definition of attributes is the following:

Listing 2.6: attributes to variables

[<varname >]:< attributename >= <attribute >

The choice of the attribute name is up to the user but you should try to follow the COARDSstandard. For the description of the whole data set, global attributes are used. In this case< varname > is missing, but not the colon in front of the attribute name!

In the COARDS standard, the following conventions are applied:

< attributename > attributetype meaning

Frequent attributes of variables

units character string units of data < varname >long name -”- description of < varname >comment -”- commentvalid (min|max) float or integer allowed min/max values

Frequent global attributes

title character/string title of dataauthor -”- author name of data coll.institution -”- address of authorhistory -”- history of filesource -”- source of data

Attributes which are special to restart files (not in COARDS)

fdate integer yyyymmdd; start date of experiment/dt start

ftime -”- hhmmss; start time of experiment/dt start

vdate -”- yyyymmdd; rerun datevtime -”- hhmmss; rerun timenstep -”- number of time steps between

(fdate, ftime) and (vdate, vtime).A restart of ECHAM6 will crash ifthere is any inconsistency!

timestep real time step in seconds

Leading zeros are omitted in the variables fdate, ftime, vdate and vtime!


The data itself are written into the cdl file using the following syntax:

Listing 2.7: data

<varname >= val1 , val2 , ..., valn;

val1, val2, valn are the values of < varname > and have the type of variable < varname >.

The order of the dimension in cdl is like in the C programming language:

• The last index varies fastest.

• The unlimited dimension has to vary slowest and consequently has to be the first index.

In the Fortran programming language, it is the inverse.

There is also the possibility to write comments into the cdl files which will not appear in thenetcdf file. Comments have to be included after

Listing 2.8: comment

//

Here is a full example of a cdl file example.cdl:

netcdf example

dimensions:

time = unlimited ,

lon= 8 ,

lat = 5;

variables:

double lat(lat);

lat:long_name = "latitude" ;

lat:units = "degrees_north" ;

double lon(lon);

lon:long_name = "longitude" ;

lon:units = "degrees_east" ;

double time(time);

time:long_name = "time" ;

time:units = "day since 1978 -01 -01 12:00:00" ;

time:calendar = "gregorian" ;

double st(time ,lat ,lon);

st:long_name = "surface temperature" ;

st:units = "degrees Celsius" ;

// global attributes

:author="Sebastian Rast ([email protected])" ;

:institution="MPI fuer Meteorologie Hamburg" ;

:created="20061006" ;

:history="example file for echam course 2006";

data:

lon = -180., -135., -90., -45., 0., 45., 90., 135;

//0, 45., 90., 135., 180., 225., 270., 315. ;

lat = 85., 45., 0., -45., -85.;


time = 0, 0.5;

st =

//st(1,1,1-8)

-15., -20., -18., -19., -10., -8., -20., -18.,

//st(1,2,1-8)

10., 12., 8., 11., 8., 3., 15., 12.,

//st(1,3,1-8)

25., 30., 28., 32., 27., 25., 23., 24.,

//st(1,4,1-8)

35., 32.1, 38., 31., 29., 25., 18., 21.,

//st(1,5,1-8)

-25., -30., -28., -29., -30., -38., -40., -28.,

//st(2,1,1-8)

-12., -10., -15., -12., -9., -10., -15., -19.,

//st(2,2,1-8)

10., 12., 8., 11., 8., 3., 15., 12.,

//st(2,3,1-8)

25., 30., 32., 31., 21., 23., 25., 28.,

//st(2,4,1-8)

45., 42.1, 38., 31., 34., 35., 28., 31.,

//st(2,5,1-8)

-21., -25., -28., -39., -30., -38., -40., -58.

;

2.3.2 ncdump

The program ncdump provides ASCII text representation (of parts) of a netcdf file in the cdlformat on the standard output device (screen). The syntax of the call of this program on thecommand line including the most important options is:

Listing 2.9: ncdump

ncdump [-c|-h] [-v <var >,<var1 >, ...] [-b language] [-n name]

<inputfile >

-c:

Gives header information and values of variables which are also dimensions only.

-h:

Gives header information only; attention: -c and -h are mutually exclusive.

-v < var1 >,< var2 >,. . . :

Gives header information and values of < var1 >, < var2 >, . . . only. The variablelist must not contain any space (blank); this option does not work with either -c or-h.


-b < language >:

< language > = f or F for Fortran = c or C for C. Index information is added forFortran (indices begin with 1) or c (indices begin with 0) but the order of data onthe screen is the same for Fortran and C.

-n < name >:

Set < name > as the name of the cdl-dataset in the ASCII output.

2.3.3 ncgen

This program generates either binary netcdf files from cdl format files or it generates Fortran/Ccode from cdl files which can produce those netcdf-files.

Syntax of usage:

Listing 2.10: ncgen

ncgen [-b] [-o] <netcdf -fname > [-c] [-f] <inputfile >

no option:

Performs a syntax check of the cdl format file only. If errors are detected, errormessages are sent to the standard output device (screen).

-b:

Creates a binary netcdf file that will be named<name>.nc according to<name>.ncin the cdl-file. Warning: Any existing netcdf file with the same name will beoverwritten without any message.

-o

<netcdf-fname> is used instead of <name>. The -b option can be omitted

-c:

Generates C–code that can create the netcdf file corresponding to the cdl file.C-code is written to the standard output device.

-f:

Generates Fortran-code that can create the netcdf file corresponding to the cdlfile. Fortran code is written to the standard output device.

Note that the creation of C or Fortran code with the -c and -f options is interesting for smallfiles only because all data is included into the program code.


2.4 Climate data operators (cdo’s)

The fastest way to work with your output data in various formats and to perform variousanalysis calculations on your data is to use the climate data operators (cdo’s). There is athorough documentation on http://code.zmaw.de/projects/cdo by the author of the climatedata operators (U. Schulzweida).

2.4.1 File formats

The cdo’s can handle various file formats like grib, netcdf, netcdf2, service, extra. You canconvert files from one file format to another by the command

Listing 2.11: Change of file format

cdo -f<fmt > copy <ifile > <ofile >

<fmt> is the format of the output file <ofile> and is one of grb, nc, nc2, srv, ext for theabove formats, respectively.

2.4.2 General structure of cdo commands

Use of a single operator

The syntax of the usage of one single operator is

Listing 2.12: Call of single cdo operator

cdo <options > <operator > <ifile[s]> [<ofile >]

In the example above, copy is the operator and -f <fmt> is an option to the copy operator.

2.4.3 Use of several operators in one command

The result of the application of a certain operator to a file can be considered as a new input filefor another operator. E.g.:

Listing 2.13: Pipeline cdo operators

cdo <op3 > -<op2 > <ifile2 > -<op1 > <ifile1 > <ofile >

In the above command, -<op1> <ifile1> and -<op2> <ifile2> act as input files for theoperator <op3> that is an operator needing two input files and writing the results to <ofile>.Note the “-” sign in front of the operators <op1> and <op2> that are preparing the input for<op3>. The advantage of such a “pipelining” mechanism is that the results of <op1> and <op2>

are not written to the disk and therefore the i/o is reduced compared to a formulation in threeseparated cdo calls. It is possible to use up to 20 operators in such a pipelining mechanism. Thenumber of outputfiles of an operator with preceeding “-” sign has to be clear before execution(this is not the case for all file splitting commands for example).

2.4. CLIMATE DATA OPERATORS (CDO’S) 27

2.4.4 Operators for getting information about files

cdo infov <ifile>: Provides information about the content of <ifile>. Warning: The re-ported mean values of fields are arithmetic mean values over all latitude/longitude gridboxes only, not an area weighted mean.

cdo pardes <ifile>: Prints a pretty list of code numbers and variable names of the variablesin file <ifile>.

cdo griddes <ifile>: Prints a description of the horizontal grid.

cdo vct <ifile>: gives a list of the (Ai, Bi)i=1,L+1 coefficients giving the index in the formi′ := i− 1.

2.4.5 Selection and renaming of variables

cdo selname,<var1>,<var2>,<var3>,...,<varn> <ifile> <ofile>: writes the variables<var1>,...,<varn> into file <ofile>. Mind the comma between the operator selname

and the first variable <var1>. For selecting codes, there is an operator selcode.

cdo chname,<var1>,<var2> <ifile> <ofile>: <var1> will be renamed by <var2> in the out-put file <ofile>.

2.4.6 Interpolation

There are three different types of interpolation: (i) with respect to the horizontal grid (latitudesand longitudes), (ii) with respect to the vertical coordinate, and (iii) with respect to the timeaxis. We only discuss the interpolation with respect to the horizontal grid and the verticalcoordinate.

Horizontal interpolation (on model levels)

cdo remapbil,remapbic,<grid> <ifile> <ofile>: The operator remapbil performs a bi-linear interpolation, the operator remapbic performs a bicubic interpolation. The operatoris accompanied by a grid descriptor <grid> of the new horizontal grid. For the Gaussiangrid used by ECHAM6, use t31grid, t63grid, t127grid, t255grid where the num-ber means the spectral resolution. For regular grids, it is sufficient to give the number oflongitudes and latitudes in the form r<lon>x<lat>, e.g. cdo remapbil,r200x100.

Remarks: The interpolation is circular meaning that there are no gaps at 0E longitude.There is no extrapolation or circular interpolation towards the poles. Instead, missing values(−9 × 1033) are attributed to those grid points. The horizontal grid interpolation operatorremapcon conserves the integral over the sphere.


Vertical interpolation

Interpolation to pressure levels. The ml2pl operator performs an interpolation of variablesgiven on σ–hybrid coordinates (model levels) to given pressure levels.

Listing 2.14: Interpolation to pressure levels

cdo ml2pl ,<plevels > <ifile > <ofile >

<plevels>: comma separated list of pressure levels in Pa (not hPa!).

<ifile>: Must contain either the variable aps (code 134) or lsp (code 152) for the surfacepressure; the file must not contain other data than grid point variables.

<ofile>: The outputfile contains the values of all grid point variables depending on thehorizontal coordinate at the given pressure levels <plevels>. If extrapolation wouldbe required at some points, a missing value (−9× 1033) will be written to the outputfile <ofile>.

Listing 2.15: Interpolation to model levels

cdo remapeta ,<vct >[,<oro >] <ifile > <ofile >]

Interpolation to a different system of σ–hybrid coordinates. <vct>: ASCII file contain-ing the definition of the (Ai, Bi)i=1,L+1 coefficients of the new σ–hybrid coordinates.This file can be generated by the command: cdo vct <gridfile> > <vct> where<gridfile> is a file containing a sample of the new grid. The format is exactly asthe output of the vct operator.

<oro>: name of a file containing the orography of the new system of σ–hybrid coordinatesas represented by the surface geopotential geosp. This file is optional. If it is notpresent the orography described by geosp of <ifile> is used.

<ifile>: The input file must contain asp or lsp, the 3–dimensional temperature in avariable called t, the orography in geosp and the <ifile> must only contain gridpoint variables. Warning: The 3–d temperature is often stored in a variable calledst. You have to rename it before you can apply the remapeta operator by thecommand cdo chname,st,t <ifile> <ofile>.

2.4.7 Conversion from spectral space to grid point space

Grid point space — Spectral space

The general circulation program ECHAM6 calculates the time evolution of functions ft, t ≥ 0that “live” on a grid of points in S2 × R+, where

S2 represents the spherical surface of the Earth and

R+ is the vertical co-ordinate. The functions ft normally have values in the real numbers asthey are representing some physical quantities. In mathematical notation, these functions arethen represented by:

2.4. CLIMATE DATA OPERATORS (CDO’S) 29

ft :

S2 × R+ → R(λ, ϕ, p) 7→ ft(λ, ϕ, p)

for times t > 0

The basic equations of atmosphere dynamics lead to partial differential equations which involvenot only the time derivative of such functions but also derivatives with respect to location.For a solution of these partial differential equations it is very helpful to develop ft in sphericalharmonics:

ft(λ, ϕ, p) =∞∑l=1

l∑m=−l

Y ml (ϕ, λ)(Ft)

ml (p) (2.1)

This leads to algebraic equations or differential equations for the coefficients (Ft)ml that are more

easily tractable.

Nevertheless, the cloud physics, chemistry, aerosol physics, soil processes, and the transport oftracers is better described in terms of functions defined on S2 × R+ (the so–called grid pointspace) than in terms of a series of spherical harmonics (so–called spectral space). This leads totwo parts in ECHAM6: A first part in which atmospheric dynamics is calculated in spectral spaceand a second part in which all other calculations are done in grid point space. Consequently,some of the functions have to be transformed in each time step from the grid point to the spectralspace and vice versa. Such transformations are computationally slow because they involve thecalculation of many integrals. The choice of a certain grid (Gaussian grid) in S2×R+ and specialalgorithms allow to do these transformations faster. The Gaussian grid consists of equally spacedlongitude points λi, i = 1, nlon and latitude points ϕi, i = 1, nlat which are spaced according tothe zeros of some Legendre polynomials. These zeros are not equidistant.

Some of the output variables (3d–temperature, divergence and vorticity of the wind field, loga-rithm of the surface pressure) are in spectral space and have to be transformed into grid pointspace by some postprocessing procedure.

Transformations from spectral to grid point space with cdo’s

Transformation: Simple transformation from spectral space to grid point space.

Listing 2.16: Spectral to grid point space

cdo sp2gp <ifile > <ofile >

transforms all spectral variables of <ifile> into grid point space.

Wind calculation: In the ECHAM6 output, only the divergence and the vorticity of the windfield are present but not the u and v components of the wind. These can be calculatedfrom the divergence sd (code 155) and vorticity svo (code 138) by the command

cdo dv2uv <ifile > <ofile >

Chapter 3

Programming in ECHAM6

3.1 Simplified flow chart of ECHAM6

We present a simplified flow chart of ECHAM6 in Figure 3.1. All ∗ subm∗ routines are frommodule mo submodel interface.

3.2 Submodel interface in ECHAM6

The submodel interface of ECHAM6 consists of 15 interface routines collected in mo submodel interface

(or mo submodel) which may be used for the coupling of submodels to ECHAM6 (see also theflow chart, Figure 3.1).

Listing 3.1: submodel interface

setsubmodel (mo_submodel)

init_subm

init_subm_memory

stepon_subm

physc_subm_ 1,2,3,4

radiation_subm_ 1,2

radheat_subm

vdiff_subm

cuflx_subm

cloud_subm

free_subm_memory

These interface routines are adequate for submodels which need to introduce additional tracers.For such tracers, a transport mechanism is built in that works like for water vapour. Additionalprocesses between these tracers can be easily introduced as far as these processes can be describedinvolving one grid box only without the influence of the processes in the neighbouring gridboxes. A typical example is the introduction of chemical processes into the atmosphere forwhich the chemical species are transported tracers which undergo certain chemical reactions.In a particular simulation box, the chemical reactions depend on the boxes above only whichmay influence the radiation and therefore the photolysis in the particular box. The reactionsthemselves involve only the species in the particular simulation box. Another typical example

31

32 CHAPTER 3. PROGRAMMING IN ECHAM6

Figure 3.1: Flow chart of ECHAM6

master

control

initialize

setsubmodel

init subm

finish tracer definition

init memory

xt burden init mem

init subm memory

iorestart or ioinitial

xt initialize

stepon

bc list read

stepon subm

scan1

gpc

physc

physc subm 1

radiation

psrad interface

radiation subm 1

radiation subm 2

vdiff

vdiff subm

radheat

radheat subm

physc subm 2

cucall — cumastrh,t — cuflx

cuflx subm

cloud

cloud subm

physc subm 3

xt burden

ocean coupling, prepare hydrological discharge

physc subm 4

scan2

free memory

free subm memory

end

Start of model and parallelization

All initializations including time manager (events)

(mo submodels), register submodels

initialize submodels here

(mo tracer), sets tracer properties

(mo memory streams)

(mo tracer processes), burden diagnostics

allocate submodel memory here

(mo tracer), initialization of tracers

loop over time steps, read boundary conditions (sst, sic)

(mo boundary condition), read boundary conditions for submodels

good for reading of files at regular time intervals

spectral calculations, advection, loop over grid point blocks

grid point calculations

“physcis” in grid boxes or columns

submodel calculations before radiation

(mo radiation)

(mo psrad interface)

optical properties of aerosols e.g.

diagnostics

vertical diffusion

surface emissions, deposition

heating of aerosols

physics of submodels

aerosol cloud interaction

burden diagnostics

submodel diagnostics

(mo memory streams)

deallocate submodel memory, reset all default values

3.3. INTRODUCTION OF A SUBMODEL 33

is the introduction of aerosols into ECHAM6. Nevertheless, there are applications to whichthe submodel interface is not adapted. In particular, it is difficult to introduce tracers thatare interacting with the land surface model or an ocean model that is coupled to ECHAM6.Even though, we will mainly discuss the submodel interface but also the most important datastructures for grid point fields, some aspects of the “time manager” for the programming ofevents, and some general programming techniques in ECHAM6. We illustrate the programmingtechniques by a basic example of the introduction of one tracer that will be emitted in a certaingrid box and transported in the atmosphere. The corresponding submodel consists of only onemodule which is called mo transport.

As a general programming guide line, one should formulate submodels in one (or few) modulesthe subroutines of which are called by the subroutines collected in mo submodel interface.You should follow the programming style of the “icon programming guide” [1].

Each submodel must have a switch to switch it off completely. This has to be the default.

3.3 Introduction of a submodel

Your new submodel has to be registered by ECHAM6. There are two variables associated witheach submodel: (i) a switch <lyrsub> and (ii) an identification number <id yrsub> that is aninteger number identifying the submodel.

(i) A submodel switch has to be a member of the namelist SUBMODELCTL that is defined insubmodelctl.inc. Furthermore, it must have the default value .FALSE. so that the sub-model is switched off by default. The submodel switch must be available on all processorsand for all modules or subroutines of the model. Proceed in four steps:

– Declare the switch and assign the default value in mo submodel:logical:: <lyrsub>=.false.

– Make it public in mo submodel so that it can be used by other subprograms:public:: <lyrsub>

– Add the switch to the list of namelist entries inecham-6.2.01/include/submodelctl.inc

– Send it to all processors (in subroutine setsubmodel of mo submodel):CALL p bcast(<lyrsub>,p io)

Remark: The subprogram p bcast is defined in mo mpi. In that latter module, you canfind subroutines which make the use of MPI easier.

The subprogram p bcast is an overloaded subprogram that can send real 0,1,2,3,4-d fields,integer 0,1,2,3,4-d fields, or boolean 0,1,2,3,4-d fields and characters.

The following syntax applies:

Listing 3.2: p bcast

p_bcast (var , p_source[,comm])

– var: Variable of one of the types mentioned above


– p source: Number of the source processor

example: p io (defined in mo mpi) is the number of the i/o–processor

– comm: number of processor receiving the data

default: all processors

(ii) Declare an identification index variable <id yrsub> in mo submodel:public:: <id yrsub>integer:: <id yrsub>

(iii) Register your submodel in setsubmodel of mo submodel. The registration has to be doneif and only if the submodel is active, i.e. if <lyrsub>=.true.:IF (<lyrsub>) CALL new submodel(’yrsubname’,<id yrsub>)The name of the submodel ’yrsubname’ can be chosen freely by yourself.

(iv) Print the activity status of your newly introduced submodel in setsubmodel of mo submodel.The status should be printed whether the submodel is active or not.CALL print status (’<submodel message>’,<lyrsub>)Here, the <submodel massage> can be the submodel name or any string describing yoursubmodel further.

3.4 Introduction of your own namelist

The data which are needed in your submodel are either read from files (in this case you shouldprefer netcdf format as for all input fields of ECHAM6) or from a namelist group (namelist).Namelists are good for reading small amount of data. You can introduce your own namelist bythe following steps.

(i) Create first a module mo <yrsub> that will contain the global variables and subprogramsof your submodel. Similar to the submodel switch <lyrsub> in mo submodel, declare thevariables of your namelist in mo <yrsub> and assign default values.

(ii) Your namelist variables that you have declared in mo <yrsub> must now become mem-bers of a namelist group. For this purpose, create a new “include file” with file name<yrnamelist>ctl.inc (in directory echam-6.2.01/include). In this file, set up yournamelist by

Listing 3.3: yrnamelistctl

NAMELIST /<yrnamelist >ctl/ comma separated list of variables

For an example, see the runctl namelist.

(iii) Now, we are ready to code the reading of the namelist. In general, reading variableshas two steps: The reading on the i/o–processor and the distribution of the data to allprocessors. In order to read your namelist, create a subroutine init <yrsub> in yoursubmodel module mo <yrsub>. This actual submodel initialization routine has to becalled in the dummy subroutine init subm of the submodel interface routines collectedin mo submodel interface. As you can see from the simplified flow chart of ECHAM6(Figure 3.1), init subm will be called in the subroutine initialize before ECHAM6 enters

3.4. INTRODUCTION OF YOUR OWN NAMELIST 35

the loop over the time steps or any spectral calculation. This means, that it is only calledat the beginning of any model run (or if lforcererun=.true. at the time when the restartfiles are written). The initialization of your model has to be done if and only if your modelis switched on. Therefore, your call in init subm has to be:

IF (lyrsubmodel) CALL init_ <yrsub >

Do not forget that both, your switch <lyrsub> and your subroutine init <yrsub> must beknown to init subm. This means that you have to introduce a respective “use”–statementinto init subm:

USE mo_submodel , ONLY: <lyrsub

USE mo_ <yrsub >, ONLY: init_ <yrsub >

The “ONLY” attribute tells init subm that it has to use the definitions of <lyrsub> andinit <yrsub> from mo submodel and mo <yrsub> only. This adds some security to thecode: If not all public variables of mo <yrsub> are known to init subm they can notaccidentally interfere with definitions of other submodels that are also used in init subm.Futhermore, the code is easier to read: It is clear which definitions of any module in aUSE–statement are known to the actual subprogram.ECHAM6 provides a number of subroutines and variables which make it easier to reada namelist and distribute the values of the variables to any processor after reading. Inparticular, you will need the following variables, functions, and subroutines:

– from module mo mpi:

∗ p io: integer variable giving the number of the processor that is doing the I/O.

∗ p parallel io: logical that is .true. only on processor p io, .false. otherwise.With this logical, you can always detect whether you are on the i/o–processor ornot.

∗ p parallel: logical that is .true. on all processors if a simulation in parallel modeon more than one processor is performed.

∗ p bcast: A subroutine that is easy to use for sending variables to various proces-sors. We already discussed it in the previous paragraph.

– from module mo namelist:

∗ open nml: function that opens the namelist file. Result: logical file number.

∗ position nml: This function searches your namelist group in the file namelist.echamand positions the cursor at the beginning of the particular namelist. The pro-gram is then ready to read the variables of this namelist group. Result: descriptorvariable about state of the search of the namelist group.

– constants from modules mo namelist:

∗ positioned (=0) if namelist was found and cursor could be positioned at thebeginning of the namelist.

∗ missing (=1) namelist is missing

∗ length error (=2) length of namelist name is longer than in the definition ofyrnamelistctl.inc

∗ read error (=3) general read error

– from module mo exception:

∗ message: write a message to standard output

∗ finish: write a message to error output and exit ECHAM6


(iv) Syntax of subroutine and function calls

– p bcast see the listing 3.3

Listing 3.4: function open nml

open_nml (’fname ’)

opens the file fname. The result of this function is the logical file number as an integernumber.

Listing 3.5: function position nml

position_nml (’yrnamelistctl ’, inml [,status = ierr])

Searches for namelist yrnamelistctl in the file with logical file number inml and po-sitions the cursor. The variable status is optional and contains an error code. Atexit of the function postition nml, the error code has one of the following values:positioned, missing, length error, read error. It is good practice to ask for thestatus of this error code and to write a message to the outputfile or to abort theprogram giving an appropriate message.

Listing 3.6: subroutine message

message (’subroutine_name ’, ’message ’)

subroutine name is the name of the subroutine which sends message message.subroutine name and message are of type string.

Listing 3.7: subroutine finish

finish (’subroutine_name ’, ’message ’)

subroutine name is the name of the subroutine in which ’finish’ was encountered.Before finishing the ECHAM6 program, the message message is written onto thestandard error output device.subroutine name and message are of type string.

For example, in the case of the transport module, we will have:

Listing 3.8: init transport

MODULE mo_transport

! ...

! ... Here you have to declare your variables of the namelist

! ... Set also default values

! ...

SUBROUTINE init_transport

! ...

INCLUDE ’transportctl.inc’

IF (p_parallel_io) THEN

3.5. READ IN FIELDS FROM NETCDF FILES 37

inml=open_nml(’namelist.echam’) ! open namelist file under

file number inml

nnml=position_nml (’transportctl ’, inml , status=ierr)

SELECT CASE (ierr)

CASE (positioned)

read (nnml , transportctl)

CASE (missing)

CALL finish (’transport ’, ’namelist transportctl ’//

’is missing in namelist.echam’)

! ...

END SELECT

ENDIF

CALL p_bcast (var1 , p_io) ! send variables which are

!... ! needed on each processor

CALL p_bcast (varn , p_io) ! to all processors

!...

END SUBROUTINE init_transport

END MODULE mo_transport

3.5 Read in fields from netcdf files

In many cases it is necessary to read in gridded data from files (netcdf files) e.g. for emissionfluxes or special surface properties for dust emissions. There are two possibilities to read griddeddata into ECHAM6:

• There is a so called “external field processor”. This field processor allows to read variousgridded variables (1,2,3–dimensional) once or in regular time intervals. The respectivesubprograms are collected in the module mo external field processor.

• In the module mo read netcdf77 you find a collection of subroutines that can read (hy-perslabs) of 0,1,2,3–d fields. The order of the dimensions of a field in the netcdf–file and inECHAM6 can be different. Automatic reordering of the dimensions is performed. Theseroutines are much more basic than those of the external field processor and should be usedby exception only.

example:

read var hs nf77 3d: reads 3d–hyperslab of a 4d–field from a netcdf file. The order ofdimensions in the file can be different from that of the field in ECHAM6.

3.6 Derived data types

In Fortran 90, it is possible to combine variables of different types under one name. So, you canaccess a collection of integer, character and real variables with one single name and pass themunder this name into subprograms like an array. Such a complex data structure has first to bedeclared in a type statement. Then, you can declare variables of this new data structure.

The syntax of the declaration of a new data type by a type statement is as follows:


Listing 3.9: type statement

type <typename >

declaration of var1 (e.g. integer :: iv1)

! ...

declaration of varn

end type [<typename >]

Then, you can declare variables of type < typename > by

Listing 3.10: type

type (typename) [,attributes] :: <varlist >

A variable var of < varlist > has then n “components”

Listing 3.11: components

var%var1

var%var2

! ...

var%varn

Each component can be handled separately or var can be treated as an integral entity by usingits name (e.g. for passing it into subprograms, in write–statements).

A good example are vectorfields which represent functions f : Rm → Rn, meaning that we willrepresent each component fi : Rm → R, i = 1, . . . , n by anm–dimensional field in FORTRAN 90.For the full description of f , we need a set of n such m–dimensional vectors which we can storein a data structure containing each component fi, i = 1, . . . , n as the components of the datastructure. The horizontal wind field is an example with m = 3 and n = 2 (the u and vcomponents depend both on the longitudes, latitudes, and levels).

Listing 3.12: vectorfield

type vectorfield

! nlon: number of longitudes

! nlev: levels

! nlat: latitudes

real , dimension (nlon , nlev , nlat) :: u, v

end type vectorfield

type (vectorfield) :: hwind

In this example, hwind%u, hwind%v contain the u and v components of the horizontal windfields.The components hwind%u, hwind%v are themselves three–dimensional arrays (vectors).

3.7 Tracers in ECHAM6

Tracers are substances that are transported in the atmosphere but they also interact in variousprocesses among each other. There are two different kinds of information needed:

3.7. TRACERS IN ECHAM6 39

• Transport properties: There are physico–chemical properties that influence the transportof species in the atmosphere. E.g. the sedimentation of aerosols depends on their densityand shape.

• Physico–chemical properties that are needed for the description of the interaction amongvarious tracers, e.g. the reactivity of chemical species with the surface influences the drydeposition velocity of chemical species.

The same physico–chemical properties may be shared by several tracers. E.g.: It is possibleto define several carbon monoxide tracers depending on the source region (so called “tagged”tracers). They all share the same molecular weight, solubility in water, and reactivity. In order toavoid any double specification of the physico–chemical properties, there is a data structure for thespecification of physico–chemical properties of species separated from the tracer data structure.For each tracer, a link to the data structure containing the physico–chemical properties can beestablished.

The definition of tracers has therefore two steps:

• add the physico–chemical properties to a data structure of “chemical species”.

• add your tracer to the tracer data structure including a link to the species data structure.

Remark: The data structure speclist of type t species is defined in module mo species. Thisdata structure is used for complex tracer submodels mainly and will not be discussed here. Insimple cases, the sole tracer data structure is sufficient.

The right place to define your tracers is the subroutine init <yrsub>.

3.7.1 Data types associated with tracers in ECHAM6

trlist of data type t trlist. The data type t trlist is declared in mo tracdef.f90. Thisdatatype contains information about all tracers and for individual tracers.

Information about all tracers:

Listing 3.13: trlist

type t_trlist

integer :: ntrac !actual number of tracers ,

!maximum jptrac =200

integer :: anywetdep ,anydrydep

!...

type(t_trinfo):: ti(jptrac)

end type t_trlist

trlist%ntrac is the number of currently defined tracers in ECHAM6. Note thatthere is a maximum number of jptrac= 200 tracers (jptrac is defined in mo tracdef).

The parameters trlist%anywetdep,anydrydep, ... describe whether there is atleast one tracer of all tracers that needs the calculation of wet deposition or dry de-position. If there is at least one tracer that undergoes wet or dry deposition,trlist%anywetdep,anydrydep> 0, otherwise trlist%anywetdep,anydrydep=0.


Remark: The general properties of the whole set of tracers are set by ECHAM6automatically. You can use this information but you should not change it! Theinformation about individual tracers is collected in trlist%ti(i), i = 1, ntrac. Eachindividual component trlist%ti(i) of the variable trlist is itself a variable of aderived data type. In that case, the data type is t trinfo. The components of thedata type t trinfo will be discussed below.

• t trinfo

Listing 3.14: t trinfo

type(t_trinfo)

!name of tracer

character (len= 24):: basename

!subname of tracer

character (len= 24):: subname

!concatenation of basename and subname (be aware

!of short -cuttings because of len= 24)

character (len= 24):: fullname

!other variables:

character (len= *) :: units !units of tracer concentration

! should be mass mixing ratio (kg/kg)

integer :: ntran !perform advective transport

! default: ntran=1 (do adv. transport)

integer :: nvdiff !perform vertical diffusion

! default: nvdiff =1

integer :: nconv !perform convective transport

! default: nvdiff =1

integer :: nsedi !perform sedimentation

! default: nsedi=0

integer: :: nint !perform time integration

! default: nint=1

integer: :: ninit !mode of tracer initialization

!see text , default: ninit =3

real :: vini !initialisation value or

!spatially uniform initialization

!of tracer in an initial run ,

!default: vini =0.

! ...

end type (t_trinfo)

Every of these variables can be set when a new tracer is defined by a call to the subroutinenew tracer. If any of the transport variables ntran, nvdiff, nconv, nsedi or the timeintegration variable nint is set to 0, it means that the respective action is switched off,1 means that it is switched on. Modifications of these variables later than at the call ofnew tracer must be done by a direct access of these variables in the derived data type.

3.7. TRACERS IN ECHAM6 41

The tracer data structure is very flexible and can even host a tracer that is neither trans-ported nor integrated over time, but profits from all memory and i/o facitilities of tracersin echam. You may use it for a “climatology” in that way.

• Values of the mass mixing ratio of tracers.In ECHAM6, the value of the mass mixing ratio of the tracers has to be stored for theactual time step and the time step before (t−∆t). They are stored in different modules:mo memory gl: xt: 4–dimensional array for the mass mixing ratios of all tracers at theactual time step (and the prediction for the next time step, time “t”).mo memory g1a: xtm1: 4–dimensional array for tracer values at previous time step (time“t−∆t”). In ECHAM6, it is often referred to t−∆t as “m1”.You should avoid to work with these tracer fields directly and use the methods describedbelow instead. In a lot of cases, you will also find the following variables in the parameterlists of subroutines which can be used:pxt: tracer mass mixing ratio at time t. The p at the beginning indicates that it is a variableof a parameter list in a subroutine or function (according to the ICON programming guidelines).pxtm1: tracer mass mixing ratio at time t−∆t (“m1”).pxtte: tracer tendency ∆x, so that x(t+ ∆t) = x(t) + ∆x(t) ∗∆t.

Subroutines for working with tracers

• The get tracer subroutine

Listing 3.15: get tracer

get_tracer (<name >[, ...])

This routine gives back pointers to the tracer mass mixing ratios at time t or t − ∆t oftracer <name>; in addition, you can also obtain the index of the tracer in the tracer list.An error code is generated, if the tracer does not exist. For a more detailed parameterlist, see [5].

Warning: the routine is slow, because of comparisons of strings. Use it in the initializationphase of your submodel but not in every time step. Store your pointers as module variablesinstead.

• The new tracer subroutine

Listing 3.16: new tracer

new_tracer (<name >, <modulename >[, ...])

Defines a new tracer with name <name> for the module <modulename> of a certainsubmodel. It should be used after an inquiry whether the tracer already exists using the(get tracer) subroutine.

Tips and Tricks

• Improvement of the performance of your submodel.You can speed up your submodel routines considerably when you do not call get tracer ateach time step. Instead, you can save the tracer index in a module variable and use itdirectly in the access of the respective elements of the tracer fields pxt, pxtm1, or pxtte.


• Initialization of tracers.Currently, there are three possibilities in ECHAM6: Initialization by a spatially uniformmass mixing ratio (set vini in the call of new tracer), initialization by reading massmixing ratios from a file through the external field processor, or initialization by readingthe mass mixing ratios from a restart file. We may whish that these three possibilities arehierarchically ordered so that if the tracer concentration is not present in the restart filewe read values from another file or as a last possibility, initialize by a constant. Thesethree switches are condensed into one integer variable ninit.

Indeed, there is the possibility to combine 31 logical variables in one integer nint ≥ 0 in 32bit binary representation. In such a representation, each power of two for 20, . . . , 231 cantake the value 0 or 1. Thus, each power of two can be interpreted as .false. or .true..We make the following assignment:

20 21 22 23 . . . 231 sign of integer

0/1 0/1 0/1 0/1

constant restart initial

if nint = 1⇒ initialization with a constantif nint = 1 + 2 = 3⇒ initialization with a constant or restart.

etc.

In ECHAM6, the hierarchy is: restart → values from file → constant. Apparently, thehierarchy does not simply correlate with the actual position of the corresponding power oftwo in ninit.

It is recommended, to use the predefined constants restart, initial, constant by a usestatement

USE mo_tracer , ONLY: restart , initial , constant

instead of “hardcoded” numbers because the actual values of restart, initial, constantmay change.

3.8 Representation of 3–d variables in ECHAM6

Before we can discuss how to work with 3–d grid point variables we have to learn how they arerepresented (and distributed) on the processors. In the following, we restrict our discussion to 2–d variables since the vertical dimension can just be added without any change of the structure ofthe following. The most basic representation of grid point variables g : S2 → R is the definitionof a 2–dimensional array (Gij)i=1,nlon

j=1,nlat

in which the values (Gij)i=1,nlonj=1,nlat

of g are ordered from north to south and from 0E to 360E:

j(latitudes) latitudes from N → S; longitudes from 0E 360E

i(longitudes)

3.8. REPRESENTATION OF 3–D VARIABLES IN ECHAM6 43

ECHAM6 is a parallelized and vectorized program. The calculation takes place on severalprocessors. We can distribute the latitudes in several chunks on different processors so thatonly a subdomain of the whole globe is on each processor. On the other hand, when loopingover latitudes of a certain chunk, the number of different longitudes at each latitude may betoo low or too high for having the optimal speed of computation on vector machines. We mayfind that the length of the longitude vector is too short for having optimal computational speed.Therefore, in addition to the distribution of the latitudes onto different processors, we can applya transformation on the chunk of latitudes on each processor that combines several latitudesinto one block that is longer than the original “longitude vector”. This means that there arelongitudes of several latitudes in one block (line). More generally, we can define a transformationT npromag that performs a reordering of longitudes and latitudes. In the following, we will restrict

ourselves to one processor only.

T (nproma)g (Gij)i=1,nlon

j=1,nlat

=: (Gi,j)i=1,kproma(j)j=1,ngpblks

(3.1)

with kproma(j) ≤ nlon ∗ nlat for j = 1, . . . , ngpblks (n grid point blocks) andnproma = max1≤j≤ngpblks(kproma(j)). Scanning the elements of (Gij)i=1,nlon

j=1,nlat

, we successively

fill (Gi,j)i=1,kproma(j)j=1,ngpblks

with maximum block length until Gnlon,nlat. Only the last block may be

shorter than the maximum length nproma.

Schematically: In the figure below, we present how the longitudes are fitted into the blocks.

r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r rr r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r rr r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r6 6

first longitude block second longitude block (continued in 2nd gripoint block)

?

gridpoint block

third longitude block

The maximum block length can be set in the runctl namelist by

Listing 3.17: nproma

nproma= <number >

We give a less abstract example for nlon = 3 and nlat = 2:

-

-

66

-

(1,2) (2,2) (3,2)

(1,1) (2,1) (3,1)

Tg

j

i

(2,2) (3,2)

(1,2) (2,2)

(1,1)

(1,1)

(2,1)

ngpblks= 2

(2,1)

(3,1)

(3,1)

(1,2)

(4,1)

kproma(2)= 2

kproma(1)= 4lat

lon


In ECHAM6, there is a loop over all blocks jrow=1,ngpblks in the routine scan1 in which thegrid point calculations gpc are called (see also Figure 3.1).

Listing 3.18: scan1

DO jrow=1, ngpblks

!...

CALL gpc(jrow)

END DO

Below this subroutine, you can find krow=jrow and kproma in the parameter list of most ofthe called subroutines. Here, kproma is the actual length of each block j=1, ..., ngpblks. Forj=1,...,ngpblks-1, kproma(j)= nproma, the maximum vector length you gave in the namelist.The last block may be shorter as in the example above in which the last block has a lengthof only 2 elements. Most parameter lists also contain kbdim= max1≤j≤ngpblks(kproma(j)). Thevariable kbdim should be used in the declaration of arrays in the subprograms since it gives themaximum number of elements in the blocks.

In general, not the whole globe is on one processor but only a sub–domain. The transforma-tion T nproma

g is applied to each (rectangular) sub–domain on each processor separately. Thesub–domains normally consist of latitude bands, but these latitude bands are not necessarilycontiguous.

For some applications, we have to know the latitude and longitude of a particular grid boxon one processor. There is no simple relationship between the indices of the decomposed field(Gi,j)i=1,kproma(j)

j=1,ngpblks

and the corresponding longitudes and latitudes, but ECHAM6 provides fields

that contain this information.

We first discuss the information about the transformation T npromag and the distribution of fields

on several processors, in ECHAM6 also called the decomposition of the fields. The decom-position is defined in mo decomposition in the data structure local decomposition of typepe decomposed:

Listing 3.19: pe decomposed

type pe_decomposed

integer :: nlon ! number of longitudes (global)

integer :: nlat ! number of latitudes (global)

integer :: nlev ! number of levels

integer :: nm ! triangular truncation (T21 , T31 , T42 ,

...)

...

integer :: nprocb ! before nproma decomposition , each

! longitude band is distributed on

! nprocb processors

integer :: nproca ! before nproma decomposition ,

! each latitude

! band is distributed on nproca

! processors

...

3.9. WORKING WITH 2 AND 3–DIMENSIONAL VARIABLES IN ECHAM6 45

integer :: nglat ! number of latitudes on

! particular processor

integer :: nglon ! number of longitudes on

! particular processor

...

integer :: ngpblks ! number of blocks on processor

integer :: nproma ! (max) number of points in each block

integer :: npromz ! length of the last block

...

end type pe_decomposed

The information about longitudes and latitudes in a loop over the blocks can be retrieved bythe use of variables of the module mo geoloc. This module provides for 1 ≤ i ≤ kproma(j) and1 ≤ j ≤ ngpblks the following two dimensional fields:

i) philat d(i,j)

1 ≤ i ≤ kproma(j)

1 ≤ j ≤ ngpblks : latitude in degrees.

ii) ilat(i,j): index of latitude in global latitude vector ordered from N → S.

iii) philon 2d(i,j): longitude in degrees

iv) ilon(i,j): index of longitude in global longitude vector ordered from 0E → 360E.

Summary: Until now, we can introduce new submodels with their respective name lists, weknow how to introduce new tracers, and we learned some aspects about the representation oftwo dimensional longitude–latitude fields in ECHAM6. Nevertheless, we do not know how tointroduce a new two or three dimensional field and how to distribute it among the processors.We have to learn how we can do the “decomposition” by the distribution to the processors andthe transformation T nproma

g . This will be the subject of the next section.

3.9 Working with 2 and 3–dimensional variables in ECHAM6

In the previous section, we restricted the discussion primarily to 2–dimensional variables g :S2 → R. We will now include 3–dimensional variables

f :

S2 × R+ → R(λ, ϕ, p) 7→ f(λ, ϕ, p)

The order of the dimensions for 2– and 3–dimensional variables in ECHAM6 is lon, [lev,] lat.

We already know examples of 3–dimensional variables, e.g. the tracer mass mixing ratios xt,xtm1, and the tracer tendency xtte. All these variables are fields depending on(1:nlon,1:nlev,1:nlat) in the global part of the program and on(1:local decomposition%nproma, 1:local decomposition%nlev, 1:local decomposition%ngpblks) on


each processor. For the mass mixing ratios of tracer and the tracer tendencies, the distribution

to the processors and the T (nproma)g transformation is done automatically. If you define your own

fields which you need on one processor (for initialization or calculation of global integrals) and

“decomposed” on the processors, you have to do the distribution and T (nproma)g tranformation

by hand.

Recipe:

i) Declare a 2– or 3–dimensional field on one processor (typically the i/o–processor):

Listing 3.20: zfield

REAL(dp), POINTER :: zfield (1:nlon ,[1:nlev ,]1: nlat)

which must be a pointer

ii) Declare and allocate on each processor a local field

Listing 3.21: zlocalfield

REAL(dp) :: zlocalfield (:,[:,]:)

ALLOCATE(zlocalfield(local_decomposition%nproma ,

[local_decomposition%nlev ,] local_decomposition%

ngpblks))

iii) Distribute the field zfield to all processors and apply the T npromag transformation by the

use of scatter gp:

Listing 3.22: scatter gp

call scatter_gp(zfield , zlocalfield , global_decomposition)

Use scatter gp from mo transpose and global decomposition from mo decomposition.The data structure global decomposition contains the comprehensive information of thedistribution and the T nproma

g transformation. There is also a routine for collecting a“decomposed” data field from all processors to a global field on one processor:

Listing 3.23: gather gp

gather_gp

For further subroutines handling 2d– or 3d–arrays, see the “user guide” [5].

3.9.1 Global diagnostic

In general, it is recommended to do some diagnostics during the model runs, because it istime consuming to work with the archived data. On the other hand, global operations needcommunication between the processors. During these calculations, the other processors risk towait. This tends to slow down your program. It is therefore recommended to restrict the use ofglobal operations to larger time intervals (e.g. daily or monthly).

3.9. WORKING WITH 2 AND 3–DIMENSIONAL VARIABLES IN ECHAM6 47

ECHAM6 provides various functions in mo global op.f90 but no written documentation isavailable. In this file, you will find functions for calculating global sums, the global minimum,or global maximum. There are also functions for the calculation of zonal and meridional means,minima, and maxima.

Example: Global sum.

We illustrate the use of these diagnostic functions by the calculation of the global sum of a2–dimensional field:

Listing 3.24: sum global of 2–dim. field

USE mo_global_op , ONLY: sum_global

USE mo_decomposition , ONLY: ldc=>local_decomposition

REAL :: zfield2 (1: ldc%nproma ,1: ldc%ngpblks)

gs = sum_global(zfield2)

The global sum of the field zfield2 will be stored in gs. The field zfield2 is a decomposed 2–dimensional field.

The function sum global is an overloaded function and can therefore also be used for the calcu-lation of the global sum of a 3–dimensional field. The result will be a vector and contain theglobal sum for each level:

Listing 3.25: sum global of 3–dim. field

USE mo_global_op , ONLY: sum_global

USE mo_decomposition , ONLY: ldc=>local_decomposition

REAL :: zfield3 (1: ldc%nproma ,1:n,1: ldc%ngpblks)

gs(1:n) = sum_global(zfield3)

In this case, n > 0 is arbitrary, but will usually be the levels n = ldc%nlev.

It calculates for each 1 ≤ k ≤ n (e.g. for each level) the global sum:

gs(k) =∑

allproc.

∑1 ≤ i ≤ kproma(j)

1 ≤ j ≤ ngpblks

zfield3(i, k, j)processor

The global total of a 3–dimensional field can be calculated by performing sums over columns oneach processor and then applying sum global to the resulting 2–dimensional field.

For the proper integrals over the globe you need the grid box area:

Listing 3.26: use grid box area

USE mo_geoloc , ONLY: gboxarea_2d ! grid box area in m^2


3.9.2 Vertical coordinates:

In section 1.3.1, the vertical resolution of ECHAM6 was introduced. In order to calculate theactual pressure at the midpoint of a certain model layer or its interfaces we need the coefficients(Ai, Bi)i=1,nlev

, and (Ai, Bi)i=1,nlev+1. You can calculate these coefficients in the following way:

Listing 3.27: σ-levels

USE mo_control , ONLY: nvclev , nlev , vct

! coefficients for layer interfaces hyai , hybi

hyai = vct (1: nvclev)

hybi = vct(nvclev+1, 2* nvclev)

! coefficients for the layers hyam , hybm

do jlev = 1, nlev

hyam(jlev) = 0.5 * (hyai(jlev) + hyai(jlev +1))

hybm(jlev) = 0.5 * (hybi(jlev) + hybi(jlev +1))

endo

In ECHAM6, hyam, hybm contain Ai, Bi, respectively and hyai, hybi contain Ai, Bi, respectively.

3.10 Recursive derived data types

Before we can proceed to discuss “streams” for which all the above transformations are doneautomatically, we have to discuss recursive derived data types. In Fortran 90, it is possible toconstruct recursively defined data types. In this way, it is possible to create very complex datastructures. In particular, it is possible to concatenate derived data structures in a “list of infinitelength”. This means that we do not need to know how long this list is (how many elements itwill contain) at compile time of the program because the list will be constructed during run timeand can be of arbitrary length. One possible application is the definition of a list which containsthe information about the tracers in each list element. This can be very complex informationfor each list element like the mass mixing ratio, the chemical properties, the name, and chemicalformula of the tracer. When we need to define a new tracer during the runtime of the program,we just append it to the end of the existing list. In the following example, we will present arecursive list each element of which contains the mass mixing ratio of the tracer only. So, wedefine a data type that contains a 3d–field for the mass mixing ratio of the tracer but in additiona pointer that can point to the next element of the list (Remark: Tracers in echam are actuallynot stored in recursive data structures but tracers are an easy to demonstrate example):

Listing 3.28: recursive data types

type tracer

real , allocatable :: xtfield (:,:,:)

type (tracer), pointer :: next

end type tracer

Let us now define three variables of type tracer

Listing 3.29: variables of type tracer

type (tracer), pointer :: xtracer , firsttracer , lattertracer

3.11. STREAMS 49

Using the pointer structure of these variables, we can construct a “chain” of tracers of arbitrarylength by the following pointer construct:

Listing 3.30: tracer list

nullify (xtracer , firsttracer , lattertracer)

allocate (firsttracer)

allocate (firsttracer%xtfield (nlon , nlev , nlat))

!this allocates a 3--dim field for the mass mixing ratio

lattertracer -> firsttracer

do i=2, ntrac ! ntrac is the number of tracers

allocate (xtracer)

allocate (xtracer%xtfield (nlon , nlev , nlat))

lattertracer%next -> xtracer

lattertracer -> xtracer

enddo

We go step by step through the above lines of code: firsttracer will be associated for i=1, sofirsttracer%xtfield contains the 3d–field of the mass mixing ratio of tracer 1.

Furthermore, lattertracer points to firsttracer, so that we save in the variablelattertracer the location where the data of firsttracer are stored in the memory.

For i=2:The first two allocate statements reserve memory for another xtracer component and its asso-ciated 3d–field of mass mixing ratio. The third statement in the loop now connects the %next

component of firsttracer which was intermediately stored in lattertracer to the actual(second) tracer. This assures that we can get the actual second tracer by firsttracer%next.In a last step, we link lattertracer to the actual tracer so that we will be able to associate the%next component of the actual (second) tracer in a subsequent step to the new (third) tracer.

If we continue our recursive chain over further steps, we see thatfirsttracer%next%next%...%next%xtfield (containing %next n–times) is the 3d–field of then+ 1st tracer of our recursive pointer structure.

One of the most important data structures in ECHAM6 is constructed as a recursive pointerstructure: The so–called (output) streams are just recursive pointer structures containing theinformation of all the output variables that will be written into one file. Thus, a “stream” inthe sense of ECHAM6 is just a recursive pointer structure.

3.11 Streams

Streams in ECHAM6 are the data structure which is used for the variables that are written toan output file. But they are useful in many other cases also. We saw in the previous sectionthat we have to do the decomposition of a 2– or 3– dimensional variable (so the distributionto the processors and the transformation with T nproma

g ) by hand if we need the global anddecomposed version of this variable. It is also difficult to write the values of such a variable fordebugging for example. These things are done automatically if we include the variable into an


ECHAM6–stream. A comprehensive description of streams can be found in the “user guide” [5].The stream datatype contains a recursive pointer structure.

Only 2,3,4–dimensional real arrays can be stored in a stream.

The stream itself has the data type t stream which is defined in mo linked list and it has thepointer attribute. We summarize the main components of the t stream data type:

Listing 3.31: t stream

type t_stream

! name of stream

character (len =16) :: name

! recursive pointer structure contains

! arbitrarily many variables

type (list_element), pointer :: first_list_element

!...

! variables , describing the output properties of the whole

stream.

end type t_stream

The type list element contains information about the properties of each (output) variable in thestream. (Output) variables in a stream are called stream elements.

Working with streams

• There are subroutines with which you can handle entire streams. They are all defined inmo memory base.

– new stream:

creates a new stream and sets the basic stream properties; on exit, it gives back apointer of type t stream.

– get stream:

gives back a pointer of type t stream to an existing stream.

– default stream setting:

sets general properties of an existing stream.

– add stream element:

adds a new element to an existing stream (adds 2,3,4–dimensional variables to astream) and sets its properties. You need a pointer to an existing stream and you getback a pointer to the new stream element on exit.

• Routines which are provided for the access and modification of existing stream elements(also defined in mo memory base):

– get stream element:

gives back a pointer to an existing stream element.

– set stream element info:

sets the properties of a certain stream element.

3.11. STREAMS 51

Tips and tricks

• Similar to the usage of the subroutines that handle tracers, you should avoid repeatedget stream or get stream element calls. These tend to slow down your program consid-erably because of string comparisons.

Listing 3.32: laccu

• laccu = .true. !default: .false.

This switch can be used for the calculation of mean values. Contrary to what you mayexpect, no automatic accumulation is done, if laccu = .true.. The respective variableor all variables of a steam are only devided by the number of seconds of the output intervalwhen the variable(s) are written to the output file. Then, the variable(s) are set to 0 inECHAM6. No sum over the time steps is performed automatically! This means, thatyou have to program the calculation of the sum over the time steps yourself. We give anexample of the calculation of mean values of the ozone mass mixing ratio over the outputinterval. In a first step, you have to create a respective stream and add a stream elementfor the mean ozone mass mixing ratio.

Listing 3.33: add stream element

USE mo_memory_base , ONLY: newstream , add_stream_element

USE mo_linked_list , ONLY: t_stream

USE mo_time_event , ONLY: io_time_event

TYPE(t_stream), POINTER :: ozone_mean

TYPE(io_time_event) :: outfrq

REAL(dp), POINTER :: ptr_o3 (:,:,:)

call new_stream(

ozone_mean , ! pointer of type t_stream

’ozone_mean ’, ! name as included in ouput filename

interval= outfrq ! variable of type io_type_event

! see next section

)

call add_stream_element(

ozone_mean , ! stream

’o3_mean ’, ! name of stram element

ptr_o3 , ! pointer to stream element , 3d-

! real variable

laccu =.true.

)

In a certain subroutine, you may accumulate the ozone mass mixing ratio for the calculationof the mean value over the output interval. Such a diagnostic may be done in a subroutinewritten by yourself and called in physc subm 4 of mo submodel interface.

Listing 3.34: accumulation over time

USE mo_time_control , ONLY: delta_time

! o3_mm: your variable in the program containing


! the instantaneous mass mixing ratio of ozone

ptr_o3= ptr_o3 + o3_mm * delta_time ! accumulation

3.12 Time control in ECHAM6

In this section, we will discuss the various time format definitions and how calculations on thetime axis (e.g. for the definition of events) can be performed. ECHAM6 uses the “prolepticGregorian” calendar. This means that the calendar is an extension of the present Gregoriancalendar backward in time even to dates prior to the introduction of the Gregorian calendarin 1582. This calendar goes back to 4712/01/01 12:00:00 UTC time B.C.. This calender hasmonths of unequal length and includes leap years. The calender is closely linked to the orbit ofthe Earth. The standard orbit in ECHAM6 corresponds to the proleptic Gregorian calendar.Nevertheless, there are other orbits available for paleo-calculations. A time in the prolepticGregorian calender can be represented in various ways leading to the following definitions ofdate–time (DT) variables. All DT variables in ECHAM6 are of one of the following data typesthe definitions of which can be found in mo time conversion:

Listing 3.35: time days

type time_days

! ...

integer :: day ! day counted since 1/1/1

integer :: second ! second in the day [0, 86399]

end type time_days

Listing 3.36: time intern

type time_intern

! ...

integer :: ymd ! ‘year month day ‘ of

! proleptic Gregorian calendar

! (leading zeros omitted);

! e.g. 2001008 is the 8th of Oct. 200.

integer :: hms ! ‘hour minute second ‘ of ymd

! (leading zeros omitted);

end type time_intern

Listing 3.37: time native

type time_native

! ...

integer :: year , month , day , hour , minute , second

end type_native

One can also use an array of 6 elements containing year, month, hour, minute, second.

3.12. TIME CONTROL IN ECHAM6 53

Usage of these DT–variables.

For the derived data types time days, time intern, andtime native, a direct access of the components is not possible because they are declared being“PRIVATE”. Instead, they are accessible by the use of subprograms defined in mo time conversion.The reason for this is the fact that you can easily create a date in these formats that is not valid.Then, all subroutines using such an invalid DT variable would fail. In order to avoid this, allthe subroutines changing one of the components of the DT variables test whether the resultingdates are correct. ECHAM6 provides the following subroutines:

• Set a DT variable of type time days, time native or time intern by the overloaded routinetc set. e.g. for

Listing 3.38: tc set

type(time_native) :: my_date

call tc_set(kyear , kmonth , kday , khour , kminute , ksecond ,

mydate)

This call of tc set will search for the special routine set native that actually sets a variableof type time native using the input variables kyear, kmonth, kday, khour, kminute, andksecond.

• Conversion of a variable of one time format into another:

There are 3× 2 = 6 possible non–trivial conversions which can all be performed by a calloftc convert(var1,var2), var1, var2 being of one of the 3 types. The types of var1 and var2must be different!

• Getting components of a DT variable

The components of a DT variable can be retrieved by a call to the subroutine tc get. Forthe exact interface, look into the user guide [5] or intomo time conversion directly.

• Comparison of DT variables

You can also perform comparisons among DT variables using certain operators. E.g. inorder to investigate whether a certain date is before or after a second date. Fortran90provides the possibility to overload intrinsic Fortran90 functions like “<” or “==”. Thismeans that you can extend the domain of these operators to work on user defined datatypes also like on integers, real numbers or characters as in standard Fortran90. Then,you can use these operators also for the comparison of user defined data types. Such anextension was programmed for the DT variables. The usage is as follows:

Listing 3.39: overloaded operators

USE mo_time_conversion , ONLY: operator(<),operator (==),

operator(>)

TYPE(time_native) :: var1 , var2

! ...

IF (var1 < var2) THEN

!...


The comparison works with all DT variable types (time native,time intern,time days) but thetwo arguments of the operator have to be of the same type.

Information about the actual date and time in ECHAM6.

There are three variables in which the time and date of the previous (t − ∆t), the actual (t),and the next time step (t+ ∆t) are stored. These variables are declared in mo time control:

Listing 3.40: date and time variables for time steps

type(time_days) :: previous_date , current_date , next_date

Variables describing repeated events.

In the last two sections, we learned something about the representation of absolute date andtime in ECHAM6. In this paragraph, we will discuss the data structure associated with repeatedevents. We saw in the discussion of the namelist (section 1.3.3) how we have to give the infor-mation about repeated events to ECHAM6. We had an integer number and the unit, describingthe frequency of the event. In addition, some keyword had to be given which describes theposition of the repeated events relative to the absolute time axis. The underlying data structureis defined in mo time event:

Listing 3.41: io time event

type io_time_event

integer :: counter ! interval

character(len =20) :: unit ! unit

character(len =20) :: adjustment ! adjustment

integer :: offset ! offset

end type io_time_event

With the help of this data structure, we can define a variable outfrq that will describe the outputfrequency of a stream for example.

Listing 3.42: outfrq

type(io_time_event) :: outfrq

A variable of such a type can be read from the namelist like all the other variables describingrepeated events (putdata, putrerun) but we have then to communicate it to all processors. Forthis purpose, there is a special subroutine p bcast event defined in mo time control.f90:

Listing 3.43: p bcast event

USE mo_time_control , ONLY: p_bcast_event

call p_bcast_event(outfrq , pe_io)

Then, the variable outfrq can be used in the definition of a new stream for which you can thusapply your own output frequency read from the namelist.

3.13. RADIATION IN ECHAM6 55

Table 3.1: Optical properties and their corresponding arrays

optical property corresponding array wavelength range

τ aer tau sw vr solarω aer piz sw vr solarg aer cg sw vr solarτeff aer tau lw vr thermal

3.13 Radiation in ECHAM6

The radiation calculation has two parts: (i) The calculation of the radiative transfer in the solarwavelength range. This is the radiation that has the sun as origin and is scattered and absorbedby the gases and aerosols in the atmosphere or reflected at the surface. Although the maximumof energy is emitted around 550 nm, a good representation of the solar radiation also involveswavelengths in the infrared. Currently, 14 wavelength bands between 200 nm and 12195 nm areused. (ii) The calculation of the radiation transfer in the thermal wavelength range. This is theradiation that has as origin the surface of the earth or the atmosphere itself. Since each bodyat a temperature above absolute zero emits electromagnetic radiation, the surface of the earthbut also the atmosphere itself are a source of electromagnetic radiation. The maximum energyis emitted at much longer wavelengths than in the case of the sun and the calculation of theradiative transfer including scattering and absorption in the atmosphere requires the choice ofdifferent wavelenghts. Currently, 16 wavelength bands between 3078 nm and 1 000 000 nm areused.

The interface to the radiative transfer calculation is the subroutine psrad interface that is calledinside radiation of the module mo radiation. The subroutine psrad interface is defined in themodule mo psrad interface. Inside psrad interface, the optical properties of the atmosphereare set. In particular, the optical properties of aerosols in the atmosphere are passed insidepsrad interface to the various subroutines performing the radiative transfer calculation for thesolar and thermal spectral range.

As mentioned above, in particular aerosols scatter or absorb the radiation and have highlyvariable concentrations in the air. In order to introduce their influence on radiation, you firsthave to determine their optical properties aerosol optical depth τ , single scattering albedo ω,and asymmetry factor g for the solar radiation and the (effective) aerosol optical depth τeff

(excluding the absorption) in the thermal wavelength range. In the case of a single type ofaerosols, τeff = τ(1 − ω). You can set these quantities or add additional aerosol optical depthby the use of the usual mixing rules to the respective variables in radiation subm 1 of thesubmodel interface (mo submodel interface). Table 3.1 gives the optical properties and theircorresponding array names.

The infix sw in the array names means solar wavelength range, the infix lw stands for the thermalwavelength range. The suffix vr indicates that the order of model layers is reversed, meaningthat the layers are ordered in these arrays such that the geometrically lowest layer (just abovethe surface) has the lowest index and we proceed to the top of the atmosphere in contrast to allother parts of ECHAM6.

If you like to add new aerosols to existing ones (e.g. for the simulation of a volcanic eruption thatadds aerosols to the background), use the following mixing rules for aerosol species j = 1, . . . , N :


τ =N∑j=1

τj solar

ω =

N∑j=1

τjωj

τsolar

g =

N∑j=1

τjωjgj

τωsolar

τeff =

N∑j=1

τj(1− ωj) thermal

Bibliography

[1] L. Bonaventura, M. Esch, H. Frank, M. Giorgetta, Th. Heinze, P. Korn, L. Korn-blueh, D. Majewski, A. Rhodin, P. Rıpodas, B. Ritter, D. Reinert, and U. Schulzweida.ICON Programming Standard. Max Planck Institute for Meteorology, Bundesstr. 53,D–20146 Hamburg, https://code.zmaw.de/projects/icon/repository/entry/trunk/icon-dev/doc/style/icon standard.pdf edition, December 2012.

[2] M.A. Giorgetta, J. Jungclaus, Ch.H. Reick, St. Legutke, J. Bader, M. Bttinger, V. Brovkin,T. Crueger, M. Esch, K. Fieg, K. Glushak, V. Gayler, H. Haak, H.-D. Hollweg, T. Ily-ina, St. Kinne, L. Kornblueh, D. Matei, Th. Mauritsen, U. Mikolajewicz, W. Mueller,D. Notz, F. Pithan, Th. Raddatz, S. Rast, R. Redler, E. Roeckner, H. Schmidt, R. Schnur,J. Segschneider, K.D. Six, M. Stockhause, C. Timmreck, J. Wegner, H. Widmann, K.-H.Wieners, M. Claussen, J. Marotzke, and Bjorn Stevens. Climate and carbon cycle changesfrom 1850 to 2100 in mpi–esm simulations for the coupled model intercomparison projectphase 5. Journal of Advances in Modeling Earth Systems, 5(3):572–597, 2013.

[3] M.A. Giorgetta, E. Roeckner, T. Mauritsen, J. Bader, T. Crueger, M. Esch, S. Rast, L. Ko-rnblueh, H. Schmidt, S. Kinne, C. Hohenegger, B. Mobis, T. Krismer, K.-H. Wieners, andB. Stevens. The atmospheric general circulation model ECHAM6 — Model description.Max Planck Institute for Meteorology, Bundesstr. 53, D–20146 Hamburg, October 2013.Distributed with the model.

[4] S. Kinne, D. O’Donnell, Ph. Stier, S. Kloster, K. Zhang, H. Schmidt, S. Rast, M. Giorgetta,T.F. Eck, and B. Stevens. MAC–v1: A new global aerosol climatology for climate studies.Journal of Advances in Modeling Earth Systems, 5(2):704–740, 2013.

[5] S. Rast, R. Brokopf, S.-K. Cheedela, M. Esch, V. Gayler, I. Kirchner, L. Kornbluh,A. Rhodin, H. Schmidt, U. Schulzweida, and K.-H. Wieners. User manual for ECHAM6.Max Planck Institute for Meteorology, Bundesstr. 53, D–20146 Hamburg, echam–6.2.00–guide–2.0 edition, June 2014. Distributed with the model.

[6] B. Stevens, M. Giorgetta, M. Esch, Th. Mauritsen, T. Crueger, S. Rast, M. Salzmann,H. Schmidt, J. Bader, K. Block, R. Brokopf, I. Fast, St. Kinne, L. Kornblueh, U. Lohmann,R. Pincus, Th. Reichler, and E. Roeckner. Atmospheric component of the mpi–m earthsystem model: Echam6. Journal of Advances in Modeling Earth Systems, 5(2):146–172,2013.

57

58 BIBLIOGRAPHY

Listings

1.1 Load compilers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.2 basic compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Dependencies: create new Makefile . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.4 Namelist syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.5 out datapath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.6 offset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.7 putrerun . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1 netcdf syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2 dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3 example for dimension listing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.4 variable definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.5 Example: definition of a variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.6 attributes to variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.7 data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.8 comment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.9 ncdump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.10 ncgen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.11 Change of file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.12 Call of single cdo operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.13 Pipeline cdo operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.14 Interpolation to pressure levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.15 Interpolation to model levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.16 Spectral to grid point space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.1 submodel interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

59

60 LISTINGS

3.2 p bcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.3 yrnamelistctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4 function open nml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.5 function position nml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.6 subroutine message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.7 subroutine finish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.8 init transport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.9 type statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.10 type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.11 components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.12 vectorfield . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.13 trlist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.14 t trinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.15 get tracer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.16 new tracer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.17 nproma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.18 scan1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.19 pe decomposed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.20 zfield . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.21 zlocalfield . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.22 scatter gp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.23 gather gp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.24 sum global of 2–dim. field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.25 sum global of 3–dim. field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.26 use grid box area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.27 σ-levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.28 recursive data types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.29 variables of type tracer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.30 tracer list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.31 t stream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.32 laccu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

LISTINGS 61

3.33 add stream element . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.34 accumulation over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.35 time days . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.36 time intern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.37 time native . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.38 tc set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.39 overloaded operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.40 date and time variables for time steps . . . . . . . . . . . . . . . . . . . . . . . . 54

3.41 io time event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.42 outfrq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.43 p bcast event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

using and programming echam6| a rst introduction · currently, the newest released version is...

Documents