climate data operators (cdo) · introduction and history • a single command with hundreds of...
TRANSCRIPT
![Page 1: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/1.jpg)
Climate Data Operators (CDO)
https://code.zmaw.de/projects/cdo/wiki/Cdo
![Page 2: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/2.jpg)
Introduction and History • A single command with hundreds of operators
• CDO was inspired by NCO – providing a range of climate data-related operations through the command-line
• Designed to operate on netCDF3/4, GRIB1/2 primarily
• Much functionality can be used for any NetCDF/gridded data
• Very efficient for specific tasks
• Manages memory effectively
![Page 3: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/3.jpg)
CDO: a command-line tool CDO is a single command-line tool. It is used as follows: $ cdo <operator> [options] <files>
![Page 4: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/4.jpg)
Types of CDO Operator 1. File information and file operations 2. Selection and Comparison 3. Modification of metadata 4. Arithmetic operations 5. Statistical analysis 6. Regression and Interpolation 7. Vector and spectral Transformations 8. Formatted I/O 9. Climate indices
![Page 5: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/5.jpg)
Many, many operators CDO breaks down its functionality into individual operators. There are over 650 at the time of writing this. For example: showstdname Show standard names sellonlatbox Select a longitude/latitude box setmissval Set a new missing value monadd Add monthly time series zonstd Zonal standard deviation eca_hd Heating degree days per time period
![Page 6: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/6.jpg)
Getting help on an operator To see help just type: cdo -h <operator> $ cdo -h showparam NAME showformat, showcode, showname, showstdname, showlevel, showltype, showyear, showmon, showdate, showtime, showtimestamp - Show variables, levels or times SYNOPSIS <operator> ifile DESCRIPTION This module prints the format, variables, levels or times of the input dataset.
![Page 7: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/7.jpg)
Operators: 1. File Information $ cdo infov ifile
This is an example result of a dataset with one 2D variable over 3 time steps: -1 : Date Time Varname Level Size Miss : Minimum Mean Maximum 1 : 1987−01−31 12:00:00 SST 0 2048 1361 : 232.77 266.65 305.31 2 : 1987−02−28 12:00:00 SST 0 2048 1361 : 233.64 267.11 307.15 3 : 1987−03−31 12:00:00 SST 0 2048 1361 : 225.31 267.52 307.67
![Page 8: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/8.jpg)
Operators: 1. File Information $ cdo showtimestamp ggas2014121200_00-18.nc 2014-12-12T00:00:00 2014-12-12T06:00:00 2014-12-12T12:00:00 2014-12-12T18:00:00 $ cdo pardes ggas2014121200_00-18.nc -1 CI Sea-ice cover [(0 - 1)] -2 SSTK Sea surface temperature [K] -3 MSL Mean sea-level pressure [Pa] -4 TCC Total cloud cover [(0 - 1)] -5 U10 10 metre U wind component [m s**-1] -6 V10 10 metre V wind component [m s**-1] -7 SKT Skin temperature [K]
![Page 9: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/9.jpg)
Operators: 1. File Information It is simple to see a summary of the numerical difference between 2 files: $ cdo diff tas_rcp45_2055_ann_95p_change.nc tas_rcp45_2055_ann_05p_change.nc Date Time Param Level Size Miss : S Z Max_Absdiff Max_Reldiff 1 : 2065-12-30 12:00:00 -1 0 64800 0 : T F 16.693 0.99997 1 of 1 records differ
![Page 10: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/10.jpg)
Operators: 1. File Operations The full list of file operations covers: • Copying and concatenating • Merging fields/times • Splitting fields by variable/level/grid/time
Some examples follow
![Page 11: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/11.jpg)
Operators: 1. File Operations To copy a file and convert the output to NetCDF: $ cdo −f nc copy ifile ofile.nc To merge all files along the time axis: $ cdo mergetime ifile1 ifile2 ifile3 ofile
![Page 12: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/12.jpg)
Operators: 2. Selection Data can be selected by a number of different methods including: • Variable code, name or attribute • Levels • Time • Spatial region (latitude/longitude)
![Page 13: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/13.jpg)
Operators: 2. Selection To select variables with ids: "SSTK" and "CI": $ cdo selname,SSTK,CI infile.nc outfile.nc
To select a lat/lon bounding box: $ cdo sellonlatbox,120,−90,20,−20 infile.nc outfile.nc
To select a date/time range: $ cdo seldate,2014-12-12T12:00:00, 2015-01-31T18:00:00 infile.nc outfile.nc
![Page 14: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/14.jpg)
Operators: 3. Modification To set the time axis to 1987-01-16 12:00 with an increment of one month for each time step use: $ cdo settaxis,1987-01-16,12:00,1mon ifile ofile
To invert the latitudes of a 2D field from N-to-S to S-to-N use: $ cdo invertlat ifile ofile
![Page 15: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/15.jpg)
Operators: 4. Arithmetic To calculate the square root for all field elements use: $ cdo sqrt ifile ofile
To sum all input fields with the constant -273.15 use: $ cdo addc,-273.15 ifile ofile
![Page 16: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/16.jpg)
Operators: 5. Statistics To compute the zonal mean of all input fields use: $ cdo zonmean ifile ofile
Assume an input dataset has monthly means over several years. In order to compute seasonal means from monthly means the first two months must be skipped: $ cdo timselmean,3,2 ifile ofile
![Page 17: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/17.jpg)
Operators: 5. Statistics Multi-year daily running percentile values: • To calculate a percentile the minimum and
maximum bounds must be known. Hence the signature of the command is:
$ cdo ydrunpctdl,p,nts infile minfile maxfile outfile
This operator writes running percentile values for each day of year in infile to outfile. The minimum and maximum bounds are provided in minfile and maxfile.
![Page 18: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/18.jpg)
Operators: 5. Statistics So to calculate a multi-year daily running percentile values you need multiple steps: $ cdo ydrunmin,5 ifile minfile $ cdo ydrunmax,5 ifile maxfile $ cdo ydrunpctl,90,5 ifile minfile maxfile ofile
![Page 19: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/19.jpg)
Operators: 6. Interpolation To remap all fields bilinear to a T42 Gaussian grid type: $ cdo remapbil,t42grid ifile ofile
To interpolate hybrid model level data to pressure levels of 925, 850, 500 and 200 hPa use: $ cdo ml2pl,92500,85000,50000,20000 ifile ofile
![Page 20: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/20.jpg)
Operators: 9. Climate Indices To get the largest number of consecutive dry days of a time series of daily precipitation amounts use: $ cdo eca_cdd rrfile ofile
Where rrfile is a time series of daily precipitation amounts RR, then counted is the largest number of consecutive days where RR is less than 1 mm.
![Page 21: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/21.jpg)
Operators: 9. Climate Indices
To get the number of tropical nights of a time series of daily minimum temperatures use: $ cdo eca_tr tnfile ofile
Where tnfile is a time series of daily minimum temperatures TN, then counted is the number of days where TN > T. The number T is an optional parameter with default T = 20°c.
![Page 22: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/22.jpg)
What else can CDO do? That was a whistle-stop tour of some of the functionality provided by CDO. There are other useful features worth knowing about.
![Page 23: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/23.jpg)
CDO Command-line switches CDO has many generic command-line switches that can be used with (most) operators. Here is a selection:
-a Generate an absolute time axis -f <format> Specify format ("grb", "nc", "nc4" etc) -m <val> Set the default missing value -Q Sort netCDF variable names -r Generate a relative time axis -s Silent mode -V Version of CDO -v Verbose - print extra details. -z zip Deflate compression of netCDF4 vars.
![Page 24: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/24.jpg)
Combining operators
All operators with a fixed number of input streams and one output stream can pipe the result directly to an other operator. The operator must begin with "-", in order to combine it with others. This can improve the performance by: • Reducing unnecessary disk I/O • Parallel processing
![Page 25: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/25.jpg)
Combining operators: example 1
For example, we can replace: $ cdo timavg ifile1 tmp1 $ cdo dayavg ifile2 tmp2 $ cdo sub tmp2 tmp1 ofile $ rm tmp1 tmp2
With... $ cdo sub -dayavg ifile2 -timavg ifile1 ofile
![Page 26: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/26.jpg)
Combining operators: example 2 The three commands to calculate a multi-year daily running percentile values you need multiple steps: $ cdo ydrunmin,5 ifile minfile $ cdo ydrunmax,5 ifile maxfile $ cdo ydrunpctl,90,5 ifile minfile maxfile ofile
…can be replaced with a single command: $ cdo ydrunpctl,90,5 ifile −ydrunmin ifile −ydrunmax ifile ofile
![Page 27: Climate Data Operators (CDO) · Introduction and History • A single command with hundreds of operators • CDO was inspired by NCO – providing a range of climate data-related](https://reader030.vdocuments.net/reader030/viewer/2022021612/5b509b227f8b9a5a6f8ed2ad/html5/thumbnails/27.jpg)
Further information
CDO Home page: https://code.zmaw.de/projects/cdo CDO Tutorial: https://code.zmaw.de/projects/cdo/wiki/Tutorial CDO Documentation: https://code.zmaw.de/projects/cdo/wiki#Documentation