© crown copyright met office an introduction to linux, pp utilities, cdo and ncl bangkok workshop,...

© Crown copyright Met Office

An Introduction to Linux, PP utilities, CDO and NCLBangkok Workshop, 26th February– 1st March 2013


• What is Linux?

• Useful commands

• Bash

• PP utilties

• CDO

• NCL

• Useful websites

Contents


Tux the penguin, mascot of Linuxhttp://www.linux.org/

• A form of the Unix operating system.

• Freely downloadable from the internet.

What is Linux?


Directory structure

//

binbin devdev homehome rootroot usrusr

dir1dir1 dir2dir2


Navigating the file system

cd Change Directory

• cd home/dir1 to get to the dir1 directory

• cd .. to go up one directory (parent directory)

• cd $HOME or cd ~ to navigate back to your personal directory no matter where you are

• cd - goes to the former working directory

pwd Print Working Directory to display the path of the current directory


List directory contents

ls Lists files and directories

• ls -a lists all files, including hidden ones.

• ls -l displays the file list in long format, including file details like size, time stamp, and owner.

• ls -F adds / after the name for directories, an * for executables, and @ for linked files.

• ls -r reverses the sort order (alphabetic or time).

• ls -t sorts the list by the time each file was created.


Copying and renaming files

cp Copy

• cp file1 file2 makes a copy of file1 in same directory

• cp file1 /dir2/ makes a copy of the file in dir2

mv To move or rename files

• mv file1 file001 to rename file1 to file 001

• mv file001 dir2 to move file001 into dir2


Deleting files and directories

• rm file1 removes the file.

Directories

• mkdir dir3 makes a new directory.

• rmdir -r dir1 deletes the directory and the r flag is used to delete everything underneath it.


Bash

Some useful stuff

• Tab completion

• To copy double click with left mouse button and paste by pressing middle mouse button.

Some keyboard shortcuts


Regular expressions

*

• ls e* lists all files that start with e

• ls *pdf lists all files that end in pdf

?

• ls file?.dat will list files such as file1.dat and file7.dat but will not list file001.dat


Other Basic Commands

• cat Concatenate and display

• less Can move through a file when viewing it

• man Manual

• touch Makes a new file

• clear Clears the terminal screen

• emacs Editor

• pico Editor

• vi Editor

• acroread Acrobat Reader


PP Utilities, CDO and NCL


What are they?

• CDO, NCL and PRECIS utilities are tools for data analysis and visualisation.

• Different tool are useful for different activities e.g. CDO and PRECIS tools for analysis and NCL for visualisation.

• In the worksheets these tools are used together to process, analyse and visualise PRECIS output.


PRECIS utilities

• Designed for use with PRECIS output (which is in Met Office PP format.)

• Can be used with Linux and Windows.

• Can be typed directly into the command line, or scripted.

• Extensive range of different utilities available, see http://www.metoffice.gov.uk/precis/tutorials

• For example – calculate the JJA mean:

ppstats –H “mon>=6 and mon<=8” –o jja.mean.pp –t mean infile.pp


CDO – Climate Data Operators (1)

• Developed by the Max Planck Institute for use with GRIB 1/2, netCDF 3/4, SERVICE, EXTRA and IEG data.

• Designed specifically for climate and NWP data analysis, there are more than 600 operators available.

• Can be run on Linux, Windows, MasOS and others.

• As with PP utilities can be typed directly into the command line, or scripted.

• For example – calculate the JJA mean:

cdo timmean –selmon,6/8 infile.nc jja.mean.nc


•Free to download and documentation and support forums can be found at https://code.zmaw.de/projects/cdo

CDO – Climate Data Operators (2)


NCL – NCAR Command Language (1)

• Developed by NCAR (Nation Center for Atmospheric Research) for use with GRIB 1/2, netCDF 3/4, shape files and ascii data.

• Free interpreted language designed for scientific data processing and visualization.

• Can be run on Linux, Windows, MasOS and others.

• Can be typed directly into the NCL command line (very time consuming), or scripted.

• Produces plots of publishable quality.



• Extensive documentation and hundreds of example scripts and plots at http://www.ncl.ucar.edu/index.shtml



• Looks more complex than it is.


Using tools together

• The worksheets combine all three of these tools to post process and analysis PRECIS data.

1. Use PP utilities to remove rim and convert from PP to netCDF format.

2. Use CDO to analyse the netCDF data.

3. Use NCL to visualise (and process) the data.


Worksheets

1 – File locations and pre-processing (into to Linux and using PP utilities.)

2a- Introduction to using CDO for analysis and NCL for visualisation (validation.)

2b- Creating future climate scenarios and analysing change (using CDO and NCL.)

3-Preparing data for GLAM using CDO.


Some useful websites

• Doctor Bobs Lowfat Linux http://lowfatlinux.com/

• Getting started with Linux http://www.linux.org/lessons/beginner/toc.html

• Unix Tutorial for beginners http://www.ee.surrey.ac.uk/Teaching/Unix/index.html


Questions

© crown copyright met office an introduction to linux, pp utilities, cdo and ncl bangkok workshop,...

Documents