python programming guide for earth...
TRANSCRIPT
Python programming guide for
Earth Scientists
Maarten J. Waterloo and Vincent E.A. Post
Amsterdam Critical Zone Hydrology group
September 2015
Cover page: Meteorological tower in an abandoned agricultural field near Va-
gos, North Portugal, June 2008. Meteorological measurements are done for esti-
mating evapotranspiration rates.
Contents
1 Why use Python? 5
2 Basics of Python 7
2.1 Starting up Python . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Python through a text terminal . . . . . . . . . . . . . . . 7
2.1.2 IDLE integrated development environment . . . . . . . . 9
2.1.3 SPYDER integrated development environment . . . . . . 10
2.2 Working with Python . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Commenting your code . . . . . . . . . . . . . . . . . . . 10
2.2.2 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Using Python as a calculator . . . . . . . . . . . . . . . . 11
2.2.4 Use of modules to load functions . . . . . . . . . . . . . . 12
2.2.5 Variables and data types . . . . . . . . . . . . . . . . . . 14
2.2.6 Formatting of numbers using string format conversion specifiers 17
2.2.7 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.8 Combining or splitting arrays of data . . . . . . . . . . . 21
3 Python scripts 24
3.1 What is a script? . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Example script . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 Useful snippets of code . . . . . . . . . . . . . . . . . . . . . . . 27
4 Program flow control 29
4.1 Boolean tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2 The IF statement . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.3 The FOR and WHILE statements . . . . . . . . . . . . . . . . . . 30
5 Functions in Python 33
5.1 Basic functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.1 Help() function . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.2 Find() function . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.3 Mod() and floor() functions . . . . . . . . . . . . . . . . 36
5.1.4 Range functions . . . . . . . . . . . . . . . . . . . . . . 36
3
5.2 Statistical functions . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3 Meteorological and hydrological functions . . . . . . . . . . . . . 38
5.3.1 General meteorological functions: meteolib.py . . . . . . 38
5.3.2 Evaporation functions: evaplib.py . . . . . . . . . . . . . 39
5.4 Defining your own functions . . . . . . . . . . . . . . . . . . . . 39
5.4.1 Defining a named function . . . . . . . . . . . . . . . . . 39
5.4.2 Lambda functions . . . . . . . . . . . . . . . . . . . . . . 42
6 Object oriented programming in Python – Classes 43
7 File and directory manipulations 46
7.1 Directory manipulations . . . . . . . . . . . . . . . . . . . . . . 46
7.2 Working with ASCII data files . . . . . . . . . . . . . . . . . . . 48
7.2.1 Reading data from file with loadtxt() . . . . . . . . . . . . 48
7.2.2 Reading data from file with genfromtxt() . . . . . . . . . 50
7.2.3 Saving Python variables to file . . . . . . . . . . . . . . . 52
7.3 Working with NetCDF files . . . . . . . . . . . . . . . . . . . . . 53
8 Working with dates and times 54
9 Plotting data in a graph 57
9.1 Basics of plotting graphs . . . . . . . . . . . . . . . . . . . . . . 57
9.2 More complicated graphs . . . . . . . . . . . . . . . . . . . . . . 62
9.3 Interactive plotting of figures . . . . . . . . . . . . . . . . . . . . 62
9.4 Changing text fonts in a figure . . . . . . . . . . . . . . . . . . . 62
9.5 Creating subplots and a second y-axis . . . . . . . . . . . . . . . 63
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Chapter 1
Why use Python?
Students normally do most their calculations in spreadsheet programs such as MS
Excel or, if you do not have an MS Office license, the alternative open source
spreadsheet program Calc from Sun’s OpenOffice suite or the The Document Foun-
dation’s LibreOffice suite. Working with spreadsheets is easy to learn and it works
well when you deal with few data. However, suppose that you have long time series
of meteorological data, or 10 Hz data wind/temperature from a Sonic anemometer
(about 12 MB of data generated per day). In a spreadsheet program the maximum
column length would then be reached very quickly so that you would have to divide
your data over different sheets. This would be very inefficient. It is also compli-
cated to build a complex computer model in a spreadsheet to simulate hydrological
and micro-meteorological processes, such as groundwater flow, erosion, or release
of greenhouse gases. Another issue is that when you get additional data, you will
have to start copying all cells containing equations and you have to manually mod-
ify graphs that you made earlier.
Because of these limitations, professionals revert to writing dedicated computer
programs or scripts that can do the processing of data, do model calculations au-
tomatically and generate updated graphs. The most common compiled languages
for programming are the C programming language and its derivatives C++ or C#,
Java and Fortran, to name just a few. These languages are very powerful but have
the disadvantage that commands are not directly evaluated. Instead, the following
procedure has to be followed:
1. you have to write the program,
2. when you finish this, you need to compile it to create an executable,
3. you can then execute the program to get your results, but you may find some
bugs and have to start over again.
The advantages of compiled languages are that they very efficient in coding, have
little overhead and are therefore fast in execution. A disadvantage is that if you
do not have the program source code, porting the program to a different operating
system is difficult.
5
Another branch of programming languages are the interpreted languages. In-
terpreted programming languages are designed to be processed by an application
that translates your plain text instructions into actions inside the target application.
Here, your commands can be typed in and are directly evaluated by the interpreter
when you press the ENTER key, thus without the need for prior compilation. A
number of consecutive commands can be saved in an ASCII text file, which is
then called a script. Examples of interpreted languages are Matlab, Python and R.
This also means that as long as you have any of these programs installed on your
computer, you can run the script independent of the operating system.
A Matlab license is quite costly and you may not have Matlab available at
home. A good alternative is the open source Python language. Python is a very
versatile open-source programming language that can be used to process data in a
rather easy way. This document aims to teach you basic Python programming and
give you hints and examples on how to use Python in your scientific career.
Chapter 2
Basics of Python
2.1 Starting up Python
There are several distributions of Python that install the most frequently used
packages for you on a Microsoft Windows operating system machine. We use
the Python(x,y) distribution (http://www.pythonxy.com). If you want to install
Python on your computer, please visit http://python.hydrology-amsterdam.nl for
download and installation instructions. Alternatively, you can download and use
the Anaconda (https://store.continuum.io/cshop/anaconda/) or Enthought Python
(http://www.enthought.com) editions. NOTE: if you intend to use equations in your
graphs, you will need to install LATEX (see http://thesistools.hydrology-amsterdam.nl/)
on your computer as well.
If you use one of the Linux operating system distributions, such as my preferred
Debian distribution (https://www.debian.org/), Python comes pre-installed but you
may need to install additional packages such as python-matplotlib, python-pylab or
python-scipy. You can use Synaptic (as superuser, Figure 2.1) to see which python
packages are available and to install these.
There are various ways to interface and program with Python and three of these
are discussed below, appearing in order of increased complexity, flexibility and
ease of use.
2.1.1 Python through a text terminal
The most simple method to work with Python – that is if you are familiar with
Python already – is through a text terminal window. A text terminal, or text con-
sole, or terminal for short, is an interface to the computer that allows you to enter
text commands on the command line and displays output from the computer. In
Windows you open a command-line interpreter window by executing cmd.exe.
If you start up Python (or the more sophisticated IPython command shell win-
dow) from a terminal by entering Python (or ipython) at the terminal command-line
prompt, Python will start up and you will see some text identifying the version of
Python that is installed and the Python prompt (>>>), as shown in Figure 2.2.
7
Figure 2.1: Installing Python packages on a Debian OS distribution using the
Synaptic package manager.
Figure 2.2: Running Python from within a Debian Linux terminal window. Python
has started up and several commands have been entered at the Python prompt
(>>>).
Figure 2.3: IDLE shell window for working with Python.
The interfacing with Python is done in plain ASCII (American Standard Code
for Information Interchange) text that is typed in. If you use a Python terminal, a
separate ASCII text editor, such as the fancy Xemacs editor or the simple Notepad
or Wordpad text editors can be used to store your commands for later re-use.
2.1.2 IDLE integrated development environment
An easier way to interface with Python than through a terminal window is by IDLE.
IDLE is an Integrated Development Environment (IDE) for Python. This means
that it incorporates an editor for Python scripts and a command window with menus
for Python operations, such as running scripts or debugging (i.e. identifying errors
in your script). You can start Idle up from the start menu through Python(x,y), or
by typing ”idle” in a terminal command window. You will then see a shell window
similar to that in Figure 2.3. The >>> sign that you see at the start of the last line
in Figure 2.3 is called the prompt of the command line and indicates that Python is
ready to receive commands. In the following sections a short explanation of Python
programming is presented and we encourage you to try the commands out on your
computer.
Figure 2.4: Spyder integrated development environment for working with Python.
2.1.3 SPYDER integrated development environment
A more elaborate, Matlab-type of integrated development environment with ad-
vanced editing, interactive testing and debugging capabilities is provided in Python(x,y)
and is called Spyder. Spyder opens a single window (Figure 2.4) with various
menus and several screens for script editing, command interfacing, displaying your
variables, functions, etc. and a help function search field. In most cases, you may
prefer using Spyder, rather than the more austere methods discussed above, to use
Python.
2.2 Working with Python
When you started up Python you can enter commands in the shell command win-
dow and look at the output. We will be storing commands sequentially in a text file,
which is then called a Python script. You can find more about scripts in Chapter 3.
2.2.1 Commenting your code
If you insert a command in the Python shell window (Figure 2.3), you are writing
a single line of program code. Often, you will need to type in several lines of com-
mands, or statements to get the result that you need. Commenting your program,
or code, is extremely important as it allows you, or somebody else who has to work
with your code, to better understand what your code does. It is therefore very im-
portant to make a habit of explaining and commenting everything that you do in
your code!
There are two ways to include comments in program code. A one line comment
can be made using # at the beginning of the line, whereas a multi-line comment is
made using matching pairs of three apostrophes (”’ This is a multi-line comment
”’).
>>> # T h i s i s a one l i n e comment
>>> x = 3 # We a s s i g n a v a l u e o f 3 t o x
Multi-line comments are made by placing text between three single quotes (”’) in
a row:
>>> ’ ’ ’ T h i s i s a m u l t i l i n e comment
t h a t s p a n s
3 l i n e s ’ ’ ’
’ T h i s i s a m u l t i l i n e comment \n t h a t s p a n s \n 3 l i n e s ’
>>>
Assignment
Write two comment lines in the shell window. The first is a multi-line comment
and contains a text explaining the purpose of your script (make something up) and
the second line is a single line comment with the date and your name. Now open a
new text file in IDLE (open > new window) and copy these lines into the text file
and then save the file as yourname.py. This is now your script file. You can execute
or run this file by selecting the Run > run module commands that are displayed in
the top menu of your script file in Idle (see Figure 2.3), or pressing the ”run” button
in Spyder. Note that running the file with comments does not produce any output
in the shell window.
2.2.2 Operators
Python has operators that allow you to do basic calculations or work otherwise
with variables numbers or variables. The most well-known ones are the standard
mathematical operators (e.g. +, -, *, /), the ** power function operator, the %
modulus (returns remainder of division) and the \\ floor operators (returns division
result without decimal part).
Other operators are for comparison of values, such as == (equal to), != (not
equal to), logical operators such as and) and or, bitwise operators (e.g. the binary
& and | (or) operators, or membership operators (in, not in).
2.2.3 Using Python as a calculator
Python can be used as a very fancy calculator. You can do calculations with values
or with variables containing data. For instance, you could make a division by
typing:
>>> 3 . 0 / 4 . 0
0 . 7 5
>>>
We can also use variables in our calculations:
>>> a =3 .0
>>> b =4 .0
>>> a / b
0 . 7 5
>>>
Note that if we want to use scientific or mathematical functions, such as cos(),
sin() or sqrt(), we get an error message telling us that these functions are not de-
fined. In order to use these functions we need to load a module containing these
functions first (see Section 2.2.4 below).
>>> cos ( 3 0 )
T raceback ( most r e c e n t c a l l l a s t ) :
F i l e ”<c o n s o l e>” , l i n e 1 , in <module>
NameError : name ’ cos ’ i s not d e f i n e d
>>>
Assignment
Using Python as a calculator, try to calculate the following in Python. Take a good
look at the output and keep the answers in mind while you read the next sections.
s i n ( 3 . 1 4 ) # t a k e t h e s i n e o f 3 . 1 4
2∗∗4 # F our th power o f 2
2 .∗∗4 # F our th power o f 2 . 0
3+4
3.0−4
3 . 0 / 4
3 / 4 . 0
3 . / 4
3 / 4
2.2.4 Use of modules to load functions
You will have noticed from the assignment above that Python does know how to
make a division, but does not know the command sin(x. Several of the basic math-
ematical functions are defined in Python. However, more advanced functions have
often been defined in special modules that serve as function libraries. These mod-
ules are normal Python .py files that contain many function definitions and can be
opened in your editor to see the contents. Popular modules are the math with math-
ematical functions, the scipy module with scientific Python functions, the datetime
module that contains date and time functions and the Pylab module with the mat-
plotlib library of functions for plotting of data. To use a function from such a
module, you have to import the module first. There are several ways to import a
module or a function in a module using the import and from–import statements:
1. import X: imports module X with all its functions and creates a reference
to it in the current namespace. You can now use function func(), defined
in module X, by calling X.func(). Functions with the same name can be
defined in different libraries. For instance, the function sin() is defined in the
module math, but also in the SCIentific PYthon scipy module. Using import
math and import scipy would give you two options for the same function.This
is the preferred way of importing functions as you always know for sure
which implementation of the function you use. As an example for the sin(x)
function, with x in radians, as defined in the math and scipy modules we see:
>>> import math
>>> import s c i p y
>>> math . s i n ( 2 )
0.90929742682568171
>>> s c i p y . s i n ( 2 )
0.90929742682568171
2. from X import *: imports module X, and creates references in the cur-
rent name space to all public objects (i.e. functions) defined in the module,
but not to objects starting with defined in the module. You can now use
the name of the function, without the module name (which is not defined),
to call the function but the X.func() command will not work here:
>>> from math import ∗>>> cos ( 0 . 7 5 )
0.7316888688738209
>>> math . cos ( 0 . 7 5 )
T raceback ( most r e c e n t c a l l l a s t ) :
NameError : name ’ math ’ i s not d e f i n e d
Note that if func() was already defined in another module imported earlier
(for instance the scipy module), it will be overwritten and the latest imported
version will be used. This may create some confusion as two different func-
tions with the same name could potentially be available in your program.
3. from X import func1, func2, func3: This imports module X
and only provides references to func1, func2 and func3 to the user, but not
to the other functions defined in module X.
>>> from math import s in , cos , s q r t
>>> t a n ( 0 . 5 )
T raceback ( most r e c e n t c a l l l a s t ) :
NameError : name ’ t a n ’ i s not d e f i n e d
>>> from math import t a n
>>> t a n ( 0 . 5 )
0.54630248984379048
4. Y = import (’X’): this is like using import X but you can pass the
module name X to an arbitrary string (called Y here) and assign the module
to variable Y.
>>> N = i m p o r t ( ’numpy ’ )
>>> N. cos ( 0 . 7 5 )
0.7316888688738209
As said before, the preferred way is to use the first option (import X) as the other
options may create confusion, for instance when functions with the same name
(cos, sin) are defined in multiple modules (defined in the modules math, numpy –
numerical python module, scipy, etc.).
Two function libraries (modules) have been developed by M.J. Waterloo for use
in Hydrology and micro-meteorology. These are the meteolib.py module with stan-
dard meteorological functions and the evaplib.py module that has several functions
for calculating evaporation. See Section 5.3 for more on these modules.
Assignments
1. Import all functions from the the scipy module using method 2, then calculate
the square root (sqrt() function) of 2
2. Import the math module using method 1 and calculate the square root of 2.
3. Import only the function sqrt() from the numpy module and calculate the
square root of 2
4. Use method 4 to import the math module as M and calculate the square root
of 2
2.2.5 Variables and data types
Variables can be used to store data, such as numbers (1, 20.4, etc.), character strings
(’this is a text string’) or date information (’2002-03-11’, 2002, etc.). Variable
names consist of upper or lower case characters and numbers. Names cannot con-
tain spaces, commas, +, -, *, / and some other special characters. It is good practice
to use variable names that are recognisable, for instance a, b, x, y, z zero, alpha,
LAI, Lambda E, dummy 1, dummy 2, etc. are proper variable names. The lower
case characters i, j, n and m are by convention used in counters. At the prompt we
can assign values to variables a and b by typing the following commands:
>>> 3 ∗ 5
15
>>> a = 5
>>> a
5
>>>
b = 3
>>> b
3
>>> a ∗ b
15
>>>
We can look up the value of a by simply typing a at the prompt, as is done in
the above. We can also do calculations with these variables and use Python as a
calculator:
>>> c = a ∗ b
>>> c
15
>>>d = a / b
>>>d
1
>>>
In the assignment in Section 2.2.3 you have calculates 3/4 and may have no-
ticed that you got a value of 0 as the answer, rather than 0.75. Note again above
that the division of a by b gives a value of 1, instead of 1.66666666. This has to
do with how Python treats values. In the first case, the values 3 and 4 are both
integers, and the resulting division is therefore also an integer (0). If either one of
these values is a real value (float, 3., 3.0, 4., 4.0), then the division of 3.0 by 4.0
will also report a float (0.75). The type of value (integer, float, etc.) therefore is
important for the outcome of your calculations!
The same holds for the variable types. We again defined a and b as integer data
types by assigning them integer values (3, 5), instead of real or float data types
(3.0, 5.0). Multiplying or dividing two integer values with each other will return
an integer, but when one of these values is a float, the result of the calculation will
also be a float. We can define a or b as floats by adding a decimal point to the value
using statements:
>>> a = 5 . 0
>>> a
5 . 0
>>>d = a / b
>>>d
1.6666666666666667
>>>
Alternatively, you can convert an integer to a float or vice versa using the float()
and int() functions:
>>> f l o a t ( b )
3 . 0
>>>i n t ( a )
5
>>>
Integer and float types are two of the simple data types defined in Python. Other
simple data types dare the boolean bool, i.e. true or false) and complex (1+0j) data
types. The above variables only held one value and were therefore assigned simple
data types. You can see what type a variable is using the type() function:
>>> type ( b )
<type ’ f l o a t ’>
>>>
To handle complex data, such as variables holding multiple values and/or text,
Python also know a number of complex data types called tuple, string, unicode,
list, set, frozenset and dictionary. The tuple can hold a mix of values and text that
are separated by commas, and these values cannot be changed:
>>> t e s t 1 = ( 1 , 2 . 0 , 5 , (3+2 j ) , ’ t h e dog i s on t h e r o o f ’ )
>>> t e s t 1
( 1 , 2 . 0 , 5 , (3+2 j ) , ’ t h e dog i s on t h e r o o f ’ )
>>> type ( t e s t 1 )
<type ’ t u p l e ’>
>>>
The string type (defined as a ”string of characters”) only holds text:
>>> t e s t 2 = ’ t h e dog i s on t h e r o o f ’
>>> t e s t 2
’ t h e dog i s on t h e r o o f ’
>>> type ( t e s t 2 )
<type ’ s t r ’>
>>>
Another useful data type in Python is the dictionary type. This data type con-
sists of unordered key:value pairs that can for instance be used to use a kwargs
dictionary variable to pass arguments to a function. Dictionary values can be of
any type and can be duplicated in a dictionary, but the keys have to be unique and
can consist of strings, numbers or tuples. A dictionary can be made using the dict()
statement, for instance the following could be used to pass different arguments to
the genfromtxt() function to import data from a file (Section 7.2.1):
kwargs = d i c t ( d e l i m i t e r =” , ” ,\m i s s i n g v a l u e s ={” \”NAN\” ” } ,\s k i p h e a d e r =5 ,\s k i p f o o t e r =0 ,\f i l l i n g v a l u e s ={−9999} ,\c o n v e r t e r s = {0: s t r p d a t e 2 n u m ( ’\”%Y−%m−%d %H:%M:%S\” ’
) }\)
Another way is to make the dictionary using key:value pairs enclosed by a
matching pair of {}:
m u s i c r a t i n g ={ ’ John Denver ’ : 3 . 0 , \’ Frank Zappa ’ : 9 . 0 , \’ J i m i Hendr ix ’ : 9 . 5 , \’ J a n i s J o p l i n ’ : 8 . 0 , \’ Lady Gaga ’ : 6 . 0 , \’ Rumer ’ : 6 . 5 \}
>>>m u s i c r a t i n g
{ ’ J i m i Hendr ix ’ : 9 . 5 , ’ Rumer ’ : 6 . 5 , ’ Lady Gaga ’ : 6 . 0 , ’ J a n i s
J o p l i n ’ : 8 . 0 , ’ Frank Zappa ’ : 9 . 0 , ’ John Denver ’ : 3 . 0}>>> type ( m u s i c r a t i n g )
<type ’ d i c t ’>
>>>
2.2.6 Formatting of numbers using string format conversion speci-
fiers
We often do calculations with numbers. For instance, we have measured water
level H and want to use a rating curve equation (Q=CHx) to calculate discharge
Q, where C and x are the rating curve coefficients. Suppose that H = 0.35 m,
C = 0.357 and x = 1.327, we can then calculate Q:
>>> H=0.35
>>> C=0.357
>>> x =1.327
>>> Q=C∗H∗∗x
>>> p r i n t (Q)
0.088643421532
Table 2.1: String format conversion specifiers
Specifier Output format
d Signed integer decimal
i Signed integer decimal
o Unsigned octal
u Unsigned decimal (obsolete)
x Unsigned hexadecimal (lowercase)
X Unsigned hexadecimal (uppercase)
e Floating point exponential format (lowercase)
E Floating point exponential format (uppercase)
f Floating point decimal format
F Floating point decimal format
g Same as ”e” if exponent is greater than -4 or less than precision, ”f” otherwise
G Same as ”E” if exponent is greater than -4 or less than precision, ”F” otherwise
c Single character (accepts integer or single character string)
r String (converts any Python object using repr()1)
s String (converts any Python object using str())
% No argument is converted, results in a ’%’ character in the result
You will note that when printing, we get a lot of decimals for the calculated Q.
There are several ways to control the output format. To format a number for print-
ing or saving to file, we convert it to a string and we can use string format con-
version specifiers to choose between decimal notation, scientific notation, etc., and
also control how many decimals we want to print. In other ways, there are controls
to format the printing of a number. We do this by converting the number into a
string using the str() command. There are several format specifiers that control
the output as shown in Table 2.1. Several examples for formatting of the value of
Q are given below. The general syntax is the format specifier (for instance ’%.4g’
for general with four decimals) followed by a % operator and the variable Q:
>>> p r i n t ( s t r ( ’ %.4g ’ % Q) )
0 .08864
>>> p r i n t ( s t r ( ’ %.2 f ’ % Q) )
0 . 0 9
>>> p r i n t ( s t r ( ’ %.4 f ’ % Q) )
0 .0886
>>> p r i n t ( s t r ( ’ %.2 e ’ % Q) )
8 . 8 6 e−02
>>> p r i n t ( s t r ( ’ %.2E ’ % Q) )
8 . 8 6E−02
>>> p r i n t ( s t r ( ’ %.4 f ’ % Q) )
0 .0886
>>> p r i n t ( s t r ( ’ %.4 i ’ % Q) )
1repr(object) returns a string containing a printable representation of an object, i.e. the same value
yielded by conversions (surrounded by reverse quotes)
0000
>>> s t r ( ’ %0.3 f ’ % Q)
’ 0 .089 ’
>>> s t r ( ’ %10.3 f ’ % Q)
’ 0 .089 ’
The string formatting operator, % (the percent sign) uses two inputs, i.e. the string
with formatting characters ’%.2E’ and the data Q to be formatted, separated by the
percent sign, thus ’%.4i’ % Q. The number before the dot (.) in the string with
formatting characters specifies the total number of characters to be used, whereas
the number following the dot in the string with formatting characters specifies the
number of decimals (e.g. %5.1f would give a value of five characters (can be filled
by spaces for values lower than 1000) with 1 decimal ’1234.5’ or ’ 34.5’). Note
that the conversion to integer ’(%.4i’ %) of this float gives four zeros due to the .4
in the format specifier (%.4i). Note that the str() statement changes the type of the
variable to string (str) type, in this case from float to str as shown below.
>>> type (Q)
<type ’ f l o a t ’>
>>> type ( s t r ( ’ %.2 f ’ % Q) )
<type ’ s t r ’>
>>>
We can also use the str() statement end-format specifiers to format two or more
numbers at a time:
>>> p r i n t ( s t r ( ’ %0.4 f \%.2e ’ % (Q,H) ) )
0 .0886 3 . 5 0 e−01
2.2.7 Arrays
Most of our hydrological and meteorological data consist of measurements made
over time, such as water level data, sapflow data, conductivity measurements, etc.
With meteorological data collected by our Campbell 21X datalogger, we have one
row of data, such as radiation, temperature, relative humidity, air pressure, etc at
every time interval. In most cases individual data values will be of type float, for
instance a temperature of 27.82 ◦C, or net radiation of 256.9 W m−2. The various
rows and columns of data can be considered an array or matrix, where every row
represents a line of data collected at a certain date and time. This means that this
array would be of the tuple type.
Each value in an array can be accessed by its index numbers. Say that we
define the array t data as an array of 5 rows and 6 columns containing year, day,
time, temperature and temperature variance and standard deviation data:
>>> # Impor t t h e s c i p y module t o work w i t h a r r a y s
>>> from s c i p y import ∗>>> # Now d e f i n e t h e a r r a y t d a t a
>>> t d a t a = a r r a y ( [ [ 2 0 0 7 , 1 2 1 , 1 4 3 5 , 2 2 . 9 5 , . 2 8 7 , . 5 3 6 ] ,
[ 2 0 0 7 , 1 2 1 , 1 4 4 0 , 2 2 . 4 5 , . 1 8 8 , . 4 3 4 ] ,
[ 2 0 0 7 , 1 2 1 , 1 4 4 5 , 2 1 . 8 4 , . 2 0 7 , . 4 5 5 ] ,
[ 2 0 0 7 , 1 2 1 , 1 4 5 0 , 2 1 . 9 0 , . 2 6 5 , . 5 1 4 ] ,
[ 2 0 0 7 , 1 2 1 , 1 4 5 5 , 2 2 . 3 4 , . 4 7 6 , . 6 9 ] ] )
>>> t d a t a
a r r a y ( [ [ 2 .00700000 e +03 , 1.21000000 e +02 , 1.43500000 e +03 ,
2.29500000 e +01 , 2.87000000 e−01 , 5.36000000 e−01] ,
[ 2 .00700000 e +03 , 1.21000000 e +02 , 1.44000000 e +03 ,
2.24500000 e +01 , 1.88000000 e−01 , 4.34000000 e−01] ,
[ 2 .00700000 e +03 , 1.21000000 e +02 , 1.44500000 e +03 ,
2.18400000 e +01 , 2.07000000 e−01 , 4.55000000 e−01] ,
[ 2 .00700000 e +03 , 1.21000000 e +02 , 1.45000000 e +03 ,
2.19000000 e +01 , 2.65000000 e−01 , 5.14000000 e−01] ,
[ 2 .00700000 e +03 , 1.21000000 e +02 , 1.45500000 e +03 ,
2.23400000 e +01 , 4.76000000 e−01 , 6.90000000 e −01] ] )
>>>
Each value in a two-dimensional array is identified by indices [x,y], where x rep-
resents the row number and y the column number. The first row and column have
indexes of zero (0). Hence, the upper left value (2007) is t data[0,0], whereas the
lower right value (row 5, column 6, value = 0.69) is t data[4,5].
>>> t d a t a [ 0 , 0 ] # f i r s t v a l u e i n a r r a y
2007 .0
>>> t d a t a [ 4 , 5 ] # l a s t v a l u e i n a r r a y
0 . 6 9
We can select an entire column or row of data using the colon (:). Say that we
want to assign the fourth column with temperature data to the variable T , then we
would use the statement t = t data[:,3].
>>> t d a t a [ : , 3 ] # f o u r t h column i n a r r a y
a r r a y ( [ 2 2 . 9 5 , 2 2 . 4 5 , 2 1 . 8 4 , 2 1 . 9 , 2 2 . 3 4 ] )
>>> t d a t a [ 1 , : ] # Second row i n a r r a y
a r r a y ( [ 2 .00700000 e +03 , 1.21000000 e +02 , 1.44000000 e +03 ,
2.24500000 e +01 , 1.88000000 e−01 , 4.34000000 e−01])
>>> T = t d a t a [ : , 3 ] # Put t e m p e r a t u r e v a l u e s i n v a r i a b l e T
If we use a minus (-) sign in the index, we start counting from the final rows or
columns of the array. As such t data[-1,-1] gives us the last, lower right value, in
the array (0.69) and t data[:,-2] gives us the column before the last one in the array.
>>> t d a t a [ : , −1]
a r r a y ( [ 0 . 5 3 6 , 0 . 4 3 4 , 0 . 4 5 5 , 0 . 5 1 4 , 0 . 6 9 ] )
>>> t d a t a [ : , −2]
a r r a y ( [ 0 . 2 8 7 , 0 . 1 8 8 , 0 . 2 0 7 , 0 . 2 6 5 , 0 . 4 7 6 ] )
We can also place a colon (:) before an array index to make more selective
choices. If we want only the first three columns of an array, we can use t data[:,:3]
to select these:
>>> t d a t a [ : , : 3 ]
a r r a y ( [ [ 2 0 0 7 . , 1 2 1 . , 1 4 3 5 . ] ,
[ 2 0 0 7 . , 1 2 1 . , 1 4 4 0 . ] ,
[ 2 0 0 7 . , 1 2 1 . , 1 4 4 5 . ] ,
[ 2 0 0 7 . , 1 2 1 . , 1 4 5 0 . ] ,
[ 2 0 0 7 . , 1 2 1 . , 1 4 5 5 . ] ] )
>>>
Using a negative sign will provide everything except the columns/rows counted
from the end of the array. With t data having six columns, we could select only the
first column by:
>>> t d a t a [ : , : − 5 ]
a r r a y ( [ [ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ] )
>>>
Finally, we could also select everything except the first and last columns by:
>>> t d a t a [ : , 1 : −1 ]
a r r a y ( [ [ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ,
[ 2 0 0 7 . ] ] )
>>>
These tricks are of good use when providing exact index values is difficult because
the size of the array is not known, or changes during program execution.
2.2.8 Combining or splitting arrays of data
It is sometimes necessary to combine two arrays in a single array, for instance if you
have a 1-dimensional array with soil depth, and another array with soil chemical
data. The merging of two arrays is called stacking and can be done using the hstack
or vstack() instructions.
For example, if we wish to add one row to an existing matrix, i.e. stack the
matrices vertically or row on row, we can use vertical stack with the vstack() state-
ment. All arrays in the sequence must have the same shape along all but the first
axis.
The vstack() or hstack() statements take exactly one argument called tup, which
contains the matrices to be stacked. For example, if we have two matrices a and bof 3 columns, we can stack them as follows with the argument being (a,b):
>>> from s c i p y import ∗>>> a = a r r a y ( [ 0 , 0 , 0 ] )
>>> b = a r r a y ( [ [ 1 , 1 , 1 ] , [ 2 , 2 , 2 ] ] )
>>> c = v s t a c k ( ( a , b ) )
>>> c
a r r a y ( [ [ 0 , 0 , 0 ] ,
[ 1 , 1 , 1 ] ,
[ 2 , 2 , 2 ] ] )
Splitting a data array is also possible, using vsplit() or hsplit(). These com-
mands have two arguments, being the array to be slit and the indices or sections
where the split is desired.
You can use vstack() and hstack() statements to rebuild arrays divided by vs-
plit() and hsplit(), respectively.
Assignment
Make three variables a, b and c with respective values of 10.3, 5 and 137.289. Now
calculate:
• a+b+c
• a/b-c
• (b+c)/a
• 2*b+5.1/c+a*15.9
• √a
• log b
NOTE: The log and square root functions are defined in the math module or the
scipy (scientific python). To access the sqrt() and log() functions you need to load
one of these modules first by executing the following command in IDLE:
>>> from s c i p y import ∗>>>
Store these commands in the script file that you made earlier. If you run the
script file, no output will be produced. You can produce output using the print
command by specifying
p r i n t a+b+c
p r i n t s q r t ( a )
in your script. Please try this for the above calculations...
Now define a variable called dummy that contains an array with two rows of
three values each. The first row is 2.0, 5.6, 3.13 and the second row 17.3, 25.8, 1.6.
• What answers do you get when you type dummy[0,2], dummy[:,0] and dummy[1,:]
in the shell window?
• Which column is selected if you do dummy[:,-1]?
• What data do you select with dummy[:-1,:]?
• What data do you select with dummy[:,:-2]?
• What data do you select with dummy[1:-1,:]?
• Multiply the first column of dummy by 3.0.
• Make a new variable dummy2 that holds the second row of dummy divided
by 2.0
Chapter 3
Python scripts
3.1 What is a script?
Using the command line in Idle to execute statements works fine if we only have
a few commands to give, and we do not have to repeat these often. However, for
some processing tasks, we have many commands that we want to issue in sequence.
In addition, we might like to repeat our calculations on different data at a later time,
or have a web application processing data for publishing on-line. When we start
up IDLE, we open a shell window (Figure reffig:idle), in which we see a >>>prompt that Idle is ready to receive commands. To make things easier, we could
also collect a sequence of these same commands in a text file, which is then called
a script file. Such a script file serves as a kind of program and always has a name
with a .py extension by convention.
Python scripts are just text files and can therefore be created in any text edi-
tor. However, programming is made easier if the editor recognises Python state-
ments, does automatic indenting and colourises text, such as comments or flow
control text, for clarity. Here we shall use the rather simple built-in Idle editor
window, but an example of using the very versatile JEdit programmer’s text editor
(http://www.jedit.org) for editting script files for different programming languages
is shown in Figure 3.1. To open an editor window in IDLE, choose File > New.
An editor window similar to that in Figure 3.2 will then open.
3.2 Example script
An example of a simple script file listing the sequence of commands to create the
sub-plot in Figure 9.5 is given in Figure 3.2. This script can be run from the editor
by opening the examplescript.py file (Download this file for experimenting with
Python from http://python.hydrology-amsterdam.nl) and then from the menu select
Run > Run module. You will have to type show() in the Idle command window to
display the figure. You now have the basis to start programming in Python.
24
Figure 3.1: JEdit programmer’s text editor showing the formatting for a python
script file.
Figure 3.2: Idle editor window showing example script.
3.3 Debugging
The process of removing errors (bugs) in your script is called debugging. There
are two forms of errors, those that impact the syntax of the code (forgotten :, },
etc.) and those that are syntactically correct but produce wrong output. The latter
ones are usually much more difficult to find. These would include, for instance,
erroneous use of a variable name in an equation or use of wrong unit conversions
(e.g. multiply by 100 instead of 1000). Most program codes contain such errors,
which sometimes only appear under certain conditions, or are never detected at all
because the output seems reasonable.
The Python compiler will stop when an error in the program code syntax is
detected. When you run a script and an error occurs, a message will be displayed
in the Python shell window. The error may have been caused in your script, but
may also be transferred to Pylab, Math, Scipy or other function libraries that you
have imported in your script. When variables or functions have been defined in the
code that has been executed before the error occurred, these will remain available in
memory and can be therefore accessed/used by typing in commands at the prompt
in the shell window. This is very useful for debugging your script, as you can often
detect why a bug occurred by looking at the variables that have loaded.
3.4 Useful snippets of code
When you are using Python regularly, you will often be recycling snippets of code.
A few examples have been given earlier already, such as how to load an ASCII data
file and assign variables to the columns:
# Use some o p e r a t i n g s y s t e m and s c i p y r o u t i n e s
from os import c h d i r , getcwd , l i s t d i r
from s c i p y import l o a d t x t
# A s s i g n work ing d i r e c t o r y t o v a r i a b l e w o r k d i r
w o r k d i r = ’ i : / s v a e m e t e o / p y s c r i p t s ’
# Load data from d a t a f i l e . t x t i n v a r i a b l e da ta
d a t a = l o a d t x t ( d a t a f i l e . t x t , comments= ’ # ’ , d e l i m i t e r = ’\ t ’ )
# A s s i g n f i r s t column t o v a r i a b l e da ta1
d a t a 1 = d a t a [ : , 0 ]
You can use the Run > Run module menu entries in the Idle editor to run
a script. Alternatively, within the Idle shell window, or any other Python shell
window, a script can be executed using the following command:
>>> e x e c f i l e ( ” s c r i p t . py ” )
A script file can also be executed independently from Idle or Python using the
following command in a terminal window:
watm@pc06−091:˜\ $ python s c r i p t . py
Chapter 4
Program flow control
4.1 Boolean tests
Suppose that we want to know which values in the t data array defined in Sec-
tion 2.2.7 are non-temperatures. We know that temperatures will not become
higher than 50 ◦C, hence we could say that values over 50 ◦C would not be true
temperatures, but erroneous values that would need correction. The conditional
statement t data> 50 would then indicate these values as Boolean types, i.e. True
for those that fulfil the condition and and False for those that do not satisfy the
condition. Below some examples of conditional statements.
>>> # S e t v a r i a b l e a t o 1
>>> a=1
>>> # Check i f a e q u a l s 1
>>> a==1
True
>>> a==2
F a l s e
>>> # Now check t h e t d a t a a r r a y f o r da ta l a r g e r than 50
>>> t d a t a > 5 0 . 0
>>> a r r a y ( [ [ True , True , True , F a l se , F a l se , F a l s e ] ,
[ True , True , True , F a l se , F a l se , F a l s e ] ,
[ True , True , True , F a l se , F a l se , F a l s e ] ,
[ True , True , True , F a l se , F a l se , F a l s e ] ,
[ True , True , True , F a l se , F a l se , F a l s e ] ] , d t y p e = bool )
>>>
The following statements use conditional statements to execute some part of
code depending on whether the condition is True or False. Note that the conditional
statement is followed by a colon (:) and that the code to be executed needs to
be indented. The statement continues during indentation and only ends when the
indentation is stopped.
29
4.2 The IF statement
If statements are useful to perform an action based on whether a condition is met
or not. For instance, the code below sets a solar radiation Rs variable at -10.0 W
m−2 and then tests if Rs < 0 W m−2. If the condition is met (true, Rs < 0) it will
set Rs = 0.0 and print a short text and the value of Rs. If not true, it prints ”Solar
radiation larger than or equal to zero.”:
>>> Rs = −10.0
>>> i f Rs < 0 :
Rs = 0 . 0
p r i n t ’ N i g h t t i m e s o l a r r a d i a t i o n = ’ , Rs
e l s e :
p r i n t ’ S o l a r r a d i a t i o n l a r g e r t h a n or e q u a l t o z e r o . ’
Night ime s o l a r r a d i a t i o n = 0 . 0
>>>
Note that the conditional statement Rs<0 is ended with a colon (:). You could
also include elif : statements between the if and else statements to make further
distinctions, for instance for very bright sunlight conditions:
>>> Rs = −10.0
>>> i f Rs < 0 :
Rs = 0 . 0
p r i n t ’ N i g h t t i m e s o l a r r a d i a t i o n = ’ , Rs
e l i f Rs > 1000:
p r i n t ’ Very b r i g h t s u n l i g h t ! ’
e l s e :
p r i n t ’ S o l a r r a d i a t i o n l a r g e r t h a n or e q u a l t o z e r o . ’
Night ime s o l a r r a d i a t i o n = 0 . 0
>>>
4.3 The FOR and WHILE statements
The for or while statements are used to loop through some code. The for loop is a
looping statement that iterates over a sequence of objects. We can create a set of
sequential data using the range() function.
>>> help ( range )
Help on b u i l t −in f u n c t i o n range in module b u i l t i n :
range ( . . . )
range ( [ s t a r t , ] s t o p [ , s t e p ] ) −> l i s t of i n t e g e r s
Re turn a l i s t c o n t a i n i n g an a r i t h m e t i c p r o g r e s s i o n
of i n t e g e r s . range ( i , j ) r e t u r n s [ i , i +1 , i +2 , . . . ,
j −1]; s t a r t ( ! ) d e f a u l t s t o 0 . When s t e p i s given ,
i t s p e c i f i e s t h e i n c r e m e n t ( or decrement ) . For
example , range ( 4 ) r e t u r n s [ 0 , 1 , 2 , 3 ] . The end
p o i n t i s o m i t t e d ! These a r e e x a c t l y t h e v a l i d
i n d i c e s f o r a l i s t of 4 e l e m e n t s .
>>> range ( 5 )
[ 0 , 1 , 2 , 3 , 4 ]
>>>
Now let’s use this in a for loop, the else statement is optional and is executed when
the i becomes outside the defined range:
>>> f o r i in range ( 1 , 1 0 , 2 ) :
p r i n t ’ i = ’ , i
e l s e :
p r i n t ’We r e a c h e d t h e end of t h e loop ’
i = 1
i = 3
i = 5
i = 7
i = 9
We r e a c h e d t h e end of t h e loop
>>>
Suppose that we do not want to print these values but store them in a list, together
with a second value that is twice as large as i. We can define a list data type variable
with the statement testlist = []. We would then have to append each line of new
values created in the loop and temporarily stored in the ”dummy” variable to the
list of previous values stored in the variable testlist using a testlist.append(dummy)
instruction. The resulting list can then be converted to an array data type using the
array() function. This can be done as follows:
>>> from s c i p y import ∗>>> t e s t l i s t = [ ] # D e f i n e a v a r i a b l e t o s t o r e your v a l u e s
>>> f o r i in range ( 1 , 10 , 2 ) :
j = 2 . 0 ∗ i
dummy = i , j
# append t h e dummy v a r i a b l e t o p r e v i o u s v a l u e s
t e s t l i s t . append (dummy )
>>> t e s t l i s t
[ ( 1 , 2 . 0 ) , ( 3 , 6 . 0 ) , ( 5 , 1 0 . 0 ) , ( 7 , 1 4 . 0 ) , ( 9 , 1 8 . 0 ) ]
>>> t e s t l i s t = a r r a y ( t e s t l i s t ) # c o n v e r t l i s t o f v a l u e s t o a r r a y
>>> t e s t l i s t
a r r a y ( [ [ 1 . , 2 . ] ,
[ 3 . , 6 . ] ,
[ 5 . , 1 0 . ] ,
[ 7 . , 1 4 . ] ,
[ 9 . , 1 8 . ] ] )
>>>
This procedure is very useful for calculating daily averages, sums, standard devia-
tions, etc.
The while is another looping statement that repeatedly executes statements as
long as a condition is true. It can also have an optional ELSE clause. In the
following example we assign a value of 5 to a. In the while statement we increase
a with one and print out the text a is lower than 10 every time an iteration is made.
As you can see below, the loop stops when a reaches 10.
>>> a = 5 # S e t a t o 5
>>> wh i l e a < 1 0 :
p r i n t a , ’ i s l ower t h a n 10 ’
a = a + 1 # I n c r e a s e a by 1
5 i s l ower t h a n 10
6 i s l ower t h a n 10
7 i s l ower t h a n 10
8 i s l ower t h a n 10
9 i s l ower t h a n 10
>>>
The break and continue statements also allow looping but will not be discussed
here.
Chapter 5
Functions in Python
We have already been introduced to the float(), int(), type(), sqrt() and log() func-
tions in the above, but there are many more (predefined) functions in Python. Be-
low are some examples.
5.1 Basic functions
5.1.1 Help() function
An important function is the help() function, that provides you information about
a function. Let’s take the pow() function as an example:
>>> help ( pow )
Help on b u i l t −in f u n c t i o n pow in module b u i l t i n :
pow ( . . . )
pow ( x , y [ , z ] ) −> number
With two arguments , e q u i v a l e n t t o x∗∗y . With t h r e e
arguments , e q u i v a l e n t t o ( x∗∗y ) \% z , b u t may be
more e f f i c i e n t ( e . g . f o r l o n g s ) .
>>> pow ( 3 , 2 )
9
>>> 3∗∗2 .0
9 . 0
>>>
This tells you that pow(x,y) will give you x to the power of y (as does x ∗ ∗y)
and that the function is built into Python and in this case returns an integer (9). As
mentioned earlier a lot of functions are defined in function libraries, such as in the
math module (sin(), cos(), tan(), pow() and sqrt() functions, pi, e constants, etc.).
A function library is called a module. The pow() function is also defined in the
33
Figure 5.1: Documentation screen in Spyder that provides explanation of functions
in Python.
module math, that has to be loaded before you can use the power function using
from math import pow, or load all functions in math by using from math import *.
>>> from math import pow
>>> help ( pow )
Help on b u i l t −in f u n c t i o n pow in module math :
pow ( . . . )
pow ( x , y )
Re turn x∗∗y ( x t o t h e power of y ) .
>>> pow ( 3 , 2 )
9 . 0
>>>
You can see which functions the math module holds by typing in help(math).
In Spyder, there is a separate screen with a tab labelled doc (Figure 5.1) for
documentation, that provides documentation for functions. In addition, next to the
tools menu entry a question mark (?) provides access to addition documentation of
several important Python modules.
Assignment
Consult help for the functions sqrt, log, rand, len and math and write down what
each of these functions does.
5.1.2 Find() function
The find() function is defined in the pylab module, which should be imported be-
fore you can use this function. The find() function returns the indices from an
array where the condition that you use in the function is TRUE. It will do so in
an 1-dimensional array. Remember that we assigned temperature to the variable Tbefore. Suppose that we want to know when the temperature is higher than 22.0◦C, we could then issue the following commands:
>>> T
a r r a y ( [ 2 2 . 9 5 , 2 2 . 4 5 , 2 1 . 8 4 , 2 1 . 9 , 2 2 . 3 4 ] )
>>> Thigh = f i n d (T > 2 2 . 0 )
>>> Thigh
a r r a y ( [ 0 , 1 , 4 ] )
>>>
This shows you that T[0], T[1] and T[4] satisfy the condition of being larger than
22.0 ◦C. Suppose that we only want these values to be converted to K. We would
then do:
>>> T
a r r a y ( [ 2 2 . 9 5 , 2 2 . 4 5 , 2 1 . 8 4 , 2 1 . 9 , 2 2 . 3 4 ] )
>>> T k e l v i n = T [ f i n d ( T > 2 2 . 0 ) ] + 273 .16
>>> T k e l v i n
a r r a y ( [ 296 .1 , 295 .6 , 2 9 5 . 4 9 ] )
>>>
The above shows that we can use the find() function within other functions, which
is very useful.
An alternative to find() is to use a condition as an array index. If we want to
find the temperatures after, for instance, 14:45 h, we could do:
>>> # Find t e m p e r a t u r e s f o r t i m e a f t e r 14:45 h
>>> t d a t a [ t d a t a [ : , 2 ] >1 4 4 5 , 3 ]
a r r a y ( [ 2 1 . 9 , 2 2 . 3 4 ] )
>>>
This is often more elegant than using the find() function.
An application would be the following. Suppose that you have measured so-
lar radiation (short-wave) at half-hourly intervals over several days and that you
know that these values should be zero at night, but are sometimes small negative
or positive numbers instead. If you would like to set values of Rs smaller than 5
W m−1 to zero, you could do the following, where the Rs variable holds an array
of half-hourly solar radiation values:
>>> Rs [ Rs<5]=0
or
>>> Rs [ f i n d ( Rs<5) ]=0
5.1.3 Mod() and floor() functions
Sometimes there is the need to separate the decimal part from the integer part of
a number. One such example is the conversion of data logger time, stored as year,
day of year and hour (2008, 157, 1430), to a decimal time that is needed for plotting
of the data.
The floor() function in the math module will return the integer part of a number,
i.e. the largest integer value less than or equal to that number, whereas the mod()
in the scipy module returns the modulo, i.e. the decimal part. The mod() function
takes two arguments, the number (n) and a divisor (d). Mod(n,d) returns the dif-
ference between n and the largest multiple of d smaller than n. With a divisor dof 1, mod(n,1) will return the decimal part of the number n. You can also use the
modulo operator (%) instead of the mod() function. :
>>> from math import f l o o r
>>> from s c i p y import mod
>>> f l o o r ( 3 . 5 )
3 . 0
>>> mod ( 3 . 5 , 1 )
0 . 5
>>> 3 . 5 % 1
0 . 5
>>>
5.1.4 Range functions
Ranges of values can be created using the range(), arange() and the linspace()
functions. The range() function creates a list type, whereas the arange() function
creates an array of integers and linspace() function an array of float values:
>>> x=range ( 0 , 10)
>>> x
[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ]
>>> type ( x )
<type ’ l i s t ’>
>>> y = a r a n g e ( 0 , 10)
>>> y
a r r a y ( [ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] )
>>> type ( y )
<type ’ numpy . n d a r r a y ’>
>>> z = l i n s p a c e ( 1 , 6 , 5 )
>>> z
a r r a y ( [ 1 . , 2 . 2 5 , 3 . 5 , 4 . 7 5 , 6 . ] )
>>> type ( z )
<type ’numpy . n d a r r a y ’>
>>>
You can also use a third, additional argument in the range() and arange() functions
to influence the steps:
>>> x= a r a n g e ( 0 , 10 , 2 )
>>> x
a r r a y ( [ 0 , 2 , 4 , 6 , 8 ] )
>>>
5.2 Statistical functions
We often need to describe the statistics of our dataset. We can use the average(),
std(), sum() max() and min() functions to get the average, standard deviation, sum,
maximum and minimum values, respectively. These functions are defined in the
module scipy so you have to import this module first. Let’s do this for the variable
T again.
>>> from s c i p y import ∗>>> a v e r a g e (T )
22.295999999999999
>>> s t d ( T )
0.40450463532572778
>>> sum ( T )
111.47999999999999
>>> max ( T )
22.949999999999999
>>> min ( T )
21 .84
>>>
It is also possible to do linear regression using the polyfit() function from the
scipy module. The polyfit() function is based on least-square analysis and takes
three arguments, x-data, y-data and a number n describing the polynomial order
(1 = linear regression, 2 = quadratic, 3 = cubic, etc.). In the linear case (y =ax + b) it returns two coefficients, the slope a and the intercept b. You call the
function for linear analyses as: a,b = polyfit(xdata,ydata,1). This function will not
provide you with any further statistics. If you need such detailed statistics (r2, F-
test, etc.) then you could download and use the module ols.py, which you can get
at http://www.scipy.org/Cookbook/OLS.
5.3 Meteorological and hydrological functions
We have defined several meteorological functions and evaporation formulae in
the meteolib.py and evaplib.py modules. You can download these modules from
http://python.hydrology-amsterdam.nl/.
5.3.1 General meteorological functions: meteolib.py
The following functions are defined in meteolib.py:
• date2doy(): converts a date (dd, mm, yyyy) to a ”day of year” (1-366) value.
• sun NR(): calculates daylength and extraterrestrial radiation input, used for
calculating evaporation
• es calc(): Calculate saturation vapour pressures
• ea calc(): Calculate actual vapour pressures
• vpd calc(): Calculate vapour pressure deficits
• Delta calc(): Calculate slope of vapour pressure curve
• L calc(): Calculate latent heat of vapourisation
• cp calc(): Calculate specific heat
• gamma calc(): Calculate psychrometric constant
• rho calc(): Calculate air density
• pottemp(): Calculate potential temperature (1000 hPa reference pressure)
The latent heat of vapourisation (λ) depends only on temperature. To calculate λfrom temperature variable T we would do:
>>> from m e t e o l i b import ∗>>> L = L c a l c (T )
>>> L
a r r a y ( [ 2445736.025975 , 2446919.4761 , 2448363 .2852525 ,
2448221 .2712375 , 2447179 .8 3 5 1 2 75 ] )
>>>
Use the help() function to get information about the other functions in the meteolib
module.
5.3.2 Evaporation functions: evaplib.py
A second library, evaplib.py was developped by M.J. Waterloo and contains func-
tions to calculate daily evaporation rates according to the Penman open water
evaporation [E0(); Penman, 1948, 1956, 1963, Valiantzas, 2006], the FAO Penman
Monteith reference evaporation [ET0pm(); Allen et al., 1998], the Dutch Makkink
reference crop evaporation [Em(); de Bruin, 1987] and the Priestley Taylor evapo-
ration [Ept(); Priestley and Taylor, 1972]. The following statement calculates the
Priestley Taylor evaporation for a temperature of 21.65 ◦C, a relative humidity of
67%, air pressure of 101300 Pa, net radiation input of 18200000 J m−2 day−1 and
a soil heat flux of 600000 J m−2 day−1 .
>>> from e v a p l i b import ∗>>> Ept ( 2 1 . 6 5 , 6 7 . 0 , 1 0 1 3 0 0 , 1 8 2 0 0 0 0 0 , 6 0 0 0 0 0 )
6.3494561161280778
>>>
The evaplib.py library can be downloaded from ecohydro.falw.vu.nl.
5.4 Defining your own functions
If you need to execute several statements or calculations more than once, it may be
a good idea to define your own function. You can define two types of functions in
Python. The first type is named and is defined using the def – return keywords. You
can store these function scripts in a function module from which you can import
these for repeated use within a second script. The second type are anonymous
functions, called lambda functions. Both function types will be discussed below.
5.4.1 Defining a named function
The definition of a traditional named function starts with the def statement fol-
lowed by the function name and, optionally between parenthesis, a list of its argu-
ments called the function argument list. The arguments are any input values that
you want to pass on to the function. For example, lets define a function linreser-
voir(). The discharge recession can often be described by the following equation:
Qt = Q0e−
1
kt (5.1)
Now suppose that you want to calculate Qt, for which you would need Q0, k and
t as the functions arguments. You could then define the function linreservoir()
as follows, where we import the exponential function from the scipy within the
function and we state in the function argument list that Q0 and k are of type float:
d ef l i n r e s e r v o i r ( q0 = f l o a t ( ) , k = f l o a t ( ) , t ) :
from s c i p y import exp # i m p o r t e x p o n e n t i a l f u n c t i o n
q t = q0 ∗ exp(−1 / k ∗ t )
re tu rn q t
Note that the function statements are all indented and that the final statement (return
qt) provides output of the function. If we save this function in a script file called
linres.py, we can import the function linreservoir() as follows:
>>> from l i n r e s import l i n r e s e r v o i r
>>> l i n r e s e r v o i r ( 1 0 0 . 0 , 3 0 . 0 , 67)
10.717059875230674
>>>
The above works only with single values as input. If you want to work with arrays
of time values as input, you can specify this in the function argument list:
# i m p o r t modules
from s c i p y import a r r a y
d ef l i n r e s e r v o i r ( q0= f l o a t ( ) , k= f l o a t ( ) , t = a r r a y ( [ ] ) ) :
from s c i p y import exp
q t = q0 ∗ exp(−1 / k ∗ t )
re tu rn q t
If we now use the function with t as an array, we als get an array of qt as output:
>>> t = a r r a y ( [ 1 , 2 , 3 , 4 ] )
>>> t
a r r a y ( [ 1 , 2 , 3 , 4 ] )
>>> q = l i n r e s e r v o i r ( 1 0 0 . 0 , 5 0 . 0 , t )
>>> q
a r r a y ( [ 98 .01986733 , 96 .07894392 , 94 .17645336 , 9 2 . 3 1 1 6 3 4 6 4 ] )
>>>
It is always very important to document your programs well so that other people
may use or modify them. The first thing that springs to mind is adding a help
entry text in your function. This means that if you would type help(linreservoir)
it would show you how to use the function and what its inputs and outputs are.
Adding a help entry is easy. Just insert the help text as a multi-line comment after
the function definition, as shown by the example below.
# i m p o r t modules
from s c i p y import a r r a y
d ef l i n r e s e r v o i r ( f l o a t ( q0 ) , f l o a t ( k ) , t = a r r a y ( [ ] ) ) :
’ ’ ’ The f u n c t i o n l i n r e s e r v o i r ( ) c a l c u l a t e s t h e
d i s c h a r g e a t t ime t from an i n i t i a l
d i s c h a r g e Q( t =0) and a r e c e s s i o n
c o n s t a n t
I n p u t :
−−−−−−−−−t : ( a r r a y of ) t ime
q0 : i n i t i a l d i s c h a r g e v a l u e
k : r e c e s s i o n c o n s t a n
Outpu t :
−−−−−−−−−q t : ( a r r a y of ) d i s c h a r g e v a l u e s a t t ime t
Example :
−−−−−−−−−>>> l i n r e s e r v o i r ( 1 5 . 0 , 3 0 . 0 , 20 )
2.6359713811572676
’ ’ ’
# Impor t r e l e v a n t Python modules
from s c i p y import exp
# Do t h e c a l c u l a t i o n s
q t = q0 ∗ exp(−1 / k ∗ t )
re tu rn q t
If we now invoke the help function (e.g. help(linreservoir)) we get the following
output with our help text:
>>> help ( l i n r e s e r v o i r )
Help on f u n c t i o n l i n r e s e r v o i r in module m a i n :
l i n r e s e r v o i r ( q0 = 0 . 0 , k = 0 . 0 , t = a r r a y ( [ ] , d t y p e = f l o a t 6 4 ) )
The f u n c t i o n l i n r e s e r v o i r ( ) c a l c u l a t e s t h e
d i s c h a r g e a t t ime t from an i n i t i a l
d i s c h a r g e Q( t =0) and a r e c e s s i o n c o n s t a n t
I n p u t :
−−−−−−−−−t : t ime
q0 : i n i t i a l d i s c h a r g e v a l u e
k : r e c e s s i o n c o n s t a n t
Outpu t :
−−−−−−−−−q t : d i s c h a r g e a t t ime t
Examples :
−−−−−−−−−>>> l i n r e s e r v o i r ( 1 5 . 0 , 3 0 . 0 , 20 )
7.70125678548888
>>>
Many meteorological functions are defined like this in the meteolib and evaplib
modules (see Sections 5.3.1 and 5.3.2).
5.4.2 Lambda functions
In contrast to the named functions, lambda functions are anonymous functions that
are not attached to a name during runtime of your script. They are defined as
follows using the lambda construct where the function outputs the argument (x) to
the power 3:
>>> power3 = lambda x : x∗∗3
>>> power3
<f u n c t i o n <lambda> a t 0 xb9f0ca4>
>>> power3 ( 3 )
27
>>>
You can use more than one argument in a lambda function:
>>> powerxy = lambda x , y : x∗∗3 + y∗∗2
>>> powerxy ( 3 , 4 )
43
>>>
These lambda functions can be part of named functions.
Chapter 6
Object oriented programming in
Python – Classes
In the previous chapters you have learned the basics of Python. This included data
types, use of variables and function definitions that combined in a script form a
program that can be used to perform certain actions. In many computer models,
parts of code are implemented with minor changes at different places in the pro-
gram. This implies that if such a part of code is changed or used differently, the
change in function code should be implemented in different places in the program,
making it subject to errors. It is much more efficient to develop code as an object
in one place, that serves as a recipe use elsewhere in the program. A computer
language that can do this, such as Python, is called an object-oriented language.
For instance, if you want to develop a program that defines functions for soils,
you have to deal with differences between soil types, such as for example course
grained, clay and peat soils. If you have very simple functions you could use a
dictionary data type (Section 2.2.5) to achieve this:
>>> s o i l t y p e = { ’ c o u r s e ’ : ’ e x e c u t e program code f o r c o u r s e s o i l s ’ ,\’ c l a y ’ : ’ e x e c u t e program code f o r c l a y s o i l s ’ ,\’ p e a t ’ : ’ e x e c u t e program code f o r p e a t s o i l s ’ ,\}
>>> s o i l t y p e . g e t ( ’ p e a t ’ , ’ s o i l t y p e unknown ’ )
’ e x e c u t e program code f o r p e a t s o i l s ’
>>> s o i l t y p e . g e t ( ’ s i l t ’ , ’ s o i l t y p e unknown ’ )
’ s o i l t y p e unknown ’
>>>
You could also achieve the same using if – elif –else statements (Section 4.2)
that would select the appropriate function code:
>>> s o i l = ’ sand ’
>>> i f s o i l == ’ c o u r s e ’ :
p r i n t ’ e x e c u t e f u n c t i o n code f o r c o u r s e s o i l s ’
e l i f s o i l == ’ c l a y ’ :
43
p r i n t ’ e x e c u t e f u n c t i o n code f o r c l a y s o i l s ’
e l i f s o i l == ’ p e a t ’ :
p r i n t ’ e x e c u t e f u n c t i o n code f o r p e a t s o i l s ’
e l s e :
p r i n t ’ unknown s o i l t y p e ’
unknown s o i l type
>>>
It is very likely that the functions used for these different soils show similarity and
you therefore have to edit three functions if you make modifications.
Object oriented programming makes it possible to develop a kind of general
recipe or prototype for a set of functions that can have several instances with dif-
ferent functioning, i.e. for each of the soil types, in your program. This recipe can
contain constants, variables and functions. The latter are then called the methods
of the object.
A prototype of an object can be created in Python by defining a Class type.
Such a class type contains variables, properties and methods that characterize the
objects of the class. The attributes of a class are called data members and meth-
ods that can be accessed through a . (dot) notation. For instance if a class is
constructed with the name of address, address.street might show the street name
variable, whereas address.city would yield the name of the city.
Below is a simple program using a soil() class with two functions for calculat-
ing arbitrary soil values K and H:
’ ’ ’ T h i s i s a c l a s s f o r h a n d l i n g s o i l d a t a
An i n s t a n c e of t h i s c l a s s can be c r e a t e d by c a l l i n g s o i l (C) ,
where C i s a c o n s t a n t used i n t h e c a l c u l a t i o n s . C v a r i e s
between s o i l t y p e s .
W r i t t e n by M. J . W ate r loo as an example .
’ ’ ’
c l a s s s o i l ( o b j e c t ) :
# D e f i n e an i n i t i a l v a r i a b l e t h a t i s r e l a t e d t o each c l a s s
# i n s t a n c e
d ef i n i t ( s e l f , C) :
# Get a c o n s t a n t C f o r each c l a s s i n s t a n c e c r e a t e d
s e l f . C = C
d ef calcK ( s e l f , t h e t a ) :
# C a l c u l a t e a K v a r i a b l e a c c o r d i n g t o m o i s t u r e s t a t u s
# t h e t a (0−1) and t h e c o n s t a n t C
K = 1 / t h e t a ∗ s e l f . C
re tu rn K
d ef calcH ( s e l f , t h e t a ) :
# C a l c u l a t e water p r e s s u r e based on s o i l m o i s t u r e t h e t a
H = s e l f . C / t h e t a ∗ 100
re tu rn H
# Make s e v e r a l i n s t a n c e s o f t h e c l a s s f o r each s o i l t ype , n o t e
# t h a t C v a r i e s
c l a y = s o i l ( 0 . 0 5 )
sand = s o i l ( 0 . 7 )
# C a l c u l a t e s o i l p r o p e r t i e s K and H
k c l a y = c l a y . calcK ( 0 . 4 )
ksand = sand . calcK ( 0 . 4 )
p r i n t ’K f o r c l a y a t t h e t a = 0 . 4 i s : ’ , k c l a y
p r i n t ’K f o r sand a t t h e t a = 0 . 4 i s : ’ , ksand
In the above first a soil() class and functions in the class are defined, where def
init (self,C): is done when the instance of the soil class is initiated. In this case
C varies between soil types and is used to calculate K and H based on a θ input
variable. Then an instance of soil is made for clay with C = 0.05 by calling clay
= soil(0.05) (and also for sand with a different C value) and K is calculated for
θ = 0.4. The same is done for sand. When we run the script we get:
>>>
K f o r c l a y a t t h e t a = 0 . 4 i s : 0 . 125
K f o r sand a t t h e t a = 0 . 4 i s : 1 . 7 5
>>>
Note that the outcome differs between soil types for the same value of θ. We could
now easily add an instance for peat by including:
p e a t = s o i l ( 0 . 1 )
We then have three instances, one for each soil type, that we can call to calculate
corresponding K values.
Chapter 7
File and directory manipulations
7.1 Directory manipulations
When you start you have to create a directory to work in. Please create a directory
called svae python on one of the drives of your computer. Now start Python by
opening Idle. To change change your working directory you have to import the OS
module first. The OS (Miscellaneous operating system interfaces) module contains
functions that pertain to commands for your operating system, such as changing
directories, making directories, listing files, etc. Some examples are given below.
To import this module type on the command prompt:
>>> import os
>>>
Now you can use the os.getcwd() function to see where you are (/home/watm is my
home directory under unix):
>>> os . getcwd ( )
’ / home / watm ’
>>>
You can change to another directory using the os.chdir() function. To change to an-
other directory use the os.chdir() command, for instance type os.chdir(’C:\svae python’)
in MS Windows, or if you work under unix os.chdir(’/home/watm/svae python’).
It is often easy to define a variable, for instance called ”dataroot”, holding the name
of your working directory. For unix:
>>> d a t a r o o t = ’ / home / watm / s v a e p y t h o n ’
>>> os . c h d i r ( d a t a r o o t )
>>> os . getcwd ( )
’ / home / watm / s v a e p y t h o n ’
>>>
46
or in MS windows:
>>> d a t a r o o t = ’ c :\ s v a e p y t h o n ’
>>> os . c h d i r ( d a t a r o o t )
>>> os . getcwd ( )
’ c :\ s v a e p y t h o n ’
>>>
To list all files in the current working directory use os.listdir(os.getcwd()):
>>> os . l i s t d i r ( os . getcwd ( ) )
[ ’ m e t e o l i b . py ’ , ’ meteo2008 . t x t ’ , ’ m e t e o s c r i p t . py ’ ]
>>>
If you have a long directory path and file name, such as /home/watm/svae python/data.txt
the os.path module provides possibilities to split or join these paths and file names
or file name extensions. For instance, if you have the variable /home/watm/s-
vae python/data.txt and you want to separate the filename from the directory path,
you can do:
>>> import os
>>> pathname , f i l e n a m e = os . p a t h . s p l i t ( ’ / home / watm / s v a e p y t h o n / d a t a .
t x t ’ )
>>> pathname
’ / home / watm / s v a e p y t h o n ’
>>> f i l e n a m e
’ d a t a . t x t ’
>>>
If you want to split the path (/home/watm/svae python), file name (data) and its
extension you could use os.path.splitext()
>>>p a t h f i l e n a m e , extname=os . p a t h . s p l i t e x t ( ’ / home / watm / s v a e p y t h o n /
d a t a . t x t ’ )
>>> p a t h f i l e n a m e
’ / home / watm / s v a e p y t h o n / d a t a ’
>>> extname
’ . t x t ’
>>>
These commands are very useful for in case you want to place files in a different
directory than where these were originally, or to change file names.
Assignment
Based on the above, add a few lines of program code to your personal script file
that sets your working directory and gives a list of the files in this directory.
7.2 Working with ASCII data files
7.2.1 Reading data from file with loadtxt()
Often we get our data from automatic data registration units, or data loggers. These
data loggers store the values from the instruments attached to them in an ACSCII
or text file. The values in this file are separated by a delimiter, which usually is
a space ( ), a tab ( ) or a comma (,). The latter file is often called a comma
separated values file and has .csv as its file name extension. Comments in such
files are often preceded by the pound (#) symbol. Output for a VU water level data
logger is, for example:
# DOY day month year hr min s e c H
124 04 05 09 10 15 30 +00002
124 04 05 09 10 30 30 −00005
124 04 05 09 10 45 30 −00010
124 04 05 09 11 00 30 −00020
The data on the first line represents the day of year (124, doy), day (04), month
(05), year (09), hour (10), minutes (15), seconds (30) and the water level in 1/100of a cm (+00002). Note that all values are separated by spaces.
If these data were stored in a file called wl.txt, we can read these into an array
variable (here called wldata) using the loadtxt() function that is defined in the scipy
(Scientific Python) or pylab modules:
>>> from s c i p y import l o a d t x t
>>> w l d a t a = l o a d t x t ( ’ wl . t x t ’ , comments= ’ # ’ , d e l i m i t e r = ’ ’ )
>>>
Note that you cannot have missing values in the file, so every column must have
the same length and you should not have empty lines at the end of the file. If you
have missing data in one or more columns, you could fill these locations first with
-9999, a value commonly used to indicate missing data. Now we have an array in
which the different columns are stored. The first header lines (#) was skipped due
to the comments=’#’ argument in the loadtxt() function. The first column of the
array (wldata[:,0]) holds the DOY, and the last column (wldata[:,7]) the water level
data. You can now perform calculations with the data:
>>> wl = [ ]
>>> wl = w l d a t a [ : , 7 ] / 1 0 0 0 0 . 0
The water level is now stored in the variable wl and its unit is now m, rather than
in tenths of a mm.
Assignment
1. Use the loadtxt() function (defined in the pylab or scipy modules) to read
the file wl.txt (available at http://ecohydro.falw.vu.nl/python) and place the
data in a variable named wldata. Note that the first line in wl.txt contains a
comment with the header for each column.
2. Now assign each column in wldata to individual variables as defined in the
header, i.e. JD, DD, MM, etc.
3. Calculate the average value of your variable H (the water level)
4. Select all H values after day 160
5. The actual water level H was 0.31 m at the beginning of the measurements
when the logger H was set to zero and the logger H values are in 0.1mm units.
Correct H so that it represents the water level in m.
The Campbell Scientific CR1000 data logger that we use for our meteorologi-
cal measurements stores data differently than the VU water level data logger. Here
is an example of the data output of the logger:
”TOA5” , ” MyDatalogger ” , ”CR1000” , ” E3686 ” , ”CR1000 . S td . 2 7 ” , ”CPU:
c r 1 0 0 0 m e t e r o l o g y v 1 . 1 2 P o r t u g a l 2 0 1 4 . CR1” , ” 28966 ” , ” f a s t t a b l e
”
”TIMESTAMP” , ”RECORD” , ”AvgTCa” , ” s tdTCa ”
”TS” , ”RN” , ”Deg C” , ”Deg C”
” ” , ” ” , ”Avg” , ” S td ”
”2014−06−05 1 5 : 0 5 : 0 0 ” , 0 1 , ”NAN” , ”NAN”
”2014−06−05 1 5 : 1 0 : 0 0 ” , 0 2 , ”NAN” , ”NAN”
”2014−06−05 1 5 : 1 5 : 0 0 ” , 0 3 , 2 7 . 0 1 , 1 . 7 9 8
”2014−06−05 1 5 : 2 0 : 0 0 ” , 0 4 , 2 4 . 2 5 , 0 . 9 8
We see that it is a comma separated values (csv) file, the first few lines iden-
tify the logger and the meaning of the data in the four columns (”TIMESTAMP”,
”RECORD”, ”AvgTCa”, ”stdTCa”), their units, etc., followed by data lines, which
all start with a date string (”2014-06-05 15:05:00”), float values (27.01) and in
some cases ”NAN”, which is the abbreviation for Not A Number for when the
sensor had not yet been connected.
If you use the loadtxt() function as above, it will give you an error telling you
that it can only read float values. As such we have to extend loadtxt() arguments to
tell it:
1. that we should skip the first four lines of the file because these contain a
header with text
2. that the first column should be translated to a date
3. that if it encounters ”NAN” it should convert it to a number indicating miss-
ing values, usually -9999.
Loadtxt() has different arguments to do this. To skip the first four lines we use
the skiprows argument in loadtxt():
>>> from s c i p y import l o a d t x t
>>> d a t a = l o a d t x t ( ’ f i l e n a m e . d a t ’ ,\comments= ’ # ’ ,\d e l i m i t e r = ’ , ’ ,\s k i p r o w s =4)
>>>
Loadtxt() also has an argument to translate certain input it reads to a different
output. In this case, we want to convert the date string in the first column (column
0) to a date number that Python can use for plotting. This is done with the loadtxt()
converters= argument that we apply to column 0 and telling it what format the date
string is (”%Y-%m-%d %H:%M:%S”) and how to translate it using the matplotlib
strpdate2num() function.
from m a t p l o t l i b . d a t e s import s t r p d a t e 2 n u m # m a t p l o t l i b d a t e
f u n c t i o n
m e t e o d a t a = l o a d t x t ( ’ f i l e n a m e . d a t ’ ,\comments= ’ # ’ ,\d e l i m i t e r = ’ , ’ ,\s k i p r o w s =4 ,
c o n v e r t e r s = {0: s t r p d a t e 2 n u m ( ’\”%Y−%m−%d %H:%M
:%S\” ’ ) }\)
Finally we have to make sure that the ”NAN” values are converted to indicate
missing values (-9999).
7.2.2 Reading data from file with genfromtxt()
The genfromtxt() function in the scipy is a more versatile alternative for scipy.loadtxt()
for importing data from a text file.
Suppose that we are working with an HBV model input file ptq.txt, which is
an ASCII text file that starts with a 2-line header followed by input data in four
tab-separated columns being a date, precipitation P , temperature T and discharge
Q:
Dinke l c a t c h m e n t 1976−2010
Date P T Q
19760101 9 . 4 3 . 7 0 .260
19760102 1 6 . 7 4 . 3 0 .430
19760103 1 3 . 3 5 . 0 1 .418
19760104 4 . 3 2 . 0 1 .625
19760105 1 . 3 5 . 1 1 .708
19760106 1 5 . 9 3 . 7 1 .739
19760107 0 . 3 7 . 5 1 .875
19760108 0 . 1 6 . 5 NaN
19760109 0 . 0 6 . 9 1 .668
We could read the file as follows:
import s c i p y
import m a t p l o t l i b . d a t e s
kwargs= d i c t ( d e l i m i t e r = ’\ t ’ ,\d e l e t e c h a r s = ’ ’ ,\c o n v e r t e r s = {0: m a t p l o t l i b . d a t e s . s t r p d a t e 2 n u m ( ’%Y%m%d ’ ) } ,\s k i p h e a d e r =1 ,\names=True ,\s k i p f o o t e r =0 ,\f i l l i n g v a l u e s =−9999 ,\m i s s i n g v a l u e s ={ ’ \”NaN\” ’}\)
p t q d a t a = s c i p y . g e n f r o m t x t ( d a t a f i l e , ∗∗ kwargs )
We have first stored the arguments for scipy.genfromtxt in the kwargs dict dic-
tionary data container type variable (Section 2.2.5) that can store keys and their
corresponding values (e.g. delimiter=’,’). The converters key is used to convert
the date to a matplotlib.dates date object (pylab numerical date value) and missing
values are identified as NaN (Not a Number).
In the statement scipy.genfromtxt(datafile, **kwargs) statement point to the
previously declared kwargs dictionary using **. We have seen in Section 2.2.3
that the ** operator is used as a power operator (x**4 raises x to the power of
four), but now ** is used for keyword argument unpacking. In this statement **
therefore unpacks the kwargs dictionary into separate keyword options as defined
in the kwargs dict() statement. Note that a singe ** before a variable (e.g. *args)
would mean argument unpacking. So both * and ** are used to unpack data struc-
tures (e.g. lists of arguments or keywords) of the arguments that these precede.
Executing the above statements gives:
>>> p t q d a t a
a r r a y ( [ ( 7 2 1 3 5 4 . 0 , 9 . 4 , 3 . 7 , 0 . 2 6 ) , ( 7 2 1 3 5 5 . 0 , 1 6 . 7 , 4 . 3 , 0 . 4 3 ) ,
( 7 2 1 3 5 6 . 0 , 1 3 . 3 , 5 . 0 , 1 . 4 1 8 ) , . . . , ( 7 3 3 9 8 3 . 0 , 8 . 7 , 1 7 . 2 ,
nan ) ,
( 7 3 3 9 8 4 . 0 , 0 . 0 , 1 9 . 5 , nan ) , ( 7 3 3 9 8 5 . 0 , 0 . 1 , 1 9 . 3 , nan ) ] ,
d t y p e = [ ( ’ Date ’ , ’<f8 ’ ) , ( ’P ’ , ’<f8 ’ ) , ( ’T ’ , ’<f8 ’ ) , ( ’Q’ , ’<
f8 ’ ) ] )
>>>
The data is now in a ndarray with formats according to dtype and we can address,
for instance, the entire date column, the first date value only, and the fifth T value
by:
>>> p t q d a t a [ ’ Date ’ ]
a r r a y ( [ 7 2 1 3 5 4 . , 7 2 1 3 5 5 . , 7 2 1 3 5 6 . , . . . , 7 3 3 9 8 3 . , 7 3 3 9 8 4 . ,
7 3 3 9 8 5 . ] )
>>> p t q d a t a [ ’ Date ’ ] [ 0 ]
721354 .0
>>> p t q d a t a [ ’T ’ ] [ 4 ]
5.0999999999999996
>>>
7.2.3 Saving Python variables to file
Suppose that we have one or more variables that we would like to save in files,
such as calculated daily evaporation rates. Python allows saving such variables
to an ASCII text file using the open() function to open a text file and the close()
function after we have written the data. The following example opens the file
d:/mydata/asu pe.txt, as defined in the variable pe outfile for writing (’w’), then
writes two lines with comments starting with the # character and ending with a
newline (\n). Then we create single line lists (filelist=[]) in which we append the
day number dayno[n], a tab (filelist.append(’\t’)), sum of rainfall (dsumP[n]) as a
string number with format xxxxx.xx (’%5.2f’ %), then another tab and the evapo-
ration E0[n]. These are written to the file with fileout.writelines(filelist). When all
the lines have been written (n = len(dayno)) we close the file with fileout.close().
The len() function provides the length of a string, list or array as a number. In this
case it provides the number of rows in the dayno variable.
p e o u t f i l e = ’ d : / mydata / a s u p e . t x t ’
f i l e o u t = open ( p e o u t f i l e , ’w’ )
f i l e o u t . w r i t e ( ’ # C u i e i r a s Reserve , Manaus\n ’ )
f i l e o u t . w r i t e ( ’ # Date , R a i n f a l l [mm] , Penman E0 [mm]\ n ’ )
f o r i in range ( 0 , l e n ( dayno ) ) :
f i l e l i s t = [ ]
f i l e l i s t . append ( s t r ( i n t ( dayno [ i ] ) ) )
f i l e l i s t . append ( ’\ t ’ )
f i l e l i s t . append ( s t r ( ’ %5.2 f ’ % f l o a t ( dsumP [ i ] ) ) )
f i l e l i s t . append ( ’\ t ’ )
f i l e l i s t . append ( s t r ( ’ %5.2 f ’ % f l o a t ( E0 [ i ] ) ) )
f i l e l i s t . append ( ’\n ’ )
f i l e o u t . w r i t e l i n e s ( f i l e l i s t )
f i l e o u t . c l o s e ( )
Assignment
Save the variables from wldata, with the corrected H value, in a text file named
wlm.txt.
7.3 Working with NetCDF files
Network Common Data Form (netCDF) machine-independent binary data format,
with software libraries that support the creation, access, and sharing of array-
oriented scientific data. The format is maintained by the University Corporation
for Atmospheric Research (UCAR) and is often used for storing scientific data in
a uniform way.
Python knows various software libraries (modules) that have commands to cre-
ate and access data stored in this format. One of these is PyNIO (http://www.pyngl .ucar.edu/Nio.shtml),
developed by NCAR’s Computational and Information Systems Laboratory, which
also reads other common formats (GRIB, HDF, etc.). There is also a visualisation
module (PyNGL).
An alternative is the netcdf4-python module, which has recently been inte-
grated in Python(x,y). To make a new NetCDF file you can use the Dataset()
function and variables can be added to the file and accessed using the createVari-
able() function, where the time variable in the example below is a one-dimensional
array of values (1 column) of type float:
>>> from netCDF4 import D a t a s e t
>>> t e s t c d f f i l e = D a t a s e t ( ’ t e s t . nc ’ , ’w’ , format= ’NETCDF4 ’ )
>>> p r i n t t e s t c d f f i l e . f i l e f o r m a t
NETCDF4
>>> t i m e s = t e s t c d f f i l e . c r e a t e V a r i a b l e ( ’ t ime ’ , ’ f8 ’ , ( ’ t ime ’ , ) )
>>> t e s t c d f f i l e . c l o s e ( ) # Close t h e f i l e
Consult the netcdf4-python documentation for more information.
A simple library with netCDF functions has been developed by Ko van Huisst-
eden and is available at http://ecohydro.falw.vu.nl/python.
Chapter 8
Working with dates and times
In hydrology we often work with time series. The datetime module provides the
tools to conveniently work with dates and times. Detailed information about the
functions in the datetime module is given in http://docs.python.org/library/datetime.html.
We have to import the module first. If we need the current date or date and time
we can do this using the datetime.date() and datetime.now() functions:
>>> from d a t e t i m e import ∗>>> d a t e . t o d a y ( )
d a t e t i m e . d a t e (2009 , 6 , 1 )
>>> d a t e t i m e . now ( )
d a t e t i m e . d a t e t i m e (2009 , 6 , 1 , 16 , 54 , 44 , 395137)
In the above example, the year (2009) is followed by the month (6), day (1), hour
(16), minutes (54), seconds (44) and microseconds (395137). We can also have an
ordinal value of the date, which in Python is the number of days since 01-01-0001.
For the first of June 2009 this would be:
>>> d a t e . t o o r d i n a l ( d a t e . t o d a y ( ) )
733559
>>>d a t e . f r o m o r d i n a l (733559)
d a t e t i m e . d a t e (2009 , 6 , 1 )
We often have data where the year, month, day, and perhaps time (in hour, min-
utes, seconds) are provided in separate columns (e.g. 2009 10 29 15 31 03). There
are several ways to get the day of year (doy, 1–366) from these date values. The
first one is to use the date2doy() function that is defined in the meteolib function
library. For instance if we have a number of rows with year (YYYY), month (MM)
and day of month (DD) values, we could call the following to assign day of year
values to the variable doy:
doy= da t e2doy (YYYY,MM,DD)
54
We can also use the datetime module to combine these date values into a date-
time object using the following code, where the year is again given in the variable
YYYY, the month as MM and the day as DD.
>>> import d a t e t i m e
>>>YYYY=2000
>>>MM=9
>>>DD=18
>>>datum = d a t e t i m e . d a t e (YYYY,MM,DD)
>>>datum
d a t e t i m e . d a t e (2000 , 9 , 18)
With the function strftime() we can now convert these dates to other values, such
as the day of year as a decimal number (doy, 0-366, %j format), the week number
(%U), weekday as a decimal number (%w, 0 (Sunday) – 6), etc. This is shown
below:
>>>datum . s t r f t i m e ( ’%j ’ )
’ 262 ’
>>> datum . s t r f t i m e ( ’%U’ )
’ 38 ’
>>> datum . s t r f t i m e ( ’%w’ )
’ 1 ’
So let’s do something useful. This script reads in a file with three columns con-
taining the year, month and day of the month. It then creates a datetime object that
is stored in an array called alldata and converts the date to the day of year using
the strftime() function. We use the int to convert the string that results from the
strftime() function to an integer doy number. We use append to add data to the end
of the arrays.
>>>from s c i p y import l o a d t x t
>>>import d a t e t i m e
>>>d a t a f i l e = ’ t e s t . t x t ’
>>># D e f i n e d e l i m i t e r
>>>d e l i m i t e r = ’ , ’
>>># Load data from f i l e
>>>d a t a = l o a d t x t ( d a t a f i l e , comments= ’ # ’ , d e l i m i t e r = d e l i m i t e r )
>>># P lace each column i n i t s own v a r i a b l e
>>>YYYY= d a t a [ : , 0 ]
>>>MM= d a t a [ : , 1 ]
>>>DD= d a t a [ : , 2 ]
>>># D e f i n e a r r a y s h o l d i n g da ta
>>>a l l d a t a = [ ]
>>>doy = [ ]
>>># Loop t h r o u g h data t o c o n v e r t t o d a t e t i m e o b j e c t
>>># and doy v a l u e s
>>>f o r i in range ( 0 , l e n ( d a t a ) ) :
>>> datum = d a t e t i m e . d a t e (YYYY[ i ] ,MM[ i ] ,DD[ i ] )
>>> a l l d a t a . append ( datum )
>>> doy . append ( i n t ( datum . s t r f t i m e ( ’%j ’ ) ) )
>>> p r i n t doy
[ 3 3 5 , 336 , 337]
Chapter 9
Plotting data in a graph
9.1 Basics of plotting graphs
The pylab module allows you to plot your data in many different ways and save the
image for later use in a report. In order to do so you first have to import the pylab
module
>>> import p y l a b
>>>
This allows you to create very nice graphics. For meteorological data, it is often
useful to plot a variable, such as wind speed, temperature or radiation against time.
In order to do so we use the plot() function in the pylab module. Let’s use the
temperature data that were introduced in Section 2.2.7 as an example for plotting.
We assigned the temperature data to a variable T , whereas the time data was given
in the third column of the array (i.e. t data[:,2]). We can plot the data by simply
defining a figure (pylab.figure(1), giving the pylab.plot() command followed by the
pylab.show() command:
>>> p y l a b . f i g u r e ( 1 )
<m a t p l o t l i b . f i g u r e . F i g u r e i n s t a n c e a t 0 xb6d571ec>
>>> p y l a b . p l o t ( t d a t a [ : , 2 ] , T )
[<m a t p l o t l i b . l i n e s . Line2D i n s t a n c e a t 0 xb5ee42ec >]
>>> p y l a b . show ( )
This will create a figure window as in Figure 9.1. If you want to issue additional
commands in Idle, close the figure window first. We can fancy things up a bit by
giving the graph a label for the x-axis:
>>> x l a b e l ( ’ Time ’ )
<m a t p l o t l i b . t e x t . Text i n s t a n c e a t 0 xb5de64cc>
57
Figure 9.1: Plot of temperature data versus time using the plot command of the
pylab module.
Figure 9.2: Plot of temperature data versus time, now with x-axis label and title.
This will result in the graph shown in Figure 9.2. For the y-axis label we need a
superscript in the ◦C unit for temperature. For this we need to invoke LATEX style
code, where the \circ command will create the small circle and adding the ∧ will
put format it as superscript. We need to set the following first to activate LATEX text
interpretation:
>>> r c ( ’ t e x t ’ , u s e t e x = True )
>>>
Then we need to use the ylabel() command with LATEX text. In LATEX, a formula
starts and ends with the $ symbol. Text between the $ signs will be formatted as an
equation, which means that spaces are ignored. If you need a space you should use
the tilde (∼) symbol between two words. We often want normal text in the label,
and then use a superscript, for instance, in the unit. We can explicitly state that the
text should be in normal roman-font text font by placing it in a \rm environment,
i.e. between accolades. We also have to place an r before the label text to tell
Python that this is a raw string so it does not interpret Python escape commands,
such as \n that would make a new line, or \t that would insert a tab in Python. So
the ylabel() statement could look like this:
>>> y l a b e l ( r ’ ${\rm T empera tu re ˜ [ } ˆ \ c i r c {\ rm C]} $ ’ , f o n t s i z e =16)
<m a t p l o t l i b . t e x t . Text i n s t a n c e a t 0 xb4e93bec>
>>>
to get Temperature [◦C], or
>>> y l a b e l ( r ’ ${\rm Wind ˜ speed ˜ [m˜ s }ˆ{−1} {\ rm ]} $ ’ , f o n t s i z e =16)
<m a t p l o t l i b . t e x t . Text i n s t a n c e a t 0 xb4e93bec>
>>>
to get Wind speed [m s−1]. Note again the use of ∼ where we need spaces in
the text. In these examples we also increased the font to 16 points by the fontsize
statement in the ylabel() command, as this makes the figure better readable in your
report. The result is shown in Figure 9.3. We also need to increase the font sizes
of the x- and y-ticks, using xticks(fontsize=15) and yticks(fontsize=15) commands,
and increase the linewidth, include a legend label and specify a legend. We should
then save the figure in the image directory of our report. All commands to create
Figure 9.4 are listed below (without Python responses).
>>> f i g u r e ( 1 )
>>> p l o t ( t d a t a [ : , 2 ] , T , l a b e l = ’ Ai r T empera tu re ’ , l i n e w i d t h = 1 . 5 )
>>> x l a b e l ( ’ Time ’ , f o n t s i z e =16)
>>> y l a b e l ( r ’ ${\rm T empera tu re ˜ [ } ˆ \ c i r c {\ rm C]} $ ’ , f o n t s i z e =16)
>>> x t i c k s ( f o n t s i z e =15)
>>> y t i c k s ( f o n t s i z e =15)
>>> l e g e n d ( )
>>> s a v e f i g ( ’ t e m p e r a t u r e . eps ’ , d p i =300)
Figure 9.3: Plot of temperature data versus time, now with x- and y-axis labels, the
latter with fontsize=16.
>>> show ( )
Inserting the saved figure (as encapsulated postscript, eps, file) in your report re-
sults in Figure 9.5.
You can also use markers, where the ’o’ represents a closed circle, a ’+’ gives
you a plus marker, etc. By default, markers are filled but you can create open
dots and determine the fill and edge colours using the facecolors and edgecolors
options. For example we can create a scatter plot graph displaying x and y val-
ues that we obtain from the random module in scipy. The scipy.random.randn()
function gives you random samples from the standard normal distribution (a nor-
mal distribution whit average value of 0 and variance of 1). The s= option in the
pylab.scatter function represents the size of the markers.
>>> import p y l a b
>>> import s c i p y
>>> x= s c i p y . random . randn ( 1 0 0 )
>>> y= s c i p y . random . randn ( 1 0 0 )
>>> a= s c i p y . random . randn ( 1 0 0 )
>>> b= s c i p y . random . randn ( 1 0 0 )
>>> f i g u r e ( 1 )
>>> p y l a b . s c a t t e r ( x , y , s =50 , f a c e c o l o r s = ’ none ’ , e d g e c o l o r s = ’ b ’ )
>>> p y l a b . s c a t t e r ( a , b , s =70 , f a c e c o l o r s = ’ y ’ , e d g e c o l o r s = ’ g ’ )
Figure 9.4: Plot of temperature data versus time, with x- and y-axis labels (fontsize
16), x- and y-ticks fontsize 15, a legend and a linewidth of 1.5 instead of the default
1.0.
Figure 9.5: Saved EPS figure of temperature data versus time, with x- and y-axis
labels (fontsize 16), x- and y-ticks fontsize 15, a legend and a linwidth of 1.5
instead of the default 1.0.
−4 −3 −2 −1 0 1 2 3 4−4
−3
−2
−1
0
1
2
3
4
Figure 9.6: Example of the use of the pylab.scatter function with open and closed
markers of different colours.
>>> p y l a b . show ( )
The above commands would result in a plot similar to that shown in Figure 9.6.
9.2 More complicated graphs
It is sometimes necessary to present your data in more complicated graphs. For
instance, if you have spatially varying data (e.g. topography, groundwater head
variations, variations in NO3 concentrations) you often want to plot contours using
these data. In these cases the data consist of x and y coordinates that determine the
location, and a third variable (z) that varies spatially.
A contour plot can be made using the meshgrid() and contour() commands.
9.3 Interactive plotting of figures
Instead of the show() command that displays figures, we can also use the ion()
function for plotting figures interactively while running your Python script. The
ion() function is defined in the pylab and this module should therefore be imported
before you can include the ion() statement in your script.
9.4 Changing text fonts in a figure
We can also change the default font and other parameters in one go by issueing
the following command before we start creating the figure. We do this with the
rcParams.update() function. In the script below we define a variable newdefaults
that lists how figures are saved (.png, .eps, etc) by specifying the backend vari-
able, and the font sizes of axis labels (axes.labelsize variable, ticks (xtick.labelsize,
ytick.labelsize, text (text.fontsize and legend (legend.fontsize. We then use rc-
Params.update(newdefaults) to effectuate the update.
>>> n e w d e f a u l t s = { ’ fontname ’ : ’Roman ’ ,
’ backend ’ : ’ eps ’ ,
’ axes . l a b e l s i z e ’ : 16 ,
’ t e x t . f o n t s i z e ’ : 16 ,
’ l e g e n d . f o n t s i z e ’ : 15 ,
’ x t i c k . l a b e l s i z e ’ : 14 ,
’ y t i c k . l a b e l s i z e ’ : 14}>>> r cP arams . u p d a t e ( n e w d e f a u l t s )
9.5 Creating subplots and a second y-axis
If we want subplots in a figure, we can use the subplot() command:
>>> f i g u r e ( 2 ) # Crea te second f i g u r e
>>> s u b p l o t ( 2 1 1 ) # P l o t t e m p e r a t u r e da ta i n upper p l o t
>>> p l o t ( t d a t a [ : , 2 ] , T , l a b e l = ’ Ai r T empera tu re ’ , l i n e w i d t h = 1 . 5 )
>>> y l a b e l ( r ’ ${\rm T empera tu re ˜ [ } ˆ \ c i r c {\ rm C]} $ ’ )
>>> s u b p l o t ( 2 1 2 ) # P l o t s t a n d a r d d e v i a t i o n s i n lower p l o t
>>> p l o t ( t d a t a [ : , 2 ] , t d a t a [ : , 4 ] , l a b e l = ’ S t . dev . T ’ , l i n e w i d t h
= 1 . 5 )
>>> y l a b e l ( r ’ ${\rm S t . ˜ dev . ˜ T empera tu re ˜ [ } ˆ \ c i r c {\rm C]} $ ’ )
>>> x l a b e l ( ’ Time ’ )
>>> s a v e f i g ( ’ temp2 . eps ’ , d p i =300)
This creates Figure 9.7.
For plotting time series with a large difference between y-values, such as for
temperature (0–30 ◦C) and relative humidity (60–100%), it is often convenient
to use a second y-axis for plotting the latter. This can be done with the twinx()
command. Basically, twinx() creates a new set of axes over the existing figure.
The temperature data would then plot on the first set of axes (left y-axis), whereas
the relative humidity would be plotted on the second created set of axes (right y-
axis).
The following script code would create a single figure with the temperature
plotted on the left y-axis, and its standard deviation on the right axis as in Fig-
ure 9.8.
# Crea te f i g u r e 3 , now w i t h two y−axes
f i g u r e ( 3 )
# c r e a t e l a b e l s f o r t h e x− and l e f t y−a x i s
x l a b e l ( r ’ ${\ rm Time}$ ’ )
y l a b e l ( r ’ ${\ rm T empera tu re ˜ [ } ˆ \ c i r c {\rm C]} $ ’ )
# P l o t T v e r s u s t ime , use red c o l o u r ( r ) and round marker ( o )
Figure 9.7: Saved EPS figure of temperature and standard deviation data versus
time using the subplot() function.
p l o t ( t ime , T , ’ ro ’ , l a b e l = ’ T empera tu re ’ )
# P lace a l e g e n d
l e g e n d ( l o c =1)
# Use t w i n x ( ) t o c r e a t e a second s e t o f axes on t o p o f t h e f i r s t
s e t
tw inx ( )
# Crea te l a b e l f o r t h e r i g h t y−a x i s
y l a b e l ( r ’ ${\rm S t . ˜ dev . ˜ T empera tu re ˜ [ } ˆ \ c i r c {\rm C]} $ ’ )
# Now p l o t s t a n d a r d d e v i a t i o n o f T v e r s u s t i m e
p l o t ( t ime , s tdT , l a b e l = ’ S t . dev ’ )
# P lace a l e g e n d f o r t h e second s e t o f axes
l e g e n d ( l o c =2)
# Save t h e f i g u r e as EPS f i l e
s a v e f i g ( ’ twoaxes . eps ’ , d p i =300)
Figure 9.8: Saved EPS figure of time series of temperature and standard deviation
of temperature using the twinx() function to create two y-axes.
Bibliography
R. Allen, L.S. Pereira, D. Raes, and M. Smith. Crop Evaporation -
Guidelines for computing crop water requirements, volume 56 of FAO
Irrigation and Drainage Paper. FAO, Rome, Italy, 1998. URL
http://www.fao.org/docrep/x0490e/x0490e00.htm.
H.A.R. de Bruin. From Penman to Makkink. In J.C. Hooghart, editor, Evaporation
and Weather, pages 5–31, The Hague, 1987. TNO Committee on Hydrological
Research. Proceedings and Information 39.
H. L. Penman. Natural evaporation from open water, bare soil and grass. Proceed-
ings of the Royal Society, 193:120–146, 1948. Series A.
H. L. Penman. Evaporation: An introductory survey. Neth. J. Agric. Sci., 4:9–29,
1956.
H. L. Penman. Vegetation and hydrology. Technical communication 53, Common-
wealth Agricultural Bureaux, Farnham Royal, Bucks, UK, 1963.
C. H. B. Priestley and R. J. Taylor. On the assessment of surface heat flux and
evaporation using large-scale parameters. Monthly Wheater Review, 100:81–92,
1972.
J.D. Valiantzas. Simplified versions for the Penman evaporation equation using
routine weather data. Journal of Hydrology, 331(3–4):690–702, 2006. doi:
10.1016/j.jhydrol.2006.06.012.
66
Index
>>> Python prompt, 7, 9, 24
∼, 58
{\rm }, 58
e number, 33
pi number, 33
**, 50
** operator, 11, 50
**kwargs, 50
*args, 50
.py file extension, 24
[] list definition, 31, 51
# (Pound) sign, 11
% operator, 18
LATEX coding, 58
LATEX font use, 61
LATEX use in figures, 58
import (), 14
Raw string r, 58
Anaconda Python, 7
append() function, 31, 51, 54
arange() function, 36
array data type, 31
array() function, 31
ASCII text editor, 9
ASCII text file operations, 51
axes.labelsize variable, 62
backend variable, 62
Boolean data type, 16, 29
Break statement, 32
Bugs, 27
Calculator, 15
Class data members, 44
Class methods, 44
Class type, 44
close() function, 51
cmd.exe, 7
Comma separated values (csv), 48
Command line, 7, 9
Commenting code, 10
Complex data type, 16
Conditional statement, 29
Continue statement, 32
contour() plot function, 61
converters=, loadtxt argument, 49
cos() function, 12, 33
cp calc() function, 38
createVariable() function, 52
csv file, 48
Dataset() function, 52
date, 53
Date and time formatting, 54
date2doy() function, 38
Datetime module, 13, 53
datetime.date() function, 53
datetime.now() function, 53
Daylength function, 38
Debugging, 9, 27
def statement, 39
Delta calc() function, 38
dict(), 17, 50
Dictionary data type, 16, 50
dictionary data type, 16
dtype, 51
E0() function, 39
ea calc() function, 38
edgecolors, 59
Editor, 9
elif, 43
67
else, 43
ELSE statement, 31, 32
Em() function, 39
Enthought Python, 7
Ept() function, 39
Error, 27
Error message, 27
es calc() function, 38
ET0pm() function, 39
Evaplib function library, 39
Evaplib module, 14, 38, 39, 42
Evaporation functions, 38
Executing a script, IDLE, 27
Executing script files, 27
Exponential function, 39
Extraterrestrial radiation function, 38
facecolors, 59
False, 29
figure() function, 56
File load, 27
File saving, 51
find() function, 35
float, 52
Float data type, 15
float() function, 16
Floating point decimal format, 18
Floating point exponential format, 18
fontsize, 58
FOR loop, 30
FOR statement, 30
Format of number, 18
from–import, 13
Frozenset data type, 16
Function argument list, 39, 40
Function definition, 39
Function library, 12, 33
gamma calc() function, 38
genfromtxt() function, 17, 49
GRIB file format, 52
HDF file format, 52
Help function, 10, 41
Help text, 40, 41
help() function, 33
hsplit(), 22
hstack(), 21
IDE, 9
IDLE, 9
Idle editor, 24
IDLE editor window, 24
IDLE shell window, 9, 24
IF statement, 30
If statement, 43
import, 13
Importing modules, 34
Indentation, 29
Instance, 44
int() function, 16, 54
Integer, 15
Integer data type, 15
Integrated Development Environment, 9
interactive plotting, 61
ion() function, 61
Ipython, 7
JEdit programmer’s text editor, 24
Keyword argument unpacking, 50
L calc() function, 38
Lambda function, 42
lambda function, 39
legend() function, 58
legend.fontsize variable, 62
len() function, 51
Linear reservoir function, 39
linewidth, 58
linspace() function, 36
List data type, 16, 31, 51
list data type, 31
Load a function, 34
loadtxt(), skiprows, 49
log() function, 22
Makkink evaporation, 39
Math module, 13, 22, 33, 34
Mathematical functions, 12
matplotlib, 49
Matplotlib library, 13
matplotlib.dates, 50
meshgrid() function, 61
Meteolib function library, 53
Meteolib functions, 38
Meteolib module, 14, 38, 42
Meteorological functions, 38
methods, 44
Modules, 12, 33
Multi-line comment, 11
NaN, not a number, 50
NAN—hyperpage, 48
ndarray, 51
netCDF, 52
NetCDF file format, 52
netcdf4-python module, 52
Network Common Data Form, 52
Not A Number, 48
Notepad, 9
Numerical python module, 14
numpy module, 14
Object, 43, 44
Object-oriented language, 43
open() function, 51
Operator, 11
Operator, &, 11
Operator, and, 11
Operator, binary |, 11
Operator, binary &, 11
Operator, floor, 11
Operator, modulus, 11
Operator, or, 11
Operators, bitwise, 11
Operators, comparison, 11
Operators, logical, 11
Operators, mathematical, 11
Operators, membership, 11
os.path module, 46
os.path.splitext(), 46
Output format, 18
Penman open water evaporation, 39
Penman-Monteith reference evaporation,
39
plot() function, 56
Plotting of variables, 56
pottemp() function, 38
pow() function, 33
Priestley-Taylor evaporation, 39
Program code, 10
Prompt, 9, 27
Pylab module, 13, 35, 56, 61
pylab.scatter, 59
PyNGL, 52
PyNIO module, 52
Python compiler, 27
Python script, 10, 24
Python shell window, 10
Python(x,y), 7
random module, 59
range() function, 30, 36
rcParams.update function, 61
repr(), 18
rho calc() function, 38
Run, 11
Run module, 27
Scatterplot, 59
Scientific functions, 12
Scientific Python module, 22
Scipy, 49
scipy, 14, 59
Scipy module, 13, 22
scipy module, 39
scipy.genfromtxt() function, 49
Script file, 24
Script, execution, 27
Second y-axis, 62
Set data type, 16
show() function, 56
Signed integer decimal, 18
sin() function, 12, 33
skiprows, loadtxt() argument, 49
Space in LATEX, 58
Spyder development environment, 10
sqrt() function, 12, 22
Square root function, 22
Stacking, 21
Statement, 10
str(), 18, 19
strftime() function, 54
String, 18
String data type, 16
String format conversion specifier, 18
strpdate2num() function, 49
subplot() function, 62
sun NR() function, 38
Superscript use in figures, 58
Syntax error, 27
tan() function, 33
Terminal, 7
Text console, 7
Text terminal, 7
text.fontsize variable, 62
time, 53
True, 29
Tuple data type, 16
twinx() function, 62
Type, 15
type() function, 16
Unicode data type, 16
Unpack data structures, 50
Unsigned decimal, 18
Unsigned hexadecimal, 18
Unsigned octal, 18
Using subplots, 62
Variable names, 14
Vertical stack, 22
vpd calc() function, 38
vsplit(), 22
WHILE statement, 30, 32
Wordpad, 9
writelines() function, 51
Xemacs, 9
xlabel() function, 56
xticklabelsize variable, 62
xticks() function, 58
ylabel() function, 58
ytick.labelsize variable, 62
yticks() function, 58