monet documentation - readthedocs.org

of 100 /100
MONET Documentation Barry Baker Jan 26, 2022

Author: others

Post on 14-Feb-2022

3 views

Category:

Documents


0 download

Embed Size (px)

TRANSCRIPT

MONET Documentation2 Reference 5 2.1 Overview: Why MONET? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.2 Gallery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2.1 Required dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1.1 For parallel computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2.1.2 For plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.2 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 MONET XArray Accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3.1 Initializing the Accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 Interpolation Accessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.2.1 Find Nearest Lat lon point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4 Installing on WCOSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5 Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.1 How to read AQS data from PAMS and do a quick analysis . . . . . . . . . . . . . . . . . . 14 2.5.1.1 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.5.2 Loading IMPROVE Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.5.2.1 Trend Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.5.2.2 KMEANS Clustering using scikit-learn . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.3 How to Open and View CMAQ simulations - HI Volcano Example . . . . . . . . . . . . . . 27 2.5.3.1 Plotting on a map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.5.3.2 Finding nearest lat lon point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.5.3.3 Pairing with AirNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.5.3.4 Overlaying Observations on Contour Plots . . . . . . . . . . . . . . . . . . . . . . 34
2.5.4 NOAA NESDIS AOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.5.4.1 VIIRS EDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.5.4.2 VIIRS EPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.5.5 FV3-CHEM in MONET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5.5.1 Convert nemsio2nc4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5.5.2 Convert fv3grib2nc4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.5.5.3 Using MONET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.6 Get in touch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.6.1 Get in touch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
i
2.6.2 API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.6.2.1 monet package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.6.2.2 monet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Bibliography 89
MONET Documentation
MONET is an open source project and Python package that aims to create a common platform for atmospheric composition data analysis for weather and air quality models.
MONET was developed to evaluate the Community Multiscale Air Quality Model (CMAQ) for the NOAA National Air Quality Forecast Capability (NAQFC) modeling system. MONET is designed to be a modularized Python package for (1) pairing model output to observational data in space and time; (2) leveraging the pandas Python package for easy searching and grouping; and (3) analyzing and visualizing data. This process introduces a convenient method for evaluating model output. MONET processes data that is easily searchable and that can be grouped using meta- data found within the observational datasets. Common statistical metrics (e.g., bias, correlation, and skill scores), plotting routines such as scatter plots, timeseries, spatial plots, and more are included in the package. MONET is well modularized and can add further observational datasets and different models.
Our goals is to provide easy tools to retrieve, read and combine datasets in order to speed scientific research. Currently, MONET is able to process several models and observations related to air composition and meteorology.
Please site our work.
What’s New
MONET v2.2.0 has been released. MONET has re-engineered the way it deals with multidimensional observations or model output by using an Xarray Accessor giving MONET a flexible and intuitive way of expanding Xarray for multidimensional geospatial information commonly used in meteorology, climate and air quality all while making it easier on the user to use MONET and add to it.
MONET also underwent a major restructure with v2.2.0. All I/O functions have been moved to a sister repository https://github.com/noaa-oar-arl/monetio.
Features include:
• Xarray Accessor for both xarray.DataArray and xarray.Dataset using the .monet attribute
• Pandas Accessor for pandas.DataFrame using the .monet attribute
• vertical interpolation using python-stratify (https://github.com/SciTools-incubator/python-stratify) using the .monet.stratify function
• spatial interpolation using .monet.remap including: - Nearest neighbor finder - Constant latitude interpolation - Constant longitude interpolation - remap DataArray to current grid using pyresample nearest neighbor or xesmf - remap entire dataset to current grid using pyresample nearest neighbor or xesmf - find nearest i,j or lat lon - interpolate to constant latitude or longitude
• Simplified combine tool to combine point source data with multidimensional xarray objects
CHAPTER 2
Reference
Baker, Barry; Pan, Li. 2017. “Overview of the Model and Observation Evaluation Toolkit (MONET) Version 1.0 for Evaluating Atmospheric Transport Models.” Atmosphere 8, no. 11: 210
2.1 Overview: Why MONET?
2.1.1 Features
Retrieving, loading, and combining data and putting into a common format is the core of MONET. MONET uses the pandas and xarray_ data formats for data analysis.
• Open point observations in a common format. pandas excels at working with tabular data or point measurements. It is used for time series analysis and statistical measures.
• Open model and satellite data in a common format. xarray_ is used when N-dimensional arrays are needed.
• Retrieving observational datasets for given time and space.
• Efficiently combine/interpolate model and observational datasets.
• Provide easy plotting using proven tools in python
• Perform statics between model runs or observations or models and observations.
6 Chapter 2. Reference
MONET Documentation
2.1.2 Gallery
MONET Documentation
2.2 Installation
• numpy (1.11 or later)
• pandas (0.18.0 or later)
• xarray (0.10 or later)
• netcdf4
• matplotlib
• seaborn 1 MONET has dropped support for python 2.7 and requires python 3.6+. For more information see the following references: – Python 3 Statement – Tips on porting to Python 3
10 Chapter 2. Reference
2.2.1.2 For plotting
2.2.2 Instructions
MONET itself is a pure Python package, but some of it’s dependencies may not be. The simplest way to install MONET is to install it from the conda-forge feedstock:
$ conda install -c conda-forge monet
This will install all of the dependencies needed by MONET and MONET itself.
xesmf is an optional dependency and can be installed easily from conda-forge, Note xesmf is not available on windows due to the dependency on esmpy and esmf.:
$ conda install -c conda-forge monet
To install MONET from source code you must install with pip. This can be done directly from the github page:
$ pip install git+https://github.com/noaa-oar-arl/MONET.git
or you can manually download it from GitHub and install it using the setup.py:
$ git clone https://github.com/noaa-oar-arl/MONET.git $ cd MONET $ pip install .
2.3 MONET XArray Accessor
MONET can add georeferecing tools to xarray‘s data structures. These tools can be accessed via a special .monet attribute, available for both xarray.DataArray and xarray.Dataset objects after a simple import monet in your code.
2.3.1 Initializing the Accessor
All you have to do is import monet after you import xarray.
import xarray as xr import monet as m
cmaqfile = monet.__path__ + '/../data/aqm.t12z.aconc.ncf'
2.3.2 Interpolation Accessors
The MONET accessor provides several useful interpolation routines including: Getting the nearest point to a given latitude and longitude, interpolating to a constant latitude or longitude, interpolating to a vertical levels, and remapping entire 2D :py:class::~xarray.DataArray or :py::class::xarray.DataSet.
2.3.2.1 Find Nearest Lat lon point
To find the nearest latitude longitude point you just need to use the .monet.nearest_latlon method. In this example we will continue to use the CMAQ test file above. We will find the closest grid points to lat=20.5, lon=-157.4.
c.monet.nearest_latlon(lat=20.5,lon=-157.4)
<xarray.Dataset> Dimensions: (time: 48, x: 1, y: 1, z: 1) Coordinates:
* time (time) datetime64[ns] 2018-05-17T12:00:00 2018-05-17T13:00:00 ... latitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)> longitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)>
Dimensions without coordinates: x, y, z Data variables:
O3 (time, z, y, x) float32 dask.array<shape=(48, 1, 1, 1), chunksize=(48, 1, → 1, 1)>
12 Chapter 2. Reference
MONET Documentation
Notice that the length of the dimensions are now (time: 48, x: 1, y: 1, z: 1). If you wanted to only find the nearest location for a single variable you can use the accessor on the DataArray.
c.O3.monet.nearest_latlon(lat=20.5,lon=-157.4)
<xarray.DataArray 'O3' (time: 48, z: 1, y: 1, x: 1)> dask.array<shape=(48, 1, 1, 1), dtype=float32, chunksize=(48, 1, 1, 1)> Coordinates:
* time (time) datetime64[ns] 2018-05-17T12:00:00 2018-05-17T13:00:00 ... latitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)> longitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)>
Dimensions without coordinates: z, y, x Attributes:
long_name: O3 units: ppbV var_desc: Variable O3 _FillValue: nan
2.4 Installing on WCOSS
If you have access to the NOAA WCOSS machines you can create your own python environments very easily using intel-python.:
module load ips/19.0.5.281
Then it is suggested to create a .condarc file that lives in your $HOME folder to point to a location that will house your conda environments. Below is a sample .condarc file.
channels: - intel - conda-forge - defaults
envs_dirs: - /gpfs/dell2/emc/verification/noscrub/User.Name/conda/envs
pkgs_dirs: - /gpfs/dell2/emc/verification/noscrub/User.Name/conda/pkgs
Next you should start a new environment by cloning the default environment to a new name. This can be done in a single command.
conda create -n myenv --clone="/usrx/local/prod/intel/2019UP05/intelpython3"
A prompt should come up and tell you to activate the environment you just created, myenv.
:: source activate myenv
From here you can install any package the same way you could on regular anaconda installations.
conda install -c conda-forge monet
2.4. Installing on WCOSS 13
MONET Documentation
2.5 Tutorial
2.5.1 How to read AQS data from PAMS and do a quick analysis
The first tutorial for MONET will load data from the AQS data, specifically the PAMS dataset, and show to reshape the dataframe and do a statistical analysis with different meteorological and composition data. For more examples, please see http://monet-arl.readthedocs.io/en/develop/. This will later be able to be found on the github page and readthedocs.
We will first begin by importing monet and a few helper classes for later
import numpy as np # numpy import pandas as pd # pandas from monet.obs import * # observations from MONET import matplotlib.pyplot as plt # plotting import seaborn as sns # better color palettes import cartopy.crs as ccrs # map projections import cartopy.feature as cfeature # politcal and geographic features sns.set_style('ticks') # plot configuration sns.set_context("talk") # plot configure for size of text
Now we have all the imports we could need lets load some data. Most of the PAMS data is on daily data so lets add the kwarg daily=True to the call. We will also create this for the year 2015 and 2016. Some variables that may be valuable are the VOCS, ozone, NO2, NOX, temperature. For all of the measurments available please see https://aqs.epa.gov/aqsweb/airdata/download_files.html
dates = pd.date_range(start='2004-01-01',end='2004-12-31') df = aqs.add_data(dates,daily=True,param=['VOC','OZONE'], download=True)
Retrieving: daily_VOCS_2004.zip https://aqs.epa.gov/aqsweb/airdata/daily_VOCS_2004.zip
Retrieving: daily_44201_2004.zip https://aqs.epa.gov/aqsweb/airdata/daily_44201_2004.zip
/anaconda3/lib/python3.6/site-packages/python/core/interactiveshell.py:2963: →DtypeWarning: Columns (0) have mixed types. Specify dtype option on import or set →low_memory=False. exec(code_obj, self.user_global_ns, self.user_ns)
/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py:189: →SettingWithCopyWarning: (continues on next page)
(continued from previous page)
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/ →indexing.html#indexing-view-versus-copy self._setitem_with_indexer(indexer, value)
Now we have the data in aqs.df and a copy of it in df. . . just in case. So lets take a look at it.
#df.head() aqs.df.head()
Notice that in this printed format it obscures some of the dataframe columns from view. Lets see what they are!
from numpy import sort for i in sort(df.columns): # loop over the sorted columns and print them
print(i)
(continues on next page)
state_code state_name time time_local tribe_name units variable
We have lots of columns but this is actually the long format (data is stacked on variable). Data analysis could be done easier in a wide format. So lets use a utility function in MONET to aid with reshaping the dataframe.
from monet.util import tools new = tools.long_to_wide(df) new.head()
Lets see how many ISOPRENE sites there are. We will drop the NaN values along the ISOPRENE column and then find the unique siteid’s and look at the shape of them
new.dropna(subset=['ISOPRENE']).siteid.unique().shape
(140,)
Now as you can see we have lots of columns that is sorted by time and siteid. But what measurements are included in the dataframe? Let’s see all the new columns generated from pivoting the table.
from numpy import sort for i in sort(new.columns):
print(i)
(continues on next page)
(continues on next page)
18 Chapter 2. Reference
obs observation_count observation_percent parameter_code parameter_name poc pollutant_standard sample_duration site_num siteid state_code state_name time time_local tribe_name units variable
Now as you can see we have lots of columns that is sorted by time and siteid. This can be very useful as we can now do some direct comparisons using the dataframe. Lets get a description of the dataset first so we can see some averages and ranges of the different chemical species.
new.describe()
This gives us a format that allows simple statistics and plots using pandas, matplotlib, and seaborn. For time series it is often useful to have the index as the time. Lets do that
new.index = new.time new['OZONE_ppb'] = new.OZONE * 1000. new.OZONE_ppb.mean()
27.581303457170307
2.5.1.1 Plotting
As you can see the data is now indexed with the UTC time. Lets make a time series plot of the average ISOPRENE.
f,ax = plt.subplots(figsize=(10,4)) # this is so we can control the figure size. new.ISOPRENE.resample('D').mean().plot(ax=ax)
<matplotlib.axes._subplots.AxesSubplot at 0x1c2bb1f860>
2.5. Tutorial 19
MONET Documentation
This is quite noisy with the daily data. Lets resample in time to every month using the average Isoprene concentration to weekly and monthly.
f,ax = plt.subplots(figsize=(10,4)) # this is so we can control the figure size. new.ISOPRENE.resample('D').mean().plot(ax=ax, label='daily') new.ISOPRENE.resample('W').mean().plot(ax=ax, label='weekly') new.ISOPRENE.resample('M').mean().plot(ax=ax, label='monthly') plt.ylabel('ISOP') plt.legend() sns.despine()
Where are these measurements. Lets plot this on a map and see where it is. We can use a utility plotting function in monet to generate the plot
20 Chapter 2. Reference
from monet import plots ax = plots.draw_map(states=True, extent=[-130,-60,20,50], resolution='10m') # get only where ISOPRENE is not NAN isop = new.dropna(subset=['ISOPRENE']) ax.scatter(isop.longitude, isop.latitude)
<matplotlib.collections.PathCollection at 0x1c47460f60>
There are monitors all across the US with many in TX, CA, New England and the Mid-Atlantic.
What if we wanted to do a linear regression between two variables. Lets say OZONE and temperature. To do this we will use the statsmodels package. It is a robust library for curve fitting. For specific information for this module look here https://www.statsmodels.org/stable/index.html
import statsmodels.api as sm # load statsmodels api #first clean of nan values fit_df = new[['ISOPRENE','OZONE']].dropna() x = fit_df.ISOPRENE y = fit_df.OZONE result = sm.OLS(y,x).fit() print(result.summary())
fit_df.plot.scatter(x='ISOPRENE',y='OZONE') plt.plot(x,result.predict(x),'--r')
OLS Regression Results ============================================================================== Dep. Variable: OZONE R-squared: 0.248 Model: OLS Adj. R-squared: 0.248 Method: Least Squares F-statistic: 1.833e+05 Date: Tue, 19 Jun 2018 Prob (F-statistic): 0.00 Time: 09:31:12 Log-Likelihood: 1.2390e+06 No. Observations: 556733 AIC: -2.478e+06 Df Residuals: 556732 BIC: -2.478e+06
(continues on next page)
Df Model: 1 Covariance Type: nonrobust ==============================================================================
coef std err t P>|t| [0.025 0.975] ------------------------------------------------------------------------------ ISOPRENE 0.0057 1.33e-05 428.134 0.000 0.006 0.006 ============================================================================== Omnibus: 205153.426 Durbin-Watson: 0.007 Prob(Omnibus): 0.000 Jarque-Bera (JB): 2465877.353 Skew: -1.433 Prob(JB): 0.00 Kurtosis: 12.904 Cond. No. 1.00 ==============================================================================
Warnings: [1] Standard Errors assume that the covariance matrix of the errors is correctly →specified.
[<matplotlib.lines.Line2D at 0x1c1e7f0f28>]
new.to_csv('/Users/barry/Desktop/new.csv')
2.5.2 Loading IMPROVE Data
In this tutorial we will read and load data from the IMPROVE aerosol network (http://vista.cira.colostate.edu/Improve/)
To do this we need to first download the data from the colstate.edu website. There are a few format requirements needed to be compatible with MONET. First go to http://views.cira.colostate.edu/fed/DataWizard/Default.aspx to
22 Chapter 2. Reference
• Select your parameters
• Fields
– skinny format
– delimited (note you will need to pass this onto the open command, ‘,’ by default)
After downloading we can then read the data. Here we included the EPACode and State to add additional meta data stored on the EPA auxilary files used in the EPA AQS and AirNow datasets in MONET. Let’s make a few imports from monet and some other libraries that will aid us later.
from monet.obs import improve import matplotlib.pyplot as plt from monet.util import tools from monet.plots import * import pandas as pd
Now we will load the data (in this case the file is ‘2018620155016911Q1N0Mq.txt’)
df = improve.add_data('/Users/barry/Desktop/20186258419212M0xuxu.txt', add_meta=False, → delimiter=',') from numpy import NaN df['obs'].loc[df.obs < 0] = NaN
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/ →indexing.html#indexing-view-versus-copy self._setitem_with_indexer(indexer, value)
df.head(20)
Now this is in the long pandas format. Lets use the monet.util.tools.long_to_wide utility to reformat the dataframe into a wide format.
from monet.util import tools df1 = tools.long_to_wide(df) df1.head()
Lets now plot some of the different measurements with time from a site. In this case we will look at the PHOE1 site in Phoenix, Arizona.
acad1 = df1.loc[df1.siteid == 'ACAD1'] acad1.head()
2.5.2.1 Trend Analysis
Let’s look at SIf as an example from ACAD1.
acad1.index = acad1.time
<matplotlib.axes._subplots.AxesSubplot at 0x1c286eb4a8>
Now this is good but lets resample to see if we can see a trend.
ax = acad1.resample('W').mean().plot(y='SIf', figsize=(14,5), label = 'weekly') ax = acad1.resample('M').mean().plot(y='SIf', ax=ax, label='monthly')
24 Chapter 2. Reference
q = acad1.SIf.copy() for i in range(1000):
q = q.rolling(10, min_periods=1, win_type='triang',center=True).mean() ax = acad1.resample('M').mean().plot(y='SIf', figsize=(14,4), label='monthly') q.resample('M').mean().plot(ax=ax,label='KZ Filter') plt.legend()
<matplotlib.legend.Legend at 0x1c290de080>
2.5.2.2 KMEANS Clustering using scikit-learn
Clustering algorithms can be very useful in finding signals within data. As an example we are going to use the KMeans algorithm from scikit-learn (http://scikit-learn.org/stable/modules/clustering.html#k-means) to analyse dust signals using the improve data. First we need to import some tools from sklearn to use in our analysis.
from sklearn.preprocessing import RobustScaler #to scale our data from sklearn.cluster import KMeans # clustering algorithm
2.5. Tutorial 25
First we want to separate out different variables that may be useful such as Si, PM2.5, PM10, Fe, and SOILf. We will also need to drop any NaN values so let us go ahead and do that.
dfkm = df1[['SIf','MF','MT','FEf','SOILf']].dropna() dfkm.head()
Usually, with sklearn it is better to scale the data first before putting it through the algorithm. We will use th Ro- bustScaler to do this.
X_scaled = RobustScaler().fit(dfkm).transform(dfkm)
Now we will define our clustering algorithm to have 2 clusters. You may need to adjust this as this is just a starting point for further analysis.
km = KMeans(n_clusters=2).fit(X_scaled)
The clusters can be found under km.labels_ . These are integers representing the different clusters.
clusters = km.labels_
Lets plot this so that we can see where there is dust.
plt.figure(figsize=(10,6)) plt.scatter(dfkm.SOILf,dfkm.MT,c=clusters,edgecolor='k') plt.xlabel('SOILf') plt.ylabel('PM10')
MONET Documentation
2.5.3 How to Open and View CMAQ simulations - HI Volcano Example
In this example, we will learn how to open CMAQ simulations using MONET, view the data on a map, view cross sections of the data, and compare to surface observations (AirNow) for the May 2018 Hawaiian volcanic eruption. First, import MONET and several helper functions for later.
from monet.obs import airnow from monet.models import cmaq import matplotlib.pyplot as plt from monet.util import tools import pandas as pd
Now the data can be downloaded from the MONET github page in the MONET/data directory. We will assume you already have this downloaded and will proceed. Open the simulation. As of right now we still require that a seperate grdcro2d (grddot2d) file be loaded for the mass points (dot points) using the grid kwarg.
conc = '/Users/barry/Desktop/MONET/data/aqm.t12z.aconc.ncf' c = cmaq.open_dtaset(flist=conc)
The cmaq object will return a xarray.Dataset but it also still lives inside of the cmaq object in cmaq.dset. We may use a combination of both the dataset (c) directly and the cmaq calls to gather or aggrigate species in the concentration file.
c
<xarray.Dataset> Dimensions: (DATE-TIME: 2, VAR: 41, time: 48, x: 80, y: 52, z: 1) Coordinates:
* time (time) datetime64[ns] 2018-05-17T12:00:00 2018-05-17T13:00:00 ... longitude (y, x) float64 -161.9 -161.8 -161.7 -161.6 -161.5 -161.4 ... latitude (y, x) float64 17.76 17.76 17.77 17.77 17.77 17.77 17.78 ...
Dimensions without coordinates: DATE-TIME, VAR, x, y, z Data variables:
TFLAG (time, VAR, DATE-TIME) int32 ... O3 (time, z, y, x) float32 28.296675 28.461445 28.483536 ... NO2 (time, z, y, x) float32 0.0072615896 0.007076008 0.0070880884 ... NO (time, z, y, x) float32 1.6760728e-08 1.6089658e-08 ... NO3 (time, z, y, x) float32 0.0022378012 0.0022129775 ... N2O5 (time, z, y, x) float32 8.560217e-06 8.241847e-06 ... HNO3 (time, z, y, x) float32 0.1326946 0.13388994 0.1343063 ... HONO (time, z, y, x) float32 6.2205704e-06 5.9552485e-06 ... PNA (time, z, y, x) float32 2.1638462e-07 1.9233373e-07 ... CO (time, z, y, x) float32 72.599915 72.56304 72.599266 ... FORM (time, z, y, x) float32 0.3351323 0.3413216 0.34457546 ... ALD2 (time, z, y, x) float32 0.13579664 0.12625198 0.1253843 ... PAN (time, z, y, x) float32 0.00054027676 0.00047953427 ... NTR (time, z, y, x) float32 0.0057188044 0.0053887735 0.005364259 ... XO2N (time, z, y, x) float32 2.1813818e-05 2.1018073e-05 ... SO2 (time, z, y, x) float32 0.009011468 0.009855208 0.009888159 ... ASO4I (time, z, y, x) float32 0.00071987335 0.0006097436 ... ASO4J (time, z, y, x) float32 0.6792807 0.68186235 0.6739671 ... ANH4I (time, z, y, x) float32 5.4321783e-05 4.6263896e-05 ... ANH4J (time, z, y, x) float32 0.051220056 0.051677197 0.05048342 ... ANO3I (time, z, y, x) float32 9.547329e-16 8.1884126e-16 ... ANO3J (time, z, y, x) float32 8.933346e-13 9.0928e-13 5.418493e-05 ...
2.5. Tutorial 27
MONET Documentation
AORGAI (time, z, y, x) float32 1.4356241e-10 1.308317e-10 ... AORGAJ (time, z, y, x) float32 1.2230534e-05 1.1428401e-05 ... AORGPAI (time, z, y, x) float32 6.7508597e-07 6.614316e-07 ... AORGPAJ (time, z, y, x) float32 0.05786852 0.058112975 0.05817498 ... AORGBI (time, z, y, x) float32 6.7122006e-09 6.5227788e-09 ... AORGBJ (time, z, y, x) float32 0.0005746193 0.00057244353 ... AECI (time, z, y, x) float32 2.4381194e-07 2.3888646e-07 ... AECJ (time, z, y, x) float32 0.014596849 0.014644699 0.014672826 ... A25I (time, z, y, x) float32 ... A25J (time, z, y, x) float32 ... NUMATKN (time, z, y, x) float32 ... NUMACC (time, z, y, x) float32 ... SRFATKN (time, z, y, x) float32 ... SRFACC (time, z, y, x) float32 ... AH2OI (time, z, y, x) float32 ... AH2OJ (time, z, y, x) float32 ... ACLI (time, z, y, x) float32 ... ACLJ (time, z, y, x) float32 3.5000003e-13 3.5000003e-13 ... ANAI (time, z, y, x) float32 8.394818e-17 8.4039525e-17 ... ANAJ (time, z, y, x) float32 0.2311452 0.23135261 0.23287562 ... PM25 (time, z, y, x) float32 1.0354732 1.0388906 1.0314605 ... PM10 (time, z, y, x) float32 1.0354732 1.0388906 1.0314605 ... CLf (time, z, y, x) float32 3.5000003e-13 3.5000003e-13 ... NAf (time, z, y, x) float32 0.2311452 0.23135261 0.23287562 ... NOy (time, z, y, x) float32 0.0001484681 0.00014906164 ... NOx (time, z, y, x) float32 1.6760728e-11 1.6089657e-11 ... NO3f (time, z, y, x) float32 8.9428936e-13 9.100988e-13 ... NH4f (time, z, y, x) float32 0.051274378 0.05172346 0.050527096 ... SO4f (time, z, y, x) float32 0.68000054 0.6824721 0.67454904 ...
Attributes: IOAPI_VERSION: $Id: @(#) ioapi library version 3.1 $ ... EXEC_ID: ???????????????? ... FTYPE: 1 CDATE: 2018142 CTIME: 135716 WDATE: 2018142 WTIME: 135716 SDATE: 2018137 STIME: 120000 TSTEP: 10000 NTHIK: 1 NCOLS: 80 NROWS: 52 NLAYS: 1 NVARS: 41 GDTYP: 2 P_ALP: 19.0 P_BET: 21.0 P_GAM: -157.5 XCENT: -157.5 YCENT: 20.53 XORIG: -480000.0 YORIG: -312000.0 XCELL: 12000.0
28 Chapter 2. Reference
MONET Documentation
YCELL: 12000.0 VGTYP: 1 VGTOP: 200.0 VGLVLS: [1. 0.089794] GDNAM: AQF_HI UPNAM: OPACONC VAR-LIST: O3 NO2 NO NO3 ... FILEDESC: Concentration file output ... HISTORY: proj4_srs: +proj=lcc +lat_1=19.0 +lat_2=21.0 +lat_0=20.53 +lon_0=-1... area: Area ID: MONET_Object_GridnDescription: IOAPI area_def ... mapping_tables: {'improve': {}, 'aqs': {'OZONE': ['O3'], 'PM2.5': ['PM25...
Notice that this looks like the ncdump of the file except that there are seperate coordinates including the latitude and longitude and the time as numpy.datetime64 objects. Also included is the proj4 string, a pyresample area grid and default mapping tables to several different observational datasets.
2.5.3.1 Plotting on a map
It is often useful to be able to view the data on a map. Let’s view a random time slice of SO2 (we will view time 20 hours into the simulation).
c.SO2[15,0,:,:].monet.quick_map()
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.4s
<matplotlib.collections.QuadMesh at 0x1c27910860>
2.5. Tutorial 29
MONET Documentation
Now this doesn’t look very pretty. First it isn’t on a map, the color scale isn’t good as we cannot really see any of the data. To fix this we will add a map using the MONETAccessor and use the robust=True kwarg.
c.SO2[15,0,:,:].monet.quick_map(robust=True)
Better but we can still do much more. There is low concentrations on most of this map making it hard to notice the
30 Chapter 2. Reference
MONET Documentation
extremely high values and the SO2 data is in ppmv and not ppbv as normally viewed as. Also, a logscale may be better fo this type of data as it goes from 0-20000 ppbv rather than a linear scale.
from matplotlib.colors import LogNorm # convert to ppbv so2 = c.SO2[15,0,:,:] so2.where(so2 > 0.1).monet.quick_map(robust=True, norm=LogNorm())
Now let’s us view serveral time slices at once. We will average in time (every 8 hours) to give us 6 total subplots.
so2 = c.SO2[:,0,:,:] * 1000. so2_resampled = so2.resample(time='8H').mean('time').sortby(['y', 'x'],ascending=True) p = so2_resampled.plot.contourf(col_wrap=3,col='time',x='longitude',y='latitude', →robust=True,figsize=(15,10),subplot_kws={'projection': ccrs.PlateCarree()}) extent = [so2.longitude.min(),so2.longitude.max(),so2.latitude.min(),so2.latitude. →max()] for ax in p.axes.flat:
draw_map(ax=ax,resolution='10m',extent=extent)
2.5.3.2 Finding nearest lat lon point
Suppose that we want to find the model data found at a point (latitude,longitude). Use the MONETAccessor again
so2.monet.nearest_latlon(lat=20.5,lon=157.5).plot(figsize=(12,6)) plt.xlabel('') plt.tight_layout()
32 Chapter 2. Reference
2.5.3.3 Pairing with AirNow
It is often useful to be able to pair model data with observational data. MONET uses the pyresample library (http: //pyresample.readthedocs.io/en/latest/) to do a nearest neighbor interpolation. First let us get the airnow data for the dates of the simulation. We will also rotate it from the raw AirNow long format (stacked variables) to a wide format (each variable is a seperate column)
from monet.util import tools df = airnow.add_data(so2.time.to_index()) df = tools.long_to_wide(df)
Aggregating AIRNOW files... Building AIRNOW URLs... [########################################] | 100% Completed | 1.3s [########################################] | 100% Completed | 1.4s [########################################] | 100% Completed | 1.4s [########################################] | 100% Completed | 1.5s [########################################] | 100% Completed | 1.5s [########################################] | 100% Completed | 12.3s [########################################] | 100% Completed | 12.4s [########################################] | 100% Completed | 12.4s [########################################] | 100% Completed | 12.5s [########################################] | 100% Completed | 12.6s
Adding in Meta-data
Now let us combine the two. This will return the pandas dataframe with a new column (model).
df_combined = so2.monet.combine_point(df,col='SO2')
(continues on next page)
[########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
Let’s look at the distributions to see if the two overlap to get a general scence of performance.
df_so2 = df.loc[(df.variable == 'SO2') & (df.state_name == 'HI')].dropna(subset=['obs →']) import seaborn as sns f,ax = plt.subplots(figsize=(12,7)) sns.kdeplot(df_so2.obs, ax=ax, clip=[0,500]) sns.kdeplot(df_so2.model,ax=ax, clip=[0,500])
<matplotlib.axes._subplots.AxesSubplot at 0x1c290e94a8>
2.5.3.4 Overlaying Observations on Contour Plots
Now let’s put a time slice on a map. Let’s look back to the time step ‘2018-05-18 03:00’,
from monet.plots import * ax = draw_map(states=True, resolution='10m', figsize=(15,7)) so2_now = so2.sel(time='2018-05-18 03:00') p = so2_now.plot(x='longitude',y='latitude',ax=ax, robust=True,norm=LogNorm(),cbar_ →kwargs={'label': 'SO2 (ppbv)'}) vmin,vmax = p.get_clim() cmap = p.get_cmap() d = df_so2.loc[df_so2.time == '2018-05-18 03:00'] plt.scatter(d.longitude.values,d.latitude.values,c=d.obs,cmap=cmap,vmin=vmin, →vmax=vmax)
34 Chapter 2. Reference
MONET Documentation
{'figsize': (15, 7), 'subplot_kw': {'projection': <cartopy.crs.PlateCarree object at →0x1c27451d00>}} [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.4s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.4s
<matplotlib.collections.PathCollection at 0x1c2ae207b8>
Not bad but again we can do a little better with the scatter plot. It’s hard to see the outlines of the observations when there is high correlation, the sizes may be a little large
ax = draw_map(states=True, resolution='10m', figsize=(15,7))
so2_now = so2.sel(time='2018-05-18 03:00') p = so2_now.plot(x='longitude',y='latitude',ax=ax, robust=True,norm=LogNorm(),cbar_ →kwargs={'label': 'SO2 (ppbv)'}) vmin,vmax = p.get_clim() cmap = p.get_cmap() d = df_so2.loc[df_so2.time == '2018-05-18 03:00'] plt.scatter(d.longitude.values,d.latitude.values,c=d.obs,cmap=cmap,s=100,edgecolors='k →',lw=.25, vmin=vmin,vmax=vmax)
{'figsize': (15, 7), 'subplot_kw': {'projection': <cartopy.crs.PlateCarree object at →0x1c2b0ead00>}} [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.1s
(continues on next page)
(continued from previous page)
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.4s [########################################] | 100% Completed | 0.5s
<matplotlib.collections.PathCollection at 0x1c2b43c5f8>
61.45120441800254
2.5.4 NOAA NESDIS AOD
NOAA NESDIS has an operational data product for the aerosol optical depth from the VIIRS satellite. There are two products, the viirs edr and viirs eps available from ftp://ftp.star.nesdis.noaa.gov/pub/smcd/.
2.5.4.1 VIIRS EDR
The VIIRS EDR data is an aerosol optical depth product available at 0.1 and 0.25 degree resolution. This VIIRS EDR product also does not include the Blue Sky algorithm to retrieve over bright surfaces such as the Sahara. Lets open the data on a single day at first, in this case ‘2018-07-05’.
36 Chapter 2. Reference
from monet.sat import nesdis_edr_viirs edr = nesdis_edr_viirs.open_dataset('2018-07-05') print(edr)
<xarray.DataArray 'VIIRS EDR AOD' (time: 1, y: 1800, x: 3600)> array([[[nan, nan, ..., nan, nan],
[nan, nan, ..., nan, nan], ..., [nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan]]], dtype=float32)
Coordinates:
* time (time) datetime64[ns] 2018-07-05
* y (y) int64 0 1 2 3 4 5 6 7 ... 1793 1794 1795 1796 1797 1798 1799
* x (x) int64 0 1 2 3 4 5 6 7 ... 3593 3594 3595 3596 3597 3598 3599 latitude (y, x) float64 -89.88 -89.88 -89.88 -89.88 ... 89.88 89.88 89.88 longitude (y, x) float64 -179.9 -179.8 -179.7 -179.6 ... 179.7 179.8 179.9
Attributes: units: long_name: Aerosol Optical Depth source: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/jhuang/npp.viirs.aero...
edr is now a xarray.DataArray for that day. The nesdis_edr_viirs module downloads data to the current directory. To download this into a different directory you can supply the datapath= keyword if needed. To quickly view this you can use the monet accessor.
edr.monet.quick_map(robust=True)
<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1022a77470>
The EDR data is available in two resolutions. By default monet will download the 0.1 degree dataset. If you would like the 0.25 degree dataset you can pass the kwarg resolution='low'.
edr = nesdis_edr_viirs.open_dataset('2018-07-05', resolution='low') print(edr)
2.5. Tutorial 37
MONET Documentation
<xarray.DataArray 'VIIRS EDR AOD' (time: 1, y: 720, x: 1440)> array([[[nan, nan, ..., nan, nan],
[nan, nan, ..., nan, nan], ..., [nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan]]], dtype=float32)
Coordinates:
* time (time) datetime64[ns] 2018-07-05
* y (y) int64 0 1 2 3 4 5 6 7 8 ... 712 713 714 715 716 717 718 719
* x (x) int64 0 1 2 3 4 5 6 7 ... 1433 1434 1435 1436 1437 1438 1439 latitude (y, x) float64 -89.88 -89.88 -89.88 -89.88 ... 89.88 89.88 89.88 longitude (y, x) float64 -179.9 -179.6 -179.4 -179.1 ... 179.4 179.6 179.9
Attributes: units: long_name: Aerosol Optical Depth source: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/jhuang/npp.viirs.aero...
Notice that the dimensions changed from 1800x3600 to 720x1440.
Open Multiple Days
If you want to open multiple days in a sinlge call you could use the open_mfdataset. Lets grab the first nine days of July 2018.
import pandas as pd dates = pd.date_range(start='2018-07-01',end='2018-07-09') edr = nesdis_edr_viirs.open_mfdataset(dates)
print(edr)
<xarray.DataArray 'VIIRS EDR AOD' (time: 9, y: 1800, x: 3600)> array([[[nan, nan, ..., nan, nan],
[nan, nan, ..., nan, nan], ..., [nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan]],
...,
[[nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan], ..., [nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan]],
[[nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan], ..., [nan, nan, ..., nan, nan],
(continues on next page)
38 Chapter 2. Reference
[nan, nan, ..., nan, nan]]], dtype=float32) Coordinates:
* y (y) int64 0 1 2 3 4 5 6 7 ... 1793 1794 1795 1796 1797 1798 1799
* x (x) int64 0 1 2 3 4 5 6 7 ... 3593 3594 3595 3596 3597 3598 3599 latitude (y, x) float64 -89.88 -89.88 -89.88 -89.88 ... 89.88 89.88 89.88 longitude (y, x) float64 -179.9 -179.8 -179.7 -179.6 ... 179.7 179.8 179.9
* time (time) datetime64[ns] 2018-07-01 2018-07-02 ... 2018-07-09 Attributes:
units: long_name: Aerosol Optical Depth source: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/jhuang/npp.viirs.aero...
We can visualize these in a seaborn FacetGrid through xarray. For more information on FacetGrid in xarray plotting please look here: http://xarray.pydata.org/en/stable/plotting.html#faceting
import cartopy.crs as ccrs # map projections and coastlines cbar_kwargs=dict(orientation='horizontal',pad=0.1, aspect=30) d = edr.plot.pcolormesh(x='longitude',y='latitude',col='time',col_wrap=3,
figsize=(12,12),robust=True,cbar_kwargs=cbar_kwargs, subplot_kws={'projection':ccrs.PlateCarree()})
for ax in d.axes.flat: ax.coastlines()
2.5. Tutorial 39
2.5.4.2 VIIRS EPS
The VIIRS EPS data includes the Blue Sky algorithm in the AOD calculation. The same methods are available as with the nesdis_edr_viirs methods.
from monet.sat import nesdis_eps_viirs eps = nesdis_eps_viirs.open_dataset('2018-07-05') print(eps)
<xarray.DataArray 'VIIRS EPS AOT' (time: 1, y: 720, x: 1440)> array([[[nan, nan, ..., nan, nan],
[nan, nan, ..., nan, nan], ..., [nan, nan, ..., nan, nan], [nan, nan, ..., nan, nan]]], dtype=float32)
Coordinates: latitude (y, x) float64 89.88 89.88 89.88 89.88 ... -89.88 -89.88 -89.88 longitude (y, x) float64 -179.9 -179.6 -179.4 -179.1 ... 179.4 179.6 179.9
* time (time) datetime64[ns] 2018-07-05 Dimensions without coordinates: y, x
(continues on next page)
40 Chapter 2. Reference
eps.monet.quick_map(robust=True)
<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c3406d080>
Notice that there are AOD values over deserts such as the Sahara, Australia, northern China, Mongolia and the Middle East
2.5.5 FV3-CHEM in MONET
FV3-CHEM is the experimental implementation of the NASA Gocart model into NOAA’s Next Generation Global Prediction System. This tutorial will explain how to open read and use MONET to explore output from the FV3- CHEM system.
Currently, FV3 outputs data in three formats, i.e., nemsio, grib2, and netcdf. There are slight issues with both the nemsio and grib2 data that will be explained below but needless to say they need to be converted with precossors.
2.5.5.1 Convert nemsio2nc4
The first format that needs to be converted is the nemsio data. This is a binary format used within EMC that is planned to be phased out in favor for netCDF. As a work around a converter was created to convert the binary nemsio data into netCDF, nemsio2nc4.py.
This is a command-line script that converts the nemsio data output by the FV3 system to netcdf4 files using a combi- nation of mkgfsnemsioctl found within the fv3 workflow and the climate data operators (cdo). This tool is available on github at https://github.com/bbakernoaa/nemsio2nc4
2.5. Tutorial 41
nemsio2nc4.py --help usage: nemsio2nc4.py [-h] -f FILES [-n NPROCS] [-v]
convert nemsio file to netCDF4 file
optional arguments: -h, --help show this help message and exit -f FILES, --files FILES
input nemsio file name (default: None) -v, --verbose print debugging information (default: False)
Example Usage
If you want to convert a single nemsio data file to netcdf4, it can be done like this:
nemsio2nc4.py -v -f 'gfs.t00z.atmf000.nemsio' mkgfsnemsioctl: /gpfs/hps3/emc/naqfc/noscrub/Barry.Baker/FV3CHEM/exec/mkgfsnemsioctl cdo: /naqfc/noscrub/Barry.Baker/python/envs/monet/bin/cdo Executing: /gpfs/hps3/emc/naqfc/noscrub/Barry.Baker/FV3CHEM/exec/mkgfsnemsioctl gfs. →t00z.atmf000.nemsio Executing: /naqfc/noscrub/Barry.Baker/python/envs/monet/bin/cdo -f nc4 import_binary →gfs.t00z.atmf000.nemsio.ctl gfs.t00z.atmf000.nemsio.nc4 cdo import_binary: Processed 35 variables [1.56s 152MB]
To convert multiple files you can simple use the hot keys available in linux terminals.
nemsio2nc4.py -v -f 'gfs.t00z.atmf0*.nemsio'
2.5.5.2 Convert fv3grib2nc4
Although there are several ways in which to read grib data from python such as pynio, a converter was created due to the specific need to distinguish aerosols which these readers do not process well.
fv3grib2nc4.py like nemsio2nc4.py tool is a command line tool created to convert the grib2 aerosol data to netcdf files. fv3grib2nc4.py will create seperate files for each of the three layer types; ‘1 hybrid layer’, ‘entire atmosphere’, and ‘surface’. These are the three layers that currently hold aerosol data. The tool is available at https://github.com/ bbakernoaa/fv3grib2nc4
fv3grib2nc4.py --help usage: fv3grib2nc4.py [-h] -f FILES [-v]
convert nemsio file to netCDF4 file
optional arguments: -h, --help show this help message and exit -f FILES, --files FILES
input nemsio file name (default: None) -v, --verbose print debugging information (default: False) ```
### Example Usage
If you want to convert a single grib2 data file to netcdf4, it can be done like this:
fv3grib2nc4.py -v -f 'gfs.t00z.master.grb2f000' wgrib2:
(continues on next page)
42 Chapter 2. Reference
/nwprod2/grib\_util.v1.0.0/exec/wgrib2 Executing: /nwprod2/grib\_util.v1.0.0/exec/wgrib2 gfs.t00z.master.grb2f000 -match "entire atmosphere:" -nc\_nlev 1 -append -set\_ext\_name 1 -netcdf gfs.t00z.master.grb2f000.entire\_atm.nc Executing: /nwprod2/grib\_util.v1.0.0/exec/wgrib2 gfs.t00z.master.grb2f000 -match "1 hybrid level:" -append -set\_ext\_name 1 -netcdf gfs.t00z.master.grb2f000.hybrid.nc Executing: /nwprod2/grib\_util.v1.0.0/exec/wgrib2 gfs.t00z.master.grb2f000 -match "surface:" -nc\_nlev 1 -append -set\_ext\_name 1 -netcdf gfs.t00z.master.grb2f000.surface.nc\`\`\`
To convert multiple files you can simple use the hot keys available in linux terminals.
fv3grib2nc4.py -v -f 'gfs.t00z.master.grb2f0*'
2.5.5.3 Using MONET
Using MONET with FV3-Chem is much like using MONET with other model outputs. It tries to recognize where the files came from (nemsio, grib2, etc. . . .) and then processes the data, renaming coordinates (lat lon to latitude and longitude) and processing varaibles like geopotential height and pressure if available. First lets import monet and fv3chem from MONET
import matplotlib.pyplot as plt from monet.models import fv3chem
To open a single file
f = fv3chem.open_dataset('/Users/barry/Desktop/temp/gfs.t00z.atmf006.nemsio.nc4') print(f)
* time (time) datetime64[ns] 2018-07-01T06:00:00
* x (x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1
* y (y) float64 89.28 88.36 87.42 86.49 ... -87.42 -88.36 -89.28
* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0 longitude (y, x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1 latitude (y, x) float64 89.28 89.28 89.28 89.28 ... -89.28 -89.28 -89.28
Data variables: ugrd (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> vgrd (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> dzdt (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> delz (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> tmp (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> dpres (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> spfh (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),
→chunksize=(1, 64, 192, 384)> (continues on next page)
2.5. Tutorial 43
(continued from previous page)
clwmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
rwmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
icmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
snmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
grle (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
cld_amt (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
o3mr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
so2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
sulf (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
dms (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
msa (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
pm25 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
bc1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
bc2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
oc1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
oc2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
dust1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
dust2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
dust3 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
dust4 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
dust5 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
seas1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
seas2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
seas3 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
seas4 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
pm10 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)>
pressfc (time, y, x) float32 dask.array<shape=(1, 192, 384), chunksize=(1, 192, → 384)>
hgtsfc (time, y, x) float32 dask.array<shape=(1, 192, 384), chunksize=(1, 192, → 384)>
geohgt (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384), →chunksize=(1, 64, 192, 384)> (continues on next page)
44 Chapter 2. Reference
Attributes: CDI: Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/... Conventions: CF-1.6 history: Thu Dec 20 17:46:09 2018: cdo -f nc4 import_binary gfs.t00z... CDO: Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/...
Notice this object f has dimensions of (time,z,y,x) with 2d coordinates of latitude and longitude. You can get more infomation on single varaibles such as pm25 simply by printing the varaible.
print(f.pm25)
<xarray.DataArray 'pm25' (time: 1, z: 64, y: 192, x: 384)> dask.array<shape=(1, 64, 192, 384), dtype=float32, chunksize=(1, 64, 192, 384)> Coordinates:
* time (time) datetime64[ns] 2018-07-01T06:00:00
* x (x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1
* y (y) float64 89.28 88.36 87.42 86.49 ... -87.42 -88.36 -89.28
* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0 longitude (y, x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1 latitude (y, x) float64 89.28 89.28 89.28 89.28 ... -89.28 -89.28 -89.28
Attributes: long_name: model layer
Here units are not included because it is not stored in the nemsio format.
Quick Map Plotting
Now one of the main things that will need to be done is plotting on a map. This can be done quickly using the functionallity in MONET. In this example we will plot the first layer PM2.5 at time 2018-07-01.
f.pm25[0,0,:,:].monet.quick_map()
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c164f2dd8>
2.5. Tutorial 45
MONET Documentation
Adjusting the scale is simple by suppling vmin and vmax. Lets set a minimum of 0 AOD and maximum of 0.5.
f.pm25[0,0,:,:].monet.quick_map(vmin=0,vmax=.5)
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c18cbf6d8>
Now we have all the control that xarray has built into their plotting routines. For example, lets have a descrete colorbar with 10 levels, levels=10, and let it determine the levels by throwing out the top and bottom 2% of values using
46 Chapter 2. Reference
f.pm25[0,0,:,:].monet.quick_map(levels=10,robust=True)
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c1905ce48>
Now there are a lot of very low values, since this is at the beginning of the simulation so lets mask out values less than 0.015 AOD.
f.pm25.where(f.pm25 > 0.015)[0,0,:,:].monet.quick_map(levels=10,robust=True)
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c19635240>
2.5. Tutorial 47
MONET Documentation
For more information on plotting with xarray and matplotlib some useful links are shown below
• Xarray Plotting
• Matplotlib Colorbars
Nearest neighbor
Monet has some extra functionality that may be useful for exploritory studies such as nearest neighbor finder. Lets find the nearest neighbor to NCWCP (38.972 N, 76.9245 W).
nn = f.pm25.monet.nearest_latlon(lat=38.972,lon=-76.9245) print(nn)
<xarray.DataArray 'pm25' (time: 1, z: 64)> dask.array<shape=(1, 64), dtype=float32, chunksize=(1, 64)> Coordinates:
* time (time) datetime64[ns] 2018-07-01T06:00:00 x float64 283.1 y float64 38.81
* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0 longitude float64 283.1 latitude float64 38.81
Attributes: long_name: model layer
Now we can do a quick plot of this vertically, since it was a single time step.
nn.plot(aspect=2,size=5)
48 Chapter 2. Reference
[<matplotlib.lines.Line2D at 0x1c194702e8>]
Now this is a simple plot but it is usually valuable to view the vertical coordinate on the y-axis.
nn.plot(y='z',aspect=2,size=5)
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
[<matplotlib.lines.Line2D at 0x1c1c0bbac8>]
2.5. Tutorial 49
MONET Documentation
Now this is not very useful because the vertical coordinate right now is just the layer number. Lets get the geopoential height at this location and add it as a coordinate to plot.
geohgt = f.geohgt.monet.nearest_latlon(lat=38.972,lon=-76.9245).squeeze() geohgt
<xarray.DataArray 'geohgt' (z: 64)> dask.array<shape=(64,), dtype=float32, chunksize=(64,)> Coordinates:
time datetime64[ns] 2018-07-01T06:00:00 x float64 283.1 y float64 38.81
* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0 longitude float64 283.1 latitude float64 38.81
Attributes: long_name: Geopotential Height units: m
nn['z'] = geohgt.values nn.plot(y='z') plt.ylim([0,3000])
[########################################] | 100% Completed | 0.3s [########################################] | 100% Completed | 0.4s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
(0, 3000)
Constant Latitude and Longitude
Sometimes it may be useful to see a latitudinal or longitudinal cross section. This feature is included in monet through the xr.DataArray.monet accessor. Lets take a constant latitude at 10 degrees N.
pm25_constant_lat = f.pm25.monet.interp_constant_lat(lat=10., method='bilinear') pm25_constant_lat
Create weight file: bilinear_192x384_192x1.nc [########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
<xarray.DataArray 'pm25' (time: 1, z: 64, x: 192, y: 1)> array([[[[8.107865e-02],
..., [4.765707e-02]],
[[9.984020e-05], ..., [9.982444e-05]]]])
Coordinates: longitude (x, y) float64 0.0 1.88 3.76 5.64 ... 353.4 355.3 357.2 359.1 latitude (x, y) float64 10.0 10.0 10.0 10.0 10.0 ... 10.0 10.0 10.0 10.0
* time (time) datetime64[ns] 2018-07-01T06:00:00
* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0 Dimensions without coordinates: x, y
(continues on next page)
Attributes: regrid_method: bilinear
Like before lets go ahead and get the geopotential height along this latitude.
geoght_constant_lat = f.geohgt.monet.interp_constant_lat(lat=10., method='bilinear') pm25_constant_lat['geohgt'] = geoght_constant_lat
[########################################] | 100% Completed | 0.7s [########################################] | 100% Completed | 0.8s
Let us plot the 2D cross track (height vs longitude).
pm25_constant_lat.plot(x='longitude',y='geohgt',robust=True,ylim=1000,aspect=2,size=5) plt.ylim([0,50000])
(0, 50000)
Calculate Pressure Levels
By default pressure levels are not calculated due to processing time, however, MONET does include a helper function in the fv3chem module, fv3chem.calc_nemsio_pressure.
f = fv3chem.calc_nemsio_pressure(f) f
(continues on next page)
52 Chapter 2. Reference
[########################################] | 100% Completed | 0.1s [########################################] | 100% Completed | 0.2s
<xarray.DataArray 'press' (time: 1, z: 64, y: 192, x: 384)> array([[[[1.002220e+03, ..., 1.002230e+03],
..., [6.768812e+02, ..., 6.770022e+02]],
Coordinates:
* time (time) datetime64[ns] 2018-07-01T06:00:00
* x (x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1
* y (y) float64 89.28 88.36 87.42 86.49 ... -87.42 -88.36 -89.28
* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0 longitude (y, x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1 latitude (y, x) float64 89.28 89.28 89.28 89.28 ... -89.28 -89.28 -89.28
Attributes: units: mb long_name: Mid Layer Pressure
2.6 Get in touch
• Ask questions, suggest features or view source code on GitHub.
Help & Reference
Ask questions, suggest features or view source code on GitHub.
If an issue arrises please post on the GitHub issues.
2.6.2 API
cmap: colormap instance, eg. cm.jet. N: number of colors.
Example x = resize(arange(100), (5,100)) djet = cmap_discretize(cm.jet, 5) imshow(x, cmap=djet)
monet.plots.colorbars.colorbar_index(ncolors, cmap, minval=None, maxval=None, dtype=’int’, basemap=None)
monet.plots.mapgen module
map utilities
Short summary.
• ax (type) – Description of parameter ax (the default is None).
• natural_earth (bool) – Description of parameter natural_earth (the default is True).
• coastlines (bool) – Description of parameter coastlines (the default is True).
• states (bool) – Description of parameter states (the default is True).
• countries (bool) – Description of parameter countries (the default is True).
• state_resolutions (bool) – Description of parameter state_resolutions (the default is ‘10m’).
• extent ([lon_min,lon_max,lat_min,lat_max]) – Description of parameter extent (the default is None).
Returns Description of returned object.
Return type type
Parameters
• col (type) – Description of parameter col (the default is ‘obs’).
• title (type) – Description of parameter title (the default is None).
• label (type) – Description of parameter label (the default is None).
54 Chapter 2. Reference
• ax (type) – Description of parameter ax (the default is ax).
• **kwargs (type) – Description of parameter **kwargs.
Returns Description of returned object.
Return type type
monet.plots.plots.make_spatial_plot(modelvar, m, dpi=None, plotargs={}, ncolors=15, dis- crete=False)
monet.plots.plots.normval(vmin, vmax, cmap)
monet.plots.plots.scatter(df, x=None, y=None, title=None, label=None, ax=None, **kwargs) Short summary.
Parameters
• x (type) – Description of parameter x (the default is ‘obs’).
• y (type) – Description of parameter y (the default is ‘model’).
• **kwargs (type) – Description of parameter **kwargs.
Returns Description of returned object.
Return type type
monet.plots.plots.taylordiagram(df, marker=’o’, col1=’obs’, col2=’model’, label1=’OBS’, la- bel2=’MODEL’, scale=1.5, addon=False, dia=None)
monet.plots.plots.timeseries(df, x=’time’, y=’obs’, ax=None, plotargs={}, fillargs={’alpha’: 0.2}, title=”, ylabel=None, label=None)
Short summary.
• col (type) – Description of parameter col (the default is ‘Obs’).
• ax (type) – Description of parameter ax (the default is None).
• sample (type) – Description of parameter sample (the default is ‘H’).
• plotargs (type) – Description of parameter plotargs (the default is {}).
• fillargs (type) – Description of parameter fillargs (the default is {}).
• title (type) – Description of parameter title (the default is ‘’).
• label (type) – Description of parameter label (the default is None).
Returns Description of returned object.
Return type type
MONET Documentation
monet.plots.taylordiagram module
class monet.plots.taylordiagram.TaylorDiagram(refstd, scale=1.5, fig=None, rect=111, la- bel=’_’)
Bases: object
Taylor diagram: plot model standard deviation and correlation to reference (data) sample in a single-quadrant polar plot, with r=stddev and theta=arccos(correlation).
add_contours(levels=5, **kwargs) Add constant centered RMS difference contours.
add_sample(stddev, corrcoef, *args, **kwargs) Add sample (stddev,corrcoeff) to the Taylor diagram. args and kwargs are directly propagated to the Figure.plot command.
Module contents
monet.plots.taylordiagram(df, marker=’o’, col1=’obs’, col2=’model’, label1=’OBS’, la- bel2=’MODEL’, scale=1.5, addon=False, dia=None)
monet.util package
Parameters
• df (pd.DataFrame) – Description of parameter df.
• lay (iterable, default = [0]) – Description of parameter lay.
• radius (integer or float, default = 12e3) – Description of parameter radius.
Returns
Parameters
• lay (iterable, default = [0]) – Description of parameter lay.
56 Chapter 2. Reference
• radius (integer or float, default = 12e3) – Description of parameter radius.
Returns
Parameters
• lay (iterable, default = [0]) – Description of parameter lay.
• radius (integer or float, default = 12e3) – Description of parameter radius.
Returns
Return type pandas.DataFrame
monet.util.combinetool.combine_da_to_height_profile(da, dset, ra- dius_of_influence=12000.0)
This function will combine an xarray.DataArray to a 2d dataset with dimensions (time,z)
Parameters
• da (xarray.DataArray) – Description of parameter da.
• dset (xarray.Dataset) – Description of parameter dset.
Returns returns the xarray.Dataset with the da added as an additional variable.
Return type xarray.Dataset
Parameters
• latitude (float) – latitude for constant
Returns
Parameters
• latitude (float) – latitude for constant
2.6. Get in touch 57
MONET Documentation
Parameters
Returns
Parameters
• longitude (2d xarray.DataArray) – Longitude -> must be from -180 -> 180 and monotoni- cally increasing
• latitude (2d xarray.DataArray) – Latitude -> must be from -90 -> 90 and monotonically increasing
Returns
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
monet.util.stats module
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
2.6. Get in touch 59
MONET Documentation
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
monet.util.stats.HSS(obs, mod, minval, maxval) Heidke Skill Score (1 is perfect - below zero means no confidence)
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
monet.util.stats.ME(obs, mod, axis=None) Mean Gross Error (model and obs unit)
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
2.6. Get in touch 61
MONET Documentation
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
monet.util.stats.MdnE(obs, mod, axis=None) Median Gross Error (model and obs unit)
62 Chapter 2. Reference
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
2.6. Get in touch 63
MONET Documentation
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
2.6. Get in touch 65
MONET Documentation
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
2.6. Get in touch 67
MONET Documentation
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
68 Chapter 2. Reference
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
monet.util.stats.RMSE(obs, mod, axis=None) Root Mean Square Error (model unit)
Parameters
MONET Documentation
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
2.6. Get in touch 71
MONET Documentation
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
72 Chapter 2. Reference
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
2.6. Get in touch 73
MONET Documentation
Returns Description of returned object.
Return type type
monet.util.stats.circlebias(b) Short summary.
Returns Description of returned object.
Return type type
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Returns Description of returned object.
Return type type
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
monet.util.tools.calc_annual_ave(df, col=None)
monet.util.tools.get_relhum(temp, press, vap)
monet.util.tools.kolmogorov_zurbenko_filter(df, col, window, iterations) KZ filter implementation series is a pandas series window is the filter window m in the units of the data (m = 2q+1) iterations is the number of times the moving average is evaluated
monet.util.tools.linregress(x, y)
MONET Documentation
Module contents
DESCRIPTION
This package contains the main routines for estimating variables related to the Monin-Obukhov (MO) Similar- ity Theory, such as MO length, adiabatic correctors
for heat and momentum transport. It requires the following package.
monet.met_funcs.calc_L(ustar, T_A_K, rho, c_p, H, LE) Calculates the Monin-Obukhov length.
Parameters
• T_A_K (float) – air temperature (Kelvin).
• rho (float) – air density (kg m-3).
• c_p (float) – Heat capacity of air at constant pressure (J kg-1 K-1).
• H (float) – sensible heat flux (W m-2).
• LE (float) – latent heat flux (W m-2).
Returns L – Obukhov stability length (m).
Return type float
Parameters zoL (float) – stability coefficient (unitless).
Returns Psi_H – adiabatic corrector factor fof heat transport (unitless).
Return type float
Returns Psi_M – adiabatic corrector factor fof momentum transport (unitless).
Return type float
monet.met_funcs.calc_c_p(p, ea) Calculates the heat capacity of air at constant pressure.
Parameters
• p (float) – total air pressure (dry air + water vapour) (mb).
• ea (float) – water vapor pressure at reference height above canopy (mb).
Returns c_p
Return type heat capacity of (moist) air at constant pressure (J kg-1 K-1)
References
based on equation (6.1) from Maarten Ambaum (2010): Thermal Physics of the Atmosphere (pp 109).
monet.met_funcs.calc_delta_vapor_pressure(T_K) Calculate the slope of saturation water vapour pressure.
Parameters T_K (float) – temperature (K).
Returns s – slope of the saturation water vapour pressure (kPa K-1)
Return type float
Parameters T_A_K (float) – Air temperature (Kelvin).
Returns Lambda – Latent heat of vaporisation (J kg-1).
Return type float
monet.met_funcs.calc_lapse_rate_moist(T_A_K, ea, p) Calculate moist-adiabatic lapse rate (K/m)
Parameters
• T_A_K (float or numpy array) – air temperature at reference height (K).
• ea (float or numpy array) – water vapor pressure at reference height (mb).
• p (float or numpy array) – total air pressure (dry air + water vapour) at reference height (mb).
Returns Gamma_w – moist-adiabatic lapse rate (K/m)
Return type float or numpy array
2.6. Get in touch 77
MONET Documentation
Parameters
• ea (float or numpy array) – water vapor pressure at reference height (mb).
• p (float or numpy array) – total air pressure (dry air + water vapour) at reference height (mb).
Returns r – mixing ratio (-)
References
http://glossary.ametsoc.org/wiki/Mixing_ratio
Parameters z (float) – height above sea level (m).
Returns p – air pressure (mb).
Return type float
Parameters
• c_p (float) – heat capacity of (moist) air at constant pressure (J kg-1 K-1).
• p (float) – atmopheric pressure (mb).
• Lambda (float) – latent heat of vaporzation (J kg-1).
Returns psicr – Psicrometric constant (mb C-1).
Return type float
Parameters
• p (float) – total air pressure (dry air + water vapour) (mb).
• ea (float) – water vapor pressure at reference height above canopy (mb).
• T_A_K (float) – air temperature at reference height (Kelvin).
Returns rho – density of air (kg m-3).
Return type float
References
based on equation (2.6) from Brutsaert (2005): Hydrology - An Introduction (pp 25).
monet.met_funcs.calc_richardson(u, z_u, d_0, T_R0, T_R1, T_A0, T_A1) Richardson number.
Estimates the Bulk Richardson number for turbulence using time difference temperatures.
Parameters
• z_u (float) – Wind speed measurement height (m).
• d_0 (float) – Zero-plane displacement height (m).
• T_R0 (float) – radiometric surface temperature at time 0 (K).
• T_R1 (float) – radiometric surface temperature at time 1 (K).
• T_A0 (float) – air temperature at time 0 (K).
• T_A1 (float) – air temperature at time 1 (K).
Returns Ri – Richardson number.
Return type float
Parameters
• lat (float) – latitude of the site (degrees).
• long (float) – longitude of the site (degrees).
• stdlng (float) – central longitude of the time zone of the site (degrees).
• doy (float) – day of year of measurement (1-366).
• ftime (float) – time of measurement (decimal hours).
Returns
monet.met_funcs.calc_theta_s(xlat, xlong, stdlng, doy, year, ftime) Calculates the Sun Zenith Angle (SZA).
Parameters
2.6. Get in touch 79
MONET Documentation
• xlong (float) – longitude of the site (degrees).
• stdlng (float) – central longitude of the time zone of the site (degrees).
• doy (float) – day of year of measurement (1-366).
• year (float) – year of measurement .
• ftime (float) – time of measurement (decimal hours).
Returns theta_s – Sun Zenith Angle (degrees).
Return type float
References
Adopted from Martha Anderson’s fortran code for ALEXI which in turn was based on Cupid.
monet.met_funcs.calc_u_star(u, z_u, L, d_0, z_0M) Friction velocity.
Parameters
• z_u (float) – wind speed measurement height (m).
• L (float) – Monin Obukhov stability length (m).
• d_0 (float) – zero-plane displacement height (m).
• z_0M (float) – aerodynamic roughness length for momentum transport (m).
References
Parameters T_K (float) – temperature (K).
Returns ea – saturation water vapour pressure (mb).
Return type float
monet.met_funcs.flux_2_evaporation(flux, T_K=293.15, time_domain=1) Converts heat flux units (W m-2) to evaporation rates (mm time-1) to a given temporal window
Parameters
• flux (float or numpy array) – heat flux value to be converted, usually refers to latent heat flux LE to be converted to ET
• T_K (float or numpy array) – environmental temperature in Kelvin. Default=20 Celsius
• time_domain (float) – Temporal window in hours. Default 1 hour (mm h-1)
Returns ET – evaporation rate at the time_domain. Default mm h-1
Return type float or numpy array
80 Chapter 2. Reference
obj Description of attribute obj.
Type type
Returns Description of returned object.
Return type type
Parameters
Returns Description of returned object.
Return type type
Parameters lon (float) – Latitude on which to interpolate to
Returns DataArray of at constant longitude
Return type DataArray
Returns DataArray of at constant longitude
Return type DataArray
is_land(return_xarray=False) checks the mask of land and ocean.
Parameters return_xarray (bool) – If True, return the data array with the ocean values set to NaN. If False, return a numpy boolean array of the land (True).
Returns
MONET Documentation
is_ocean(return_xarray=False) checks the mask of land and ocean.
Parameters return_xarray (bool) – If True, return the data array with the land values set to NaN. If False, return a numpy boolean array of the ocean (True).
Returns
Return type xarray.DataArray or numpy.array
nearest_ij(lat=None, lon=None, **kwargs) Uses pyresample to intepolate to find the i, j index of grid with respect to the given lat lon.
Parameters
• **kwargs (dict) – pyresample kwargs for nearest neighbor interpolation
Returns Returns the i (x index) and j (y index) of the given latitude longitude value
Return type i,j
nearest_latlon(lat=None, lon=None, cleanup=True, esmf=False, **kwargs) Uses xesmf to intepolate to a given latitude and longitude. Note that the conservative method is not available.
Parameters
Returns Description of returned object.
Return type type
quick_contourf(map_kws={}, roll_dateline=False, **kwargs) This function takes an xarray DataArray and quickly cerates a figure using cartopy and the matplotlib contourf
Parameters
• map_kws (dictionary) – kwargs for monet.plots.mapgen.draw_map
• roll_dateline (bool) – roll_dateline is meant to help with global datasets that the longitudes range from 0 to 360 instead of -180 to 180. Otherwise a white line appears at 0 degrees.
• **kwargs – kwargs for the xarray.DataArray.plot.contourf function
Returns axes
Return type type
quick_imshow(map_kws={}, roll_dateline=False, **kwargs) This function takes an xarray DataArray and quickly cerates a figure using cartopy and the matplotlib imshow. Note that this should only be used for regular grids.
Parameters
• map_kws (dictionary) – kwargs for monet.plots.mapgen.draw_map
• roll_dateline (bool) – roll_dateline is meant to help with global datasets that the longitudes range from 0 to 360 instead of -180 to 180. Otherwise a white line appears at 0 degrees.
82 Chapter 2. Reference
Returns axes
quick_map(map_kws={}, roll_dateline=False, **kwargs) This function takes an xarray DataArray and quickly cerates a figure using cartopy and the matplotlib pcolormesh
Parameters
• map_kws (dictionary) – kwargs for monet.plots.mapgen.draw_map
• roll_dateline (bool) – roll_dateline is meant to help with global datasets that the longitudes range from 0 to 360 instead of -180 to 180. Otherwise a white line appears at 0 degrees.
• **kwargs – kwargs for the xarray.DataArray.plot.pcolormesh function
Returns axes
remap_nearest(data, **kwargs)
Interpolates from another grid (data) to the current grid of self using pyresample. it assumes that the dimensions are ordered in y,x,z per
pyresample docs
• da (xarray DataArray or xarray DataSet) – Object to be interpolated
• radius_of_influence (float or integer) – radius of influcence for pyresample in meters.
Returns resampled object on current grid.
Return type xarray.DataArray
remap_xesmf(dataarray, method=’bilinear’, **kwargs) remaps from another grid to the current grid of self using xesmf
Parameters
• radius_of_influence (float or integer) – radius of influcence for pyresample in meters.
Returns resampled object on current grid.
Return type xarray.DataArray
Parameters
Returns Description of returned object.
Return type type
MONET Documentation
Parameters
Returns Description of returned object.
Return type type
tidy(lon_name=’longitude’) Tidy’s DataArray–wraps longitudes and sorts lats and lons
Returns The tidy object
Return type xr.DataArray
window(lat_min=None, lon_min=None, lat_max=None, lon_max=None, rectilinear=False) Function to window, ie select a specific region, given the lower left latitude and longitude and the upper right latitude and longitude
Parameters
• lat_min (float) – lower left latitude .
• lon_min (float) – lower left longitude.
• lat_max (float) – upper right latitude.
• lon_max (float) – upper right longitude.
• rectilinear (bool) – flag if this is a rectilinear lat lon grid
Returns returns the windowed object.
Return type xr.DataArray
Returns Description of returned object.
Return type type
Parameters xray_obj (type) – Description of parameter xray_obj.
obj Description of attribute obj.
Type type
Parameters name (str) – time variable name.
Returns Description of returned object.
Return type xarray.DataArray
Parameters
Returns Description of returned object.
Return type type
Parameters lon (float) – Latitude on which to interpolate to
Returns DataArray of at constant longitude
Return type DataArray
Returns DataArray of at constant longitude
Return type xr.Dataset or xr.DataArray
is_land(