Download - MONET Documentation - readthedocs.org

MONET Documentation

Barry Baker

Jan 26, 2022

Getting Started

1 What’s New 3

2 Reference 52.1 Overview: Why MONET? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Gallery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2.1 Required dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1.1 For parallel computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2.1.2 For plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.2 Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3 MONET XArray Accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3.1 Initializing the Accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3.2 Interpolation Accessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3.2.1 Find Nearest Lat lon point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4 Installing on WCOSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.5 Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.5.1 How to read AQS data from PAMS and do a quick analysis . . . . . . . . . . . . . . . . . . 142.5.1.1 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.5.2 Loading IMPROVE Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.5.2.1 Trend Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.5.2.2 KMEANS Clustering using scikit-learn . . . . . . . . . . . . . . . . . . . . . . . . 25

2.5.3 How to Open and View CMAQ simulations - HI Volcano Example . . . . . . . . . . . . . . 272.5.3.1 Plotting on a map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5.3.2 Finding nearest lat lon point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.5.3.3 Pairing with AirNow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.5.3.4 Overlaying Observations on Contour Plots . . . . . . . . . . . . . . . . . . . . . . 34

2.5.4 NOAA NESDIS AOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.5.4.1 VIIRS EDR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.5.4.2 VIIRS EPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.5.5 FV3-CHEM in MONET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.5.5.1 Convert nemsio2nc4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.5.5.2 Convert fv3grib2nc4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422.5.5.3 Using MONET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.6 Get in touch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.6.1 Get in touch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

i

2.6.2 API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.6.2.1 monet package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.6.2.2 monet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Bibliography 89

Python Module Index 91

Index 93

ii

MONET Documentation

MONET is an open source project and Python package that aims to create a common platform for atmosphericcomposition data analysis for weather and air quality models.

MONET was developed to evaluate the Community Multiscale Air Quality Model (CMAQ) for the NOAA NationalAir Quality Forecast Capability (NAQFC) modeling system. MONET is designed to be a modularized Python packagefor (1) pairing model output to observational data in space and time; (2) leveraging the pandas Python package foreasy searching and grouping; and (3) analyzing and visualizing data. This process introduces a convenient methodfor evaluating model output. MONET processes data that is easily searchable and that can be grouped using meta-data found within the observational datasets. Common statistical metrics (e.g., bias, correlation, and skill scores),plotting routines such as scatter plots, timeseries, spatial plots, and more are included in the package. MONET is wellmodularized and can add further observational datasets and different models.

Our goals is to provide easy tools to retrieve, read and combine datasets in order to speed scientific research. Currently,MONET is able to process several models and observations related to air composition and meteorology.

Please site our work.

Getting Started 1

MONET Documentation

2 Getting Started

CHAPTER 1

What’s New

MONET v2.2.0 has been released. MONET has re-engineered the way it deals with multidimensional observationsor model output by using an Xarray Accessor giving MONET a flexible and intuitive way of expanding Xarray formultidimensional geospatial information commonly used in meteorology, climate and air quality all while making iteasier on the user to use MONET and add to it.

MONET also underwent a major restructure with v2.2.0. All I/O functions have been moved to a sister repositoryhttps://github.com/noaa-oar-arl/monetio.

Features include:

• Xarray Accessor for both xarray.DataArray and xarray.Dataset using the .monet attribute

• Pandas Accessor for pandas.DataFrame using the .monet attribute

• vertical interpolation using python-stratify (https://github.com/SciTools-incubator/python-stratify) using the.monet.stratify function

• spatial interpolation using .monet.remap including: - Nearest neighbor finder - Constant latitude interpolation -Constant longitude interpolation - remap DataArray to current grid using pyresample nearest neighbor or xesmf- remap entire dataset to current grid using pyresample nearest neighbor or xesmf - find nearest i,j or lat lon -interpolate to constant latitude or longitude

• Simplified combine tool to combine point source data with multidimensional xarray objects

3

https://github.com/noaa-oar-arl/monetio

https://github.com/SciTools-incubator/python-stratify

MONET Documentation

4 Chapter 1. What’s New

CHAPTER 2

Reference

Baker, Barry; Pan, Li. 2017. “Overview of the Model and Observation Evaluation Toolkit (MONET) Version 1.0 forEvaluating Atmospheric Transport Models.” Atmosphere 8, no. 11: 210

2.1 Overview: Why MONET?

2.1.1 Features

Retrieving, loading, and combining data and putting into a common format is the core of MONET. MONET uses thepandas and xarray_ data formats for data analysis.

• Open point observations in a common format. pandas excels at working with tabular data or point measurements.It is used for time series analysis and statistical measures.

• Open model and satellite data in a common format. xarray_ is used when N-dimensional arrays are needed.

• Retrieving observational datasets for given time and space.

• Efficiently combine/interpolate model and observational datasets.

• Provide easy plotting using proven tools in python

• Perform statics between model runs or observations or models and observations.

5

http://pandas.pydata.org

http://pandas.pydata.org

MONET Documentation

Fig. 1: Time Series

Fig. 2: Time Series of RMSE

6 Chapter 2. Reference

MONET Documentation

Fig. 3: Spatial Plots

2.1. Overview: Why MONET? 7

MONET Documentation

2.1.2 Gallery


MONET Documentation

2.1. Overview: Why MONET? 9

MONET Documentation

2.2 Installation

2.2.1 Required dependencies

• Python 2.71, 3.4, 3.5, or 3.6

• numpy (1.11 or later)

• pandas (0.18.0 or later)

• xarray (0.10 or later)

• dask

• pseudonetcdf

• xesmf (1.9.0 or later)

• netcdf4

• matplotlib

• seaborn1 MONET has dropped support for python 2.7 and requires python 3.6+. For more information see the following references:– Python 3 Statement– Tips on porting to Python 3


http://www.numpy.org/

http://pandas.pydata.org/

http://xarray.pydata.org/

http://dask.pydata.org/

https://github.com/barronh/pseudonetcdf/

https://github.com/pytroll/pyresample/

http://unidata.github.io/netcdf4-python/

https://matplotlib.org/

https://seaborn.pydata.org/

http://www.python3statement.org/

https://docs.python.org/3/howto/pyporting.html

MONET Documentation

• cartopy

2.2.1.1 For parallel computing

• dask.array (0.9.0 or later): required for

2.2.1.2 For plotting

• matplotlib: required for plotting

• cartopy: recommended for plot-maps

• seaborn: for better color palettes

2.2.2 Instructions

MONET itself is a pure Python package, but some of it’s dependencies may not be. The simplest way to installMONET is to install it from the conda-forge feedstock:

$ conda install -c conda-forge monet

This will install all of the dependencies needed by MONET and MONET itself.

xesmf is an optional dependency and can be installed easily from conda-forge, Note xesmf is not available on windowsdue to the dependency on esmpy and esmf.:

$ conda install -c conda-forge monet

To install MONET from source code you must install with pip. This can be done directly from the github page:

$ pip install git+https://github.com/noaa-oar-arl/MONET.git

or you can manually download it from GitHub and install it using the setup.py:

$ git clone https://github.com/noaa-oar-arl/MONET.git$ cd MONET$ pip install .

2.3 MONET XArray Accessor

MONET can add georeferecing tools to xarray‘s data structures. These tools can be accessed via a special .monetattribute, available for both xarray.DataArray and xarray.Dataset objects after a simple import monet in your code.

2.3.1 Initializing the Accessor

All you have to do is import monet after you import xarray.

import xarray as xrimport monet as m

cmaqfile = monet.__path__ + '/../data/aqm.t12z.aconc.ncf'

(continues on next page)

2.3. MONET XArray Accessor 11

https://scitools.org.uk/cartopy/docs/latest/

http://dask.pydata.org

http://matplotlib.org/

http://scitools.org.uk/cartopy/

https://stanford.edu/~mwaskom/software/seaborn/

MONET Documentation

(continued from previous page)

from monet.models import cmaq

c = cmaq.open_dataset(cmaqfile)

c.O3[0,0,:,:].monet.quick_map()

2.3.2 Interpolation Accessors

The MONET accessor provides several useful interpolation routines including: Getting the nearest point to a givenlatitude and longitude, interpolating to a constant latitude or longitude, interpolating to a vertical levels, and remappingentire 2D :py:class::~xarray.DataArray or :py::class::xarray.DataSet.

2.3.2.1 Find Nearest Lat lon point

To find the nearest latitude longitude point you just need to use the .monet.nearest_latlon method. In this example wewill continue to use the CMAQ test file above. We will find the closest grid points to lat=20.5, lon=-157.4.

c.monet.nearest_latlon(lat=20.5,lon=-157.4)

<xarray.Dataset>Dimensions: (time: 48, x: 1, y: 1, z: 1)Coordinates:

* time (time) datetime64[ns] 2018-05-17T12:00:00 2018-05-17T13:00:00 ...latitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)>longitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)>

Dimensions without coordinates: x, y, zData variables:

O3 (time, z, y, x) float32 dask.array<shape=(48, 1, 1, 1), chunksize=(48, 1,→˓ 1, 1)>


MONET Documentation

Notice that the length of the dimensions are now (time: 48, x: 1, y: 1, z: 1). If you wanted to only find the nearestlocation for a single variable you can use the accessor on the DataArray.

c.O3.monet.nearest_latlon(lat=20.5,lon=-157.4)

<xarray.DataArray 'O3' (time: 48, z: 1, y: 1, x: 1)>dask.array<shape=(48, 1, 1, 1), dtype=float32, chunksize=(48, 1, 1, 1)>Coordinates:

* time (time) datetime64[ns] 2018-05-17T12:00:00 2018-05-17T13:00:00 ...latitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)>longitude (y, x) float64 dask.array<shape=(1, 1), chunksize=(1, 1)>

Dimensions without coordinates: z, y, xAttributes:

long_name: O3units: ppbVvar_desc: Variable O3_FillValue: nan

2.4 Installing on WCOSS

If you have access to the NOAA WCOSS machines you can create your own python environments very easily usingintel-python.:

module load ips/19.0.5.281

Then it is suggested to create a .condarc file that lives in your $HOME folder to point to a location that will house yourconda environments. Below is a sample .condarc file.

channels:- intel- conda-forge- defaults

envs_dirs:- /gpfs/dell2/emc/verification/noscrub/User.Name/conda/envs

pkgs_dirs:- /gpfs/dell2/emc/verification/noscrub/User.Name/conda/pkgs

Next you should start a new environment by cloning the default environment to a new name. This can be done in asingle command.

conda create -n myenv --clone="/usrx/local/prod/intel/2019UP05/intelpython3"

A prompt should come up and tell you to activate the environment you just created, myenv.

:: source activate myenv

From here you can install any package the same way you could on regular anaconda installations.

conda install -c conda-forge monet

2.4. Installing on WCOSS 13

MONET Documentation

2.5 Tutorial

2.5.1 How to read AQS data from PAMS and do a quick analysis

The first tutorial for MONET will load data from the AQS data, specifically the PAMS dataset, and show to reshape thedataframe and do a statistical analysis with different meteorological and composition data. For more examples, pleasesee http://monet-arl.readthedocs.io/en/develop/. This will later be able to be found on the github page and readthedocs.

We will first begin by importing monet and a few helper classes for later

import numpy as np # numpyimport pandas as pd # pandasfrom monet.obs import * # observations from MONETimport matplotlib.pyplot as plt # plottingimport seaborn as sns # better color palettesimport cartopy.crs as ccrs # map projectionsimport cartopy.feature as cfeature # politcal and geographic featuressns.set_style('ticks') # plot configurationsns.set_context("talk") # plot configure for size of text

Now we have all the imports we could need lets load some data. Most of the PAMS data is on daily data so letsadd the kwarg daily=True to the call. We will also create this for the year 2015 and 2016. Some variables thatmay be valuable are the VOCS, ozone, NO2, NOX, temperature. For all of the measurments available please seehttps://aqs.epa.gov/aqsweb/airdata/download_files.html

dates = pd.date_range(start='2004-01-01',end='2004-12-31')df = aqs.add_data(dates,daily=True,param=['VOC','OZONE'], download=True)

Retrieving: daily_VOCS_2004.ziphttps://aqs.epa.gov/aqsweb/airdata/daily_VOCS_2004.zip

Retrieving: daily_44201_2004.ziphttps://aqs.epa.gov/aqsweb/airdata/daily_44201_2004.zip

[########################################] | 100% Completed | 26.1s[########################################] | 100% Completed | 26.2s[########################################] | 100% Completed | 26.3s[########################################] | 100% Completed | 26.4s[########################################] | 100% Completed | 26.5s[########################################] | 100% Completed | 34.8s[########################################] | 100% Completed | 34.9s[########################################] | 100% Completed | 34.9s[########################################] | 100% Completed | 35.0s[########################################] | 100% Completed | 35.1sMonitor File Path: /anaconda3/lib/python3.6/site-packages/data/monitoring_site_→˓locations.hdfMonitor File Not Found... Reprocessing

/anaconda3/lib/python3.6/site-packages/python/core/interactiveshell.py:2963:→˓DtypeWarning: Columns (0) have mixed types. Specify dtype option on import or set→˓low_memory=False.exec(code_obj, self.user_global_ns, self.user_ns)

/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py:189:→˓SettingWithCopyWarning: (continues on next page)


http://monet-arl.readthedocs.io/en/develop/

https://aqs.epa.gov/aqsweb/airdata/download_files.html

MONET Documentation


A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/→˓indexing.html#indexing-view-versus-copyself._setitem_with_indexer(indexer, value)

Now we have the data in aqs.df and a copy of it in df. . . just in case. So lets take a look at it.

#df.head()aqs.df.head()

Notice that in this printed format it obscures some of the dataframe columns from view. Lets see what they are!

from numpy import sortfor i in sort(df.columns): # loop over the sorted columns and print them

print(i)

1st_max_hour1st_max_valueaddressairnow_flagaqicbsa_namecity_namecmsa_namecollecting_agencycounty_codecounty_namedate_of_last_changedatumelevationepa_regionevent_typefirst_year_of_datagmt_offsetland_uselatitudelocal_site_namelocation_settinglongitudemethod_codemethod_namemonitor_typemsa_codemsa_namenetworksobsobservation_countobservation_percentparameter_codeparameter_namepocpollutant_standardsample_durationsite_numsiteid


2.5. Tutorial 15

MONET Documentation


state_codestate_nametimetime_localtribe_nameunitsvariable

We have lots of columns but this is actually the long format (data is stacked on variable). Data analysis could be doneeasier in a wide format. So lets use a utility function in MONET to aid with reshaping the dataframe.

from monet.util import toolsnew = tools.long_to_wide(df)new.head()

Lets see how many ISOPRENE sites there are. We will drop the NaN values along the ISOPRENE column and thenfind the unique siteid’s and look at the shape of them

new.dropna(subset=['ISOPRENE']).siteid.unique().shape

(140,)

Now as you can see we have lots of columns that is sorted by time and siteid. But what measurements are included inthe dataframe? Let’s see all the new columns generated from pivoting the table.

from numpy import sortfor i in sort(new.columns):

print(i)

1,1,2,2-TETRACHLOROETHANE1,1,2-TRICHLORO-1,2,2-TRIFLUOROETHANE1,1,2-TRICHLOROETHANE1,1-DICHLOROETHANE1,1-DICHLOROETHYLENE1,2,3-TRIMETHYLBENZENE1,2,4-TRICHLOROBENZENE1,2,4-TRIMETHYLBENZENE1,2-DICHLOROBENZENE1,2-DICHLOROPROPANE1,3,5-TRIMETHYLBENZENE1,3-BUTADIENE1,3-DICHLOROBENZENE1,4-DICHLOROBENZENE1-BUTENE1-PENTENE1st_max_hour1st_max_value2,2,4-TRIMETHYLPENTANE2,2-DIMETHYLBUTANE2,3,4-TRIMETHYLPENTANE2,3-DIMETHYLBUTANE2,3-DIMETHYLPENTANE2,4-DIMETHYLPENTANE2-METHYLHEPTANE2-METHYLHEXANE



MONET Documentation


2-METHYLPENTANE3-CHLOROPROPENE3-METHYLHEPTANE3-METHYLHEXANE3-METHYLPENTANEACETALDEHYDEACETONEACETONITRILEACETYLENEACROLEIN - UNVERIFIEDACRYLONITRILEBENZENEBENZYL CHLORIDEBROMOCHLOROMETHANEBROMODICHLOROMETHANEBROMOFORMBROMOMETHANECARBON DISULFIDECARBON TETRACHLORIDECHLOROBENZENECHLOROETHANECHLOROFORMCHLOROMETHANECHLOROPRENECIS-1,2-DICHLOROETHENECIS-1,3-DICHLOROPROPENECIS-2-BUTENECIS-2-PENTENECYCLOHEXANECYCLOPENTANEDIBROMOCHLOROMETHANEDICHLORODIFLUOROMETHANEDICHLOROMETHANEETHANEETHYL ACRYLATEETHYLBENZENEETHYLENEETHYLENE DIBROMIDEETHYLENE DICHLORIDEFORMALDEHYDEFREON 113FREON 114HEXACHLOROBUTADIENEISOBUTANEISOPENTANEISOPRENEISOPROPYLBENZENEM-DIETHYLBENZENEM-ETHYLTOLUENEM/P XYLENEMETHYL CHLOROFORMMETHYL ETHYL KETONEMETHYL ISOBUTYL KETONEMETHYL METHACRYLATEMETHYL TERT-BUTYL ETHERMETHYLCYCLOHEXANEMETHYLCYCLOPENTANE


2.5. Tutorial 17

MONET Documentation


N-BUTANEN-DECANEN-HEPTANEN-HEXANEN-NONANEN-OCTANEN-PENTANEN-PROPYLBENZENEN-UNDECANEO-ETHYLTOLUENEO-XYLENEOZONEP-DIETHYLBENZENEP-ETHYLTOLUENEPROPANEPROPYLENESTYRENESUM OF PAMS TARGET COMPOUNDSTERT-AMYL METHYL ETHERTERT-BUTYL ETHYL ETHERTETRACHLOROETHYLENETOLUENETOTAL NMOC (NON-METHANE ORGANIC COMPOUND)TRANS-1,2-DICHLOROETHYLENETRANS-1,3-DICHLOROPROPENETRANS-2-BUTENETRANS-2-PENTENETRICHLOROETHYLENETRICHLOROFLUOROMETHANEVINYL CHLORIDEaddressairnow_flagaqicbsa_namecity_namecmsa_namecollecting_agencycounty_codecounty_namedate_of_last_changedatumelevationepa_regionevent_typefirst_year_of_datagmt_offsetland_uselatitudelocal_site_namelocation_settinglongitudemethod_codemethod_namemonitor_typemsa_codemsa_namenetworks



MONET Documentation


obsobservation_countobservation_percentparameter_codeparameter_namepocpollutant_standardsample_durationsite_numsiteidstate_codestate_nametimetime_localtribe_nameunitsvariable

Now as you can see we have lots of columns that is sorted by time and siteid. This can be very useful as we can now dosome direct comparisons using the dataframe. Lets get a description of the dataset first so we can see some averagesand ranges of the different chemical species.

new.describe()

This gives us a format that allows simple statistics and plots using pandas, matplotlib, and seaborn. For time series itis often useful to have the index as the time. Lets do that

new.index = new.timenew['OZONE_ppb'] = new.OZONE * 1000.new.OZONE_ppb.mean()

27.581303457170307

2.5.1.1 Plotting

As you can see the data is now indexed with the UTC time. Lets make a time series plot of the average ISOPRENE.

f,ax = plt.subplots(figsize=(10,4)) # this is so we can control the figure size.new.ISOPRENE.resample('D').mean().plot(ax=ax)

<matplotlib.axes._subplots.AxesSubplot at 0x1c2bb1f860>

2.5. Tutorial 19

MONET Documentation

This is quite noisy with the daily data. Lets resample in time to every month using the average Isoprene concentrationto weekly and monthly.

f,ax = plt.subplots(figsize=(10,4)) # this is so we can control the figure size.new.ISOPRENE.resample('D').mean().plot(ax=ax, label='daily')new.ISOPRENE.resample('W').mean().plot(ax=ax, label='weekly')new.ISOPRENE.resample('M').mean().plot(ax=ax, label='monthly')plt.ylabel('ISOP')plt.legend()sns.despine()

Where are these measurements. Lets plot this on a map and see where it is. We can use a utility plotting function inmonet to generate the plot


MONET Documentation

from monet import plotsax = plots.draw_map(states=True, extent=[-130,-60,20,50], resolution='10m')# get only where ISOPRENE is not NANisop = new.dropna(subset=['ISOPRENE'])ax.scatter(isop.longitude, isop.latitude)

<matplotlib.collections.PathCollection at 0x1c47460f60>

There are monitors all across the US with many in TX, CA, New England and the Mid-Atlantic.

What if we wanted to do a linear regression between two variables. Lets say OZONE and temperature. To do this wewill use the statsmodels package. It is a robust library for curve fitting. For specific information for this module lookhere https://www.statsmodels.org/stable/index.html

import statsmodels.api as sm # load statsmodels api#first clean of nan valuesfit_df = new[['ISOPRENE','OZONE']].dropna()x = fit_df.ISOPRENEy = fit_df.OZONEresult = sm.OLS(y,x).fit()print(result.summary())

fit_df.plot.scatter(x='ISOPRENE',y='OZONE')plt.plot(x,result.predict(x),'--r')

OLS Regression Results==============================================================================Dep. Variable: OZONE R-squared: 0.248Model: OLS Adj. R-squared: 0.248Method: Least Squares F-statistic: 1.833e+05Date: Tue, 19 Jun 2018 Prob (F-statistic): 0.00Time: 09:31:12 Log-Likelihood: 1.2390e+06No. Observations: 556733 AIC: -2.478e+06Df Residuals: 556732 BIC: -2.478e+06


2.5. Tutorial 21

https://www.statsmodels.org/stable/index.html

MONET Documentation


Df Model: 1Covariance Type: nonrobust==============================================================================

coef std err t P>|t| [0.025 0.975]------------------------------------------------------------------------------ISOPRENE 0.0057 1.33e-05 428.134 0.000 0.006 0.006==============================================================================Omnibus: 205153.426 Durbin-Watson: 0.007Prob(Omnibus): 0.000 Jarque-Bera (JB): 2465877.353Skew: -1.433 Prob(JB): 0.00Kurtosis: 12.904 Cond. No. 1.00==============================================================================

Warnings:[1] Standard Errors assume that the covariance matrix of the errors is correctly→˓specified.

[<matplotlib.lines.Line2D at 0x1c1e7f0f28>]

Lets save this to a csv file

new.to_csv('/Users/barry/Desktop/new.csv')

2.5.2 Loading IMPROVE Data

In this tutorial we will read and load data from the IMPROVE aerosol network (http://vista.cira.colostate.edu/Improve/)

To do this we need to first download the data from the colstate.edu website. There are a few format requirementsneeded to be compatible with MONET. First go to http://views.cira.colostate.edu/fed/DataWizard/Default.aspx to


http://vista.cira.colostate.edu/Improve/

http://views.cira.colostate.edu/fed/DataWizard/Default.aspx

MONET Documentation

download data.

• Select the Raw data

• Datasets is the “IMPROVE aerosol”

• Select any number of sites

• Select your parameters

• Select the dates in question

• Fields

– Dataset (required)

– Site (required)

– POC (required)

– Date (required)

– Parameter (required)

– Data Value (required)

– Latitude (required)

– Longitude (required)

– State (optional)

– EPACode (optional)

• Options

– skinny format

– delimited (note you will need to pass this onto the open command, ‘,’ by default)

After downloading we can then read the data. Here we included the EPACode and State to add additional meta datastored on the EPA auxilary files used in the EPA AQS and AirNow datasets in MONET. Let’s make a few importsfrom monet and some other libraries that will aid us later.

from monet.obs import improveimport matplotlib.pyplot as pltfrom monet.util import toolsfrom monet.plots import *import pandas as pd

Now we will load the data (in this case the file is ‘2018620155016911Q1N0Mq.txt’)

df = improve.add_data('/Users/barry/Desktop/20186258419212M0xuxu.txt', add_meta=False,→˓ delimiter=',')from numpy import NaNdf['obs'].loc[df.obs < 0] = NaN

/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py:189:→˓SettingWithCopyWarning:A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/→˓indexing.html#indexing-view-versus-copyself._setitem_with_indexer(indexer, value)

2.5. Tutorial 23

MONET Documentation

Lets look at the dataframe

df.head(20)

Now this is in the long pandas format. Lets use the monet.util.tools.long_to_wide utility to reformat the dataframeinto a wide format.

from monet.util import toolsdf1 = tools.long_to_wide(df)df1.head()

Lets now plot some of the different measurements with time from a site. In this case we will look at the PHOE1 sitein Phoenix, Arizona.

acad1 = df1.loc[df1.siteid == 'ACAD1']acad1.head()

2.5.2.1 Trend Analysis

Let’s look at SIf as an example from ACAD1.

acad1.index = acad1.time

acad1.plot(y='SIf', figsize=(14,5))

<matplotlib.axes._subplots.AxesSubplot at 0x1c286eb4a8>

Now this is good but lets resample to see if we can see a trend.

ax = acad1.resample('W').mean().plot(y='SIf', figsize=(14,5), label = 'weekly')ax = acad1.resample('M').mean().plot(y='SIf', ax=ax, label='monthly')


MONET Documentation

Simply resampling is fine but lets try to get a signal out using a kolmogorov-zerbenko filter. See https://www.tandfonline.com/doi/pdf/10.1080/10473289.2005.10464718 for more information

q = acad1.SIf.copy()for i in range(1000):

q = q.rolling(10, min_periods=1, win_type='triang',center=True).mean()ax = acad1.resample('M').mean().plot(y='SIf', figsize=(14,4), label='monthly')q.resample('M').mean().plot(ax=ax,label='KZ Filter')plt.legend()

<matplotlib.legend.Legend at 0x1c290de080>

2.5.2.2 KMEANS Clustering using scikit-learn

Clustering algorithms can be very useful in finding signals within data. As an example we are going to use theKMeans algorithm from scikit-learn (http://scikit-learn.org/stable/modules/clustering.html#k-means) to analyse dustsignals using the improve data. First we need to import some tools from sklearn to use in our analysis.

from sklearn.preprocessing import RobustScaler #to scale our datafrom sklearn.cluster import KMeans # clustering algorithm

2.5. Tutorial 25

https://www.tandfonline.com/doi/pdf/10.1080/10473289.2005.10464718

https://www.tandfonline.com/doi/pdf/10.1080/10473289.2005.10464718

http://scikit-learn.org/stable/modules/clustering.html#k-means

MONET Documentation

First we want to separate out different variables that may be useful such as Si, PM2.5, PM10, Fe, and SOILf. We willalso need to drop any NaN values so let us go ahead and do that.

dfkm = df1[['SIf','MF','MT','FEf','SOILf']].dropna()dfkm.head()

Usually, with sklearn it is better to scale the data first before putting it through the algorithm. We will use th Ro-bustScaler to do this.

X_scaled = RobustScaler().fit(dfkm).transform(dfkm)

Now we will define our clustering algorithm to have 2 clusters. You may need to adjust this as this is just a startingpoint for further analysis.

km = KMeans(n_clusters=2).fit(X_scaled)

The clusters can be found under km.labels_ . These are integers representing the different clusters.

clusters = km.labels_

Lets plot this so that we can see where there is dust.

plt.figure(figsize=(10,6))plt.scatter(dfkm.SOILf,dfkm.MT,c=clusters,edgecolor='k')plt.xlabel('SOILf')plt.ylabel('PM10')

Text(0,0.5,'PM10')


MONET Documentation

2.5.3 How to Open and View CMAQ simulations - HI Volcano Example

In this example, we will learn how to open CMAQ simulations using MONET, view the data on a map, view crosssections of the data, and compare to surface observations (AirNow) for the May 2018 Hawaiian volcanic eruption.First, import MONET and several helper functions for later.

from monet.obs import airnowfrom monet.models import cmaqimport matplotlib.pyplot as pltfrom monet.util import toolsimport pandas as pd

Now the data can be downloaded from the MONET github page in the MONET/data directory. We will assume youalready have this downloaded and will proceed. Open the simulation. As of right now we still require that a seperategrdcro2d (grddot2d) file be loaded for the mass points (dot points) using the grid kwarg.

conc = '/Users/barry/Desktop/MONET/data/aqm.t12z.aconc.ncf'c = cmaq.open_dtaset(flist=conc)

The cmaq object will return a xarray.Dataset but it also still lives inside of the cmaq object in cmaq.dset.We may use a combination of both the dataset (c) directly and the cmaq calls to gather or aggrigate species in theconcentration file.

c

<xarray.Dataset>Dimensions: (DATE-TIME: 2, VAR: 41, time: 48, x: 80, y: 52, z: 1)Coordinates:

* time (time) datetime64[ns] 2018-05-17T12:00:00 2018-05-17T13:00:00 ...longitude (y, x) float64 -161.9 -161.8 -161.7 -161.6 -161.5 -161.4 ...latitude (y, x) float64 17.76 17.76 17.77 17.77 17.77 17.77 17.78 ...

Dimensions without coordinates: DATE-TIME, VAR, x, y, zData variables:

TFLAG (time, VAR, DATE-TIME) int32 ...O3 (time, z, y, x) float32 28.296675 28.461445 28.483536 ...NO2 (time, z, y, x) float32 0.0072615896 0.007076008 0.0070880884 ...NO (time, z, y, x) float32 1.6760728e-08 1.6089658e-08 ...NO3 (time, z, y, x) float32 0.0022378012 0.0022129775 ...N2O5 (time, z, y, x) float32 8.560217e-06 8.241847e-06 ...HNO3 (time, z, y, x) float32 0.1326946 0.13388994 0.1343063 ...HONO (time, z, y, x) float32 6.2205704e-06 5.9552485e-06 ...PNA (time, z, y, x) float32 2.1638462e-07 1.9233373e-07 ...CO (time, z, y, x) float32 72.599915 72.56304 72.599266 ...FORM (time, z, y, x) float32 0.3351323 0.3413216 0.34457546 ...ALD2 (time, z, y, x) float32 0.13579664 0.12625198 0.1253843 ...PAN (time, z, y, x) float32 0.00054027676 0.00047953427 ...NTR (time, z, y, x) float32 0.0057188044 0.0053887735 0.005364259 ...XO2N (time, z, y, x) float32 2.1813818e-05 2.1018073e-05 ...SO2 (time, z, y, x) float32 0.009011468 0.009855208 0.009888159 ...ASO4I (time, z, y, x) float32 0.00071987335 0.0006097436 ...ASO4J (time, z, y, x) float32 0.6792807 0.68186235 0.6739671 ...ANH4I (time, z, y, x) float32 5.4321783e-05 4.6263896e-05 ...ANH4J (time, z, y, x) float32 0.051220056 0.051677197 0.05048342 ...ANO3I (time, z, y, x) float32 9.547329e-16 8.1884126e-16 ...ANO3J (time, z, y, x) float32 8.933346e-13 9.0928e-13 5.418493e-05 ...

2.5. Tutorial 27

MONET Documentation

AORGAI (time, z, y, x) float32 1.4356241e-10 1.308317e-10 ...AORGAJ (time, z, y, x) float32 1.2230534e-05 1.1428401e-05 ...AORGPAI (time, z, y, x) float32 6.7508597e-07 6.614316e-07 ...AORGPAJ (time, z, y, x) float32 0.05786852 0.058112975 0.05817498 ...AORGBI (time, z, y, x) float32 6.7122006e-09 6.5227788e-09 ...AORGBJ (time, z, y, x) float32 0.0005746193 0.00057244353 ...AECI (time, z, y, x) float32 2.4381194e-07 2.3888646e-07 ...AECJ (time, z, y, x) float32 0.014596849 0.014644699 0.014672826 ...A25I (time, z, y, x) float32 ...A25J (time, z, y, x) float32 ...NUMATKN (time, z, y, x) float32 ...NUMACC (time, z, y, x) float32 ...SRFATKN (time, z, y, x) float32 ...SRFACC (time, z, y, x) float32 ...AH2OI (time, z, y, x) float32 ...AH2OJ (time, z, y, x) float32 ...ACLI (time, z, y, x) float32 ...ACLJ (time, z, y, x) float32 3.5000003e-13 3.5000003e-13 ...ANAI (time, z, y, x) float32 8.394818e-17 8.4039525e-17 ...ANAJ (time, z, y, x) float32 0.2311452 0.23135261 0.23287562 ...PM25 (time, z, y, x) float32 1.0354732 1.0388906 1.0314605 ...PM10 (time, z, y, x) float32 1.0354732 1.0388906 1.0314605 ...CLf (time, z, y, x) float32 3.5000003e-13 3.5000003e-13 ...NAf (time, z, y, x) float32 0.2311452 0.23135261 0.23287562 ...NOy (time, z, y, x) float32 0.0001484681 0.00014906164 ...NOx (time, z, y, x) float32 1.6760728e-11 1.6089657e-11 ...NO3f (time, z, y, x) float32 8.9428936e-13 9.100988e-13 ...NH4f (time, z, y, x) float32 0.051274378 0.05172346 0.050527096 ...SO4f (time, z, y, x) float32 0.68000054 0.6824721 0.67454904 ...

Attributes:IOAPI_VERSION: $Id: @(#) ioapi library version 3.1 $ ...EXEC_ID: ???????????????? ...FTYPE: 1CDATE: 2018142CTIME: 135716WDATE: 2018142WTIME: 135716SDATE: 2018137STIME: 120000TSTEP: 10000NTHIK: 1NCOLS: 80NROWS: 52NLAYS: 1NVARS: 41GDTYP: 2P_ALP: 19.0P_BET: 21.0P_GAM: -157.5XCENT: -157.5YCENT: 20.53XORIG: -480000.0YORIG: -312000.0XCELL: 12000.0


MONET Documentation

YCELL: 12000.0VGTYP: 1VGTOP: 200.0VGLVLS: [1. 0.089794]GDNAM: AQF_HIUPNAM: OPACONCVAR-LIST: O3 NO2 NO NO3 ...FILEDESC: Concentration file output ...HISTORY:proj4_srs: +proj=lcc +lat_1=19.0 +lat_2=21.0 +lat_0=20.53 +lon_0=-1...area: Area ID: MONET_Object_GridnDescription: IOAPI area_def ...mapping_tables: {'improve': {}, 'aqs': {'OZONE': ['O3'], 'PM2.5': ['PM25...

Notice that this looks like the ncdump of the file except that there are seperate coordinates including the latitude andlongitude and the time as numpy.datetime64 objects. Also included is the proj4 string, a pyresample area grid anddefault mapping tables to several different observational datasets.

2.5.3.1 Plotting on a map

It is often useful to be able to view the data on a map. Let’s view a random time slice of SO2 (we will view time 20hours into the simulation).

c.SO2[15,0,:,:].monet.quick_map()

[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.4s

<matplotlib.collections.QuadMesh at 0x1c27910860>

2.5. Tutorial 29

MONET Documentation

Now this doesn’t look very pretty. First it isn’t on a map, the color scale isn’t good as we cannot really see any of thedata. To fix this we will add a map using the MONETAccessor and use the robust=True kwarg.

c.SO2[15,0,:,:].monet.quick_map(robust=True)

Better but we can still do much more. There is low concentrations on most of this map making it hard to notice the


MONET Documentation

extremely high values and the SO2 data is in ppmv and not ppbv as normally viewed as. Also, a logscale may be betterfo this type of data as it goes from 0-20000 ppbv rather than a linear scale.

from matplotlib.colors import LogNorm# convert to ppbvso2 = c.SO2[15,0,:,:]so2.where(so2 > 0.1).monet.quick_map(robust=True, norm=LogNorm())

Now let’s us view serveral time slices at once. We will average in time (every 8 hours) to give us 6 total subplots.

so2 = c.SO2[:,0,:,:] * 1000.so2_resampled = so2.resample(time='8H').mean('time').sortby(['y', 'x'],ascending=True)p = so2_resampled.plot.contourf(col_wrap=3,col='time',x='longitude',y='latitude',→˓robust=True,figsize=(15,10),subplot_kws={'projection': ccrs.PlateCarree()})extent = [so2.longitude.min(),so2.longitude.max(),so2.latitude.min(),so2.latitude.→˓max()]for ax in p.axes.flat:

draw_map(ax=ax,resolution='10m',extent=extent)

2.5. Tutorial 31

MONET Documentation

2.5.3.2 Finding nearest lat lon point

Suppose that we want to find the model data found at a point (latitude,longitude). Use the MONETAccessor again

so2.monet.nearest_latlon(lat=20.5,lon=157.5).plot(figsize=(12,6))plt.xlabel('')plt.tight_layout()


MONET Documentation

2.5.3.3 Pairing with AirNow

It is often useful to be able to pair model data with observational data. MONET uses the pyresample library (http://pyresample.readthedocs.io/en/latest/) to do a nearest neighbor interpolation. First let us get the airnow data for thedates of the simulation. We will also rotate it from the raw AirNow long format (stacked variables) to a wide format(each variable is a seperate column)

from monet.util import toolsdf = airnow.add_data(so2.time.to_index())df = tools.long_to_wide(df)

Aggregating AIRNOW files...Building AIRNOW URLs...[########################################] | 100% Completed | 1.3s[########################################] | 100% Completed | 1.4s[########################################] | 100% Completed | 1.4s[########################################] | 100% Completed | 1.5s[########################################] | 100% Completed | 1.5s[########################################] | 100% Completed | 12.3s[########################################] | 100% Completed | 12.4s[########################################] | 100% Completed | 12.4s[########################################] | 100% Completed | 12.5s[########################################] | 100% Completed | 12.6s

Adding in Meta-data

Now let us combine the two. This will return the pandas dataframe with a new column (model).

df_combined = so2.monet.combine_point(df,col='SO2')

[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.1s


2.5. Tutorial 33

http://pyresample.readthedocs.io/en/latest/

http://pyresample.readthedocs.io/en/latest/

MONET Documentation


[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s

Let’s look at the distributions to see if the two overlap to get a general scence of performance.

df_so2 = df.loc[(df.variable == 'SO2') & (df.state_name == 'HI')].dropna(subset=['obs→˓'])import seaborn as snsf,ax = plt.subplots(figsize=(12,7))sns.kdeplot(df_so2.obs, ax=ax, clip=[0,500])sns.kdeplot(df_so2.model,ax=ax, clip=[0,500])

<matplotlib.axes._subplots.AxesSubplot at 0x1c290e94a8>

2.5.3.4 Overlaying Observations on Contour Plots

Now let’s put a time slice on a map. Let’s look back to the time step ‘2018-05-18 03:00’,

from monet.plots import *ax = draw_map(states=True, resolution='10m', figsize=(15,7))so2_now = so2.sel(time='2018-05-18 03:00')p = so2_now.plot(x='longitude',y='latitude',ax=ax, robust=True,norm=LogNorm(),cbar_→˓kwargs={'label': 'SO2 (ppbv)'})vmin,vmax = p.get_clim()cmap = p.get_cmap()d = df_so2.loc[df_so2.time == '2018-05-18 03:00']plt.scatter(d.longitude.values,d.latitude.values,c=d.obs,cmap=cmap,vmin=vmin,→˓vmax=vmax)


MONET Documentation

{'figsize': (15, 7), 'subplot_kw': {'projection': <cartopy.crs.PlateCarree object at→˓0x1c27451d00>}}[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.4s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.4s

<matplotlib.collections.PathCollection at 0x1c2ae207b8>

Not bad but again we can do a little better with the scatter plot. It’s hard to see the outlines of the observations whenthere is high correlation, the sizes may be a little large

ax = draw_map(states=True, resolution='10m', figsize=(15,7))

so2_now = so2.sel(time='2018-05-18 03:00')p = so2_now.plot(x='longitude',y='latitude',ax=ax, robust=True,norm=LogNorm(),cbar_→˓kwargs={'label': 'SO2 (ppbv)'})vmin,vmax = p.get_clim()cmap = p.get_cmap()d = df_so2.loc[df_so2.time == '2018-05-18 03:00']plt.scatter(d.longitude.values,d.latitude.values,c=d.obs,cmap=cmap,s=100,edgecolors='k→˓',lw=.25, vmin=vmin,vmax=vmax)

{'figsize': (15, 7), 'subplot_kw': {'projection': <cartopy.crs.PlateCarree object at→˓0x1c2b0ead00>}}[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.1s


2.5. Tutorial 35

MONET Documentation


[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.4s[########################################] | 100% Completed | 0.5s

<matplotlib.collections.PathCollection at 0x1c2b43c5f8>

p = df_so2.loc[df_so2.obs > 10]p.obs.std()

61.45120441800254

2.5.4 NOAA NESDIS AOD

NOAA NESDIS has an operational data product for the aerosol optical depth from the VIIRS satellite. There are twoproducts, the viirs edr and viirs eps available from ftp://ftp.star.nesdis.noaa.gov/pub/smcd/.

2.5.4.1 VIIRS EDR

The VIIRS EDR data is an aerosol optical depth product available at 0.1 and 0.25 degree resolution. This VIIRS EDRproduct also does not include the Blue Sky algorithm to retrieve over bright surfaces such as the Sahara. Lets open thedata on a single day at first, in this case ‘2018-07-05’.


ftp://ftp.star.nesdis.noaa.gov/pub/smcd/

MONET Documentation

from monet.sat import nesdis_edr_viirsedr = nesdis_edr_viirs.open_dataset('2018-07-05')print(edr)

<xarray.DataArray 'VIIRS EDR AOD' (time: 1, y: 1800, x: 3600)>array([[[nan, nan, ..., nan, nan],

[nan, nan, ..., nan, nan],...,[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan]]], dtype=float32)

Coordinates:

* time (time) datetime64[ns] 2018-07-05

* y (y) int64 0 1 2 3 4 5 6 7 ... 1793 1794 1795 1796 1797 1798 1799

* x (x) int64 0 1 2 3 4 5 6 7 ... 3593 3594 3595 3596 3597 3598 3599latitude (y, x) float64 -89.88 -89.88 -89.88 -89.88 ... 89.88 89.88 89.88longitude (y, x) float64 -179.9 -179.8 -179.7 -179.6 ... 179.7 179.8 179.9

Attributes:units:long_name: Aerosol Optical Depthsource: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/jhuang/npp.viirs.aero...

edr is now a xarray.DataArray for that day. The nesdis_edr_viirs module downloads data to the currentdirectory. To download this into a different directory you can supply the datapath= keyword if needed. To quicklyview this you can use the monet accessor.

edr.monet.quick_map(robust=True)

<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1022a77470>

The EDR data is available in two resolutions. By default monet will download the 0.1 degree dataset. If you wouldlike the 0.25 degree dataset you can pass the kwarg resolution='low'.

edr = nesdis_edr_viirs.open_dataset('2018-07-05', resolution='low')print(edr)

2.5. Tutorial 37

MONET Documentation



Coordinates:

* time (time) datetime64[ns] 2018-07-05

* y (y) int64 0 1 2 3 4 5 6 7 8 ... 712 713 714 715 716 717 718 719


Attributes:units:long_name: Aerosol Optical Depthsource: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/jhuang/npp.viirs.aero...

Notice that the dimensions changed from 1800x3600 to 720x1440.

Open Multiple Days

If you want to open multiple days in a sinlge call you could use the open_mfdataset. Lets grab the first nine days ofJuly 2018.

import pandas as pddates = pd.date_range(start='2018-07-01',end='2018-07-09')edr = nesdis_edr_viirs.open_mfdataset(dates)

print(edr)


[nan, nan, ..., nan, nan],...,[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan]],

[[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan],...,[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan]],

...,

[[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan],...,[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan]],

[[nan, nan, ..., nan, nan],[nan, nan, ..., nan, nan],...,[nan, nan, ..., nan, nan],



MONET Documentation


[nan, nan, ..., nan, nan]]], dtype=float32)Coordinates:

* y (y) int64 0 1 2 3 4 5 6 7 ... 1793 1794 1795 1796 1797 1798 1799


* time (time) datetime64[ns] 2018-07-01 2018-07-02 ... 2018-07-09Attributes:

units:long_name: Aerosol Optical Depthsource: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/jhuang/npp.viirs.aero...

We can visualize these in a seaborn FacetGrid through xarray. For more information on FacetGrid in xarray plottingplease look here: http://xarray.pydata.org/en/stable/plotting.html#faceting

import cartopy.crs as ccrs # map projections and coastlinescbar_kwargs=dict(orientation='horizontal',pad=0.1, aspect=30)d = edr.plot.pcolormesh(x='longitude',y='latitude',col='time',col_wrap=3,

figsize=(12,12),robust=True,cbar_kwargs=cbar_kwargs,subplot_kws={'projection':ccrs.PlateCarree()})

for ax in d.axes.flat:ax.coastlines()

2.5. Tutorial 39

http://xarray.pydata.org/en/stable/plotting.html#faceting

MONET Documentation

2.5.4.2 VIIRS EPS

The VIIRS EPS data includes the Blue Sky algorithm in the AOD calculation. The same methods are available as withthe nesdis_edr_viirs methods.

from monet.sat import nesdis_eps_viirseps = nesdis_eps_viirs.open_dataset('2018-07-05')print(eps)

<xarray.DataArray 'VIIRS EPS AOT' (time: 1, y: 720, x: 1440)>array([[[nan, nan, ..., nan, nan],


Coordinates:latitude (y, x) float64 89.88 89.88 89.88 89.88 ... -89.88 -89.88 -89.88longitude (y, x) float64 -179.9 -179.6 -179.4 -179.1 ... 179.4 179.6 179.9

* time (time) datetime64[ns] 2018-07-05Dimensions without coordinates: y, x



MONET Documentation


Attributes:units:long_name: Aerosol Optical Thicknesssource: ftp://ftp.star.nesdis.noaa.gov/pub/smcd/VIIRS_Aerosol/npp.vii...

eps.monet.quick_map(robust=True)

<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c3406d080>

Notice that there are AOD values over deserts such as the Sahara, Australia, northern China, Mongolia and the MiddleEast

2.5.5 FV3-CHEM in MONET

FV3-CHEM is the experimental implementation of the NASA Gocart model into NOAA’s Next Generation GlobalPrediction System. This tutorial will explain how to open read and use MONET to explore output from the FV3-CHEM system.

Currently, FV3 outputs data in three formats, i.e., nemsio, grib2, and netcdf. There are slight issues with both thenemsio and grib2 data that will be explained below but needless to say they need to be converted with precossors.

2.5.5.1 Convert nemsio2nc4

The first format that needs to be converted is the nemsio data. This is a binary format used within EMC that is plannedto be phased out in favor for netCDF. As a work around a converter was created to convert the binary nemsio data intonetCDF, nemsio2nc4.py.

This is a command-line script that converts the nemsio data output by the FV3 system to netcdf4 files using a combi-nation of mkgfsnemsioctl found within the fv3 workflow and the climate data operators (cdo). This tool is availableon github at https://github.com/bbakernoaa/nemsio2nc4

2.5. Tutorial 41

https://github.com/bbakernoaa/nemsio2nc4

MONET Documentation

nemsio2nc4.py --helpusage: nemsio2nc4.py [-h] -f FILES [-n NPROCS] [-v]

convert nemsio file to netCDF4 file

optional arguments:-h, --help show this help message and exit-f FILES, --files FILES

input nemsio file name (default: None)-v, --verbose print debugging information (default: False)

Example Usage

If you want to convert a single nemsio data file to netcdf4, it can be done like this:

nemsio2nc4.py -v -f 'gfs.t00z.atmf000.nemsio'mkgfsnemsioctl: /gpfs/hps3/emc/naqfc/noscrub/Barry.Baker/FV3CHEM/exec/mkgfsnemsioctlcdo: /naqfc/noscrub/Barry.Baker/python/envs/monet/bin/cdoExecuting: /gpfs/hps3/emc/naqfc/noscrub/Barry.Baker/FV3CHEM/exec/mkgfsnemsioctl gfs.→˓t00z.atmf000.nemsioExecuting: /naqfc/noscrub/Barry.Baker/python/envs/monet/bin/cdo -f nc4 import_binary→˓gfs.t00z.atmf000.nemsio.ctl gfs.t00z.atmf000.nemsio.nc4cdo import_binary: Processed 35 variables [1.56s 152MB]

To convert multiple files you can simple use the hot keys available in linux terminals.

nemsio2nc4.py -v -f 'gfs.t00z.atmf0*.nemsio'

2.5.5.2 Convert fv3grib2nc4

Although there are several ways in which to read grib data from python such as pynio, a converter was created due tothe specific need to distinguish aerosols which these readers do not process well.

fv3grib2nc4.py like nemsio2nc4.py tool is a command line tool created to convert the grib2 aerosol data to netcdffiles. fv3grib2nc4.py will create seperate files for each of the three layer types; ‘1 hybrid layer’, ‘entire atmosphere’,and ‘surface’. These are the three layers that currently hold aerosol data. The tool is available at https://github.com/bbakernoaa/fv3grib2nc4

fv3grib2nc4.py --helpusage: fv3grib2nc4.py [-h] -f FILES [-v]

convert nemsio file to netCDF4 file

optional arguments:-h, --help show this help message and exit-f FILES, --files FILES

input nemsio file name (default: None)-v, --verbose print debugging information (default: False) ```

### Example Usage

If you want to convert a single grib2 data file to netcdf4, it can be done like this:

fv3grib2nc4.py -v -f 'gfs.t00z.master.grb2f000' wgrib2:



https://github.com/bbakernoaa/fv3grib2nc4

https://github.com/bbakernoaa/fv3grib2nc4

MONET Documentation


/nwprod2/grib\_util.v1.0.0/exec/wgrib2 Executing:/nwprod2/grib\_util.v1.0.0/exec/wgrib2 gfs.t00z.master.grb2f000 -match"entire atmosphere:" -nc\_nlev 1 -append -set\_ext\_name 1 -netcdfgfs.t00z.master.grb2f000.entire\_atm.nc Executing:/nwprod2/grib\_util.v1.0.0/exec/wgrib2 gfs.t00z.master.grb2f000 -match"1 hybrid level:" -append -set\_ext\_name 1 -netcdfgfs.t00z.master.grb2f000.hybrid.nc Executing:/nwprod2/grib\_util.v1.0.0/exec/wgrib2 gfs.t00z.master.grb2f000 -match"surface:" -nc\_nlev 1 -append -set\_ext\_name 1 -netcdfgfs.t00z.master.grb2f000.surface.nc\`\`\`

To convert multiple files you can simple use the hot keys available in linux terminals.

fv3grib2nc4.py -v -f 'gfs.t00z.master.grb2f0*'

2.5.5.3 Using MONET

Using MONET with FV3-Chem is much like using MONET with other model outputs. It tries to recognize wherethe files came from (nemsio, grib2, etc. . . .) and then processes the data, renaming coordinates (lat lon to latitude andlongitude) and processing varaibles like geopotential height and pressure if available. First lets import monet andfv3chem from MONET

import matplotlib.pyplot as pltfrom monet.models import fv3chem

To open a single file

f = fv3chem.open_dataset('/Users/barry/Desktop/temp/gfs.t00z.atmf006.nemsio.nc4')print(f)

/Users/barry/Desktop/temp/gfs.t00z.atmf006.nemsio.nc4<xarray.Dataset>Dimensions: (time: 1, x: 384, y: 192, z: 64)Coordinates:

* time (time) datetime64[ns] 2018-07-01T06:00:00

* x (x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1

* y (y) float64 89.28 88.36 87.42 86.49 ... -87.42 -88.36 -89.28

* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0longitude (y, x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1latitude (y, x) float64 89.28 89.28 89.28 89.28 ... -89.28 -89.28 -89.28

Data variables:ugrd (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>vgrd (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>dzdt (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>delz (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>tmp (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>dpres (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>spfh (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),

→˓chunksize=(1, 64, 192, 384)>(continues on next page)

2.5. Tutorial 43

MONET Documentation


clwmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

rwmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

icmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

snmr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

grle (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

cld_amt (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

o3mr (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

so2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

sulf (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

dms (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

msa (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

pm25 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

bc1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

bc2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

oc1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

oc2 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

dust1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>





seas1 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>




pm10 (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)>

pressfc (time, y, x) float32 dask.array<shape=(1, 192, 384), chunksize=(1, 192,→˓ 384)>

hgtsfc (time, y, x) float32 dask.array<shape=(1, 192, 384), chunksize=(1, 192,→˓ 384)>

geohgt (time, z, y, x) float32 dask.array<shape=(1, 64, 192, 384),→˓chunksize=(1, 64, 192, 384)> (continues on next page)


MONET Documentation


Attributes:CDI: Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/...Conventions: CF-1.6history: Thu Dec 20 17:46:09 2018: cdo -f nc4 import_binary gfs.t00z...CDO: Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/...

Notice this object f has dimensions of (time,z,y,x) with 2d coordinates of latitude and longitude. You can get moreinfomation on single varaibles such as pm25 simply by printing the varaible.

print(f.pm25)

<xarray.DataArray 'pm25' (time: 1, z: 64, y: 192, x: 384)>dask.array<shape=(1, 64, 192, 384), dtype=float32, chunksize=(1, 64, 192, 384)>Coordinates:


* x (x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1

* y (y) float64 89.28 88.36 87.42 86.49 ... -87.42 -88.36 -89.28


Attributes:long_name: model layer

Here units are not included because it is not stored in the nemsio format.

Quick Map Plotting

Now one of the main things that will need to be done is plotting on a map. This can be done quickly using thefunctionallity in MONET. In this example we will plot the first layer PM2.5 at time 2018-07-01.

f.pm25[0,0,:,:].monet.quick_map()

[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s

<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c164f2dd8>

2.5. Tutorial 45

MONET Documentation

Adjusting the scale is simple by suppling vmin and vmax. Lets set a minimum of 0 AOD and maximum of 0.5.

f.pm25[0,0,:,:].monet.quick_map(vmin=0,vmax=.5)


<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c18cbf6d8>

Now we have all the control that xarray has built into their plotting routines. For example, lets have a descrete colorbarwith 10 levels, levels=10, and let it determine the levels by throwing out the top and bottom 2% of values using


MONET Documentation

the robust=True

f.pm25[0,0,:,:].monet.quick_map(levels=10,robust=True)


<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c1905ce48>

Now there are a lot of very low values, since this is at the beginning of the simulation so lets mask out values less than0.015 AOD.

f.pm25.where(f.pm25 > 0.015)[0,0,:,:].monet.quick_map(levels=10,robust=True)


<cartopy.mpl.geoaxes.GeoAxesSubplot at 0x1c19635240>

2.5. Tutorial 47

MONET Documentation

For more information on plotting with xarray and matplotlib some useful links are shown below

• Xarray Plotting

• Matplotlib Colorbars

Nearest neighbor

Monet has some extra functionality that may be useful for exploritory studies such as nearest neighbor finder. Letsfind the nearest neighbor to NCWCP (38.972 N, 76.9245 W).

nn = f.pm25.monet.nearest_latlon(lat=38.972,lon=-76.9245)print(nn)

<xarray.DataArray 'pm25' (time: 1, z: 64)>dask.array<shape=(1, 64), dtype=float32, chunksize=(1, 64)>Coordinates:

* time (time) datetime64[ns] 2018-07-01T06:00:00x float64 283.1y float64 38.81

* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0longitude float64 283.1latitude float64 38.81

Attributes:long_name: model layer

Now we can do a quick plot of this vertically, since it was a single time step.

nn.plot(aspect=2,size=5)

[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.1s


http://xarray.pydata.org/en/stable/plotting.html#two-dimensions

https://matplotlib.org/tutorials/colors/colormaps.html

MONET Documentation

[<matplotlib.lines.Line2D at 0x1c194702e8>]

Now this is a simple plot but it is usually valuable to view the vertical coordinate on the y-axis.

nn.plot(y='z',aspect=2,size=5)


[<matplotlib.lines.Line2D at 0x1c1c0bbac8>]

2.5. Tutorial 49

MONET Documentation

Now this is not very useful because the vertical coordinate right now is just the layer number. Lets get the geopoentialheight at this location and add it as a coordinate to plot.

geohgt = f.geohgt.monet.nearest_latlon(lat=38.972,lon=-76.9245).squeeze()geohgt

<xarray.DataArray 'geohgt' (z: 64)>dask.array<shape=(64,), dtype=float32, chunksize=(64,)>Coordinates:

time datetime64[ns] 2018-07-01T06:00:00x float64 283.1y float64 38.81

* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0longitude float64 283.1latitude float64 38.81

Attributes:long_name: Geopotential Heightunits: m

nn['z'] = geohgt.valuesnn.plot(y='z')plt.ylim([0,3000])

[########################################] | 100% Completed | 0.3s[########################################] | 100% Completed | 0.4s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s

(0, 3000)


MONET Documentation

Constant Latitude and Longitude

Sometimes it may be useful to see a latitudinal or longitudinal cross section. This feature is included in monet throughthe xr.DataArray.monet accessor. Lets take a constant latitude at 10 degrees N.

pm25_constant_lat = f.pm25.monet.interp_constant_lat(lat=10., method='bilinear')pm25_constant_lat

Create weight file: bilinear_192x384_192x1.nc[########################################] | 100% Completed | 0.1s[########################################] | 100% Completed | 0.2s

<xarray.DataArray 'pm25' (time: 1, z: 64, x: 192, y: 1)>array([[[[8.107865e-02],

...,[4.765707e-02]],

...,

[[9.984020e-05],...,[9.982444e-05]]]])

Coordinates:longitude (x, y) float64 0.0 1.88 3.76 5.64 ... 353.4 355.3 357.2 359.1latitude (x, y) float64 10.0 10.0 10.0 10.0 10.0 ... 10.0 10.0 10.0 10.0


* z (z) float64 1.0 2.0 3.0 4.0 5.0 6.0 ... 60.0 61.0 62.0 63.0 64.0Dimensions without coordinates: x, y


2.5. Tutorial 51

MONET Documentation


Attributes:regrid_method: bilinear

Like before lets go ahead and get the geopotential height along this latitude.

geoght_constant_lat = f.geohgt.monet.interp_constant_lat(lat=10., method='bilinear')pm25_constant_lat['geohgt'] = geoght_constant_lat

Overwrite existing file: bilinear_192x384_192x1.ncYou can set reuse_weights=True to save computing time.


Let us plot the 2D cross track (height vs longitude).

pm25_constant_lat.plot(x='longitude',y='geohgt',robust=True,ylim=1000,aspect=2,size=5)plt.ylim([0,50000])

(0, 50000)

Calculate Pressure Levels

By default pressure levels are not calculated due to processing time, however, MONET does include a helper functionin the fv3chem module, fv3chem.calc_nemsio_pressure.

f = fv3chem.calc_nemsio_pressure(f)f




MONET Documentation



<xarray.DataArray 'press' (time: 1, z: 64, y: 192, x: 384)>array([[[[1.002220e+03, ..., 1.002230e+03],

...,[6.768812e+02, ..., 6.770022e+02]],

...,

[[2.037354e-01, ..., 2.038117e-01],...,[3.515549e-01, ..., 3.512193e-01]]]], dtype=float32)

Coordinates:


* x (x) float64 0.0 0.9375 1.875 2.812 ... 356.2 357.2 358.1 359.1

* y (y) float64 89.28 88.36 87.42 86.49 ... -87.42 -88.36 -89.28


Attributes:units: mblong_name: Mid Layer Pressure

2.6 Get in touch

• Ask questions, suggest features or view source code on GitHub.

Help & Reference

2.6.1 Get in touch

Ask questions, suggest features or view source code on GitHub.

If an issue arrises please post on the GitHub issues.

2.6.2 API

• genindex

• modindex

• search

2.6.2.1 monet package

Subpackages

monet.plots package

2.6. Get in touch 53

https://github.com/noaa-oar-arl/monet

https://github.com/noaa-oar-arl/MONET

MONET Documentation

Submodules

monet.plots.colorbars module

colorbar helper functions

monet.plots.colorbars.cmap_discretize(cmap, N)Return a discrete colormap from the continuous colormap cmap.

cmap: colormap instance, eg. cm.jet. N: number of colors.

Example x = resize(arange(100), (5,100)) djet = cmap_discretize(cm.jet, 5) imshow(x, cmap=djet)

monet.plots.colorbars.colorbar_index(ncolors, cmap, minval=None, maxval=None,dtype=’int’, basemap=None)

monet.plots.mapgen module

map utilities

monet.plots.mapgen.draw_map(crs=None, natural_earth=False, coastlines=True, states=False,countries=True, resolution=’10m’, extent=None, figsize=(10, 5), re-turn_fig=False, **kwargs)

Short summary.

Parameters

• ax (type) – Description of parameter ax (the default is None).

• natural_earth (bool) – Description of parameter natural_earth (the default is True).

• coastlines (bool) – Description of parameter coastlines (the default is True).

• states (bool) – Description of parameter states (the default is True).

• countries (bool) – Description of parameter countries (the default is True).

• state_resolutions (bool) – Description of parameter state_resolutions (the default is ‘10m’).

• extent ([lon_min,lon_max,lat_min,lat_max]) – Description of parameter extent (the defaultis None).

Returns Description of returned object.

Return type type

monet.plots.plots module

plotting routines

monet.plots.plots.kdeplot(df, title=None, label=None, ax=None, **kwargs)Short summary.

Parameters

• df (type) – Description of parameter df.

• col (type) – Description of parameter col (the default is ‘obs’).

• title (type) – Description of parameter title (the default is None).

• label (type) – Description of parameter label (the default is None).


MONET Documentation

• ax (type) – Description of parameter ax (the default is ax).

• **kwargs (type) – Description of parameter **kwargs.


Return type type

monet.plots.plots.make_spatial_contours(modelvar, gridobj, date, m, dpi=None, save-name=”, discrete=True, ncolors=None,dtype=’int’, **kwargs)

monet.plots.plots.make_spatial_plot(modelvar, m, dpi=None, plotargs={}, ncolors=15, dis-crete=False)

monet.plots.plots.normval(vmin, vmax, cmap)

monet.plots.plots.scatter(df, x=None, y=None, title=None, label=None, ax=None, **kwargs)Short summary.

Parameters


• x (type) – Description of parameter x (the default is ‘obs’).

• y (type) – Description of parameter y (the default is ‘model’).



Return type type

monet.plots.plots.spatial(modelvar, **kwargs)

monet.plots.plots.spatial_bias_scatter(df, m, date, vmin=None, vmax=None, savename=”,ncolors=15, fact=1.5, cmap=’RdBu_r’)

monet.plots.plots.taylordiagram(df, marker=’o’, col1=’obs’, col2=’model’, label1=’OBS’, la-bel2=’MODEL’, scale=1.5, addon=False, dia=None)

monet.plots.plots.timeseries(df, x=’time’, y=’obs’, ax=None, plotargs={}, fillargs={’alpha’:0.2}, title=”, ylabel=None, label=None)

Short summary.

Parameters


• col (type) – Description of parameter col (the default is ‘Obs’).

• ax (type) – Description of parameter ax (the default is None).

• sample (type) – Description of parameter sample (the default is ‘H’).

• plotargs (type) – Description of parameter plotargs (the default is {}).

• fillargs (type) – Description of parameter fillargs (the default is {}).

• title (type) – Description of parameter title (the default is ‘’).

• label (type) – Description of parameter label (the default is None).


Return type type

monet.plots.plots.wind_barbs(ws, wdir, gridobj, m, **kwargs)


MONET Documentation

monet.plots.plots.wind_quiver(ws, wdir, gridobj, m, **kwargs)

monet.plots.taylordiagram module

Taylor diagram (Taylor, 2001) test implementation. http://www-pcmdi.llnl.gov/about/staff/Taylor/CV/Taylor_diagram_primer.htm

class monet.plots.taylordiagram.TaylorDiagram(refstd, scale=1.5, fig=None, rect=111, la-bel=’_’)

Bases: object

Taylor diagram: plot model standard deviation and correlation to reference (data) sample in a single-quadrantpolar plot, with r=stddev and theta=arccos(correlation).

add_contours(levels=5, **kwargs)Add constant centered RMS difference contours.

add_sample(stddev, corrcoef, *args, **kwargs)Add sample (stddev,corrcoeff) to the Taylor diagram. args and kwargs are directly propagated to theFigure.plot command.

Module contents

monet.plots.taylordiagram(df, marker=’o’, col1=’obs’, col2=’model’, label1=’OBS’, la-bel2=’MODEL’, scale=1.5, addon=False, dia=None)

monet.util package

Submodules

monet.util.combinetool module

monet.util.combinetool.combine_da_to_df(da, df, merge=True, **kwargs)This function will combine an xarray data array with spatial information point observations in df.

Parameters

• da (xr.DataArray or xr.Dataset) – Description of parameter da.

• df (pd.DataFrame) – Description of parameter df.

• lay (iterable, default = [0]) – Description of parameter lay.

• radius (integer or float, default = 12e3) – Description of parameter radius.

Returns

Return type pandas.DataFrame

monet.util.combinetool.combine_da_to_df_xesmf(da, df, suffix=None, **kwargs)This function will combine an xarray data array with spatial information point observations in df.

Parameters

• da (xr.DataArray) – Description of parameter da.




http://www-pcmdi.llnl.gov/about/staff/Taylor/CV/Taylor_diagram_primer.htm

http://www-pcmdi.llnl.gov/about/staff/Taylor/CV/Taylor_diagram_primer.htm

MONET Documentation


Returns


monet.util.combinetool.combine_da_to_df_xesmf_strat(da, daz, df, **kwargs)This function will combine an xarray data array with spatial information point observations in df.

Parameters

• da (xr.DataArray) – Description of parameter da.

• daz (xr.DataArray) – Description of parameter daz.




Returns


monet.util.combinetool.combine_da_to_height_profile(da, dset, ra-dius_of_influence=12000.0)

This function will combine an xarray.DataArray to a 2d dataset with dimensions (time,z)

Parameters

• da (xarray.DataArray) – Description of parameter da.

• dset (xarray.Dataset) – Description of parameter dset.

Returns returns the xarray.Dataset with the da added as an additional variable.

Return type xarray.Dataset

monet.util.interp_util module

Interpolation functions

monet.util.interp_util.constant_1d_xesmf(longitude=None, latitude=None)Creates a pyreample.geometry.SwathDefinition with a constant latitude along the longitude array. Longitudecan be a 1d or 2d np.array or xr.DataArray

Parameters

• longitude (numpy.array or xarray.DataArray) – Array of longitude values

• latitude (float) – latitude for constant

Returns

Return type pyreample.geometry.SwathDefinition

monet.util.interp_util.constant_lat_swathdefition(longitude=None, latitude=None)Creates a pyreample.geometry.SwathDefinition with a constant latitude along the longitude array. Longitudecan be a 1d or 2d np.array or xr.DataArray

Parameters

• longitude (numpy.array or xarray.DataArray) – Array of longitude values

• latitude (float) – latitude for constant


MONET Documentation

Returns


monet.util.interp_util.constant_lon_swathdefition(longitude=None, latitude=None)Creates a pyreample.geometry.SwathDefinition with a constant longitude along the latitude array. latitude canbe a 1d or 2d np.array or xr.DataArray

Parameters

• longitude – latitude for constant

• latitude (numpy.array or xarray.DataArray) – Array of longitude values

Returns


monet.util.interp_util.latlon_xarray_to_CoordinateDefinition(longitude=None,latitude=None)

Create pyresample SwathDefinition from xarray object.

Parameters

• longitude (2d xarray.DataArray) – Longitude -> must be from -180 -> 180 and monotoni-cally increasing

• latitude (2d xarray.DataArray) – Latitude -> must be from -90 -> 90 and monotonicallyincreasing

Returns

Return type pyresample.CoordinateDefinition

monet.util.interp_util.lonlat_to_swathdefinition(longitude=None, latitude=None)Short summary.

Parameters

• longitude (type) – Description of parameter longitude.

• latitude (type) – Description of parameter latitude.


Return type type

monet.util.interp_util.lonlat_to_xesmf(longitude=None, latitude=None)Creates an empty xarray.Dataset with the coordinate (longitude, latitude).

Parameters

• longitude (type) – Description of parameter longitude.

• latitude (type) – Description of parameter latitude.


Return type type

monet.util.interp_util.nearest_point_swathdefinition(longitude=None, lati-tude=None)

Creates a pyreample.geometry.SwathDefinition for a single point.

Parameters

• longitude (float) – longitude.

• latitude (float) – latitude.


MONET Documentation

Returns


monet.util.resample module

monet.util.resample.resample_stratify(da, levels, vertical, axis=1)

monet.util.resample.resample_xesmf(source_da, target_da, cleanup=False, **kwargs)

monet.util.stats module

monet.util.stats.AC(obs, mod, axis=None)Anomaly Correlation

Parameters

• obs (type) – Description of parameter obs.

• mod (type) – Description of parameter mod.

• axis (type) – Description of parameter axis.


Return type type

monet.util.stats.E1(obs, mod, axis=None)Modified Coefficient of Efficiency, E1

Parameters





Return type type

monet.util.stats.ETS(obs, mod, minval, maxval)Equitable Threat Score (1 is perfect - Range -1/3 -> 1)

Parameters



• minval (type) – Description of parameter minval.

• maxval (type) – Description of parameter maxval.


Return type type

monet.util.stats.FB(obs, mod, axis=None)Fractional Bias (%)

Parameters



MONET Documentation




Return type type

monet.util.stats.FE(obs, mod, axis=None)Fractional Error (%)

Parameters





Return type type

monet.util.stats.HSS(obs, mod, minval, maxval)Heidke Skill Score (1 is perfect - below zero means no confidence)

Parameters






Return type type

monet.util.stats.IOA(obs, mod, axis=None)Index of Agreement, IOA

Parameters





Return type type

monet.util.stats.IOA_m(obs, mod, axis=None)Index of Agreement, IOA (avoid single block error in np.ma)

Parameters





Return type type


MONET Documentation

monet.util.stats.MB(obs, mod, axis=None)Mean Bias

Parameters





Return type type

monet.util.stats.ME(obs, mod, axis=None)Mean Gross Error (model and obs unit)

Parameters





Return type type

monet.util.stats.MNB(obs, mod, axis=None)Mean Normalized Bias (%)

Parameters





Return type type

monet.util.stats.MNE(obs, mod, axis=None)Mean Normalized Gross Error (%)

Parameters





Return type type

monet.util.stats.MNPB(obs, mod, paxis, axis=None)Mean Normalized Peak Bias (%)

Parameters



• paxis (type) – Description of parameter paxis.


MONET Documentation



Return type type

monet.util.stats.MNPE(obs, mod, paxis, axis=None)Mean Normalized Peak Error (%)

Parameters






Return type type

monet.util.stats.MO(obs, mod, axis=None)Mean Observations (obs unit)

Parameters





Return type type

monet.util.stats.MP(obs, mod, axis=None)Mean Predictions (model unit)

Parameters





Return type type

monet.util.stats.MdnB(obs, mod, axis=None)Median Bias

Parameters





Return type type

monet.util.stats.MdnE(obs, mod, axis=None)Median Gross Error (model and obs unit)


MONET Documentation

Parameters





Return type type

monet.util.stats.MdnNB(obs, mod, axis=None)Median Normalized Bias (%)

Parameters





Return type type

monet.util.stats.MdnNE(obs, mod, axis=None)Median Normalized Gross Error (%)

Parameters





Return type type

monet.util.stats.MdnNPB(obs, mod, paxis, axis=None)Median Normalized Peak Bias (%)

Parameters






Return type type

monet.util.stats.MdnNPE(obs, mod, paxis, axis=None)Median Normalized Peak Bias (%)

Parameters





MONET Documentation



Return type type

monet.util.stats.MdnO(obs, mod, axis=None)Median Observations (obs unit)

Parameters





Return type type

monet.util.stats.MdnP(obs, mod, axis=None)Median Predictions (model unit)

Parameters





Return type type

monet.util.stats.NMB(obs, mod, axis=None)Normalized Mean Bias (%)

Parameters





Return type type

monet.util.stats.NMB_ABS(obs, mod, axis=None)Normalized Mean Bias - Absolute of the denominator (%)

Parameters





Return type type

monet.util.stats.NME(obs, mod, axis=None)Normalized Mean Error (%)

Parameters


MONET Documentation





Return type type

monet.util.stats.NME_m(obs, mod, axis=None)Normalized Mean Error (%) (avoid single block error in np.ma)

Parameters





Return type type

monet.util.stats.NME_m_ABS(obs, mod, axis=None)Normalized Mean Error (%) - Absolute of the denominator (avoid single block error in np.ma)

Parameters





Return type type

monet.util.stats.NMPB(obs, mod, paxis, axis=None)Normalized Mean Peak Bias (%)

Parameters






Return type type

monet.util.stats.NMPE(obs, mod, paxis, axis=None)Normalized Mean Peak Error (%)

Parameters






MONET Documentation


Return type type

monet.util.stats.NMdnB(obs, mod, axis=None)Normalized Median Bias (%)

Parameters





Return type type

monet.util.stats.NMdnE(obs, mod, axis=None)Normalized Median Error (%)

Parameters





Return type type

monet.util.stats.NMdnGE(obs, mod, axis=None)Normalized Median Gross Error (%)

Parameters





Return type type

monet.util.stats.NMdnPB(obs, mod, paxis, axis=None)Normalized Median Peak Bias (%)

Parameters






Return type type

monet.util.stats.NMdnPE(obs, mod, paxis, axis=None)Normalized Median Peak Bias (%)

Parameters


MONET Documentation






Return type type

monet.util.stats.NO(obs, mod, axis=None)N Observations (#)

Parameters





Return type type

monet.util.stats.NOP(obs, mod, axis=None)N Observations/Prediction Pairs (#)

Parameters





Return type type

monet.util.stats.NP(obs, mod, axis=None)N Predictions (#)

Parameters





Return type type

monet.util.stats.PSUTMNPB(obs, mod, axis=None)Paired Space/Unpaired Time Mean Normalized Peak Bias (%)

Parameters






MONET Documentation

Return type type

monet.util.stats.PSUTMNPE(obs, mod, axis=None)Paired Space/Unpaired Time Mean Normalized Peak Error (%)

Parameters





Return type type

monet.util.stats.PSUTMdnNPB(obs, mod, axis=None)Paired Space/Unpaired Time Median Normalized Peak Bias (%)

Parameters





Return type type

monet.util.stats.PSUTMdnNPE(obs, mod, axis=None)Paired Space/Unpaired Time Median Normalized Peak Error (%)

Parameters





Return type type

monet.util.stats.PSUTNMPB(obs, mod, axis=None)Paired Space/Unpaired Time Normalized Mean Peak Bias (%)

Parameters





Return type type

monet.util.stats.PSUTNMPE(obs, mod, axis=None)Paired Space/Unpaired Time Normalized Mean Peak Error (%)

Parameters




MONET Documentation



Return type type

monet.util.stats.PSUTNMdnPB(obs, mod, axis=None)Paired Space/Unpaired Time Normalized Median Peak Bias (%)

Parameters





Return type type

monet.util.stats.PSUTNMdnPE(obs, mod, axis=None)Paired Space/Unpaired Time Normalized Median Peak Error (%)

Parameters





Return type type

monet.util.stats.R2(obs, mod, axis=None)Coefficient of Determination (unit squared)

Parameters





Return type type

monet.util.stats.RM(obs, mod, axis=None)Mean Ratio Observations/Predictions (none)

Parameters





Return type type

monet.util.stats.RMSE(obs, mod, axis=None)Root Mean Square Error (model unit)

Parameters


MONET Documentation





Return type type

monet.util.stats.RMSEs(obs, mod, axis=None)Root Mean Squared Error (obs, mod_hat)

Parameters





Return type type

monet.util.stats.RMSEu(obs, mod, axis=None)Root Mean Squared Error (mod_hat, mod)

Parameters





Return type type

monet.util.stats.RMdn(obs, mod, axis=None)Median Ratio Observations/Predictions (none)

Parameters





Return type type

monet.util.stats.STDO(obs, mod, axis=None)Standard deviation of Observations

Parameters





Return type type


MONET Documentation

monet.util.stats.STDP(obs, mod, axis=None)Standard deviation of Predictions

Parameters





Return type type

monet.util.stats.USUTPB(obs, mod, axis=None)Unpaired Space/Unpaired Time Peak Bias (%)

Parameters





Return type type

monet.util.stats.USUTPE(obs, mod, axis=None)Unpaired Space/Unpaired Time Peak Error (%)

Parameters





Return type type

monet.util.stats.WDAC(obs, mod, axis=None)Wind Direction Anomaly Correlation

Parameters





Return type type

monet.util.stats.WDIOA(obs, mod, axis=None)Wind Direction Index of Agreement, IOA

Parameters





MONET Documentation


Return type type

monet.util.stats.WDIOA_m(obs, mod, axis=None)Wind Direction Index of Agreement, IOA (avoid single block error in np.ma)

Parameters





Return type type

monet.util.stats.WDMB(obs, mod, axis=None)Wind Direction Mean Bias

Parameters





Return type type

monet.util.stats.WDMB_m(obs, mod, axis=None)Wind Direction Mean Bias (avoid single block error in np.ma)

Parameters





Return type type

monet.util.stats.WDME(obs, mod, axis=None)Wind Direction Mean Gross Error (model and obs unit)

Parameters





Return type type

monet.util.stats.WDME_m(obs, mod, axis=None)Wind Direction Mean Gross Error (model and obs unit) (avoid single block error in np.ma)

Parameters



MONET Documentation




Return type type

monet.util.stats.WDMdnB(obs, mod, axis=None)Wind Direction Median Bias

Parameters





Return type type

monet.util.stats.WDMdnE(obs, mod, axis=None)Wind Direction Median Gross Error (model and obs unit)

Parameters





Return type type

monet.util.stats.WDNMB_m(obs, mod, axis=None)Wind Direction Normalized Mean Bias (%) (avoid single block error in np.ma)

Parameters





Return type type

monet.util.stats.WDRMSE(obs, mod, axis=None)Wind Direction Root Mean Square Error (model unit)

Parameters





Return type type

monet.util.stats.WDRMSE_m(obs, mod, axis=None)Wind Direction Root Mean Square Error (model unit) (avoid single block error in np.ma)


MONET Documentation

Parameters





Return type type

monet.util.stats.circlebias(b)Short summary.

Parameters b (type) – Description of parameter b.


Return type type

monet.util.stats.circlebias_m(b)avoid single block error in np.ma

Parameters b (type) – Description of parameter b.


Return type type

monet.util.stats.d1(obs, mod, axis=None)Modified Index of Agreement, d1

Parameters





Return type type

monet.util.stats.matchedcompressed(a1, a2)Short summary.

Parameters

• a1 (type) – Description of parameter a1.



Return type type

monet.util.stats.matchmasks(a1, a2)Short summary.

Parameters




Return type type


MONET Documentation

monet.util.stats.scores(obs, mod, minval, maxval=100000.0)Short summary.

Parameters






Return type type

monet.util.stats.stats(df, minval, maxval)Short summary.

Parameters





Return type type

monet.util.tools module

monet.util.tools.calc_24hr_ave(df, col=None)

monet.util.tools.calc_3hr_ave(df, col=None)

monet.util.tools.calc_8hr_rolling_max(df, col=None, window=None)

monet.util.tools.calc_annual_ave(df, col=None)

monet.util.tools.findclosest(list, value)

monet.util.tools.get_epa_region_bounds(index=None, acronym=None)

monet.util.tools.get_epa_region_df(df)

monet.util.tools.get_giorgi_region_bounds(index=None, acronym=None)

monet.util.tools.get_giorgi_region_df(df)

monet.util.tools.get_relhum(temp, press, vap)

monet.util.tools.kolmogorov_zurbenko_filter(df, col, window, iterations)KZ filter implementation series is a pandas series window is the filter window m in the units of the data (m =2q+1) iterations is the number of times the moving average is evaluated

monet.util.tools.linregress(x, y)

monet.util.tools.long_to_wide(df)

monet.util.tools.search_listinlist(array1, array2)

monet.util.tools.wsdir2uv(ws, wdir)


MONET Documentation

Module contents

Submodules

monet.met_funcs module

DESCRIPTION

This package contains the main routines for estimating variables related to the Monin-Obukhov (MO) Similar-ity Theory, such as MO length, adiabatic correctors

for heat and momentum transport. It requires the following package.

monet.met_funcs.calc_L(ustar, T_A_K, rho, c_p, H, LE)Calculates the Monin-Obukhov length.

Parameters

• ustar (float) – friction velocity (m s-1).

• T_A_K (float) – air temperature (Kelvin).

• rho (float) – air density (kg m-3).

• c_p (float) – Heat capacity of air at constant pressure (J kg-1 K-1).

• H (float) – sensible heat flux (W m-2).

• LE (float) – latent heat flux (W m-2).

Returns L – Obukhov stability length (m).

Return type float

References

monet.met_funcs.calc_Psi_H(zoL)Calculates the adiabatic correction factor for heat transport.

Parameters zoL (float) – stability coefficient (unitless).

Returns Psi_H – adiabatic corrector factor fof heat transport (unitless).

Return type float

References

monet.met_funcs.calc_Psi_M(zoL)Adiabatic correction factor for momentum transport.

Parameters zoL (float) – stability coefficient (unitless).

Returns Psi_M – adiabatic corrector factor fof momentum transport (unitless).

Return type float


MONET Documentation

References

monet.met_funcs.calc_c_p(p, ea)Calculates the heat capacity of air at constant pressure.

Parameters

• p (float) – total air pressure (dry air + water vapour) (mb).

• ea (float) – water vapor pressure at reference height above canopy (mb).

Returns c_p

Return type heat capacity of (moist) air at constant pressure (J kg-1 K-1)

References

based on equation (6.1) from Maarten Ambaum (2010): Thermal Physics of the Atmosphere (pp 109).

monet.met_funcs.calc_delta_vapor_pressure(T_K)Calculate the slope of saturation water vapour pressure.

Parameters T_K (float) – temperature (K).

Returns s – slope of the saturation water vapour pressure (kPa K-1)

Return type float

monet.met_funcs.calc_lambda(T_A_K)Calculates the latent heat of vaporization.

Parameters T_A_K (float) – Air temperature (Kelvin).

Returns Lambda – Latent heat of vaporisation (J kg-1).

Return type float

References

based on Eq. 3-1 Allen FAO98

monet.met_funcs.calc_lapse_rate_moist(T_A_K, ea, p)Calculate moist-adiabatic lapse rate (K/m)

Parameters

• T_A_K (float or numpy array) – air temperature at reference height (K).

• ea (float or numpy array) – water vapor pressure at reference height (mb).

• p (float or numpy array) – total air pressure (dry air + water vapour) at reference height(mb).

Returns Gamma_w – moist-adiabatic lapse rate (K/m)

Return type float or numpy array


MONET Documentation

References

http://glossary.ametsoc.org/wiki/Saturation-adiabatic_lapse_rate

monet.met_funcs.calc_mixing_ratio(ea, p)Calculate ratio of mass of water vapour to the mass of dry air (-)

Parameters

• ea (float or numpy array) – water vapor pressure at reference height (mb).

• p (float or numpy array) – total air pressure (dry air + water vapour) at reference height(mb).

Returns r – mixing ratio (-)


References

http://glossary.ametsoc.org/wiki/Mixing_ratio

monet.met_funcs.calc_pressure(z)Calculates the barometric pressure above sea level.

Parameters z (float) – height above sea level (m).

Returns p – air pressure (mb).

Return type float

monet.met_funcs.calc_psicr(c_p, p, Lambda)Calculates the psicrometric constant.

Parameters

• c_p (float) – heat capacity of (moist) air at constant pressure (J kg-1 K-1).

• p (float) – atmopheric pressure (mb).

• Lambda (float) – latent heat of vaporzation (J kg-1).

Returns psicr – Psicrometric constant (mb C-1).

Return type float

monet.met_funcs.calc_rho(p, ea, T_A_K)Calculates the density of air.

Parameters

• p (float) – total air pressure (dry air + water vapour) (mb).

• ea (float) – water vapor pressure at reference height above canopy (mb).

• T_A_K (float) – air temperature at reference height (Kelvin).

Returns rho – density of air (kg m-3).

Return type float


http://glossary.ametsoc.org/wiki/Saturation-adiabatic_lapse_rate

http://glossary.ametsoc.org/wiki/Mixing_ratio

MONET Documentation

References

based on equation (2.6) from Brutsaert (2005): Hydrology - An Introduction (pp 25).

monet.met_funcs.calc_richardson(u, z_u, d_0, T_R0, T_R1, T_A0, T_A1)Richardson number.

Estimates the Bulk Richardson number for turbulence using time difference temperatures.

Parameters

• u (float) – Wind speed (m s-1).

• z_u (float) – Wind speed measurement height (m).

• d_0 (float) – Zero-plane displacement height (m).

• T_R0 (float) – radiometric surface temperature at time 0 (K).

• T_R1 (float) – radiometric surface temperature at time 1 (K).

• T_A0 (float) – air temperature at time 0 (K).

• T_A1 (float) – air temperature at time 1 (K).

Returns Ri – Richardson number.

Return type float

References

monet.met_funcs.calc_stephan_boltzmann(T_K)Calculates the total energy radiated by a blackbody.

Parameters T_K (float) – body temperature (Kelvin)

Returns M – Emitted radiance (W m-2)

Return type float

monet.met_funcs.calc_sun_angles(lat, lon, stdlon, doy, ftime)Calculates the Sun Zenith and Azimuth Angles (SZA & SAA).

Parameters

• lat (float) – latitude of the site (degrees).

• long (float) – longitude of the site (degrees).

• stdlng (float) – central longitude of the time zone of the site (degrees).

• doy (float) – day of year of measurement (1-366).

• ftime (float) – time of measurement (decimal hours).

Returns

• sza (float) – Sun Zenith Angle (degrees).

• saa (float) – Sun Azimuth Angle (degrees).

monet.met_funcs.calc_theta_s(xlat, xlong, stdlng, doy, year, ftime)Calculates the Sun Zenith Angle (SZA).

Parameters

• xlat (float) – latitude of the site (degrees).


MONET Documentation

• xlong (float) – longitude of the site (degrees).

• stdlng (float) – central longitude of the time zone of the site (degrees).

• doy (float) – day of year of measurement (1-366).

• year (float) – year of measurement .

• ftime (float) – time of measurement (decimal hours).

Returns theta_s – Sun Zenith Angle (degrees).

Return type float

References

Adopted from Martha Anderson’s fortran code for ALEXI which in turn was based on Cupid.

monet.met_funcs.calc_u_star(u, z_u, L, d_0, z_0M)Friction velocity.

Parameters

• u (float) – wind speed above the surface (m s-1).

• z_u (float) – wind speed measurement height (m).

• L (float) – Monin Obukhov stability length (m).

• d_0 (float) – zero-plane displacement height (m).

• z_0M (float) – aerodynamic roughness length for momentum transport (m).

References

monet.met_funcs.calc_vapor_pressure(T_K)Calculate the saturation water vapour pressure.

Parameters T_K (float) – temperature (K).

Returns ea – saturation water vapour pressure (mb).

Return type float

monet.met_funcs.flux_2_evaporation(flux, T_K=293.15, time_domain=1)Converts heat flux units (W m-2) to evaporation rates (mm time-1) to a given temporal window

Parameters

• flux (float or numpy array) – heat flux value to be converted, usually refers to latent heatflux LE to be converted to ET

• T_K (float or numpy array) – environmental temperature in Kelvin. Default=20 Celsius

• time_domain (float) – Temporal window in hours. Default 1 hour (mm h-1)

Returns ET – evaporation rate at the time_domain. Default mm h-1



MONET Documentation

monet.monet_accessor module

MONET Accessor

class monet.monet_accessor.MONETAccessor(xray_obj)Bases: object

Short summary.

Parameters xray_obj (type) – Description of parameter xray_obj.

objDescription of attribute obj.

Type type

cftime_to_datetime64(name=None)Short summary.

Parameters name (type) – Description of parameter name.


Return type type

combine_point(data, suffix=None, pyresample=True, **kwargs)Short summary.

Parameters

• data (type) – Description of parameter data.

• col (type) – Description of parameter col.

• radius (type) – Description of parameter radius.


Return type type

interp_constant_lat(lat=None, lat_name=’latitude’, lon_name=’longitude’, **kwargs)Interpolates the data array to constant longitude.

Parameters lon (float) – Latitude on which to interpolate to

Returns DataArray of at constant longitude

Return type DataArray

interp_constant_lon(lon=None, **kwargs)Interpolates the data array to constant longitude.




is_land(return_xarray=False)checks the mask of land and ocean.

Parameters return_xarray (bool) – If True, return the data array with the ocean values set toNaN. If False, return a numpy boolean array of the land (True).

Returns

Return type xarray.DataArray or numpy.array


MONET Documentation

is_ocean(return_xarray=False)checks the mask of land and ocean.

Parameters return_xarray (bool) – If True, return the data array with the land values set toNaN. If False, return a numpy boolean array of the ocean (True).

Returns


nearest_ij(lat=None, lon=None, **kwargs)Uses pyresample to intepolate to find the i, j index of grid with respect to the given lat lon.

Parameters

• lat (float) – latitude in question

• lon (float) – longitude in question

• **kwargs (dict) – pyresample kwargs for nearest neighbor interpolation

Returns Returns the i (x index) and j (y index) of the given latitude longitude value

Return type i,j

nearest_latlon(lat=None, lon=None, cleanup=True, esmf=False, **kwargs)Uses xesmf to intepolate to a given latitude and longitude. Note that the conservative method is notavailable.

Parameters

• lat (type) – Description of parameter lat.

• lon (type) – Description of parameter lon.



Return type type

quick_contourf(map_kws={}, roll_dateline=False, **kwargs)This function takes an xarray DataArray and quickly cerates a figure using cartopy and the matplotlibcontourf

Parameters

• map_kws (dictionary) – kwargs for monet.plots.mapgen.draw_map

• roll_dateline (bool) – roll_dateline is meant to help with global datasets that the longitudesrange from 0 to 360 instead of -180 to 180. Otherwise a white line appears at 0 degrees.

• **kwargs – kwargs for the xarray.DataArray.plot.contourf function

Returns axes

Return type type

quick_imshow(map_kws={}, roll_dateline=False, **kwargs)This function takes an xarray DataArray and quickly cerates a figure using cartopy and the matplotlibimshow. Note that this should only be used for regular grids.

Parameters




MONET Documentation

• **kwargs – kwargs for the xarray.DataArray.plot.imshow function

Returns axes

Return type matplotlib.axes

quick_map(map_kws={}, roll_dateline=False, **kwargs)This function takes an xarray DataArray and quickly cerates a figure using cartopy and the matplotlibpcolormesh

Parameters



• **kwargs – kwargs for the xarray.DataArray.plot.pcolormesh function

Returns axes

Return type matplotlib.axes

remap_nearest(data, **kwargs)

Interpolates from another grid (data) to the current grid of self using pyresample. it assumes thatthe dimensions are ordered in y,x,z per

pyresample docs

Parameters

• da (xarray DataArray or xarray DataSet) – Object to be interpolated

• radius_of_influence (float or integer) – radius of influcence for pyresample in meters.

Returns resampled object on current grid.

Return type xarray.DataArray

remap_xesmf(dataarray, method=’bilinear’, **kwargs)remaps from another grid to the current grid of self using xesmf

Parameters

• daaarray (ndarray or xarray DataArray) – Description of parameter dset.

• radius_of_influence (float or integer) – radius of influcence for pyresample in meters.

Returns resampled object on current grid.


stratify(levels, vertical, axis=1)Short summary.

Parameters

• levels (type) – Description of parameter levels.

• vertical (type) – Description of parameter vertical.



Return type type


MONET Documentation

structure_for_monet(lat_name=’lat’, lon_name=’lon’, return_obj=True)This will attempt to restucture a given DataArray for use within MONET.

Parameters

• lat_name (type) – Description of parameter lat_name.

• lon_name (type) – Description of parameter lon_name.

• return (type) – Description of parameter return.


Return type type

tidy(lon_name=’longitude’)Tidy’s DataArray–wraps longitudes and sorts lats and lons

Returns The tidy object

Return type xr.DataArray

window(lat_min=None, lon_min=None, lat_max=None, lon_max=None, rectilinear=False)Function to window, ie select a specific region, given the lower left latitude and longitude and the upperright latitude and longitude

Parameters

• lat_min (float) – lower left latitude .

• lon_min (float) – lower left longitude.

• lat_max (float) – upper right latitude.

• lon_max (float) – upper right longitude.

• rectilinear (bool) – flag if this is a rectilinear lat lon grid

Returns returns the windowed object.


wrap_longitudes(lon_name=’longitude’)Ensures longitudes are from -180 -> 180


Return type type

class monet.monet_accessor.MONETAccessorDataset(xray_obj)Bases: object

Monet accessor to the xarray.Dataset.

Parameters xray_obj (type) – Description of parameter xray_obj.

objDescription of attribute obj.

Type type

cftime_to_datetime64(name=None)Convert cftime o numpy datetime64 objects.

Parameters name (str) – time variable name.




MONET Documentation

combine_point(data, suffix=None, pyresample=True, **kwargs)Short summary.

Parameters


• col (type) – Description of parameter col.

• radius (type) – Description of parameter radius.


Return type type

interp_constant_lat(lat=None, lat_name=’latitude’, lon_name=’longitude’, **kwargs)Interpolates the data array to constant longitude.




interp_constant_lon(lon=None, **kwargs)Interpolates the data array to constant longitude.



Return type xr.Dataset or xr.DataArray

is_land(return_xarray=False)checks the mask of land and ocean if the global_land_mask libra.

Parameters return_xarray (bool) – If True, return the data array with the ocean values set toNaN. If False, return a numpy boolean array of the land (True).

Returns


is_ocean(return_xarray=False)checks the mask of land and ocean.

Parameters return_xarray (bool) – If True, return the data array with the land values set toNaN. If False, return a numpy boolean array of the ocean (True).

Returns


nearest_ij(lat=None, lon=None, **kwargs)Uses pyresample to intepolate to find the i, j index of grid with respect to the given lat lon.

Parameters

• lat (float) – latitude in question

• lon (float) – longitude in question

• **kwargs (dict) – pyresample kwargs for nearest neighbor interpolation

Returns Returns the i (x index) and j (y index) of the given latitude longitude value

Return type i,j


MONET Documentation

nearest_latlon(lat=None, lon=None, cleanup=True, esmf=False, **kwargs)Uses xesmf to intepolate to a given latitude and longitude. Note that the conservative method is notavailable.

Parameters

• lat (type) – Description of parameter lat.

• lon (type) – Description of parameter lon.



Return type type

remap_nearest(data, radius_of_influence=1000000.0)Will remap data to the current dataset using the pyresample.kd_tree nearest neighbor interpolation.

Parameters

• data (xarray.DataArray or xarray.Dataset) – geospatial dataset that includes the latitudeand longtide coordinates

• radius_of_influence (float) – radius_of_influence kwarg for pyresample.kd_tree. Default(1e6)

Returns The interpolated xarray object

Return type xarray.Dataset or xarray.DataArray

remap_xesmf(data, **kwargs)Resample the xesmf

Parameters




Return type type

stratify(levels, vertical, axis=1)Short summary.

Parameters

• levels (type) – Description of parameter levels.

• vertical (type) – Description of parameter vertical.



Return type type

tidy(lon_name=’longitude’)Tidy’s DataArray–wraps longitudes and sorts lats and lons

Returns The tidy object



MONET Documentation

window(lat_min, lon_min, lat_max, lon_max)Function to window, ie select a specific region, given the lower left latitude and longitude and the upperright latitude and longitude

Parameters

• lat_min (float) – lower left latitude .

• lon_min (float) – lower left longitude.

• lat_max (float) – upper right latitude.

• lon_max (float) – upper right longitude.

Returns returns the windowed object.

Return type xr.DataSet

wrap_longitudes(lon_name=’longitude’)Ensures longitudes are from -180 -> 180


Return type type

class monet.monet_accessor.MONETAccessorPandas(pandas_obj)Bases: object

centerShort summary.


Return type type

cftime_to_datetime64(col=None)Short summary.

Parameters col (type) – Description of parameter col.


Return type type

get_sparse_SwathDefinition()Creates a pyreample.geometry.SwathDefinition for a single point.

Returns


remap_nearest(df, radius_of_influence=100000.0, combine=False, lat_name=None,lon_name=None)

Remap df to find nearest sites

Parameters

• df (pandas.DataFrame) – dataframe to be interpolated

• radius_of_influence (float) – kwarg for pyresample.kd_tree

Returns Returns the interpolated dataframe

Return type pandas.dataframe

rename_for_monet(df=None)Rename latitude and longitude columns in the DataFrame.

Parameters df (type) – Description of parameter df.


MONET Documentation


Return type type

to_ascii2nc_df(grib_code=126, height_msl=0.0, column=’aod_550nm’, mes-sage_type=’ADPUPA’, pressure=1000.0, qc=None, height_agl=None)

to_ascii2nc_list(**kwargs)

monet.monet_accessor.wrap_longitudes(lons)Short summary.

Parameters lons (type) – Description of parameter lons.


Return type type

Module contents

2.6.2.2 monet


Bibliography

[Brutsaert2005] Brutsaert, W. (2005). Hydrology: an introduction (Vol. 61, No. 8). Cambridge: Cambridge UniversityPress.



[Norman2000] Norman, J. M., W. P. Kustas, J. H. Prueger, and G. R. Diak (2000), Surface flux estimation usingradiometric temperature: A dual-temperature-difference method to minimize measurement errors, WaterResour. Res., 36(8), 2263-2274, http://dx.doi.org/10.1029/2000WR900033.


89

http://dx.doi.org/10.1029/2000WR900033

MONET Documentation

90 Bibliography

Python Module Index

mmonet, 88monet.met_funcs, 76monet.monet_accessor, 81monet.plots, 56monet.plots.colorbars, 54monet.plots.mapgen, 54monet.plots.plots, 54monet.plots.taylordiagram, 56monet.util, 76monet.util.combinetool, 56monet.util.interp_util, 57monet.util.resample, 59monet.util.stats, 59monet.util.tools, 75

91

MONET Documentation

92 Python Module Index

Index

AAC() (in module monet.util.stats), 59add_contours() (monet.plots.taylordiagram.TaylorDiagram

method), 56add_sample() (monet.plots.taylordiagram.TaylorDiagram

method), 56

Ccalc_24hr_ave() (in module monet.util.tools), 75calc_3hr_ave() (in module monet.util.tools), 75calc_8hr_rolling_max() (in module

monet.util.tools), 75calc_annual_ave() (in module monet.util.tools), 75calc_c_p() (in module monet.met_funcs), 77calc_delta_vapor_pressure() (in module

monet.met_funcs), 77calc_L() (in module monet.met_funcs), 76calc_lambda() (in module monet.met_funcs), 77calc_lapse_rate_moist() (in module

monet.met_funcs), 77calc_mixing_ratio() (in module

monet.met_funcs), 78calc_pressure() (in module monet.met_funcs), 78calc_Psi_H() (in module monet.met_funcs), 76calc_Psi_M() (in module monet.met_funcs), 76calc_psicr() (in module monet.met_funcs), 78calc_rho() (in module monet.met_funcs), 78calc_richardson() (in module monet.met_funcs),

79calc_stephan_boltzmann() (in module

monet.met_funcs), 79calc_sun_angles() (in module monet.met_funcs),

79calc_theta_s() (in module monet.met_funcs), 79calc_u_star() (in module monet.met_funcs), 80calc_vapor_pressure() (in module

monet.met_funcs), 80center (monet.monet_accessor.MONETAccessorPandas

attribute), 87

cftime_to_datetime64()(monet.monet_accessor.MONETAccessormethod), 81

cftime_to_datetime64()(monet.monet_accessor.MONETAccessorDatasetmethod), 84

cftime_to_datetime64()(monet.monet_accessor.MONETAccessorPandasmethod), 87

circlebias() (in module monet.util.stats), 74circlebias_m() (in module monet.util.stats), 74cmap_discretize() (in module

monet.plots.colorbars), 54colorbar_index() (in module

monet.plots.colorbars), 54combine_da_to_df() (in module

monet.util.combinetool), 56combine_da_to_df_xesmf() (in module

monet.util.combinetool), 56combine_da_to_df_xesmf_strat() (in module

monet.util.combinetool), 57combine_da_to_height_profile() (in module

monet.util.combinetool), 57combine_point() (monet.monet_accessor.MONETAccessor

method), 81combine_point() (monet.monet_accessor.MONETAccessorDataset

method), 84constant_1d_xesmf() (in module

monet.util.interp_util), 57constant_lat_swathdefition() (in module

monet.util.interp_util), 57constant_lon_swathdefition() (in module

monet.util.interp_util), 58

Dd1() (in module monet.util.stats), 74draw_map() (in module monet.plots.mapgen), 54

EE1() (in module monet.util.stats), 59

93

MONET Documentation

ETS() (in module monet.util.stats), 59

FFB() (in module monet.util.stats), 59FE() (in module monet.util.stats), 60findclosest() (in module monet.util.tools), 75flux_2_evaporation() (in module

monet.met_funcs), 80

Gget_epa_region_bounds() (in module

monet.util.tools), 75get_epa_region_df() (in module monet.util.tools),

75get_giorgi_region_bounds() (in module

monet.util.tools), 75get_giorgi_region_df() (in module

monet.util.tools), 75get_relhum() (in module monet.util.tools), 75get_sparse_SwathDefinition()

(monet.monet_accessor.MONETAccessorPandasmethod), 87

HHSS() (in module monet.util.stats), 60

Iinterp_constant_lat()

(monet.monet_accessor.MONETAccessormethod), 81

interp_constant_lat()(monet.monet_accessor.MONETAccessorDatasetmethod), 85

interp_constant_lon()(monet.monet_accessor.MONETAccessormethod), 81

interp_constant_lon()(monet.monet_accessor.MONETAccessorDatasetmethod), 85

IOA() (in module monet.util.stats), 60IOA_m() (in module monet.util.stats), 60is_land() (monet.monet_accessor.MONETAccessor

method), 81is_land() (monet.monet_accessor.MONETAccessorDataset

method), 85is_ocean() (monet.monet_accessor.MONETAccessor

method), 81is_ocean() (monet.monet_accessor.MONETAccessorDataset

method), 85

Kkdeplot() (in module monet.plots.plots), 54kolmogorov_zurbenko_filter() (in module

monet.util.tools), 75

Llatlon_xarray_to_CoordinateDefinition()

(in module monet.util.interp_util), 58linregress() (in module monet.util.tools), 75long_to_wide() (in module monet.util.tools), 75lonlat_to_swathdefinition() (in module

monet.util.interp_util), 58lonlat_to_xesmf() (in module

monet.util.interp_util), 58

Mmake_spatial_contours() (in module

monet.plots.plots), 55make_spatial_plot() (in module

monet.plots.plots), 55matchedcompressed() (in module monet.util.stats),

74matchmasks() (in module monet.util.stats), 74MB() (in module monet.util.stats), 60MdnB() (in module monet.util.stats), 62MdnE() (in module monet.util.stats), 62MdnNB() (in module monet.util.stats), 63MdnNE() (in module monet.util.stats), 63MdnNPB() (in module monet.util.stats), 63MdnNPE() (in module monet.util.stats), 63MdnO() (in module monet.util.stats), 64MdnP() (in module monet.util.stats), 64ME() (in module monet.util.stats), 61MNB() (in module monet.util.stats), 61MNE() (in module monet.util.stats), 61MNPB() (in module monet.util.stats), 61MNPE() (in module monet.util.stats), 62MO() (in module monet.util.stats), 62monet (module), 88monet.met_funcs (module), 76monet.monet_accessor (module), 81monet.plots (module), 56monet.plots.colorbars (module), 54monet.plots.mapgen (module), 54monet.plots.plots (module), 54monet.plots.taylordiagram (module), 56monet.util (module), 76monet.util.combinetool (module), 56monet.util.interp_util (module), 57monet.util.resample (module), 59monet.util.stats (module), 59monet.util.tools (module), 75MONETAccessor (class in monet.monet_accessor), 81MONETAccessorDataset (class in

monet.monet_accessor), 84MONETAccessorPandas (class in

monet.monet_accessor), 87MP() (in module monet.util.stats), 62

94 Index

MONET Documentation

Nnearest_ij() (monet.monet_accessor.MONETAccessor

method), 82nearest_ij() (monet.monet_accessor.MONETAccessorDataset

method), 85nearest_latlon() (monet.monet_accessor.MONETAccessor

method), 82nearest_latlon() (monet.monet_accessor.MONETAccessorDataset

method), 85nearest_point_swathdefinition() (in mod-

ule monet.util.interp_util), 58NMB() (in module monet.util.stats), 64NMB_ABS() (in module monet.util.stats), 64NMdnB() (in module monet.util.stats), 66NMdnE() (in module monet.util.stats), 66NMdnGE() (in module monet.util.stats), 66NMdnPB() (in module monet.util.stats), 66NMdnPE() (in module monet.util.stats), 66NME() (in module monet.util.stats), 64NME_m() (in module monet.util.stats), 65NME_m_ABS() (in module monet.util.stats), 65NMPB() (in module monet.util.stats), 65NMPE() (in module monet.util.stats), 65NO() (in module monet.util.stats), 67NOP() (in module monet.util.stats), 67normval() (in module monet.plots.plots), 55NP() (in module monet.util.stats), 67

Oobj (monet.monet_accessor.MONETAccessor attribute),

81obj (monet.monet_accessor.MONETAccessorDataset at-

tribute), 84

PPSUTMdnNPB() (in module monet.util.stats), 68PSUTMdnNPE() (in module monet.util.stats), 68PSUTMNPB() (in module monet.util.stats), 67PSUTMNPE() (in module monet.util.stats), 68PSUTNMdnPB() (in module monet.util.stats), 69PSUTNMdnPE() (in module monet.util.stats), 69PSUTNMPB() (in module monet.util.stats), 68PSUTNMPE() (in module monet.util.stats), 68

Qquick_contourf() (monet.monet_accessor.MONETAccessor

method), 82quick_imshow() (monet.monet_accessor.MONETAccessor

method), 82quick_map() (monet.monet_accessor.MONETAccessor

method), 83

RR2() (in module monet.util.stats), 69

remap_nearest() (monet.monet_accessor.MONETAccessormethod), 83

remap_nearest() (monet.monet_accessor.MONETAccessorDatasetmethod), 86

remap_nearest() (monet.monet_accessor.MONETAccessorPandasmethod), 87

remap_xesmf() (monet.monet_accessor.MONETAccessormethod), 83

remap_xesmf() (monet.monet_accessor.MONETAccessorDatasetmethod), 86

rename_for_monet()(monet.monet_accessor.MONETAccessorPandasmethod), 87

resample_stratify() (in modulemonet.util.resample), 59

resample_xesmf() (in module monet.util.resample),59

RM() (in module monet.util.stats), 69RMdn() (in module monet.util.stats), 70RMSE() (in module monet.util.stats), 69RMSEs() (in module monet.util.stats), 70RMSEu() (in module monet.util.stats), 70

Sscatter() (in module monet.plots.plots), 55scores() (in module monet.util.stats), 74search_listinlist() (in module monet.util.tools),

75spatial() (in module monet.plots.plots), 55spatial_bias_scatter() (in module

monet.plots.plots), 55stats() (in module monet.util.stats), 75STDO() (in module monet.util.stats), 70STDP() (in module monet.util.stats), 70stratify() (monet.monet_accessor.MONETAccessor

method), 83stratify() (monet.monet_accessor.MONETAccessorDataset

method), 86structure_for_monet()


TTaylorDiagram (class in monet.plots.taylordiagram),

56taylordiagram() (in module monet.plots), 56taylordiagram() (in module monet.plots.plots), 55tidy() (monet.monet_accessor.MONETAccessor

method), 84tidy() (monet.monet_accessor.MONETAccessorDataset

method), 86timeseries() (in module monet.plots.plots), 55to_ascii2nc_df() (monet.monet_accessor.MONETAccessorPandas

method), 88

Index 95

MONET Documentation

to_ascii2nc_list()(monet.monet_accessor.MONETAccessorPandasmethod), 88

UUSUTPB() (in module monet.util.stats), 71USUTPE() (in module monet.util.stats), 71

WWDAC() (in module monet.util.stats), 71WDIOA() (in module monet.util.stats), 71WDIOA_m() (in module monet.util.stats), 72WDMB() (in module monet.util.stats), 72WDMB_m() (in module monet.util.stats), 72WDMdnB() (in module monet.util.stats), 73WDMdnE() (in module monet.util.stats), 73WDME() (in module monet.util.stats), 72WDME_m() (in module monet.util.stats), 72WDNMB_m() (in module monet.util.stats), 73WDRMSE() (in module monet.util.stats), 73WDRMSE_m() (in module monet.util.stats), 73wind_barbs() (in module monet.plots.plots), 55wind_quiver() (in module monet.plots.plots), 55window() (monet.monet_accessor.MONETAccessor

method), 84window() (monet.monet_accessor.MONETAccessorDataset

method), 86wrap_longitudes() (in module

monet.monet_accessor), 88wrap_longitudes()


wrap_longitudes()(monet.monet_accessor.MONETAccessorDatasetmethod), 87

wsdir2uv() (in module monet.util.tools), 75

96 Index

Download - MONET Documentation - readthedocs.org

Top Related