New tool to ease GFS data download and processing

Juan José A. Figueroa (MARN, El Salvador)

Angelo Pascualetti (DMC, Chile) Michel Davison (WPC / NOAA, USA)

and José Gálvez (SRG / WPC / NOAA, USA)

July 29, 2014

•They simplify the download of 0.5°-GFS grib data tiles from the NCEP NOMADS server to a Windows PC/Laptop

What was developed?

•Installer includes the WingriddsV3.2* grid-analysis software plus macros/scripts developed at the International Desks for the analysis of model data

•Software Bundle: Set of Windows programs & installer

*Wingridds: Software for gridded-data analysis in Windows Systems developed by Jeff Krob. http://winweather.org/

Goal•Effort targeted at the needs of Latin America’s Forecasting community, but tool is useful anywhere

•Part of the WPC International Desks Support to WMO regions III and IV

GFS data & Latin America

Thanks to NOAA, Latin America is highly benefited by the availability of GFS model data. These are used primarily for

-Weather Forecasting

-Initialization of regional models

in countries with the capabilities

-Research

Download and Analysis Methods

•GFS Grib Data are often downloaded from-the TGFTP Server (most known/used choice)

•Once downloaded, gridded GFS data are often processed/analyzed using

-WINGRIDDS (Forecasting-Oriented Software for Windows)

-GRADS (Research-Oriented Software for LINUX)

-other software

ftp://tgftp.nws.noaa.gov

Common Limitations#1 Speed of internet connection

•Storage Space

Other•Computing Skills to play with the data-Important to make things easier so more people manage to obtain, read and analyze the data instead of just visualizing plots online

-Files can be too large if stored for research

GFS Data often used

1. WAFS (Weather Area Forecast System)

2. Global data •1-degree •0.5-degree

•All available at the tgftp server:ftp://tgftp.nws.noaa.gov

1.WAFS (Weather Area-Forecast System)

Global model data interpolated into

•1.25-degree grids•Mandatory Pressure Levels

Divided for different sectors around the globe

•WMO / NOAA / UKMET effort that started during the mid-90s to support aviation and weather forecasting

•Consists of

(smaller files instead of large global data files)

WAFS Sectors L and P

L

P

120W 30W

90N

90S

120W 30W

EQ

•Winds and temperaturesINITIALLY…

•Winds, temperatures, Omegas, RH, PrecipitationNOW

•Through 36 hrs, every 3

•1.25°-grid, Mandatory Levels

•Every 3 hrs through F36h•Every 6 hrs from F42h to F120h

•1.25°-grid, Mandatory Levels

File size: ~369 KB/forecast hour

1.WAFS

2.GFS Global DataMuch more completemore information but larger files.

VARIABLES: Winds, Temperature, Omegas, RH, Precip, Precip Type, Max/Min Temp, Vorticity, Cloud Height, 0 Degree Isotherm, etc,etc.

LEVELS: Every 50hPa plus some sigma-level data.

HOURS: Every 3hrs through 192 hrs.

FILE SIZE PER FCST HOUR:1-degree: ~16.7 MB0.5-degree: ~51.7 MB

2.GFS Global Data

1-degree 0.5-degree

Horizontal Resolution

Every 25hPa

Every 50hPa

1000 hPa

900 hPa

50 hPa

+ Some sigma (terrain-following) levels

2.GFS Global DataVertical Resolution

Dataset Inter-comparison

Recommended if you have the

capability

Resolution 1.25 ° 1.0 ° 0.5 °-Resolves meso- synoptic features

-Resolves only synoptic features

-Limited vertical resolution

WAFS Global 1° Global 0.5°File size per hour 51.7 MB16.7 MB0.37 MB

-Easy download -Lengthy download

-Broadband capability required

Other Options

• What about those who don’t have the high speed capability and would like to download the higher resolution data?

NOMADS Data Serverhttp://nomad5.ncep.noaa.gov/ncep_data/index.html

NOMADS Data Server

• Allows you to select the resolution

• Time increment

• Parameters

• Create your own sectors

Example: Mexico-Caribbean Sector (Tile)Instead of 51.7MB, 9.4MB

Advantages• Higher resolution data in smaller files,

packaged for your convenience.

• Can modify the area to suit your needs and available communication capabilities

– Can make it smaller/larger

However, you need to pick your settings by hand and submit them to the NOMADS server

Can this be made automatic?

YES!The International Desks Team has developed a software bundle that simplifiesthe download of these high-resolution sector data, with chance for full automation

Software Bundle…

(1) Installs Wingridds V3.2 including updated & documented macros/scripts developed and used at the International Desks for analysis, forecasting and training.

(2) Contains a set of scripts that allow

•automatic download of 0.5-degree GFS sector data from the NOMADS server

•automatic conversion of these into Wingridds format

Mexico-Caribbean Tile

Two sectors supported in bundle

South America Tile

9.4MB

13MB

9.4MB vs…

•51.7MB Global File in TGFTP (18%)

•233MB Global File in NOMADS (4%)

13MB vs…

•51.7MB Global File in TGFTP(25%)

•233MB Global File in NOMADS (5%)

Quick tour over the bundle

Bundle and instructions available online:

http://www.wpc.ncep.noaa.gov/ international/gfs_sector_data/

Where to get it?

Lets start a tour over the website...

http://www.wpc.ncep.noaa.gov/international/gfs_sector_data/

Requirements-Windows. Works best in Windows 7 or newer.

Advantages -Save disk space and enhance resolution simultaneously

-Faster download

-Faster processing times

-Be able to go back 12 days (data in TGFTP available only for 1 day)

-Easy installation of Wingridds with updated MACROS/scripts and documentation

Downloading & Installing-Installer (executables .exe) for 32 and 64 bits. You can install by just executing the “.exe” file.

-Alternate options: zipped folders. Unzip and extract the WINGRIDDS folder into your C:/ folder Note: Rename your C:/WINGRIDDSFolder if you have it already installed

What will happen: You will have a fresh version of Wingridds 3.2 installed. This includes all the current MACROS(scripts) used at the International Desks and their documentation.

Operating the Download Programs

Instructions available…

•On the website

•On readme files placed at “C:\WINGRIDDS\datosector_gfs\documentation” once the installer has been executed

Operating the Software

To download sector data just go to the folder“C:\WINGRIDDS\datosector_gfs\en” or “C:\WINGRIDDS\datosector_gfs\es”

Scripts in English

Scripts in Spanish

Scripts in English

-Four for Mexico-Caribbean

-Four for South America

-Available for each hour of

initialization.

NOTE: Remember that data become available about 4-hours after initialization. This means that data for 00z will become available around 04z; for 06z around 10z and so on.

Just double-click to run

C:\WINGRIDDS\datosector_gfs\en

Scripts in Spanish

-Four for Mexico-Caribbean

-Four for South America

Similar! Only differences:

• Comments are in Spanish

• The date is grabbed from your Windows System in a different manner (since in Spanish DD/MM/YYYY is used instead of MM/DD/YYYY). If needed, instructions to adjust the scripts to your Windows System and/or to fix the date explicitly are provided.

C:\WINGRIDDS\datosector_gfs\es

Just double-click to run

What will these programs do?

(1)Store the downloaded grib files in C:\WINGRIDDS\GRIB\NWS

These can be used for whatever purpose you’d like, as long as you have a means to read them.

What will these programs do?(2)Store the ready-to-use Wingridds file in the folder “C:\WINGRIDDS\GRIDDATA\”

•The extension will be “GFS255”•Can be opened with Wingridds

Bugs… what can prevent GFS data downloading?

(1)Date grabbed incorrectly from your PC/Laptop

Solution: Modify the variable “date_string” in each program (details in website).

(3)Firewall issues that block the ports used to connect with the NOMADS server

Solution: Coordinate with your network administrator.

(2)Grib-to-Wingridds converter runs before all grib files have finished downloading (in some slower connections)

Solution: Run the converter manually and stop the programs from running it by commenting the last two lines in each program.

(4)Although sector limits can be modified, the Grib-to-Wingridds converter has some issues with some sectors.

Solution: Go back to a previous sector that worked. If it works again stick to this one or try other coordinates.

This issue should be solved once the next Wingridds update becomes available.

Bugs… what can prevent GFS data downloading?

Questions?