NEAR REAL TIME INTERNET DELIVERY OF PROCESSED MODIS IMAGERY IN WISCONSIN

Samuel A. Batzli, Assistant Scientist

Jonathan W. Chipman, Assistant Scientist Paul A. Howe, Student Programmer Peter R. Weiler, Senior Programmer

Thomas M. Lillesand, Director Environmental Remote Sensing Center

University of Wisconsin-Madison Madison, WI 53706 sabatzli@wisc.edu

jchipman@wisc.edu pahowe@wisc.edu prweiler@wisc.edu tmlilles@wisc.edu

Jeffrey E. Schmaltz, Contractor (SSAI)

MODIS Rapid Response Team NASA Goddard Space Flight Center Building 33 Room G108 Code 922

Greenbelt, MD 20771 Jeff.Schmaltz@gsfc.nasa.gov

ABSTRACT

Statewide natural resource monitoring with satellite remote sensing depends on the timely availability of imagery. The Environmental Remote Sensing Center at the University of Wisconsin-Madison developed and maintains a website for the preview and distribution of near real-time MODIS (Moderate Resolution Imaging Spectroradiometer) imagery of Wisconsin, Lake Michigan, and Lake Superior. The imagery is acquired daily from both the Terra and Aqua satellites through an X-Band Direct Reception Facility maintained at the University of Wisconsin-Madison by the Space Science and Engineering Center. A series of automated processes download, clip, project, and convert level 1B HDF files into formats appropriate for state and regional agencies in Wisconsin. GeoTIFF imagery is available to view or download through our website at http://www.ersc.wisc.edu/modis/ The relatively low spatial resolution of the MODIS sensor (250m – 1km) is compensated for by its very high temporal and spectral resolution (1-2 images daily, 36 spectral bands). Not only are cloud-cover interference issues diminished, but temporally variable natural resource conditions, such as vegetation senescence and lake algal blooms, are easier to capture and study. This paper describes the development of the MODIS ImageServer hardware and software infrastructure, now in its second year of continuous operation, and its application to important natural resource problems.

INTRODUCTION

Established at the University of Wisconsin-Madison in 1970, the Environmental Remote Sensing Center (ERSC) was one of the first remote sensing facilities in the United States. Today its 35 scientists, research specialists and students are highly regarded for the development and application of cutting-edge remote sensing and geospatial technologies to the improved understanding of environmental systems. ERSC is part of the Gaylord Nelson Institute for Environmental Studies. We run three environmental monitoring graduate degree programs and focus on synergistic research and graduate training opportunities that lead to the continued advancement of innovative technologies in science, government and business.

Our recent work with satellite data centers on its application to lake water clarity monitoring. We have had success

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

1

in measuring inland lake clarity with Landsat TM and ETM+ sensors but with a 16-day repeat rate it takes up to three summers to collect a cloud-free statewide mosaic. Now we hope to take advantage of the increased temporal frequency and broader spatial coverage of MODIS imagery to expand our work.

One of the most exciting aspects of working with MODIS data is its near-real time availability and its daily repeat rate. We have discovered that the larger lake systems that we can observe with MODIS are far more dynamic than previously thought. With daily MODIS imagery, we can observe both temporal and spatial variations in turbidity and algal blooms that we would have missed if using Landsat imagery alone. The trade-off is spatial resolution. With Landsat we can observe lakes of about 10 hectares or larger. With MODIS we can look at lakes about 84 hectares or larger. Nevertheless, MODIS gives us new tools for looking at the dynamic properties of large lake systems.

One of our motivations for developing the MODIS ImageServer was to automate the otherwise tedious and time consuming tasks involved in extracting and making useful the Level 1B products generated by MODIS receiving stations. Another motivation was to customize data products to suit the needs of resource managers in Wisconsin. This means generating data products with a spatial extent covering the state of Wisconsin and that conform to the state standard Wisconsin Transverse Mercator (WTM) projection. We felt that resource managers would benefit most from the timely delivery of data products that conformed to their specifications and standards. While there are several sources of MODIS data from archives and near real time sources, the products themselves are relatively raw and not customized to end user needs. To accomplish our objectives, we were able to take advantage of the fact that the Space Science and Engineering Center at the University of Wisconsin-Madison maintains its own MODIS direct reception facility on the roof of the building we occupy.

DIRECT RECEPTION OF MODIS

About MODIS MODIS (or Moderate Resolution Imaging Spectroradiometer) is an instrument aboard the Terra (EOS AM) and

Aqua (EOS PM) satellites launched 18 December 1999 and 4 March 2002 respectively. Both satellites orbit the Earth in a sun synchronous near polar orbit. Terra's orbit around the Earth is timed so that it passes from north to south across the equator in the morning, while Aqua passes south to north over the equator in the afternoon. The MODIS instrument has a ground trace swath width of 2,300 km, wide enough to cover the entire Earth's surface every 1 to 2 days. The imagery is acquired in 36 spectral bands.

Like most Earth Observing System (EOS) satellites, both the Terra and Aqua satellites store data onboard as it is collected, uploading it to a set of receiving stations daily. However, Terra and Aqua also employ a direct broadcast method of communication. Direct broadcast (DB) is the real-time transmission of satellite data to the ground. As the Earth is being observed by satellite instruments the data is formatted and transmitted to any user below in real-time. Users who have compatible ground receiving equipment and are in direct line of sight to the satellite may receive these transmissions. The flipside of this is “direct readout.” Direct readout (DR) is the process of acquiring freely transmitted live satellite data. Both of these technologies have resulted from NASA’s desire to help make satellite data easier to acquire, process, and utilize. The goal is to foster global data exchange and scientific collaboration. It is recognized that access to near-real time local and regional environmental data ultimately benefits environmental, commercial, and public interest decision making.

DB and DR at UW

The Space Science and Engineering Center (SSEC) at the University of Wisconsin-Madison is home to the Cooperative Institute for Meteorological Satellite Studies (CIMSS). The University of Wisconsin-Madison is one of only 17 EOS direct broadcast reception sites in the United States and 77 worldwide and CIMSS is responsible for the establishment of that facility. The CIMSS direct reception tracking antenna operates in a radome on a 40-foot tower on the roof of our 15 story building (Figure 1). The antenna has a clear view of the horizon that enables it to receive satellite broadcasts covering a large part of North America (Figure 2).

ERSC has the good fortune to share building space with SSEC. We benefit from our close proximity to CIMSS. Not only are we dependent on the MODIS data flow generated by CIMSS, the success of our ImageServer processing programs is largely due to the expertise of CIMSS personnel who assisted our programmers with the Integrated Data Language (IDL) and the idiosyncrasies of the MODIS data itself.

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

2

Figure 1 (left) The Atmospheric Oceanic and Space Sciences Building at the University of Wisconsin-Madison. The MODIS receiving antenna is enclosed in a radome atop a 40-foot tower on the roof. Figure 2 (right) This map shows the receiving horizon of the SSEC antenna with a dashed circle. Diagonal lines represent orbital passes of

the Aqua and Terra satellites on one particular day.

IMAGESERVER SYSTEM AND PROCESSING DESCRIPTION

Overview of Components The ERSC MODIS ImageServer consists of three networked computers and several software applications

(Figure 3). The main processing unit is a Dell Otplix 2400 1.3ghz Pentium 4 processor with 1MB of RAM, 120GB of disk space, and a gigbit network I/O. It is dedicated to MODIS image processing. The second machine is used for data archiving, and the third computer is the main ERSC web server. The main processing machine runs Microsoft Windows XP versions of RSI’s IDL 5.6, Leica’s Imagine 8.6, and ESRI’s ArcGIS 8.3. The main controlling program “do_MODIS” is written in IDL and it sends tasks to Imagine and ArcGIS as needed while it is running.

The do_MODIS program runs in either auto or manual mode. It uses a series of sub processes to extract, clip, geolocate, and project the MODIS imagery. For each MODIS image it processes, it produces a set of four scientific and browse imagery products. It is user customizable and functions through a set of simple graphical user interfaces that offer opportunities to select products, sources, and areas of interest (AOIs) (Figures 4 and 5).

Processing (Backend)

When in auto mode, the ImageServer processing starts its work once SSEC has posted its level B1 Aqua and Terra MODIS HDF products to its FTP server. These products include 250m, 500m, and 1km resolution imagery data in Hierarchal Data Format (HDF) files and a metadata file. The program runs in a loop that checks the FTP server every 15 minutes for new data that fall within an appropriate time range for coverage of Wisconsin. When new data are present, do_MODIS spawns an FTP subroutine that first downloads the 250m and 500m data to local disk space, processes those files, and then returns to the FTP server for the 1km data. This is done to expedite the production of 250m browse product for web display.

When processing the data, the program uses the geolocations embedded in the 250m HDF file to locate and clip-out the user defined AOI in the HDF files (Figure 6). This generates a subset TIFF for each band plus an ASCII geolocation file of the latitude and longitudes for each pixel in the subset. Next the program uses ArcINFO to project the ASCII latitude/longitude file into northing/easting in the desired projection (in our case, Wisconsin

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

3

Transverse Mercator, or WTM). This new geolocation file is used to direct the resampling of the new image subset using a nearest-neighbor method. This produces single-band GeoTIFF files for each spectral band. Finally, the Imagine Modeler is used to stack the GeoTIFF layers into a set of Imagine “.img” files.

Daily regional imagery from MODIS

ERSC automated MODIS processor

“Browse Data” Archive

“Science Data” Archive

Terra and Aquasatellites

UW-MadisonSSEC

MODIS Image Server

Science applications

Figure 3. This schematic outlines the general flow of data and information from the satellite to the final products. MODIS data is ingested through the SSEC direct reception facility. ERSC processes the data and produces three

“science” products for internal archiving and one “browse” product for public distribution.

An additional step is taken when processing the 250m and 500m data. The MODIS sensor collects only two spectral bands at 250m resolution – red and near infrared. To produce a near true color image the green and blue bands from the 500m data are “sharpened” with the 250m red band and combined. The result is a 250m red, green, and blue merged GeoTIFF.

Summary of Products

The processing system produces four basic products that correspond to the spatial resolution sets of the MODIS sensor. The first is the 1 km resolution Imagine file. It includes all 36 MODIS bands in one stack and is intended for scientific use. The second is the 500m Imagine file. It includes the 7 MODIS bans available at that resolution (red, green, blue, and four infrared bands) and is also intended for scientific use. The third product is an Imagine file containing the two MODIS 250m bands and the blue and green 500m bands “sharpened” with the red band to generate a 4 band 250m file. It is stored with the science data but is not intended for analysis because the sharpening process is for visual appeal only and distorts the radiance values. The fourth product is the three band 250m merged near true color GeoTIFF product described above. Only the merged GeoTIFF is available for public download from our website.

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

4

Figure 4. In manual mode, the graphical user interface permits users to select data sets. Figure 5. Users can also select pre-defined AOIs through the interface.

Figure 6. The MODIS ImagerServer clips an area of interest (AOI) that includes Wisconsin, Lake Michigan, and Lake Superior (indicated in green)from the much larger full data swath received by SSEC (indicated in purple).

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

5

Web Interface (Front End)

Since one of the goals of ERSC MODIS ImageServer was to bringing the imagery to the web, we needed to develop an interface for display. Given the potential quantity of data to be collected and the daily updates, a dynamically updated and automated system was required. ERSC programmers developed a system using a combination of Perl, PHP, and C++ programming languages. The goal was allow a web site visitor to select a 250m resolution near true color 24-bit image from a catalog of thumbnails, to view the whole image at low resolution or a portion at a higher resolution, to view the metadata, and to download the image data file.

Several processes are performed by the web server when it receives each new 2697x3293 pixel, 24 bit GeoTIFF from the ImageServer processor. Perl scripts generate a 67x81 pixel thumbnail, update the “current image” file, generate a metadata file, and update the image catalog (Figures 7,8,9,and 10). Some web browsers can be configured to display TIFF imagery in its native format. However, at 25.4 MB these TIFFs are too large for practical web display. Therefore, ERSC programmers wrote a C++ routine that converts the GeoTIFF to a compressed 24-bit Portable Network Graphic (PNG) on the fly and delivers the image to a 680x835 pixel window in a PHP generated webpage. The webpage has controls that allow the user to pan and zoom the image through a dynamic form. Each time a user makes a request to pan or zoom, a PHP script calls the C++ program and passes corner pixel coordinates to it based on the location within the window that the user clicked their mouse. The C++ program resamples the GeoTIFF according to the requested zoom level or pan direction and returns a new PNG to the browser. The resulting graphic is always limited to 1.28 MB whether the user is viewing the full extent or a full resolution zoom level. These temporary PNG files are stored in a self-emptying temporary directory.

Figure 7. Home page for ERSC MODIS ImageServer. Figure 8. Example monthly catalog of imagery.

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

6

Figure 9. Full-resolution browse image viewer interface. Figure 10. Example metadata and download page.

ACCESS AND APPLICATION

Web Metrics

The ERSC MODIS ImageServer website receives steady visitation averaging 24 unique visitors per day viewing an average of 57 pages per day over the past 17 months (Tables 1 and 2). Our records indicate that a majority of our visitors are return visitors who have bookmarked our site. We recognize that the content of the site will necessarily limit the range of users to those interested in Wisconsin and the Great Lakes, but we have the capability to expand our coverage area or develop new website ImageServers for different AOIs.

Table 1. Visitors to ERSC MODIS ImageServer

October 2, 2002 – February 25, 2004 and Statistics for February 25, 2004 10pm

Table 2. Page Views at ERSC MODIS ImageServer October 2, 2002 – February 25, 2004 and Statistics for

February 25, 2004 10pm

Total Visitors 11,573Average Per Day 24Average Visit Length (min) 2:42Last Hour 2Today 27This Week 167

Total Page Views 33,637Average Per Day 57Average Per Visit 2.4Last Hour 8Today 72This Week 402

Application to Environmental Monitoring

The public response to the MODIS ImageServer has been very positive. Users especially like the ability to download the imagery and incorporate it into GIS projects. We have received responses from municipal water agencies that draw their drinking water from Lake Winnebago or Lake Michigan. At times they notice sediment

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

7

plumes near or over their water intake valves and have appreciated the early warning they receive from the near-real time imagery.

The temporal frequency of the data has drawn attention to the dynamics of large inland lake systems. With daily MODIS imagery, we are able for the first time to track the tremendous fluctuations in sedimentation and algal plumes and their in-lake spatial variation. These new capabilities will lead to better environmental monitoring and resource management in the future (Figure 11).

01 Oct 2001

30 Sep 2001

05 Sep 2001

28 Aug 2001

26 Aug 2001

14 Aug 2001

07 Aug 2001

03 Aug 2001

13 Jul 2001

08 Jul 2001

03 Jul 2001

0 3825 50+mg/m3

12

Estimated chlorophyll aconcentration, derived from MODIS images

MODIS satellite images of Green Bay, 2001

N

01 Oct 2001

30 Sep 2001

05 Sep 2001

28 Aug 2001

26 Aug 2001

14 Aug 2001

07 Aug 2001

03 Aug 2001

13 Jul 2001

08 Jul 2001

03 Jul 2001

20 km

Support provided by the Wisconsin Satellite Lake Observatory Initiative, a cooperative project of the UW Environmental Remote Sensing Center, the UW Center for Limnology, and the Wisconsin Department of Natural Resources. Additional support comes from the NSF North Temperate Lakes Long-Term Ecological Research Site, the NASA/UW Affiliated Research Center, and the NASA Upper Great Lakes Regional Earth Science Applications Center. Jonathan Chipman, UW Center for Limnology and Environmental Remote Sensing Center.

True-color composite MODIS images of bands 1 (red), 4 (green), and 3 (blue) with colors enhanced. Spatial resolution 500 meters.

Figure 11. Frequent cloud-free imagery of Green Bay, Wisconsin (top) can be used to derive corresponding

concentrations of chlorophyll a (bottom). This capacity will lead to better monitoring and stewardship of our water resources.

SYSTEM ISSUES AND FUTURE DIRECTIONS Overall, the ERSC MODIS ImageServer system does not require tremendous resources to operate. It does

consume a large amount of disk space over time (at the time of this writing 854 sets of products have been archived). We currently do not have a way to limit the storage to only cloud-free scenes, an option we would like to pursue.

We are investigating faster processing techniques that do not require the IDL programming language. The current system has a number of software dependencies that can cause failures if, for example, floating software licenses are temporarily unavailable. We have also faced some AOI size limitations that could be avoided with a more streamlined processing approach. We also hope to automate the development of daily chlorophyll products calibrated to in situ systems. Current limits on available in situ data have limited these efforts. In the future we hope to continue to offer processed imagery online for Wisconsin and the Great Lakes region.

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

8

Proceedings of the American Society for Photogrammetry & Remote Sensing, Spring 2004

9

ACKNOWLEDGEMENTS This work was supported in part by the Upper Midwest Regional Earth Science Applications Center under

NASA grant NAG 13-99002, by the NASA Affiliated Research Center Program, and by the NSF-funded North Temperate Lakes Long-Term Ecological Research Site. The authors would like to thank Liam Gumley of CIMSS/SSEC for his generous provision of IDL code and programming expertise pertaining to MODIS image reading and resolution merging algorithms. We would also like to acknowledge James Kuyper (NASA GSFC) for bilinear interpolations of geolocations.

SELECTED BIBLIOGRAPHY

Lindsey, R., D. Herring. (2001). MODIS: Moderate-resolution Imaging Spectroradiometer, NASA’s Earth Observing System. Adobe PDF file. NASA. Available from http://modis.gsfc.nasa.gov/about/media.html.

Gumley, L. (2002). Practical IDL Programming. Morgan Kaufmann, San Diego.