the national solar observatory digital library — a resource for space weather studies

Journal of Atmospheric and Solar-Terrestrial Physics 62 (2000) 1257–1264www.elsevier.nl/locate/jastp

The National Solar Observatory Digital Library — a resourcefor space weather studies

Frank Hill ∗, Wendy Erdwurm, Detrick Branston, Robert McGrawNational Solar Observatory, 950 N. Cherry Avenue, PO Box 26732, Tucson, AZ 85726-6732 and Sunspot, NM 88349, USA 1

Received 18 October 1999; accepted 30 November 1999

Abstract

We describe the National Solar Observatory Digital Library (NSODL), consisting of 200 GB of on-line archived solar data, aRDBMS search engine, and an Internet HTML-form user interface. The NSODL is open to all users and provides simple accessto solar physics data of basic importance for space weather research and forecasting, heliospheric research, and education.The NSODL can be accessed at the URL www.nso.noao.edu/diglib. c© 2000 Elsevier Science Ltd. All rights reserved.

Keywords: Solar databases; Solar activity; Magnetic �eld

1. Introduction

The prediction of the behavior of complex physical sys-tems, such as space weather in the heliosphere, typicallyexploits knowledge of past correlations between variousphysical parameters. These correlations can be as simple asthe observation that the number of solar ares and coronalmass ejections (CMEs) depends on the sunspot cycle, or ascomplex as the relationship between the pre- are magnetic�eld con�guration and the speed of the solar wind imping-ing on the terrestrial magnetic �eld. The development ofuseful correlations begins with the data itself. In turn, easyand e�cient access to large data sets is essential to thedevelopment of predictive models (e.g. Reardon, 1998),and improvements in this area enhance the ability to predictspace weather. Public use of the data for educational pur-poses also increases as the data become readily availableover the Internet.A key driver of space weather is the magnetic activity on

the solar surface. Flares and CMEs are powered by abrupt

∗ Corresponding author. Tel.: 1-520-318-8294; fax: 1-520-318-8278.E-mail address: [email protected] (F. Hill).1 A division of the National Optical Astronomy Observatories,

operated for the National Science Foundation by the Associationof Universities for Research in Astronomy.

changes in the magnetic �eld in the chromosphere and innercorona. The trigger for these events is presumably rooted inthe photospheric �eld con�guration. Thus, the spatial andtemporal evolution of the solar surface magnetic �eld areessential components of predictive models of space weather.In the past, the National Solar Observatory (NSO) has pro-vided several well-known and heavily used major solar datasets. In the near future, NSO will greatly enhance and ex-pand the scope of its synoptic observations applicable to theNational Space Weather Program and heliospheric research.The temporal coverage of the synoptic data from 1974 tothe present provides a consistent picture of solar behaviorover the last two solar cycles. To improve access to thesedata and further the goals of the National Space WeatherProgram, the NSO data have been organized into a DigitalLibrary placing all of the data on-line and accessible overthe Internet for all users: solar physicists, space weather re-search and prediction agencies, and the public.

2. Description of NSO data sets

The NSODL (Hill et al., 1997,1998) currently containsseveral di�erent types of data: full-disk solar magnetograms,spectroheliograms, coronal images, and high-resolutionspectra of the sun, terrestrial atmosphere, and laboratorysources. These data are freely available for use by the solar–

1364-6826/00/$ - see front matter c© 2000 Elsevier Science Ltd. All rights reserved.PII: S1364 -6826(00)00081 -X

1258 F. Hill et al. / Journal of Atmospheric and Solar-Terrestrial Physics 62 (2000) 1257–1264

Fig. 1. Some solar images in the NSODL for the date 16 June 1998. (a) Fe I 868.8-nm magnetogram; (b) Ca K 393.3-nm spectroheliogram;(c) He I 1083-nm spectroheliogram; d: H� 656.3-nm spectroheliogram.

terrestrial research community as well as educators or thepublic. The anonymous FTP server statistics show that over400 users download about 20,000 of these images per year,with the usage rapidly increasing. In addition, over 600 sci-enti�c publications have used these data. In the future theNSODL contents will be expanded to include helioseismicdata from the Global Oscillation Network Group (GONG)project (Harvey et al., 1996), historical solar images fromthe US Naval Observatory, and data from the SOLIS in-struments (Keller and NSO Sta�, 1998). Here we describein some detail the data sets most relevant to heliosphericand space weather research. Fig. 1 shows some samples ofsolar images in the NSODL for the date 16 June 1998.Image data are stored in Flexible Image Transport System

(FITS) format, designed to provide a means for convenient

exchange of data between installations whose standard in-ternal formats and hardware di�er. First developed in 1979(Wells et al., 1981), it has become quite prevalent in theastronomical community. Up-to-date information on thestandard can be found at the URL fits.gsfc.nasa.gov/.

2.1. Full-disk magnetograms

These provide basic data on the temporal and spatial evo-lution of the surface magnetic �eld. They are obtained oncea day at the NSO=Kitt Peak Vacuum Telescope (KPVT),weather and equipment permitting. The observations are ofthe line-of-sight magnetic �eld obtained in the Fe I 868.8nm spectral line with 1′′ pixels. Recently, NSO has begunto acquire daily chromospheric magnetograms using the Ca

F. Hill et al. / Journal of Atmospheric and Solar-Terrestrial Physics 62 (2000) 1257–1264 1259

II 854.2 nm spectral line. In both cases, a single 1788 ×1788 pixel image is obtained in four swaths over about 45(Fe) or 150 (Ca) min. The current instrument is the KPVTSpectromagnetograph placed in service in April 1992. Inaddition to magnetic �eld, the Spectromagnetograph dataprovide simultaneous Doppler velocity, intensity, line depth,and equivalent width images. The full-resolution magne-tograms are also available reduced in resolution to 894×894pixels and 447 × 447 pixels. From 1974 to 1992, the datawere obtained by the 512-Channel Magnetograph. Theseimages have a format of 2048 × 2048 pixels, and containthe line-of-sight magnetic �eld and a 4-bit coarse intensitymeasurement. The NSODL currently contains about 6000of these images, with a total data volume of about 50 GB.

2.2. Full-disk He I 1083 nm spectroheliograms

These are an excellent proxy for X-ray intensity imagesand show the presence of 2 × 106 K coronal temperatures.The spatial extent of coronal holes is clearly visible in theseimages. The details of these KPVT observations are verysimilar to those for the magnetograms discussed above, ex-cept that the observed quantities are intensity in the He I1083 nm spectral line and in the nearby continuum. The sizeof this data set is about 50 GB.

2.3. Full-disk H� spectroheliograms

Obtained at the NSO=Sacramento Peak Evans SolarFacility, these images show the locations of sunspots, �l-aments, prominences, and aring regions. The data aredaily (5 times per week) full-disk observations of H� in-tensity obtained with a Littrow spectrograph and 0.1-nmbandpass pre�lter. The images are recorded on �lm anddigitized to images of 2200 × 2350 pixels with a pixelsize of 1=1000 of the solar radius. Thumbnail imagesare placed on the Internet and can be found at the URLwww.sunspot.noao.edu/IMAGES/CaK Ha.html. Largerimages as well as the original data in FITS format areavailable via anonymous FTP. The digital archive recentlystarted in 1995, but �lm images are available starting in1960. About 2000 images are available.

2.4. Full-disk Ca K spectroheliograms

These images delineate the network and show plage, ac-tive regions, and sunspots. The observational details are sim-ilar except that the pre�lter is centered on the K3 minimum.About 2000 images are available. Note that over 90% of theNSO=SP spectroheliograms are stored on �lm. While �lmhas a high storage density, it must be laboriously digitizedto enable scienti�c analysis.

Fig. 2. A sample synoptic map constructed from Fe I 868.8-nmmagnetograms. The projection is sine latitude (equal areacylindrical).

2.5. Synoptic maps

The magnetograms and the He 1083-nm spectroheli-ograms covering the entire solar surface are remapped andcombined into synoptic maps of the surface magnetic �eld.These synoptic maps represent the entire surface of the sunat a given moment in time. They are constructed by remap-ping the daily observations in one of three projections:sine latitude (equal area cylindrical); latitude (equidistantcylindrical); and polar (azimuthal equidistant). The dailyremapped images are weighted, shifted to the appropriateCarrington longitude and then merged with data from otherdays to form representations of features on the solar sur-face, Fig. 2 shows a sample synoptic chart. The data arestored on an anonymous FTP server at NSO and daily im-ages placed on the Internet. Since this processing averagestogether many large daily images, the volume of this dataset is about 1 GB.

3. NSODL hardware

Originally, the data were stored on several di�erent typesof media: �lm, 9-track magnetic tape, 8-mm video tape car-tridges, 4-mmDAT cartridges, 5.25-in CD-ROMs, and mag-netic disks. Most of the data were o�-line on magnetic tapeor �lm and only available through a speci�c request to anNSO sta� member who then placed the data on-line. In addi-tion, users were advised to have their data request preciselyspeci�ed before contacting NSO sta�, requiring users to beexperts in advance. Both of these situations impeded the useof the NSO data sets.NSO has now improved the situation by migrating its

signi�cant data sets to CD-ROMs. A set of three NSM CDR100 XA jukeboxes, holding a total of 300 700-MB disks(total capacity of 210 GB), was installed in March 1996 onthe NSO data server in Tucson. Currently, the system holds a


total of 224 CD-ROMS, of which 114 contain relevant spaceweather data in the form of magnetograms, and 1083-nm,Ca K and H� spectroheliograms. The balance of the CDshold the Fourier transform spectrometer (FTS) archive ofhigh-resolution spectra.Two high-resolution (2000 DPI) Agfa DuoScan at-bed

�lm digitizers have been obtained for transferring data from�lm to on-line storage. One digitizer is located at NSO=SP,and is being used to digitize the H� and Ca K spectro-heliograms. The other digitizer, located at NSO=Tucson, iscurrently being used to transfer a historical collection ofUS Naval Observatory solar images from glass plates toCD-ROMs.

4. NSODL user interface and search tool

Simply placing the data on-line immediately improvesaccess, but primarily for the expert user who is familiar withsuch details as �le naming conventions, subdirectory names,data formats, and header contents. While these details arerelatively simple for the full-disk data sets, a search forappropriate data can be a challenge when the search volumehas a high dimensionality, and when the data are distributedover many subdirectories. We have therefore implementedtools to search the library e�ciently and download the dataeasily.The user interface to the NSODL consists of HTML forms

coupled to an Oracle RDBMS server. For images, the searchtool is capable of two types of search: query a single dataproduct using product speci�c criteria (“Product Query”),or query multiple products based upon a given date (“DateQuery”). For the data product query, the user can select thewavelength and date of the observation. Fig. 3 shows theweb page for a longitudinal magnetic �eld product query,while Fig. 4 shows the form for a date query.After entering the criteria, and pressing the “Submit

Query” button, the user is presented with a table of infor-mation and thumbnail images for all of the selected data asshown in Fig. 5. Each thumbnail image is clickable to pro-vide a higher-resolution version which may be satisfactoryfor the user’s needs. A selection button for each data �leis provided to allow further re�nement of the search basedon human interpretation of the table. If desired, anotherbutton can be used to select the entire data set. After the�nal selection is done, the user may push the “Create FTPArchive” button if the total volume of the data request isless than 200 MB. This will create a temporary anonymousFTP directory containing the selected data. Instructions foraccessing this directory are provided. The temporary direc-tory is automatically deleted after 2 days. For data requestslarger than 200 MB, the user is presented with a page fromwhich they can select a data delivery method via a physicalmedium such as 8-mm cassette, Digital Linear Tape (DLT),or CD-ROM, and provide shipping details.

5. Science with the library

The NSODL allows researchers to perform studies thatrequire e�cient access to the full record of solar surfacemagnetic �elds over the last two cycles. Three exam-ples of possible projects are spherical harmonic powerspectra, activity classi�cation and irradiance variations,and the correlations between surface �elds and spaceweather.The temporally varying spherical harmonic decom-

position of the surface magnetic �eld distribution hasbeen studied by Sten o and co-workers, and by Gokhaleand Javariah and co-workers. Sten o and Vogel (1986),Sten o (1988), and Sten o and G�udel (1988) used syn-optic charts of the surface magnetic �eld derived bycombining daily full-disk magnetograms into an esti-mate of the �eld over one Carrington rotation. Gokhaleand Javariah (1990, 1992, 1995), and Gokhale et al.(1992) applied the analysis to Greenwich sunspot loca-tions. Both studies yielded intriguing insights into thebehavior of the solar cycle, yet were limited by thenature of the data sets: the Greenwich sunspot posi-tions do not contain information about the �eld strength,and the synoptic charts e�ectively �lter out high tem-poral frequencies that may be present. By analyzingthe entire set of full-disk magnetograms contained inthe NSODL, it may be possible to identify period-icities of and phase relations between spherical har-monic components that will be useful in space weatherpredictions.Harvey andWhite (1999, 1998) are developing automated

algorithms to identify and separate the solar magnetic �eldinto di�erent structure classes such as plage, sunspots, ac-tive network, quiet network, and supergranule interiors. Thegoal of their studies is to investigate the contribution of eachclass to the observed variations in solar irradiance. How-ever, it would be of interest to apply the same classi�cationalgorithms to the entire set of full-disk magnetograms to in-vestigate the temporal evolution of the �elds themselves. Across-correlation analysis with the He I 1083-nm images andother data may shed light on the relationship between themagnetic �eld classes, the X-ray emission in the corona andCMEs that could be of use for space weather. This kind ofcorrelative study would be greatly facilitated by the NSODL.The last examples use surface magnetic �eld observations

to predict space weather. Luhmann et al. (1998, 2000) aredeveloping a technique using synoptic maps of the photo-spheric magnetic �eld as boundary conditions for potential�eld source surface models of the coronal �eld. The predic-tor is based on the idea that previously closed coronal �eldsbecome open as a consequence of the CME, and that themodels will show these changes. The statistical reliabilityof this predictor, and others such as sigmoid morphology(Can�eld et al., 2000) or �lament chirality (Martin, 2000)can be established by turning to the historical record of datacontained in the NSODL.


Fig. 3. The current NSODL web form page for a query on data product.

6. The future

The NSODL contains valuable basic solar data for spaceweather studies, developing forecasts based on historicalrecords, fundamental solar physics research, and publiceducation. Several data sets, including daily full-disk mag-netograms, spectroheliograms, and coronal maps in a vari-ety of spectral lines are available. The implementation ofthe user interface and search tool described here has greatlyfacilitated access to the archive of these data products.Several lines of development are being pursued to enhance

the capabilities of the NSODL. The archive of helioseis-mic data produced by GONG is now being integrated intothe NSODL. This archive contains helioseismic time series,spectra and frequencies; full-disk Dopplergrams and spec-troheliograms obtained every minute; and full-disk magne-tograms obtained every 20 min. The observations are taken

in the Ni I 676.8-nm line with a format of 256× 256 pixels,and are obtained around the clock from six geographicallydistributed sites. This data set began in early 1995, and isstill continuing. In early 2000, the GONG instruments willbe upgraded to a format of 1024×1024 pixels, and full-diskmagnetograms with this format will be obtained every 5 min(Harvey et al., 1998).The NSODL hardware will be substantially upgraded

in early 2001 to handle the data from the Synoptic Opti-cal Long-term Investigations of the Sun (SOLIS) project.SOLIS will replace the current KPVT with a suite ofthree instruments: a vector spectromagnetograph (VSM), afull-disk patrol (FDP) �ltergraph, and an integrated sunlightspectrometer (ISS) (Keller and NSO sta�, 1998). The corescienti�c data products from these instruments will be madefreely available to the Space Weather community throughthe NSODL. These products include: 3 per day photo-


Fig. 4. The web form page for a query on multiple products for given dates.

spheric full-disk 2048 × 2048 vector magnetograms, chro-mospheric line-of-sight magnetograms and He I 1083-nmline characteristics; 1 per day high sensitivity photosphericmagnetogram; 1=min 1024 × 1024 H� on- and o�-band,He I 1083-nm ew and Doppler images; 1=10 min continuum;1=3 h Ca K3, K2V, K2R; 1=day photospheric non-oscillatoryDoppler; and 60 high-resolution integrated sun spectra perday. Since the SOLIS core data rate is much larger (31.5GB=day) than the current volume, the NSODL hardwaremust be substantially enhanced. While the design has notyet been �nalized, the new NSODL will likely provideabout 9 TB of storage in the form of a 0.72-TB RAID, a4.2-TB DLT Jukebox, and 3.8 TB in several DVD-RAMjukeboxes.In addition to greatly expanding the data holdings of

the NSODL, the user interface could be further devel-oped. It would be useful to integrate the NSODL into a

meta-database (such as the Whole Sun Catalog, Dimitoglouet al., 1998) enabling simultaneous search and accessto several distributed solar databases. This would allow“one-stop shopping” for users of solar data. Perhaps theultimate NSODL development would be the implemen-tation of Content-Based Search & Analysis (CBSA) andCoherent Data Packaging (CDP). In CBSA, content-basedsearches using integrated image recognition, analysis, clas-si�cation, and graphical tools would allow researchers toformulate and answer speci�c research questions — forexample: “What was the weekly average solar magnetic ux between latitudes +10 and +20◦ over the last 15 yearsand how did this quantity correlate with magnetic stormsin the near-Earth environment?.” The CDP concept wouldcollect all available data pertaining to a speci�c solar eventlocalized in space and time, allowing users to quickly andeasily assemble a set of focussed information. While these


Fig. 5. The result of a successful search.

interface developments are not yet funded, their implemen-tation would result in the emergence of a powerful newtool for space weather and solar physics research — themarriage of analysis, data search and retrieval, and the dataitself into a coherent system.

Acknowledgements

This project is supported by the National Science Foun-dation, Division of Atmospheric Sciences, Solar TerrestrialResearch Program, National Space Weather Program.Many people have contributed to the NSODL. We would

like to thank Jacques Beckers, Jim Brault, Tim Henry,Amanda Jaksha, Bill Livingston, Mary McGraw, RobbieMcGraw, Larry November, Doug Rabin, Lourdes Ramirez,Ken Schatten, Nelsey Toner and Jeremy Wagner for theirvarious roles in fostering the NSODL.

References

Can�eld, R.C., Hudson, H.S., McKenzie, D.E., 2000. Sigmoidalmorphology and eruptive solar activity. Journal of Atmosphericand Solar Terrestrial Physics, in press.

Dimitoglou, G., Mendiboure, C., Reardon, K., Sanch�ez-Duarte,L., 1998. Whole sun catalog: design and implementation. In:Alissandrakis, C.E., Schmieder, B. (Eds.), Second AdvancesIn Solar Physics Euroconference — Three-DimensionalStructure Of Solar Active Regions, ASP Conference Series,Astronomical Society of the Paci�c, San Francisco, Vol. 155,pp. 297–301.

Gokhale, M.H., Javariah, J., 1990. Sunspot activity as originatingin interferences of the Sun’s global MHD oscillations. MonthlyNotices of the Royal Astronomical Society 243, 241–251.

Gokhale, M.H., Javariah, J., 1992. Global modes constituting thesolar magnetic cycle. II. Phases, ‘geometrical eigenmodes’ andcoupling of �eld behavior in di�erent latitudes. Solar Physics138, 399–410.

Gokhale, M.H., Javariah, J., 1995. Global modes constituting thesolar magnetic cycle. III. ‘Shapes’ and ‘sizes’ of the sunspot


cycles and maintenance of the MHD spectrum by energycascade. Solar Physics 156, 157–177.

Gokhale, M.H., Javariah, J., Naranyan Kutty, K., Varghese, B.A.,1992. Global modes constituting the solar magnetic cycle. I.Search for ‘dispersion relations’. Solar Physics 138, 35–47.

Harvey, J.W., Hill, F., Hubbard, R., Kennedy, J.R., Leibacher, J.W.,Pintar, J.A., Gilman, P.A., Noyes, R.W., Title, A.M., Toomre,J., Ulrich, R.K., Bhatnagar, A., Kennewell, J.A., Marquette, W.,Patr�on, J., Sa�a, O., Yasukawa, E., 1996. The Global OscillationNetwork Group (GONG) Project. Science 272, 1284–1286.

Harvey, J.W., Tucker, R., Britanik, L., 1998. High-resolutionupgrade of the GONG instruments. In: Korzennik, S., Wilson,A. (Eds.), Structure and Dynamics of the Interior of the Sunand Sun-Like Stars, ESA SP-418, pp. 209–211.

Harvey, K.L., White, O.R., 1999. Magnetic and radiative variabilityof solar surface structures. I. Image decomposition andmagnetic-intensity mapping. Astrophysics Journal 515, 812–831.

Harvey, K.L., White, O.R., 1998. Spectral irradiance and magneticstructures. In: Balasubramaniam, K.S., Harvey, J.W., Rabin,D.M. (Eds.), Synoptic Solar Physics, ASP Conference Series,Astronomical Society of the Paci�c, San Francisco, Vol. 140,pp. 247–259.

Hill, F., Branston, D., Erdwurm, W., 1997. The NationalSolar Observatory digital library. Bulletin of the AmericanAstronomical Union 29, 906.

Hill, F., Branston, D., Erdwurm, W., 1998. Current status of theNational Solar Observatory digital library. EOS Transactions ofthe American Geophysical Union 79 (17), S254.

Keller, C.U., NSO Sta�, 1998. SOLIS instrumentation aspects.In: Balasubramaniam, K.S., Harvey, J.W., Rabin, D.M. (Eds.),Synoptic Solar Physics, ASP Conference Series, Vol. 140,pp. 539–551.

Luhmann, J.G., Gosling, J.T., Hoeksema, J.T., Zhao, X.-P.,1998. The relationship between large-scale solar magnetic �eldevolution and coronal mass ejections. Journal of the GeophysicalResearch 103, 6585.

Luhmann, J.G., Li, Y., Hoeksema, J.T., Zhao, X.-P., Arge, N.,2000. The use of coronal magnetic �eld models to predictCMEs. Journal of Atmospheric and Solar Terrestrial Physics,in press.

Martin, S.F., 2000. A pilot project in forecasting the helicity ofinterplanetary magnetic clouds. Journal of Atmospheric andSolar Terrestrial Physics, in press.

Reardon, K., 1998. The role of data archives in synoptic solarphysics. In: Balasubramaniam, K.S., Harvey, J.W., Rabin, D.M.(Eds.), Synoptic Solar Physics, ASP Conference Series, Vol.140, pp. 460–467.

Sten o, J.O., 1988. Global wave patterns in the Sun’s magnetic�eld. Astrophysics and Space Science 144, 321–336.

Sten o, J.O., G�udel, M., 1988. Evolution of solar magnetic �elds:modal structure. Astronomy and Astrophysics 191, 137–148.

Sten o, J.O., Vogel, M., 1986. Global resonances in the evolutionof solar magnetic �elds. Nature 319, 285–290.

Wells, D.C., Greisen, E.W., Harten, R.H., 1981. FITS — a exible image transport system. Astronomy and Astrophysics 44,363–370.

the national solar observatory digital library — a resource for space weather studies

Documents