NOAA/NASA SSM/I PATHFINDER Pentad and Monthly Precipitation Rate Data Sets Table of Contents 1.0 INTRODUCTION 2.0 PENTAD AND MONTHLY PATHFINDER HDF FILE STRUCTURE 2.1 Pentad (Monthly)Precipitation Rate Grid 2.2 Sum of the Squared Precipitation Rates 2.3 Number of Pixels per Bin 2.4 File Description 3.0 HOW TO GET THE HDF LIBRARY AND TOOLS 4.0 HOW TO COMPILE THE PROGRAMS EXTRACTRG and GETFILEDESC 4.1 EXTRACTRG 4.2 GETFILEDESC 1.0 INTRODUCTION This README contains information on the structure of the Hierarchical Data Format (HDF) Pentad (5 days) and Monthly Precipitation Rate Grid files, instructions on how to get and install HDF on your machine, and a description of two utilities-- extractrg which extracts specified items from the pentad or monthly file, and getfiledesc which pulls out the file description and prints it to the screen. Attachment 1 is a brief discussion of the technical/scientific aspects of the data set, including pertinent references. The Marshall Space Flight Center (MSFC) Pathfinder Pentad and Monthly Precipitation Rate data sets were created using the MSFC SSM/I Pathfinder daily HDF Precipitation Rate files. A discussion of daily precipitation rate files is available from MSFC User Services at 205-922-5932 or e-mail at msfc@eos.nasa.gov. The daily data were grouped into a 1 degree longitude by 1 degree latitude array. The pixel precipitation rates were weighted using a scheme supplied by the algorithm developer; the weighting is described in Attachment 1. The array is 360 x 180 where the first dimension is longitude and the second is latitude. Array location (1,1) contains data between 180 and 179 degrees west longitude, and 89 and 90 degrees north latitude. Array location (2, 2) contains data for the 1 degrees square bounded by 179-178 degrees west longitude and 88-89 degrees north latitude. Data for each bin is accumulated for either 5 days (pentad) or one month, as appropriate. The first pentad begins with January 1 and includes data through January 5. Each succeeding pentad is generated using 5 day increments. During leap year, the pentad beginning on February 26 contains 6 days, including February 29; therefore, the other pentads are consistent with non-leap year pentads. The monthly files include all available data in the month. Attachment 2 is a list of the pentad or monthly precipitation rates which were not generated because of instrument or data problems. Each pentad or monthly file contains the precipitation rate grid in mm/day*100 (PRG), the sum of the squared of the precipitation rates (SSQ), the number of valid daily precipitation rates within the bin (NUM), and a file description. The monthly file is similar to the pentad file. Section 2 contains a description of the file structure. The MSFC Pathfinder team performed quality control (Q/C) on each file, and the details of this Q/C are available from MSFC User Services. An HDF 8-bit raster image of the precipitation rate object is available via ftp from the MSFC server. The 1 degree by 1 degree values were linearly interpolated to 0.5 degrees to generate the image, composed of 720 by 360 pixels. A color palette is attached for ease in viewing. One way to view the browse images is with the HDF Collage tool. The files were created on a Silicon Graphics VGX class computer with version 3.3, release 4 of the HDF library. If you are receiving these files on tape, they were transferred to tape with the UNIX tar facility. All files have been compressed with the IRIX UNIX compress command. The file size for an uncompressed file is 0.8 megabytes. NOTE: All discussions in this text are for row major applications written in c. If you are using the HDF FORTRAN interface to read the HDF objects, the arrays will be transposed. For example, array A(5,3) in an HDF c interface would become A(3,5) in an HDF FORTRAN interface. This will apply to all HDF data array discussions in this text. 2.0 PENTAD AND MONTHLY PATHFINDER HDF FILE STRUCTURE Each pentad or monthly precipitation rate file has the following contents: Table 1. Objects in the Pathfinder HDF Pentad (Monthly) Precipitation Rate File ITEM HDF OBJECT TYPE HDF REF No. Version Descriptor N/A 1 Pentad (or Monthly) Scientific Data Set 2 Precipitation Rate Sum of Squared Precipitation Scientific Data Set 3 Rates (unweighted) Count of valid values Scientific Data Set 4 File Description Annotation 5 Table 2 shows the flags used in the pentad and monthly precipitation rate objects. Valid calculated values are scaled before being stored. That is, the pentad or monthly precipitation rates are multiplied by 100.0 and then stored as integers; so to retrieve a pentad or monthly precipitation rate, you must divide the stored number by 100.0. Table 2 Flags For Precipitation Rates ------------------------------- | | FLAG 1 | FLAG 2 | ------------------------------- |PRG, SSQ | -10 | -20 | ------------------------------- where: PRG = Pentad or Monthly Precipitation Rate (0.0 - 2400 mm/day., unscaled) SSQ = Pentad or Monthly Precipitation Rate Squared (0.0 - 24002 mm2/day2., unscaled) FLAG 1 = No precipitation rate data was accumulated in the bin. FLAG 2 = For pentads, if more than 40% of the valid pixel precipitation values in a bin were associated with the cold surfaces and/or identified as ambiguous, the bin value is set to the -20. For monthly grids, a 20% threshold is used. Refer to Attachment 1 for a discussion on the thresholds. The following sections provide further details on each of the HDF precipitation rate objects. 2.1 Pentad (Monthly)Precipitation Rate Grid (PRG) This HDF object contains the pentad or monthly precipitation rate grid. The grid values are a weighted average of valid precipitation rate values multiplied by 100 before storing to retain a precision of 0.01 mm/day. For example, 31.5 mm/day is stored as 315. The minimum value is 0, and the maximum unscaled value is 2400. Flagged values, shown above, are stored in locations where the precipitation rate could not/should not be calculated. The array size of the grid is 360 x 180. Each value is stored as a 4 byte integer. The HDF data type used to store the precipitation rate is DFNT_INT32. It will be necessary to use this type in any software written to access the data. The HDF reference number is 2. 2.2 Sum of the Squared Precipitation Rates (SSQ) This object contains a grid of the sum of the squared precipitation rate values which are included in each bin of the grid. Daily valid precipitation rates within a bin for the specified number of days were squared and summed. These values are not weighted when squared and summed, unlike the computation of the precipitation values in the PRG object. The array size is 360 x 180. The data are stored as a 4-byte integers and HDF data type DFNT_INT32. The HDF reference number is 3. 2.3 Number of Pixels per Bin (NUM) This object contains the number of precipitation rate pixels which have been grouped into each bin in the PRG object; only valid precipitation rate values ( i.e., no flag values) are counted. The daily precipitation rate pixels are counted in each bin for the specified number of days and stored in this object. The array size is 360 x 180. The values are stored as 4-byte integers and HDF data type DFNT_INT32. The HDF reference number is 4. 2.4 File Description This object is an ASCII description of the pentad or monthly PR grid file. The description can be read with the program "getfiledesc" supplied by MSFC, after you compile it with the HDF library. Section 3.0 contains instructions on how to get the HDF library and section 4 contains information on how to compile and create +getfiledesc+. The HDF object reference number is 5. An example of a file description is: SSM/I GSCAT2 Precipitation Rates File ID = Precip.pen_87241_87245.hdf This is a LEVEL 3 product. This product is a 5-day composite grid, including Julian day 87241 through Julian day 87245. This grid includes 5 days of data. The grid is a 1-degree by 1-degree longitude/latitude grid; grid location (1,1), in the upper left corner, is located at 90 deg N latitude, 180 deg longitude. The gridbox covers 1 degree (90-89 N, 180-179 W) from that location. SSM/I PATHFINDER Software Version Number 1.0 MSFC File Structure Version Number 1.0 HDF Version Number 3.3, Release 3 MSFC Tool Set Version Numbers: extractrg 1.0 getfiledesc 1.0 This product was produced by NASA Marshall Space Flight Center. For more information on the technical content please contact Marshall Space Flight Center: Earth Observing System - Distributed Active Archive Center User Services; Phone: 205-922-5932, or via e-mail at: msfc@eos.nasa.gov 3.0 HOW TO GET THE HDF LIBRARY AND TOOLS This section gives the instructions for getting a UNIX version of HDF. 1. First connect to the NCSA ftp server. ftp.ncsa.uiuc.edu 2. Login as anonymous giving your Internet e-mail address as password. Get the README.FIRST and README files. They have most of the information referred to below plus other useful information. 3. Change to the HDF directory ftp> cd HDF There you will find several directories such as: HDF3.3r4 4. Change to the directory with the latest version, or the one you trust most. Any ALPHA release may not be your best choice. ftp> cd HDF3.3r4 5. Now you are almost there. Change directories once again to the tar directory. ftp> cd tar There you will find numerous files beginning with "HDFx.xrx.tar.Z.part-??". You can ftp the entire file from the directory using "HDFx.xrx.tar.Z.part". 6. Making sure you are in binary mode get all of these files. ftp> binary ftp> prompt ftp> mget HDF3.3r4.tar.Z.part* NOTE: You will want to come back to the top level directory and go to the Unix/Collage directory and change to the subdirectory which contains the version of collage you want to use. 7. Now return to your own system. ftp> bye 8. The files you just got contain the entire HDF library, examples, and utilities. You must concatenate them into one big file. cat HDF3.3r4.tar.Z.part*> hdf.tar.Z 9. Now uncompress the whole thing by typing. uncompress hdf.tar.Z This will yield a tar file called hdf.tar 10. Extract the contents of the tar file with the tar utility. tar -xvf hdf.tar This will create several directories including an HDF directory. 11. After the tar finishes, there will be a series of Make files in the /HDFx.xrx/hdf subdirectory. There should be one that corresponds to your type of machine. You will still have to do a little editing of the makefile to tell it where some files are, but there are ample comments to help you through. Also read the INSTALL.TOP file for further help. When you have completed the editing, run the makefile. This will install everything creating the lib/libdf.a directory and file. (Note: All utilities may not make. The team has had trouble with a few of them. You probably will not need them anyway. Just be sure that you get the libdf.a file.) When you have the lib/libdf.a file created you have the HDF library. This can then be used to compile the extractrg.c, which is included with this release. 4.0 HOW TO COMPILE THE PROGRAMS EXTRACTRG and GETFILEDESC Two programs are included with this distribution. They are extractrg.f and getfiledesc.c. Both run on the Silicon Graphics but should port easily to other platforms. A +makefile+, named +Make.pr+, is included with the distribution. It can be used to compile the programs on a UNIX platform. After making changes to the directory paths and flags, use the following UNIX command to run the make-file program: make -f Make.pr_3 all This will compile both programs, and two executables will result: +extractrg+ and +getfiledesc+ 4.1 EXTRACTRG The program, extractrg.c, extracts an HDF object from the pentad or monthly file. It produces a new HDF file with the name of the HDF object selected. The format for using the utility is: extractrg For example, the command extractrg Precip.pen_87241_87245.hdf PRG will produce a file call PRG_pen.87241. Run the program with no arguments, and it will list all of the options as shown below: ACRONYM FILE Object Type Variable Type ------- ---- -------------- ------------- PRG Precipitation Rate SDS INT32 SSQ Sum of squared PR values SDS INT32 NUM Number of Pixels in Bin SDS INT32 Where SDS is Scientific Data Set and INT32 is a 32 bit integer. Use the code when selecting an object to extract. You may select several codes at once, separated by a space. 4.2 GETFILEDESC The getfiledesc program prints the text of the HDF annotation contained in the pentad or monthly file. Its usage is getfiledesc < Pentad or Monthly PR filename>. The result should look like the example file description found in section 2.4, File Description. ATTACHMENT 1 Precipitation Rate Instantaneous global (over land and ocean) precipitation rates are calculated using the Goddard Scattering Algorithm, Version 2 documented in Adler et al. (1991, 1993, 1994). The 19 GHz horizontal and vertical, 22 GHz vertical, and 37 GHz horizontal antenna temperatures are interpolated to the higher resolution of the 85 GHz horizontal channel. These temperatures and the 85GHz horizontal temperatures are subjected to a discrimination process in which each pixel is tested for the presence of precipitation or no precipitation. At pixels with precipitation, the precipitation rates are calculated using a numerical model-based regression equation. Valid daily Pathfinder precipitation rates were weighted to account for the changes in the overlap of the pixel scenes depending on the location within the scan. The weights are a function of latitude and are recalculated about every 36 degrees (.1 Orbit). The weighted values were binned into 1 degree longitude and latitude rectangles for either 5 days or a month, and an average precipitation rate for the bin was then calculated. Counts of the ambiguous and/or cold surface pixels were kept, and if the number of the values within the bin exceeded a threshold, the pentad or monthly value for that bin was set to -20. The thresholds are 40% for the pentads and 20% for the monthlies. For a pentad, if the number of ambiguous or cold surface pixels is less than 40%, the precipitation rate for the bin is a weighted average of valid pixel precipitation rates; otherwise the bin value is set to -20. Bins in which no valid pixels are located are set to -10. Adler, Robert F., A. J. Negri, P. R. Keehn, and I. M. Hakkarinen, 1993, Estimation of Monthly Rainfall over Japan and Surrounding Waters From A Combination of Low-orbit Microwave and Geosynchronous IR Data, Journal of Applied Meteorology, vol. 32, pp. 335-356. Adler, Robert F., H.-Y. M. Yeh, N. Prasad, W.-K. Tao and J. Simpson, 1991, Microwave Simulations of a Tropical Rainfall System with a Three-Dimensional Cloud Model, Journal of Applied Meteorology, vol. 30, pp. 924-953. Adler, Robert F., G. J. Huffman, and P. R. Keehn, 1994, Global Tropical Rain Estimates From Microwave-adjusted and Geosynchronous IR Data, Remote Sensing Reviews. ATTACHMENT 2 Anomalous Pentad and Monthly Precipitation Rate Files 1987 DMSP F8 Satellite The pentad for July 30 through August 3, 1987 was not generated because Pathfinder daily precipitation rate information began on August 1, 1987. Pentad for the period October 3-7,1987 (87276-87280) has 1 day of data missing. The December 1987 monthly file was not generated because the SSM/I instrument was turned off on December 3, 1987 due to overheating. Also, pentads starting on Dec 2, 1987 (day 336) through December 31, 1987 were not generated for the same reason. 1988 DMSP F8 Satellite The pentads for the periods January 1- January 5, January 6-January 10, January 11 - January 15, 1988 was not generated because the instrument was not turned on until January 13, 1988. The January 1988 monthly file was not generated. The following pentads contained less than 5 days of data: May 6-10,1988 (88127-88131) has 3 days of data missing. September 23-27, 1988 (88267-88271) has 1 day of data missing. December 22-26, 1988 (88357-88361) has 2 days of data missing. December 27-31, 1988 (88362-88366) has 1 day of data missing.