Search

Multiwavelets are theoretically expected to perform better than traditional wavelets for image compression. Standard and new decomposition methods for multiwavelet compression have been studied in (1), where floating-point multiwavelet transforms are used for lossy image compression. In this paper we investigate both decomposition methods with a reversible integer multiwavelet transform for lossless compression of ultraspectral sounder data. This study shows that the multiwavelet compression method outperforms other well-known methods such as CALIC, SPIHT, and CCSDS IDC for lossless compression on ultraspectral data. Furthermore, the standard multiwavelet decomposition method yields better compression than the new one. We also show that the Bias- Adjusted Reordering (BAR) scheme for spectral data preprocessing yields a substantial improvement on multiwavelet compression with the average compression ratio increasing from 2.27 to 2.61.

Reversible variable-length codes (RVLCs) allow instantaneous decoding in both directions, which affords better detection of bit errors due to synchronization losses over a noisy channel. Earlier, we developed 3D wavelet reversible variable- length coding (3DWT-RVLC) for lossless compression of ultraspectral sounder data, which has significantly better error resilience than JPEG2000 Part 2 at only a small reduction in compression gain (1). To explore the feasibility of 3DWT-RVLC for real-time satellite onboard processing, we implement a memory-limited DSP version of 3DWT-RVLC. Experimental results for the 10 AIRS ultraspectral test granules show that the DSP-based 3DWT-RVLC yields an average compression ratio of 2.36, comparable to 2.51 of the original 3DWT-RVLC from our previous work.

The ultraspectral sounder data features strong correlations in disjoint spectral regions due to the same type of absorbing gases. This paper compares the compression performance of two robust data preprocessing schemes, namely Bias-Adjusted reordering (BAR) and Minimum Spanning Tree (MST) reordering, in the context of entropy coding. Both schemes can take advantage of the strong correlations for achieving higher compression gains. The compression methods consist of the BAR or MST preprocessing schemes followed by linear prediction with context-free or context-based arithmetic coding (AC). Compression experiments on the NASA AIRS ultraspectral sounder data set show that MST without bias-adjustment produces lower compression ratios than BAR and bias-adjusted MST for both context-free and context-based AC. Biasadjusted MST outperforms BAR for context-free arithmetic coding, whereas BAR outperforms MST for context-based arithmetic coding. BAR with context-based AC yields the highest average compression ratios in comparison to MST with context-free or context-based AC.

[1] This paper incorporates the latest improvements in intersatellite calibration, along with a new statistical technique, to determine the diurnal and seasonal cycles and climatic trends of 1978–2004 tropospheric temperature using Microwave Sounding Unit measurements. We also compare the latitudinal distribution of temperature trends from the surface and troposphere with each other and with model simulations for the past 26 years. The observations at the surface and in the troposphere are consistent with climate model simulations. At middle and high latitudes in the Northern Hemisphere, the zonally averaged temperature at the surface increased faster than in the troposphere while at low latitudes of both hemispheres the temperature increased more slowly at the surface than in the troposphere. The resulting global averaged tropospheric trend is +0.20 K/10 yr, with a standard error of 0.05 K/10 yr, which compares very well with the trend obtained from surface reports.

Low density parity check (LDPC) codes have the ability to transmit data at rates approaching channel capacity with very low error probability. Our study of LDPC codes on JPEG2000-compressed ultraspectral sounder data shows that for an LDPC encoder designed with certain input parameters, the LDPC decoder is able to correct all single errors and two-bit burst errors. Furthermore, our analysis on a massive variety of 3- and 4-bit burst error scenarios indicates that the locations of LDPC-uncorrected errors for 3- and 4-bit burst errors appear to be fixed, regardless of the content of the input data. We propose a look-up table based error correction method to correct errors surviving from the LDPC decoder for 3- and 4-bit burst errors.

[1] An evaluation of the temperature and moisture profile retrievals from the Atmospheric Infrared Sounder (AIRS) data is performed using more than 2 years of collocated data sets. The Aqua-AIRS retrievals, global radiosonde (RAOB) measurements, forecast data from the National Center for Environmental Prediction Global Forecasting System (NCEP_GFS), the European Center for Medium Range Forecast (ECMWF), and the operational retrievals from the NOAA 16 satellite Advanced TIROS Operational Vertical Sounder (ATOVS) instrument are used in this validation. Using RAOB observations as the reference, bias and RMS differences are computed for “sea,” “land,” and “all” categories for the AIRS retrievals and other collocated data sets. The results of the intercomparison reveal that temperature and water vapor retrievals from the AIRS are in very good agreement with the RAOBs. The RMS difference for clear-only cases over “sea” and “all” categories is close to the expected goal accuracies, namely, 1°K in 1 km layers for the temperature and better than 15% in 2-km layers for the water vapor in the troposphere. The overall RMS difference for the cloud-cleared cases is also close to the expected product goal accuracy except for a slight degradation at the surface. When AIRS and ATOVS retrievals are compared with the RAOBs, the AIRS temperature retrievals show an improvement over ATOVS of at least 0.5°K for all the accepted cases. Both the ECMWF and the NCEP_GFS forecasts match the RAOB temperatures within 1°K and water vapor within 14%. With respect to biases, the AIRS final retrieval shows a larger bias with the RAOBs relative to ATOVS, NCEP_GFS, and ECMWF. The bias is highly influenced by a larger bias contribution from “land” samples and shows a month-to-month and annual variation that correlates with the CO2 variations. This coupling suggests a need to include CO2 and possibly other trace gas climatologies in the AIRS initial guess to partially mitigate the effects in the final physical retrieval.

[1] This paper describes a unique validation data set acquired from a marine intensive observing period (IOP) conducted on board the NOAA Ship Ronald H. Brown (RHB) during the 2004 Aerosol and Ocean Science Expedition (AEROSE) in the tropical North Atlantic Ocean from 29 February to 26 March 2004. The radiometric and in situ data complement includes marine observations of the Saharan air layer (SAL), including two significant Saharan dust outbreaks over the Atlantic Ocean. Because the impact of tropospheric dust aerosols on satellite infrared (IR) radiometric observations has not yet been fully characterized, the AEROSE data are particularly valuable for IR sensor validation. Shipboard radiometric data germane to satellite validation include observations from a Marine Atmospheric Emitted Radiance Interferometer (M-AERI), a Calibrated Infrared In situ Measurement System (CIRIMS), and Microtops handheld sunphotometers. Among other things, these data provide, for the first time, coincident IR spectra of the dry, dusty SAL from both the uplooking M-AERI and the downlooking Atmospheric Infrared Sounder (AIRS) on board the Aqua satellite. In situ data collected throughout the cruise include Vaisala RS80/90 radiosondes, launched ≃3-hourly to include Aqua overpass times. The Aqua matchup profiles provide data for validation of AIRS in the presence of high dust loading, along with temperature and water vapor profile retrievals of the SAL. The frequency of sonde launches also enables validation of coincident uplooking M-AERI boundary layer profile retrievals. Preliminary analyses of the AEROSE data are presented here. Focused AEROSE validation studies are the subjects of separate papers.

Observations from the high spectral resolution Atmospheric InfraRed Sounder (AIRS) on the NASA EOS AQUA platform are providing improved information on the temporal and spatial distribution of key atmospheric parameters, such as temperature, moisture and clouds. These parameters are important for improving real-time weather forecasting, climate monitoring, and climate prediction. Trace gas products such as ozone, carbon dioxide, carbon monoxide, and methane are also derived. High spectral resolution infrared radiances from AIRS are assimilated into numerical weather prediction models. The soundings and radiances are provided in near real-time by NOAA/NESDIS to the NWP community. A significant component of the NOAA/NESDIS AIRS processing is to apply Principal Component Analysis (PCA) to the original AIRS 2000+ channel radiances. PCA is used for monitoring of the AIRS detectors, dynamic noise estimation and filtering, errant channel recovery, radiance reconstruction, and deriving an initial guess for profiles of temperature, moisture, ozone and other geophysical parameters. Since PCA has the ability to reduce the dimensionality of a dataset while retaining the significant information content, investigations are being done on its applications to AIRS data compression and archiving. Data compression is one of the key issues for the new generation of high spectral resolution satellite sensors. Our current AIRS research will allow us to provide valuable information and real-time experience to the generation of products for future sensors, such as the EUMETSAT IASI and NPOESS CrIS advanced infrared sounders. Examples of each application, along with details on the generation and application of eigenvectors are presented in this paper.

The NOAA/NESDIS Office of Research and Applications (ORA) has embarked on a pilot data stewardship project aimed at improving the data record from the Advanced Very High Resolution Radiometer (AVHRR). One part of this larger project includes the generation of a new cloud climatology from the Extended AVHRR Pathfinder Atmospheres (PATMOS-x) data set. Included within the PATMOS-x data-stream is a full suite of cloud products including various cloud amounts. This paper compares the PATMOS-x cloud amount time series for all July data (1982-2004) to the cloud amount time series from the International Satellite Cloud Climatology Project (ISCCP) and University of Wisconsin High Resolution Infrared Sounder (UW/HIRS) data sets. The results indicate that the large intersatellite discontinuities in the total amount seen in the original PATMOS are reduced in PATMOS-x. The total cloud for July time series from PATMOS-x, UW/HIRS and PATMOS show little trend over the period studied but that ISCCP time series does indicate a continuous downward trend When comparing the time series of high cloud amount, it was that PATMOS-x shows no significant trend in high cloud from 20S to 20N.

The 1Km high spatial resolution MODIS imager is used to quality control cloud-cleared radiances from the 14Km high spectral infrared sounder AIRS. This is accomplished by finding MODIS clear radiances values within the AIRS footprint.

Evaluation of the temperature and moisture profile retrievals from the Atmospheric Infrared Sounder data is performed using two years of collocated global radiosonde measurements, forecast data from the NCEP-AVN and ECMWF, and NOAA-16 operational retrievals.

PCA is a powerful tool for AIRS data compression. Eigenvector from a dependent dataset works best for data compression; those from independent datasets can be used to diagnose climate abnormal signals or high-noise in observations.

[1] This paper describes 3-hourly radiosonde observations of the Saharan air layer (SAL) acquired from the NOAA Ship Ronald H. Brown during the 2004 Aerosol and Ocean Science Expedition (AEROSE). The sampling frequency allows for unique vertical cross-sectional analyses of SAL phenomena, including dust events detected by shipboard Sun photometers, observed during March 2004. The observational analyses provide, for the first time, a coherent, 2-dimensional space-time depiction of the SAL as an expansive warm, dry, stable column located above the marine boundary layer. Midlevel easterly wind maxima are also observed to occur near the leading edge of the dry layers. The AEROSE sounding data will be useful for studies of the SAL, as well as for validation of environmental satellite sensors, especially the Aqua Atmospheric Infrared Sounder (AIRS).

The measurements from microwave imaging/sounding channels are simultaneously utilized to retrieve atmospheric temperature, water vapor and cloud water profiles. To improve accuracy, the radiative transfer model includes multiple scattering from clouds and precipitation.

Determination of atmospheric carbon requires an unprecedented precision for a satellite sounding measurement. Simulation experiments are used to assess the relative performance of the trace gas retrievals from the NASA AIRS, METOP IASI and NPOESS CrIS instruments.

The AIRS instrument was launched on the Aqua satellite in May of 2002. In addition to the core level 2 products, that include cloud cleared radiances; temperature, moisture, and ozone profiles; surface skin temperature; NDVI (from AIRS visible channels); surface spectral emissivity and reflectivity; and cloud products, the AIRS science team is also developing research algorithms for the retrieval of carbon monoxide (CO), methane (CH 4 ), and carbon dioxide (CO 2 ). These algorithms are being tested by the National Oceanographic and Atmosphere Administration (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS) in simulation and applied to real AIRS radiances. The trace gas retrievals require cloud free infrared radiances. In practice, we observe that AIRS measurements without cloud contamination occur less than 5% of the time. A key feature of the AIRS algorithm is the utilization of cloud cleared radiances that removes the effects of clouds and increases the yield of trace gas products to 50-60%. The increased yield should allow a better assessment of sources and sinks of these gases. Determination of sources and sinks of these trace gas requires an unprecedented precision for a remote sounding measurement. In addition, both the variability and errors in the trace gas products tend to be correlated with variability and errors in other products (e.g., clouds, temperature, moisture, and ozone profile). Multi-spectral, high-resolution measurements can minimize the effects of this correlation. Currently, for the AIRS products, we estimate a precision of 15% for CO, 0.5% for CO 2 and 1% for CH 4 . The remote sounding methodology for these trace gases is discussed in detail. The METOP IASI and NPOESS CrIS instruments can extend the continuity of these trace gas products over the next two decades. Simulation experiments are used to assess the relative performance of the trace gas retrievals from these sounders.

Development and testing of the IASI processing and distribution system is currently ongoing at NOAA/NESDIS/ORA. Level 1C data for 8461 channels will be available to NESDIS/NOAA from EUMETSAT shortly after MetOp 1 launch (currently scheduled for October 2005). Prior to launch, a simulation system will provide pseudo near-real time data for system testing and refinement. This will allow for a smooth and immediate system transition to the actual data processing when it becomes available. The ingested EUMETSAT level1C data will be subset both spectrally and spatially and then placed into BUFR format for a number of products including: (1) Level 1C (calibrated, apodized, and navigated) brightness temperatures, (2) cloud-cleared radiances, and (3) PCA reconstructed radiances. The subset level 1C data will be delivered within three hours of observation. System validation will consist of comparing the products to collocated radiosonde observations and model forecasts.

In this paper, we consider an application of the Empirical Mode Decomposition (EMD) introduced by Norden E. Huang in 1996 to the compression of 3D hyperspectral sounding data. The EMD is a new data analysis method which is based on expansion of the data in terms of Intrinsic Mode Functions (IMF). These IMFs are based on and derived from the data set. Since EMD adaptively represent the signal as a sum of "well behaved" amplitude/frequency modulated components, we found it very well suited for the whitening part of the compression scheme.

Since October, 2002, NESDIS has provided specially tailored radiance and retrieval products derived from Aqua AIRS and AMSU-A observations operationally (24 hours x 7 days) to a number of Numerical Weather Prediction (NWP) centers, including NCEP, ECMWF and the UK Met. Office. Two types of products are available -- thinned radiance data and full resolution retrieval products consisting of atmospheric temperature, moisture and ozone as well as surface parameters of temperature and emissivity. The radiances are thinned because of current limitations in communication bandwidth and computational resources at NWP centers. There are two types of thinning: a) spatial and spectral thinning, and b) data compression using principal component analysis (PCA). PCA is used for a) reconstructing radiances with the properties of reduced noise, b) independent instrument noise estimation, c) quality control, and d) deriving the retrieval first guess used in the AIRS processing software. The radiance products also include cloud cleared radiances. The cloud clearing procedure remove the effect of cloud contamination in partial overcast conditions and have been demonstrated to increase the amount of data that can be treated as clear from 5% to 50%. The AIRS temperature and moisture retrieval are significantly more accurate than AMSU-only retrievals in clear, cloud contaminated and cloud-cleared conditions. Most NWP centers are currently assimilating clear radiances, which we believe severely limits the impact of AIRS data. Fortunately, results presented in this paper have stimulated new upcoming experiments to test the impact of cloud-cleared radiances.

A near real-time AIRS processing and distribution system is fully operational at NOAA/NESDIS/ORA. The AIRS system went though three separate production phases: design and development, implementation, and operations. The design and development phase consisted of two years of preparation for the near real-time AIRS data. The approach was to fully emulate the AIRS measurement stream. This was accomplished by using a forecast model to represent the geophysical state and computation of simulated AIRS measurements using the characteristics of the AIRS channels. The preparation included file format development and the creation of a program to subset the radiance and product data. The implementation phase lasted over a year and involved utilizing AIRS/AMSU/HSB simulated data quasi-operationally. This simulated data was placed into deliverable files and distributed to the customers for their pre-launch preparations. The operational phase consisted of switching the simulation system to real data and is the current system status. Details of what went right and wrong at each production phase will be presented. This methodology eased the transition to operations and will be applied to other advanced sounders such as IASI and CrIS.

[1] The Microwave Sounding Units (MSU) aboard the NOAA series of polar orbiting satellites has been used by three groups to monitor the very small trend in the global tropospheric temperature over the 25-year satellite record. To obtain a homogeneous data set, each group made different calibration corrections of the MSUs in the form of fixed biases, and in some cases temperature-dependent adjustments, to each of the nine satellite instruments using data during the overlap periods. Up until now, however, the adjustments are empirically based. To improve the accuracy as well as our understanding of the error sources, this paper develops an alternate, physical approach for intercalibrating the MSU instruments. The paper develops a calibration model for the MSU instrument that includes the errors in the cold space and warm target measurements, as well as the nonlinear factor. Corrections for these calibration errors are estimated using a least squares minimization where the predictors are the differences between all 12 overlapping satellite measurements at low and high latitudes. After applying the calibration corrections, the zonally averaged differences between satellite instruments are no larger than 0.03 K, independent of latitude. It is also found that the tropospheric temperature trend derived from MSU measurements is nearly the same as the surface trend. Furthermore, it now appears that much of the earlier inconsistency between the satellite and surface measurements arises from errors in the MSU calibration correction procedure, which can artificially suppress the temperature trend.

Atmospheric InfraRed Sounder (AIRS) is a hyper-spectral infrared instrument on the EOS-Aqua satellite. Principal Component Analysis (PCA) of realistic simulations of AIRS radiances, which includes the effects of variable clouds and surface parameters, can be used to estimate and filter AIRS instrumental noise. The PCA uses noise scaled radiance, i.e. radiance divided by noise (R/N). Since the square root of the eigenvalues is equivalent to the standard deviation of the principal component score of the dependent ensemble, the square root of the eigenvalues in the R/N domain can be interpreted as a signal to the noise ratio for the new principal components (PC) or say "abstract channels". New PCs are arranged from the largest to the smallest eigenvalues. Once the R/N ratio is below unity, the signal has less contribution than noise for that PC and all the remaining PCs. This physical meaningful fact can be a criterion for radiance noise filtering. Using the linearity of PCA to the pure signal S plus ideal noise N in the R/N domain, PCA(N/N) becomes PCA(I). I is the identity matrix and hence eigenvalues of I are all equal. If reconstructing I, only k/m information can be recovered by k eigenvectors, where k = 1 .... m eigenvectors. Thus, it is easy to know how much noise merged into the reconstructed radiance and how many eigenvectors are needed for filtering noise when performing PCA in R/N domain. The number of eigenvectors to be used for noise estimating is around 60 in all sky conditions. 60 eigenvectors can filter over 97% of noise. Only 3% noise remains in the reconstructed data. Signal information can be recovered with accuracy in noise level. In addition, there is less than 5% error using PCA to estimate noise. Preliminary results from the real AIRS observation are discussed briefly.

The assimilation of Atmospheric InfraRed Sounder, Advanced Microwave Sounding Unit-A and Humidity Sounder Brazil (AIRS/AMSU/HSB) data by Numerical Weather Prediction (NWP) centers is expected to result in improved forecasts. Specially tailored radiance and retrieval products derived from AIRS/AMSU/HSB data are being prepared for NWP centers. There are two types of products – thinned radiance data and full resolution retrieval products of atmospheric and surface parameters. The radiances are thinned because of limitations in communication bandwidth and computational resources at NWP centers. There are two types of thinning: a) spatial and spectral thinning, and b) data compression using principal component analysis (PCA). PCA is also used for quality control and for deriving the retrieval first guess used in the AIRS processing software. Results show that PCA is effective in estimating and filtering instrument noise. The PCA regression retrievals show layer mean temperature ( 1 km in troposphere, 3km in stratosphere) accuracies of better than 1 K in most atmospheric regions from simulated AIRS data. Moisture errors are generally less than 15% in 2 km layers, and ozone errors are near 10% over approximately 5 km layers from simulation. The PCA and regression methodologies will be described.

Advanced infrared sounders will have thousands of spectral measurements. Eigenvectors of the data are used for data compression and retrievals of geophysical parameters

Future satellites will have the higher spectral resolutions required to provide an adequate level of vertical resolution in the soundings they produce, but the high resolution is accompanied by a large number of channels. In order for the data to be used by numerical prediction centers, the number must be reduced while maintaining the information content. This paper presents an approach for reducing the number of channels by averaging similar channels to produce “super channels” with reduced noise. To be useful, the “super channels” must be accompanied by an fast and accurate approximation of the Plank function. Up till now, this has been a limitation in the application of the approach. This paper presents a solution for adjusting the calculation at a single frequency to match the value for an average over a spectral interval. The approach will be applied to AIRS channels and the results will be presented at the meeting.

A first-order autoregressive model is used to estimate the uncertainty associated with the MSU derived global temperature trends.

The NPOESS program merges the United States military and civil operational meteorological satellite systems. The convergence of both systems result in significant cost savings and most importantly increase system performance. An example of improved performance is the inclusion of an advanced high spectral resolution infrared sounder, which will greatly improve the accuracy of satellite temperature and moisture soundings and is expected to yield significant improvements in numerical weather prediction. Further benefits include nowcasting and climate studies.

The perturbed form of the radiative transfer equation (RTE) for the simultaneous retrieval of temperature and moisture is given in Smith et al. (1985, 1991) as where ΔR is the deviation of the observed radiance from a mean or initial guess radiance. The first integral term in (1) physically represents the contribution of the Planck equivalent departure of the ambient temperature from a mean or initial guess temperature, ΔB, weighted by the vertical derivative of transmittance dτ which is evaluated for the initial condition. Any contribution to ΔR due to differences between the initial and ambient transmittance profiles, Δτ, are accounted for by the second integral term.

An estimated three billion metric tons of mineral aerosols are injected into the troposphere annually from the Saharan desert [Prospero et al., 1996]. Additionally, smoke from biomass burning sites in the savanna grasslands in sub-Saharan Africa contribute significant quantities of smaller-sized aerosols [e.g., Hobbs, 2000]. These windswept aerosols from the African continent are responsible for a variety of climate, health, and environmental impacts on both global and regional scales that span the Western Hemisphere. Unfortunately in situ measurements of aerosol evolution and transport across the Atlantic are difficult to obtain, and satellite remote sensing of aerosols can be challenging.

Pattern recognition procedures have been developed in order to improve the first-guess fields for satellite temperature retrievals. The first procedure is used to select one or more historical radiosonde temperature profiles as analog estimates of ambient thermal structure. The second procedure is used to organize a priori data into shape-coherent pattern libraries using structural information inherent in the data itself. On the basis of independent tests of about 800 temperature retrievals, it was found that: (1) the pattern recognition techniques reduced first-guess profile errors by nearly 50 percent in comparison with traditional partitioning schemes; and (2) with regression and physical-iterative retrieval algorithms, however, the effect of pattern recognition on temperature retrieval error was insignificant. Analysis of individual retrieval errors showed that poor retrievals may outweigh the potential benefits of both pattern recognition techniques.

Atmospheric temperature and water vapor profiles retrieved from the MetOp-IASI instrument were evaluated with global radiosonde measurements and ECMWF analysis. Analysis of information content embedded in these retrievals was also attempted.

This work overviews a unique complement of radiometric and in situ Saharan dust observations acquired during the 2004 Aerosol and Ocean Science Expedition (AEROSE) and their potential application for infrared hyperspectral sensor validation.

For the most part, satellite observations of climate are not presently sufficiently accurate to establish a climate record that is indisputable and hence capable of determining whether and at what rate the climate is changing. Furthermore, they are insufficient for establishing a baseline for testing long-term trend predictions of climate models. Satellite observations do provide a clear picture of the relatively large signals associated with interannual climate variations such as El Nino-Southern Oscillation (ENSO), and they have also been used to diagnose gross inadequacies of climate models, such as their cloud generation schemes. However, satellite contributions to measuring long-term change have been limited and, at times, controversial, as in the case of differing atmospheric temperature trends derived from the U.S. National Oceanic and Atmospheric Administration's (NOAA) microwave radiometers.