Search

Independent component analysis has been known for its success in blind source separation. It features a decorrelation capability beyond second-order moments. Recently ICA has been used in lossy compression for target detection in hyperspectral imager data where loss of unimportant features does not affect the detection result. For the ultraspectral sounder data, it is better to be lossless compressed for the ill-posed retrieval of geophysical parameters. In this paper we will try to use ICA in the lossless compression of ultraspectral sounder data. The compression result shows that ICA compares favorably with JPEG2000, JPEG-LS, and CCSDS IDC 5/3 on the ten standard AIRS granules dataset.

The Lloyd-Max scalar quantization is to find a set of quanta {q1, ..., qn} and a set of partition endpoints {x1,..., xn-1] that minimize the mean squared error distortion, given a probability distribution p(x). For symmetrical distributions, optimal quantizers can be nonsymmetrical. Previous studies proved that the log-concavity of p(x) is sufficient for the uniqueness of a locally optimal solution, which is therefore globally optimal. When the distribution satisfies the log-concavity condition, the optimal quantizer is symmetrical. The condition covers the Gaussian, Laplace, and Rayleigh distributions. In this paper we show that there exist asymptotic solutions for such distributions that are numerically considered as the solutions to the Lloyd-Max problem within any finite machine precision, although theoretically they are not. We use MATLAB symbolic math toolbox to list these asymptotic solutions with 21 decimal digits of accuracy, with dqn smaller than 64-bit floating point spacing to be regarded as zero. These asymptotic solutions are degenerate solutions with some quanta and endpoints approaching to infinity. For such a log-concave distribution with an even number (n = 2 m ges 6) of quantization levels, the Lloyd- Max quantization seems to yield k = lceilm/2rceil-1 degenerate solutions with n - 4, ... , n - 4 k finite quanta, respectively. Furthermore, the k degenerate solutions of the Lloyd-Max quantization with n quantizer level correspond to the optimal solutions of the Lloyd-Max quantization with n-4, ... , n-4k quantizer levels, respectively. These asymptotic solutions are locally minimum mean-square error quantizers, but none of them is the globally optimal quantizer.

Part 3 of this Special Section on High-Performance Computing in Applied Remote Sensing continues to present the state-of-the-art research in incorporating high-performance computing facilities and algorithms for effective and efficient remote sensing applications. “Land cover classification in multispectral imagery using clustering of sparse approximations over learned feature dictionaries” by Moody et al. presents results from an ongoing effort to extend neuromimetic machine vision algorithms to multispectral data using adaptive signal processing combined with compressive sensing and machine learning techniques. “Graphics processor unit accelerated finite-difference time domain method for electromagnetic scattering from onedimensional large scale rough soil surface at low grazing incidence” by Jia et al. applies the Compute Unified Device Architecture (CUDA) technology to compute an electromagnetic scattering problem on GPUs and achieves significant speedup factors. “High-performance meshing processing of remote sensing data on large displays” by Hsieh et al. presents a technique that visualizes LIDAR point clouds on a tiled display wall, termed highly interactive-parallelized display (HIPerDisplay), which has twenty 24-inch liquid-crystal displays with a total resolution of 46 Mpixels. The results also showed that the HIPerDisplay offered superior performance for the processing of large LIDAR datasets. “Real-time implementation of optimized maximum noise fraction transform for feature extraction of hyperspectral images” by Wu et al. presents a parallel implementation of the optimized maximum noise fraction (G-OMNF) transform algorithm for feature extraction of hyperspectral images on commodity graphics processing units (GPUs). The proposed approach explored the algorithm data-level concurrency and optimized the computing flow. “Hyperspectral band selection based on parallel particle swarm optimization and impurity function band prioritization schemes” by Chang et al. presents a framework for band selection in hyperspectral imagery that uses two techniques, referred to as particle swarm optimization (PSO) band selection and the impurity function band prioritization (IFBP) method. The experimental results demonstrate that the proposed PPSO/IFBP band selection method can not only improve computational speed but also offer a satisfactory classification performance. “Graphics processing unit implementation of the maximum likelihood solution to the inverse problem for retrieval of geophysical parameters from high-resolution sounder data” by Wei et al. explores the use of GPU in speedup of the maximum likelihood solution to an illposed retrieval problem. For the infrared atmospheric sounding interferometer high-resolution sounder having 8641 channels, the use of GPU on maximum likelihood estimation shows a promising speedup of 1986× compared to a single-threaded native CPU version.

This paper presents systematic lossless data compression studies conducted at Cooperative Institute of Meteorological Satellite Studies (CIMSS), University of Wisconsin-Madison in support of the real-time rebroadcast of NOAA's future GOES ultraspectral sounders. Ultraspectral sounders provide data with high spectral and spatial resolutions. Since an ultraspectral sounder could be either a grating spectrometer or a Michelson interferometer, we have investigated/developed various 2D and 3D lossless compression techniques for both grating and interferometer data. The lossless compression results are obtained and compared from wavelet/multiwavelt transform-based (e.g. JPEG2000, 3D SPIHT, MWT), prediction-based (e.g. JPEG-LS, CALIC), projection-based (e.g. Lossless PCA, Optimized Orthogonal Matching Pursuit-based Linear Prediction, PLT), and clustering-based (e.g. PPVQ, FPVQ, AVQLP) methods. Robust data preprocessing schemes (e.g. BAR, MST reordering) are also demonstrated to improve compression gains of existing state-of-the-art compression methods such as JPEG2000, 3D SPIHT, JPEG-LS, and CALIC for high-spectral-resolution data compression. Our studies show that high lossless compression gains are achievable for both grating and interferometer data.

Given a limited bandwidth, unequal error protection (UEP) via channel coding with different code rates for different important regions of bitstream may yield much less error contamination than equal error protection (EEP). We propose an optimal code rate allocation scheme that minimizes error contamination after channel and source decoding. We use JPEG2000 for source coding and turbo product code (TPC) for channel coding as an example to demonstrate this technique with ultraspectral sounder data. The statistics of erroneous pixels after TPC and JPEG2000 decoding are used to determine the optimal channel code rates for each wavelet resolution. The proposed UEP scheme significantly reduces error contamination when compared to its EEP counterpart. The optimal code rate allocation needs to be determined only once, given any predefined set of available channel codes, desired code rate, and noise level.

Satellite data transmission may possess random and burst errors. Our previous study [1] shows that 3D wavelet transform with reversible variable-length coding (3 DWT- RVLC) [1] has significantly better error resilience than JPEG 2000 Part 2 [2,3] on 1-bit random error remaining after channel decoding. In this paper, we investigate the burst error correction performance of the Digital Video Broadcasting - Second Generation (DVB-S2) standard for the 3 DWT-RVLC- compressed ultraspectral sounder data. DVB-S2 consists of Low density parity-check (LDPC) codes, which have been shown to perform close to the Shannon limit. We also study the error contamination after 3DWT-RVLC source decoding. Experimental results for the 10 standard AIRS ultraspectral test granules show that 3DWT-RVLC produces significantly fewer erroneous pixels than JPEG 2000 Part 2 due to errors remaining after DVB-S2 LDPC channel decoding of burst errors.

We propose a new low-complexity algorithm for lossless compression of hyperspectral imagery using lookup tables along with a predictor selection mechanism. We first compute a locally averaged interband scaling (LAIS) factor for an estimate of the current pixel from the co-located one in the previous band. We then search via lookup tables in the previous band for the two nearest causal pixels that are identical to the pixel co-located to the current pixel. The pixels in the current band co-located to the causal pixels are used as two potential predictors. One of the two predictors that is closest to the LAIS estimate is chosen as the predictor for the current pixel. The method pushes lossless compression of the AVIRIS hyperspectral imagery to a new high with an average compression ratio of 3.47.

JPEG-LS 1 is the new ISO/ITU standard for lossless and near-lossless compression of 2D continuous-tone images. For contemporary and future ultraspectral sounder data that features good correlations in disjoint spectral channels, we develop an MST-embedded JPEG-LS (Minimum Spanning Tree embedded JPEG-LS) for achieving higher compression gains through MST channel reordering. Unlike previous non-embedded MST work with other cost functions used only for data preprocessing, the MST-embedded JPEG-LS uniquely uses the sum of absolute median prediction errors as the cost function for MST to determine each optimal pair of predicting and predicted channels. The MST can be embedded within JPEG-LS because of the same median prediction used in JPEG-LS. The advantage of this embedding is that the median prediction residuals are available to JPEG-LS after MST channel reordering without recalculation. Numerical experiments show that the MST-embedded JPEG-LS yields an average compression ratio of 2.81, superior to 2.46 obtained with JPEG-LS for the 10 standard ultraspectral granules.

Entropy coding techniques aim to achieve the entropy of the source data by assigning variable-length codewords to symbols with the code lengths linked to the corresponding symbol probabilities. Entropy coders (e.g. Huffman coding, arithmetic coding), in one form or the other, are commonly used as the last stage in various compression schemes. While these variable-length coders provide better compression than fixed-length coders, they are vulnerable to transmission errors. Even a single bit error in the transmission process can cause havoc in the subsequent decoded stream. To cope with it, this research proposes a marker-based sentinel mechanism in entropy coding for error detection and recovery. We use arithmetic coding as an example to demonstrate this error-resilient technique for entropy coding. Experimental results on ultraspectral sounder data indicate that the marker-based error-resilient arithmetic coder provides remarkable robustness to correct transmission errors without significantly compromising the compression gains.

Contemporary and future ultraspectral sounders represent a significant technical advancement for environmental and meteorological prediction and monitoring. Given their large volume of spectral observations, the use of robust data compression techniques will be beneficial to data transmission and storage. In this paper, we propose a novel Adaptive Vector Quantization (VQ)-based Linear Prediction (AVQLP) method for ultraspectral data compression. The method is compared with several state-of-the-art methods such as CALIC, JPEG-LS and JPEG2000. The compression experiments show that our AVQLP method is the first to surpass the 4 to 1 lossless compression barrier for a selected set of AIRS ultraspectral sounder test data.

Previous study shows 3D Wavelet Transform with Reversible Variable-Length Coding (3DWT-RVLC) has much better error resilience than JPEG2000 Part 2 on 1-bit random error remaining after channel decoding. Errors in satellite channels might have burst characteristics. Low-density parity-check (LDPC) codes are known to have excellent error correction capability near the Shannon limit performance. In this study, we investigate the burst error correction performance of LDPC codes via the new Digital Video Broadcasting - Second Generation (DVB-S2) standard for ultraspectral sounder data compressed by 3DWT-RVLC. We also study the error contamination after 3DWT-RVLC source decoding. Statistics show that 3DWT-RVLC produces significantly fewer erroneous pixels than JPEG2000 Part 2 for ultraspectral sounder data compression.

The Karhunen-Loeve transform (KLT) is the optimal unitaty transform that yields the maximum coding gain. The prediction-based lower triangular transform (PLT) features the same decorrelation and coding gain properties as the KLT, but with lower complexity. Unlike KLT, PLT has the perfect reconstruction property which allows its use for lossless compression. In this paper we apply PLT to lossless compression of the ultraspectral sounder data. The experiment on the standard ultraspectral test dataset of 10 AIRS digital count granules shows that the PLT compression outperforms JPEG2000, SPIHT, JPEG-LS, and CCSDS IDC 5/3.

Most source coding techniques generate bitstream where different regions have unequal influences on data reconstruction. An uncorrected error in a more influential region can cause more error propagation in the reconstructed data. Given a limited bandwidth, unequal error protection (UEP) via channel coding with different code rates for different regions of bitstream may yield much less error contamination than equal error protection (EEP). We propose an optimal UEP scheme that minimizes error contamination after channel and source decoding. We use JPEG2000 for source coding and turbo product code (TPC) for channel coding as an example to demonstrate this technique with ultraspectral sounder data. Wavelet compression yields unequal significance in different wavelet resolutions. In the proposed UEP scheme, the statistics of erroneous pixels after TPC and JPEG2000 decoding are used to determine the optimal channel code rates for each wavelet resolution. The proposed UEP scheme significantly reduces the number of pixel errors when compared to its EEP counterpart. In practice, with a predefined set of implementation parameters (available channel codes, desired code rate, noise level, etc.), the optimal code rate allocation for UEP needs to be determined only once and can be done offline.

Multiwavelets are theoretically expected to perform better than traditional wavelets for image compression. Standard and new decomposition methods for multiwavelet compression have been studied in (1), where floating-point multiwavelet transforms are used for lossy image compression. In this paper we investigate both decomposition methods with a reversible integer multiwavelet transform for lossless compression of ultraspectral sounder data. This study shows that the multiwavelet compression method outperforms other well-known methods such as CALIC, SPIHT, and CCSDS IDC for lossless compression on ultraspectral data. Furthermore, the standard multiwavelet decomposition method yields better compression than the new one. We also show that the Bias- Adjusted Reordering (BAR) scheme for spectral data preprocessing yields a substantial improvement on multiwavelet compression with the average compression ratio increasing from 2.27 to 2.61.

Reversible variable-length codes (RVLCs) allow instantaneous decoding in both directions, which affords better detection of bit errors due to synchronization losses over a noisy channel. Earlier, we developed 3D wavelet reversible variable- length coding (3DWT-RVLC) for lossless compression of ultraspectral sounder data, which has significantly better error resilience than JPEG2000 Part 2 at only a small reduction in compression gain (1). To explore the feasibility of 3DWT-RVLC for real-time satellite onboard processing, we implement a memory-limited DSP version of 3DWT-RVLC. Experimental results for the 10 AIRS ultraspectral test granules show that the DSP-based 3DWT-RVLC yields an average compression ratio of 2.36, comparable to 2.51 of the original 3DWT-RVLC from our previous work.

The compression of ultraspectral sounder data is better to be lossless or near-lossless to avoid potential degradation of the geophysical retrieval in the associated ill-posed problem. Transform-based, prediction-based, and clustering-based methods for lossless compression of ultraspectral sounder data have been presented. It is shown that the compression ratios of ultraspectral sounder data via the standard state-of-the-art algorithms (e.g. 3D SPIHT, 2D JPEG2000, 2D CALIC, 2D JPEG-LS etc.) can be significantly improved when combining the BAR preprocessing scheme. We also report the promising compression results for the ultraspectral sounder data using various approaches such as lossless PCA and Predictive Partitioned Vector Quantization (PPVQ).

The ultraspectral sounder data features strong correlations in disjoint spectral regions due to the same type of absorbing gases. This paper compares the compression performance of two robust data preprocessing schemes, namely Bias-Adjusted reordering (BAR) and Minimum Spanning Tree (MST) reordering, in the context of entropy coding. Both schemes can take advantage of the strong correlations for achieving higher compression gains. The compression methods consist of the BAR or MST preprocessing schemes followed by linear prediction with context-free or context-based arithmetic coding (AC). Compression experiments on the NASA AIRS ultraspectral sounder data set show that MST without bias-adjustment produces lower compression ratios than BAR and bias-adjusted MST for both context-free and context-based AC. Biasadjusted MST outperforms BAR for context-free arithmetic coding, whereas BAR outperforms MST for context-based arithmetic coding. BAR with context-based AC yields the highest average compression ratios in comparison to MST with context-free or context-based AC.

Low density parity check (LDPC) codes have the ability to transmit data at rates approaching channel capacity with very low error probability. Our study of LDPC codes on JPEG2000-compressed ultraspectral sounder data shows that for an LDPC encoder designed with certain input parameters, the LDPC decoder is able to correct all single errors and two-bit burst errors. Furthermore, our analysis on a massive variety of 3- and 4-bit burst error scenarios indicates that the locations of LDPC-uncorrected errors for 3- and 4-bit burst errors appear to be fixed, regardless of the content of the input data. We propose a look-up table based error correction method to correct errors surviving from the LDPC decoder for 3- and 4-bit burst errors.

— Overdrive is commonly used to reduce the liquid-crystal response time and motion blur in liquid-crystal displays (LCDs). However, overdrive requires a large frame memory in order to store the previous frame for reference. In this paper, a lapped transform-based codec (LTC) is proposed to reduce the frame memory needed for LCD overdrive. In the latest literature, a directional prediction-based codec (DPC) employs eight directional predictions, which takes up a large percentage of the computational complexity of the codec and does not consider the de-correlation of the inter-blocks. Therefore, the LTC first uses the lapped transform to decompose the correlation of the inter-blocks in the YUV color space. A hadamard transform is then used for energy compaction. The reordered coefficients are pre-quantized and encoded using the proposed adaptive bit-plane coding (ABPC) method for a simple hardware implementation. The simulation results show that the proposed LTC significantly improves the DPC in both subjective and objective performance and outperforms the block truncation coding (BTC) and adaptive multi-level BTC (AM-BTC), which have been described in the literature.

A method 7 combining the empirical mode decomposition (EMD) 6 and the principal component analysis (PCA) was recently proposed for lossless compression of ultraspectral sounder data. In that method, data residual is obtained via the linear regression of the data against m intrinsic mode functions (IMFs) which are obtained from the EMD of the data mean, followed by the linear regression of the IMF regression error against a truncated number, n, of the corresponding principal components (PCs). In this paper we show that this two-stage (m IMFs + n PCs) linear transform approach is not as good as its counterpart two-stage (m PCs + n PCs) linear transform approach in terms of data residual and compression ratio of ultraspectral data, given the same number of IMFs and PCs used respectively at the first stage, followed by the same number of PCs used at the second stage. Mathematically, the two-stage (m PCs + n PCs) linear transform approach is equivalent to a single linear transform with (m + n) PCs. In other words, the simple PCA compression method outperforms this combined EMD and PCA compression method. This is expected because the PCA (a.k.a. the Karhunen-Loeve transform or the Hotelling transform) is known to be the optimal linear transform in the sense of minimizing the mean squared error. Our compression experiment confirms our conjecture that the role of EMD in the EMD + PCA approach is not only redundant but also degrades the compression ratios as compared to the PCA approach. We also compare the effect of using a partial set of 1502 channels against using the full set of 2107 channels, and show that the EMD + PCA approach does not break the 3:1 lossless compression barrier when compressing the full set of 2107 channels in the standard ultraspectral test data set. Our lossless PCA approach is the first to break the 3:1 compression barrier for the full set of this class of data.

A new technique called the local region of influence (LROI) scheme for supervised cloud classification of the Moderate Resolution Imaging Spectroradiometer (MODIS) is proposed. The classification of each observation is performed within the LROI, where the center of each class is calculated as a weighted average of its training class members with respect to each new observation. The probability of each class is assigned to each observation. The proposed LROI scheme is applied to the MODIS radiances observed from the scenes of clear skies, ice clouds, or water clouds. The classification results are compared with those from the maximum likelihood (ML) classification method, the multicategory support vector machine (MSVM) and the operational MODIS cloud mask algorithm. The lowest misclassification error rates show the advantage of the LROI scheme.

Nonreversible variable-length codes (e.g. Huffman coding, Golomb-Rice coding, and arithmetic coding) have been used in source coding to achieve efficient compression. However, a single bit error during noisy transmission can cause many codewords to be misinterpreted by the decoder. In recent years, increasing attention has been given to the design of reversible variable-length codes (RVLCs) for better data transmission in error-prone environments. RVLCs allow instantaneous decoding in both directions, which affords better detection of bit errors due to synchronization losses over a noisy channel. RVLCs have been adopted in emerging video coding standards--H.263+ and MPEG-4--to enhance their error-resilience capabilities. Given the large volume of three-dimensional data that will be generated by future space-borne ultraspectral sounders (e.g. IASI, CrIS, and HES), the use of error-robust data compression techniques will be beneficial to satellite data transmission. In this paper, we investigate a reversible variable-length code for ultraspectral sounder data compression, and present its numerical experiments on error propagation for the ultraspectral sounder data. The results show that the RVLC performs significantly better error containment than JPEG2000 Part 2.© (2005) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Ultraspectral sounder data is known for its huge size and sensitivity to noise in retrieving geophysical parameters. Lossless compression of it is thus desired for archival and transfer purposes. In this paper, we propose a lossless compression scheme which consists of a linear prediction, a lattice encoder, and an entropy encoder. The linear prediction serves as a whitening tool to produce a normally distributed residual. The lattice encoder can use low bit rates to approximate a Gaussian source. The entropy encoder is then used for encoding the final residual whose variance is reduced by the lattice encoder. The compression on the standard ultraspectral test dataset set of 10 AIRS granules shows that the proposed scheme outperforms DPVQ, JPEG2000 and the common compression utility BZIP2.

Research has been undertaken to examine the robustness of JPEG2000 when corrupted by transmission bit errors in a satellite data stream. Contemporary and future ultraspectral sounders such as Atmospheric Infrared Sounder (AIRS), Cross-track Infrared Sounder (CrIS), Infrared Atmospheric Sounding Interferometer (IASI), Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS), and Hyperspectral Environmental Suite (HES) generate a large volume of three-dimensional data. Hence, compression of ultraspectral sounder data will facilitate data transmission and archiving. There is a need for lossless or near-lossless compression of ultraspectral sounder data to avoid potential retrieval degradation of geophysical parameters due to lossy compression. This paper investigates the simulated error propagation in AIRS ultraspectral sounder data with advanced source and channel coding in a satellite data stream. The source coding is done via JPEG2000, the latest International Organization for Standardization (ISO)/International Telecommunication Union (ITU) standard for image compression. After JPEG2000 compression the AIRS ultraspectral sounder data is then error correction encoded using a rate 0.954 turbo product code (TPC) for channel error control. Experimental results of error patterns on both channel and source decoding are presented. The error propagation effects are curbed via the block-based protection mechanism in the JPEG2000 codec as well as memory characteristics of the forward error correction (FEC) scheme to contain decoding errors within received blocks. A single nonheader bit error in a source code block tends to contaminate the bits until the end of the source code block before the inverse discrete wavelet transform (IDWT), and those erroneous bits propagate even further after the IDWT. Furthermore, a single header bit error may result in the corruption of almost the entire decompressed granule. JPEG2000 appears vulnerable to bit errors in a noisy channel of satellite transmission, and thus has difficulty to preserve the quality of ultraspectral sounder data. A channel decoded bit error rate (BER) of 10-11 or better may be necessary for a granule error rate of 0.00116 in a compressed ultraspectral sounder data stream that is transmitted in a satellite channel. This work at The Aerospace Corporation and the University of Wisconsin, CIMSS, was under separate contracting from and performed for the National Oceanic and Atmospheric Administration (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS), a component of the U.S. Department of Commerce.

This paper presents current status of lossless compression of ultraspectral sounder data. The lossless compression results from the transform-based (e.g. JPEG2000, 3D SPIHT, and Lossless PCA), prediction-based (e.g. JPEG-LS, CALIC, and linear prediction using OOMP), and clustering-based (e.g. PVQ, DPVQ, PPVQ and FPVQ) methods are presented. The ultraspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases. Some robust data preprocessing scheme (e.g. BAR) is also demonstrated to improve compression gains of existing state-of-the-art compression methods such as JPEG2000, 3D SPIHT, JPEG-LS, and CALIC.

In lossless compression of the ultraspectral sounder data, the arithmetic coder is often used as the last stage of the compression scheme. While the arithmetic coder is good at compressing arbitrarily close to the entropy of the source, it is also known for its severe weakness to transmission noise. Even a single bit error in the transmission process can cause havoc and make the subsequent decoded stream completely useless. To cope with it, this paper adopts an arithmetic coder with a forbidden symbol and uses the maximum a posteriori (MAP) technique for decoding. Based on the ultraspectral sounder data, evaluation on the error correction capability of this error-resilient arithmetic coder will be reported.© (2005) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Improvements in weather and climate observation, analysis, and prediction will be achieved through advances of contemporary and future ultraspectral infrared sounders such as Atmospheric Infrared Sounder (AIRS), Tropospheric Emission Spectrometer (TES), Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS), and Hyperspectral Environmental Suite (HES). Given their unprecedented 3D data sizes to be generated each day, the use of robust data compression techniques will be beneficial to data transfer and archive. Lossless or near-lossless compression of this ultraspectral sounder data is desired to avoid potentially significant degradation of the geophysical parameter retrieval in an associated ill-posed inverse problem. In this paper we investigate various 2D and 3D compression techniques applicable to ultraspectral sounder data. These techniques include transform-based (JPEG2000, 3D-SPIHT), prediction-based (JPEG-LS, CALIC), and clustering-based (PVQ, DPVQ, PPVQ) compression methods. Data preprocessing schemes for compression gains are also illustrated.

The compression of three-dimensional ultraspectral sounder data is a challenging task given its unprecedented size. We develop a fast precomputed vector quantization (FPVQ) scheme with optimal bit allocation for lossless compression of ultraspectral sounder data. The scheme consists of linear prediction, bit-depth partitioning, vector quantization, and optimal bit allocation. Linear prediction serves as a whitening tool to make the prediction residuals of each channel close to a Gaussian distribution, and then these residuals are partitioned based on bit depths. Each partition is further divided into several sub-partitions with various 2/sup k/ channels for vector quantization. Only the codebooks with 2/sup m/ codewords for 2/sup k/-dimensional normalized Gaussian distributions are precomputed. A new algorithm is developed for optimal bit allocation among subpartitions. Unlike previous algorithms that may yield a sub-optimal solution, the proposed algorithm guarantees to find the minimum of the cost function under the constraint of a given total bit rate. Numerical experiments upon the NASA AIRS data show that the FPVQ scheme gives high compression ratios for lossless compression of ultraspectral sounder data.

The unprecedented size of ultraspectral sounder data makes its compression a challenging task. Ultraspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases. Previously, we proposed a reordering scheme to better explore these correlations of the ultraspectral sounder data. With this preprocessing scheme, the state-of-the-art compression algorithms such as CALIC, JPEG-LS and JPEG2000 significantly improve the compression ratios up to 15% on average. In this paper, we investigate the effects of different starting channels for spectral reordering on the lossless compression of 3D ultraspectral sounder data obtained from Atmospheric Infrared Sounder (AIRS) observations. It is shown that the compression ratios and reordering indices are dependent on the choice of the starting channel for reordering.

The compression of three-dimensional hyperspectral sounder data is a challenging task given its unprecedented size and nature. Vector quantization (VQ) is explored for the compression of this hyperspectral sounder data. The high dimensional vectors are partitioned into subvectors to reduce codebook search and storage complexity in coding of the data. The partitions are made by use of statistical properties of the sounder data in the spectral dimension. Moreover, the data is decorrelated at first to make it better suited for vector quantization. Due to the data characteristics, the iterative codebook generation procedure converges much faster and also leads to a better reconstruction of the sounder data. For lossless compression of the hyperspectral sounder data, the residual error and the quantization indices are entropy coded. The independent vector quantizers for different partitions make this scheme practical for compression of the large volume 3D hyperspectral sounder data.

Hyperspectral sounder data is used for retrieval of useful geophysical parameters which promise better weather prediction. It features two characteristics. First it is huge in size with 2D spatial coverage and high spectral resolution in the infrared region. Second it allows low tolerance of noise and error in retrieving the geophysical parameters where a mathematically ill-posed problem is involved. Therefore compression is better to be lossless or near lossless for data transfer and archive. Meanwhile medical data from X-ray computerized tomography (CT) or magnetic resonance imaging (MRI) techniques also possesses similar characteristics. It provides motivation to apply lossless compression schemes for medical data to the hyperspectral sounder data. In this paper, we explore the use of a wavelet-based lossless data compression scheme for the 3D hyperspectral data which uses in sequence a forward difference scheme, an integer wavelet transform, a Burrows-Wheeler transform and an arithmetic coder. Compared to previous work, our approach is shown to outperform the CALIC and 3D EZW schemes.

The compression of hyperspectral sounder data is beneficial for more efficient archive and transfer given its large 3-D volume. Moreover, since physical retrieval of geophysical parameters from hyperspectral sounder data is a mathematically ill-posed problem that is sensitive to the error of the data, lossless or near-lossless compression is desired. This paper provides an update into applications of state-of-the-art 2D and 3D lossless compression algorithms such as 3D EZW, 3D SPIHT, 2D JPEG2000, 2D JPEG-LS and 2D CALIC for hyperspectral sounder data. In addition, in order to better explore the correlations between the remote spectral regions affected by the same type of atmospheric absorbing constituents or clouds, the Bias-Adjusted Reordering (BAR) scheme is presented which reorders the data such that the bias-adjusted distance between any two neighboring vectors is minimized. This scheme coupled with any of the state-of-the-art compression algorithms produces significant compression gains.

The next-generation NOAA/NESDIS GOES-R hyperspectral sounder, now referred to as the HES (Hyperspectral Environmental Suite), will have hyperspectral resolution (over one thousand channels with spectral widths on the order of 0.5 wavenumber) and high spatial resolution (less than 10 km). Hyperspectral sounder data is a particular class of data requiring high accuracy for useful retrieval of atmospheric temperature and moisture profiles, surface characteristics, cloud properties, and trace gas information. Hence compression of these data sets is better to be lossless or near lossless. Given the large volume of three-dimensional hyperspectral sounder data that will be generated by the HES instrument, the use of robust data compression techniques will be beneficial to data transfer and archive. In this paper, we study lossless data compression for the HES using 3D integer wavelet transforms via the lifting schemes. The wavelet coefficients are processed with the 3D set partitioning in hierarchical trees (SPIHT) scheme followed by context-based arithmetic coding. SPIHT provides better coding efficiency than Shapiro's original embedded zerotree wavelet (EZW) algorithm. We extend the 3D SPIHT scheme to take on any size of 3D satellite data, each of whose dimensions need not be divisible by 2 N , where N is the levels of the wavelet decomposition being performed. The compression ratios of various kinds of wavelet transforms are presented along with a comparison with the JPEG2000 codec.

Hyperspectral sounder data is a particular class of data that requires high accuracy for useful retrieval of atmospheric temperature and moisture profiles, surface characteristics, cloud properties, and trace gas information. Therefore compression of these data sets is better to be lossless or near lossless. The next-generation NOAA/NESDIS GOES-R hyperspectral sounder, now referred to as the HES (Hyperspectral Environmental Suite), will have hyperspectral resolution (over one thousand channels with spectral widths on the order of 0.5 wavenumber) and high spatial resolution (less than 10 km). Given the large volume of three-dimensional hyperspectral sounder data that will be generated by the HES instrument, the use of robust data compression techniques will be beneficial to data transfer and archive. In this paper, we study lossless data compression for the HES using 3D integer wavelet transforms via the lifting schemes. The wavelet coefficients are then processed with the 3D embedded zerotree wavelet (EZW) algorithm followed by context-based arithmetic coding. We extend the 3D EZW scheme to take on any size of 3D satellite data, each of whose dimensions need not be divisible by 2N, where N is the levels of the wavelet decomposition being performed. The compression ratios of various kinds of wavelet transforms are presented along with a comparison with the JPEG2000 codec.© (2003) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Given the unprecedented volume of data (>72 Megabits per second) that will be generated by the future NOAA Geostationary Operational Environmental Satellite (GOES-R and beyond), the use of innovative data compression techniques will be essential if continuous downlink and re-broadcast from geo-orbit are to be economically feasible. A team of scientists and engineers from the Cooperative Institute for Meteorological Satellite Studies (CIMSS) of the University of Wisconsin-Madison, Offices of Research and Applications and Systems Development of NOAA/NESDIS, NASA/GSFC, and The Aerospace Corporation (a Federally Funded Research and Development Center) has been assembled to study the development of data compression for the next generation GOES sounder. This study is intended to define some feasible approaches for achieving both on-board (lossless) and ground-based (lossy or lossless) data compression. In general, on-board systems have substantially limited processing and storage capabilities, and modest compression ratios, compared with those of ground-based systems. Both highly efficient lossless and lossy algorithms therefore need to be developed to meet both on-board and ground processing objectives. In particular, innovative compression techniques for optimal quantization, transformation, coding, and decoding in interferogram or spectral domains will be essential for practical NOAA real-time operational data processing and distribution. In this presentation we will clearly define testing data sets (real and simulated), approaches, performance and feasibility of achieving hyperspectral data compression to provide a manageable data rate.

Surface and aircraft observations show that many cloud types tend to appear simultaneously at the same location but at different altitudes. Furthermore, clouds may be continuous or broken at a given cloud level within a sensor's field of view (FOV). Therefore, it is desirable that a general radiative transfer equation (RTE) can describe all the possible cloudy atmospheres for remote sensing applications. In this paper an M-level cloudy RTE is formulated. Specific diagrams and equations for up to four-level cloud systems are illustrated.

Current and future advanced atmospheric profile sounding and imaging instruments are evolving to enable global or hemispherical hyperspectral resolution measurements from space. The NASA/Navy/NOAA Geosynchronous Imaging FTS (GIFTS) for EO-3, NOAA Hyperspectral Environmental Sounder (HES) for GOES-R, and the currently operational Atmospheric Infrared Sounder (AIRS) on the NASA's Aqua Spacecraft will collect infrared high-spectral resolution/hyperspectral radiance spectra for remote sensing of the atmosphere, clouds, land, and ocean surfaces. These semi-continuous infrared high spectral- resolution/hyperspectral radiances will provide unprecedented information in the infrared region that is highly sensitive to absorption and emission of clouds. For sounding the atmospheric profiles one must perform cloud clearing or model the radiative effects of cloud explicitly if sounding is desired under cloud-contaminated conditions. We will describe the approach for modeling cloud attenuation in a fast-parameterized forward model that treats clouds as an additional absorber. Together with the usual clear forward model spectroscopic inputs, cloud altitude, effective particle size and shape and its ice or liquid water content are required input variables. Based on this efficient cloudy radiative transfer model, the simulation of the spatial and temporal coherent radiance images in three dimensions becomes possible. We will further explain how these 3-D GIFTS radiance cubes are used as test bed for a variety of trade studies.

Determining the Jacobians of the radiative transfer equation (RTE) is important to the qualities of the simultaneous retrieval of geophysical parameters from satellite radiance observations and the assimilation of radiance data into a numerical weather prediction system. Two linear forms of the RTE with analytic Jacobians are formulated. The first linear form has approximate analytic Jacobians, which involves some monochromatic approximation applied to a fast transmittance model. Unlike previous research, which lacks the transmittance Jacobian with respect to the atmospheric temperature profile, this form is complete in the sense that the transmittance Jacobians with respect to atmospheric temperature and absorbing constituent profiles are both present. The second linear form has exact analytic Jacobians derived consistently from the same fast transmittance model without using any monochromatic approximation. By numerical comparison between the two linear forms for the NOAA-12 High-Resolution Infrared Sounder, we show significant errors in the linear form with approximate analytic Jacobians. The relative absolute linearization error from the linear form with approximate analytic Jacobians is shown to be 2-4 orders of magnitude larger than that from the linear form with exact analytic Jacobians, even for the case of a 0.1% perturbation of the U.S. Standard Atmosphere. The errors unnecessarily complicate the ill-posed retrieval problem of atmospheric remote sensing and can be avoided if the correct linear form of the RTE with exact analytic Jacobians is adopted.

Visualizing light detection and ranging (LIDAR) scans consisting of millions of points on large displays poses a significant challenge. High-resolution large displays allow researchers to examine point clouds in detail. However, how to interact with point clouds rendered on large displays is a difficult problem. We present a case study that visualizes LIDAR point clouds on a tiled display wall termed highly interactive-parallelized display (HIPerDisplay). It has twenty 24-inch liquid-crystal displays with a total resolution of 46 Mpixels. Interaction between the user and the display wall is achieved by using a video camera system that is able to track the position of a hand-held light ball device. A user holds it to manipulate point clouds on the HIPerDisplay. Case studies are conducted to study the LIDAR scans of slopes in the Houshanyue mountain areas in Taiwan. Experiments were conducted to examine the advantages of using the HIPerDisplay for point clouds in data postprocessing. The experiments assess two tasks for manipulating point cloud data designed to evaluate the efficiency of the interactive devices. To evaluate the efficiency of the system, a group of 30 graduate students participated in the experiment. User surveys were performed to evaluate the efficiency of the system and to discover the users’ opinions about using the interactive device in a large display environment. The results showed that the participants preferred to perform LIDAR data operation tasks on a high-resolution large display environment rather than on a single monitor. The results also showed that the HIPerDisplay offered superior performance for the processing of large LIDAR datasets.

In recent years, satellite imaging technologies have resulted in an increased number of bands acquired by hyperspectral sensors, greatly advancing the field of remote sensing. Accordingly, owing to the increasing number of bands, band selection in hyperspectral imagery for dimension reduction is important. This paper presents a framework for band selection in hyperspectral imagery that uses two techniques, referred to as particle swarm optimization (PSO) band selection and the impurity function band prioritization (IFBP) method. With the PSO band selection algorithm, highly correlated bands of hyperspectral imagery can first be grouped into modules to coarsely reduce high-dimensional datasets. Then, these highly correlated band modules are analyzed with the IFBP method to finely select the most important feature bands from the hyperspectral imagery dataset. However, PSO band selection is a time-consuming procedure when the number of hyperspectral bands is very large. Hence, this paper proposes a parallel computing version of PSO, namely parallel PSO (PPSO), using a modern graphics processing unit (GPU) architecture with NVIDIA’s compute unified device architecture technology to improve the computational speed of PSO processes. The natural parallelism of the proposed PPSO lies in the fact that each particle can be regarded as an independent agent. Parallel computation benefits the algorithm by providing each agent with a parallel processor. The intrinsic parallel characteristics embedded in PPSO are, therefore, suitable for parallel computation. The effectiveness of the proposed PPSO is evaluated through the use of airborne visible/infrared imaging spectrometer hyperspectral images. The performance of PPSO is validated using the supervised K-nearest neighbor classifier. The experimental results demonstrate that the proposed PPSO/IFBP band selection method can not only improve computational speed, but also offer a satisfactory classification performance.

A future hyperspectral resolution remote imaging and sounding system, called the GIFTS (Geostationary Imaging Fourier Transform Spectrometer), is described. An airborne system, which produces the type of hyperspectral resolution sounding data to be achieved with the GIFTS, has been flown on high altitude aircraft. Results from simulations and from the airborne measurements are presented to demonstrate the revolutionary remote sounding capabilities to be realized with future satellite hyperspectral remote imaging/sounding systems.

The National Polar-orbiting Operational Satellite System (NPOESS) Aircraft Sounder Testbed-Interferometer (NAST-I) is one of two airborne infrared sounder systems currently being used to evaluate future spaceborne advanced sounder designs. The NAST-I instrument is a cross-track scanning Fourier Transform Spectrometer (FTS) that measures the upwelling radiation in the infrared spectrum between 645 - 2700 cm-1 (15.5 - 3.7 micrometer) at a high-spectral resolution of 0.25 cm-1. Each observation has a spatial resolution of 2.6 km from NASA's ER-2 high-altitude aircraft, which operates 20 km above the surface. Measurements from this instrument in its first year of operation have not only contributed to risk reduction studies for future IR sounders but have also provided valuable datasets from three different climate regimes. The spatial coverage, 40 km swath width, has facilitated evaluation of non-linear retrieval algorithms using high-spectral resolution information content and provided a means for further validation of infrared radiative transfer models. The capabilities of the NAST-I instrument have already been tested under varied field conditions such as tropical, mid-latitude summer, and mid- latitude winter regimes. Preliminary results from these field experiments have demonstrated favorable sounding capability under such conditions, including intensive tropical cyclone environments (Hurricane Bonnie, August 1998 and Hurricane Georges, September 1998). In addition, repeated observations over the same geographic location near Andros Island in the Bahamas have provided additional information on the temporal change and spatial distribution of water vapor responding to complex mesoscale and large-scale dynamic processes. The framework for future spaceborne IR sounders will be well established by current and future observations made by the NAST-I instrument with its capability to remotely sense atmospheric state variables and cloud radiative properties.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A new high spectral resolution (0.25 cm-1) and high spatial resolution (2.6 km) scanning (46 km swath width) Fourier Transform Spectrometer (FTS) has been built for flight on NASA high altitude (approximately 20 km) aircraft. The instrument, called the NPOESS Aircraft Sounding Testbed- Interferometer (NAST-I), has been flown during several field campaigns to provide experimental observations needed to finalize specifications and to test proposed designs for future satellite instruments; specifically, the Cross-track Infrared Sounder (CrIS) to fly on the National Polar-orbiting Operational Environmental Satellite System (NPOESS). NAST-I provides new and exciting observations of mesoscale structure of the atmosphere, including the fine scale thermodynamic characteristics of hurricanes. The NAST-I instrument is described, its excellent spectral and radiometric performance is demonstrated, and surface and atmospheric remote sensing results obtained during airborne measurement campaigns are presented.

The NASA sponsored Advanced Geosynchronous Studies (AGS) program is to conduct intensive studies to demonstrate the use of advanced new technologies and instruments on geosynchronous satellites to improve our current capabilities of monitoring the global weather, climate, and chemistry. The Geostationary Atmospheric Sounder (GAS), to be developed under AGS, is intended to demonstrate a new space-based infrared imaging interferometer that is well suited for achieving the high temporal and spatial global coverage of cloud motion, water vapor transport, thermal and moisture vertical profiles, land and ocean surface temperature, and trace gas concentrations. The AGS technology demonstrations will show the capabilities of passive infrared observations from future NOAA geostationary operational sounders. The focus of this presentation is to provide quantitative assessments of a few design configurations for the trace gases sounding feasibility from geostationary orbit. Trade-off studies of spectral, temporal, and spatial resolution are to be emphasized. Preliminary conclusions for the design of an operational geo sounder for chemistry applications will be made.

With recent technical advances in graphic processing units (GPUs), GPUs have out- performed CPUs in terms of compute capability and memory bandwidth. Many successful GPU applications to high performance computing have been reported. JPEG-LS is an ISO/IEC stan- dard for lossless image compression which utilizes adaptive context modeling and run-length coding to improve compression ratio. However, adaptive context modeling causes data depen- dency among adjacent pixels and the run-length coding has to be performed in a sequential way. Hence, using JPEG-LS to compress large-volume hyperspectral image data is quite time- consuming. We implement an efficient parallel JPEG-LS encoder for lossless hyperspectral compression on a NVIDIA GPU using the computer unified device architecture (CUDA) pro- gramming technology. We use the block parallel strategy, as well as such CUDA techniques as coalesced global memory access, parallel prefix sum, and asynchronous data transfer. We also show the relation between GPU speedup and AVIRIS block size, as well as the relation between compression ratio and AVIRIS block size. When AVIRIS images are divided into blocks, each with 64 × 64 pixels, we gain the best GPU performance with 26.3x speedup over its original CPU code. © 2012 Society of Photo-Optical Instrumentation Engineers (SPIE). (DOI: 10.1117/1.JRS.6

Distributed video coding (DVC) reduces the complexity at the encoder by using source statistics at the decoder. However, a performance gap still exists between DVC and hybrid video coders due to the inaccuracy of correlation noise (CN) modeling and inefficient exploitation of side information. We propose improvements to the performance of DVC using the two following aspects. Firstly, a progressive refinement method is proposed to improve the accuracy of CN modeling for transform domain Wyner-Ziv video coding, in which previously decoded bitplanes are exploited to progressively refine estimated CN as bitplane decoding proceeds. Secondly, a maximum likelihood pre-decoding method is also proposed to obtain a further reduction in bitrate. In our method, bitplanes are pre-decoded first using the conditional bit probability without syndrome bits. The bitrate is reduced by avoiding requests for syndrome bits for the bitplanes having strong temporal correlation. The experimental results show that our proposed methods could provide bitrate savings up to 8.26% and PSNR gains up to 0.2 dB without a significant increase in decoding complexity.

Comparing numerical simulation results with accelerograph readings is essential in earthquake investigations and discoveries. We provide a case study on the magnitude 7.6 Taiwan Chi-Chi earthquake in 1999. More than 400 seismic sensor stations recorded this event, and the readings from this event increased global strong-motion records fivefold so that the accuracy of the earthquake simulation was enhanced significantly. Direct volume rendering is used to depict the space-time relationships of numerical results and seismic readings. When earthquake simulation data are volume rendered, it reveals the sequence of seismic wave initiation, propagation, attenuation, and energy releasing events of fault ruptures so that the direction of seismic wave propagation can be observed. Both accelerograph readings and earthquake simulation data are used to generate a sequence of ground-motion maps. Stacking these maps up in sequence forms a volume data. Visual analysis of the time-varying component reveals hidden features for better comparison and evaluation. Earthquake scientists are able to obtain insights and evaluate their simulation criteria from volume rendering.

A typical LIDAR (light detection and ranging) scan contains hundreds of millions of points. As such, the visualization of LIDAR point clouds poses a significant challenge in data analysis. We propose to visualize and process LIDAR point clouds on a large display wall with an array of monitors. This provides researchers with a high-resolution display environment for looking at and studying large data sets. High-resolution large displays offer both global perspectives and local details of point clouds, which is essential in the process of data exploration. The ability to explore, conceptualize, and correlate spatial and temporal changes of topographical records is required for the development of new analytical models that capture the mechanisms contributing toward cliff erosion. Large displays driven by high-performance parallel visualization cluster allow researchers to fully interact with LIDAR point clouds of slopes in Houshanyue mountain and cliff failures observed in Solana Beach in California. In our study, cases studies of visualization based approaches were conducted using large displays in digital immersive environments. Visual analytics techniques such as delineation, segmentation, and classification of features, change detection, and annotation were used to perform erosion assessment. The results showed that the researchers can observe the temporal change of a failure mass effectively in high-resolution large display environments.

The radiative transfer for television operational vertical sounder (RTTOV) is a widely-used radiative transfer model (RTM) for calculation of radiances for satellite infrared and microwave sensors, including the 8461-channel infrared atmospheric sounding interferometer (IASI) and the 15-band Advanced Microwave Sounding Unit-A (AMSU-A). In the era of hyperspectral sounders with thousands of spectral channels, the computation of the RTM becomes more time-consuming. The RTM performance in operational numerical weather prediction systems still limits the number of used channels in hyperspectral sounders to only a few hundred. To take full advantage of such high-resolution infrared observations, a computationally efficient radiative transfer model is needed to facilitate satellite data assimilation. In this paper, we develop the parallel implementation of the RTTOV-7 IASI and AMSU-A RTMs to run the predictor module on CPUs in pipeline with the transmittance and radiance modules on NVIDIA many-core graphics processing units (GPUs). We show that concurrent execution of RTTOV-7 IASI RTM on CPU and GPU, in addition to asynchronous data transfer from CPU to GPU, allows the GPU accelerated code running on the 240-core NVIDIA Tesla C1060 to reach a speedup of 461× and 1793× for 1- and 4-GPU configurations, respectively. To compute one day's amount of 1,296,000 IASI spectra, the CPU code running on the host AMD Phenom II X4 940 CPU core with 3.0 GHz will take 2.8 days. Thus, GPU acceleration reduced running time to 8.75 and 2.25 min on 1- and 4-GPU configurations, respectively. Speedup for the RTTOV AMSU-A RTM varied from 29× to 75× for 1 and 4 GPUs, respectively.To further boost the speedup of a multispectral RTM, we developed a novel pure-GPU version of the RTTOV AMSU-A RTM where the predictor module also runs on GPUs to achieve a 96% reduction in the host-to-device data transfer. The speedups for the pure-GPU AMSU-A RTM are significantly increased to 56× and 125× for 1- and 4-GPU configurations, respectively.

Ultraspectral sounders revolutionize the way remote sensing data are collected, retrieved, and assimilated to provide better weather, climate, and environmental prediction and monitoring. This unprecedented amount of ultraspectral data increases the burden of data bandwidth and the chance of transmission noise and error contamination. It is desired that source coding of ultraspectral data also have some error-resilient capability, in addition to the error-correcting channel coding. Earlier, we developed three-dimensional wavelet reversible variable-length coding (3DWT-RVLC) for lossless compression of ultraspectral sounder data, which has significantly better error resilience than JPEG2000 Part 2 at only a small reduction in compression gain. The reversible variable-length codes allow instantaneous decoding in both directions, which affords better detection of bit errors due to synchronization losses over a noisy channel. To explore the feasibility of 3DWT-RVLC for real-time ultraspectral data processing, we implement a memory-limited digital signal processor (DSP) version of 3DWT-RVLC. Compression experiments on 10 ultraspectral test granules obtained from the NASA Atmospheric Infrared Sounder show that the memory-limited DSP-based 3DWT-RVLC is able to perform high-speed data processing at only a small reduction in compression ratio as compared to the original 3DWT-RVLC.